KR102580026B1 - Method for training neural network for Identification card recognition based on distributed database and server for the method - Google Patents

Abstract

The present invention relates to a learning method of a neural network. The neural network learning method according to the present invention includes the steps of storing personal information and non-personal information in an identification card extracted from a neural network in separate databases; Extracting an identification card image from a first database among the databases and non-personal information corresponding to the identification card image from a second database using training data of the neural network; and updating operation parameters for extraction of non-personal information of a neural network using the extracted ID image and non-personal information. According to the present invention, overall recognition performance can be improved by increasing the prediction probability of non-personal information related to object recognition corresponding to personal information in an ID card.

Description

Neural network learning method and server for identification card recognition based on distributed database {Method for training neural network for Identification card recognition based on distributed database and server for the method}

The present invention relates to a neural network learning method, and to a neural network learning method using a distributed database.

Thanks to the development of mobile communication terminals, especially smartphones, and mobile communication technology, functions that were previously performed offline can now be easily and conveniently used anytime, anywhere through mobile applications.

In particular, in recent years, in order to improve user convenience, technology has been developed and serviced to process financial transactions non-face-to-face through applications, and various methods have been devised to verify the user's identity in non-face-to-face situations. there is.

In addition, with the development of artificial intelligence technology, artificial intelligence technologies are being applied to various fields, and methods of detecting and tracking objects in images through neural network models are being developed instead of existing image processing methods. By applying this artificial intelligence technology to ID recognition, instead of allowing users to directly input their identity information, the information in the ID card image taken from the smartphone camera is recognized, and through this, the methods of authenticating the user's identity are actually applied in financial services. there is.

However, continuous updating and learning are necessary to meet the needs of adaptation to various shooting environments and lightweighting of neural network models, and it is necessary to devise a customized learning method that minimizes the use of personal information as a result of identity authentication and related recognition.

The purpose of the present invention is to propose a neural network learning method for ID recognition.

Additionally, the purpose of the present invention is to propose an efficient learning method that uses information recognized on an ID card in a past authentication process.

Additionally, the purpose of the present invention is to propose a learning method based on a database structure that manages recognized information.

In the neural network learning method according to the present invention for solving the above technical problem, the steps include storing personal information and non-personal information in the ID card extracted from the neural network in separate databases; Extracting an identification card image from a first database among the databases and non-personal information corresponding to the identification card image from a second database using training data of the neural network; and updating operation parameters for extraction of non-personal information of a neural network using the extracted ID image and non-personal information.

It is preferable that the non-personal information is a prediction result value for the presence or absence of a location or deterioration element in the ID card image of personal information that affects the extraction of personal information by the neural network.

It is preferable that the personal information is a classification result value of an object existing in the ID card.

Preferably, the non-personal information is a result of a regression analysis of the area of the object existing in the ID card.

The updating step includes freezing the loss term for the extraction result of the personal information in a loss function that defines the error of the extraction result of the neural network, and calculating parameters for extraction of non-personal information using the loss function. It is desirable to update .

The updating step preferably updates parameters for regression analysis among the layers of the neural network, and the neural network outputs the classification result using the updated parameters.

It is preferable that the second database stores and manages the corresponding non-personal information using a key value for referencing an ID card image among the personal information stored in the first database.

The neural network learning server according to the present invention for solving the above technical problem includes a database that separately stores personal information and non-personal information in the ID card extracted from the neural network; a learning data extraction unit that extracts an identification card image from a first database among the databases and non-personal information corresponding to the identification card image from a second database as the learning data of the neural network; and a learning unit that updates operation parameters for extracting non-personal information of a neural network using the extracted ID image and non-personal information.

The first database consists of an image database that stores the ID card image and a personal information database that stores classification results of objects present in the ID card extracted from the neural network, and the learning data extractor stores the image database. It is desirable to extract the ID image from .

It is preferable that the non-personal information is a prediction result value for the presence or absence of a location or deterioration element in the ID card image of personal information that affects the extraction of personal information by the neural network.

It is preferable that the learning unit fixes a loss term for the extraction result of the personal information in a loss function that defines the error of the extraction result of the neural network, and updates calculation parameters for extraction of non-personal information using the loss function. do.

Preferably, the learning unit updates parameters for regression analysis among the layers of the neural network, and the neural network outputs the classification result using the updated parameters.

It is preferable that the second database stores and manages the corresponding non-personal information using a key value for referencing an ID card image among the personal information stored in the first database.

According to the present invention, a neural network capable of recognizing various information contained in an identification card can be efficiently trained.

According to the present invention, additional learning can be performed with only partial recognition results of a neural network for ID recognition, thereby improving prediction performance when updating the neural network model.

In addition, the present invention can reduce the problem of use and leakage of personal information by extracting and learning only the portion corresponding to non-personal information among the ID recognition results stored in a closed network.

In addition, the present invention can improve overall recognition performance by increasing the prediction probability of non-personal information related to object recognition corresponding to personal information in the ID card.

Figure 1 is an exemplary diagram showing a neural network learning system according to an embodiment of the present invention.
Figure 2 is a flowchart showing a neural network learning method according to an embodiment of the present invention.
3 and 4 are diagrams illustrating output data for learning of a neural network according to an embodiment of the present invention.
Figure 5 is a diagram illustrating the configuration of a database for learning a neural network according to an embodiment of the present invention.
Figure 6 is a diagram schematically showing the learning process of a neural network according to an embodiment of the present invention.
Figure 7 is a diagram showing the configuration of a loss function for learning a neural network according to an embodiment of the present invention.
Figure 8 is a block diagram showing the configuration of a server for learning a neural network according to an embodiment of the present invention.

The following merely illustrates the principles of the invention. Therefore, a person skilled in the art can invent various devices that embody the principles of the invention and are included in the concept and scope of the invention, although not clearly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, clearly intended only for the purpose of ensuring that the inventive concept is understood, and should be understood as not limiting to the specifically listed embodiments and states. .

The above-mentioned purpose, features and advantages will become clearer through the following detailed description in relation to the attached drawings, and accordingly, those skilled in the art in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

Additionally, when describing the invention, if it is determined that a detailed description of known technology related to the invention may unnecessarily obscure the gist of the invention, the detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

1 is a diagram showing the configuration of a learning system in which a neural network learning method is performed according to an embodiment of the present invention.

Referring to FIG. 1, the learning system according to this embodiment includes a user terminal 1000 that the user 10 uses to use a non-face-to-face financial service, and the identity information of the user 10 input from the user terminal 1000. It may be configured as a learning server 100 that receives information and performs identity authentication through it.

The learning server 100 receives the ID card image 50 from the user 10 in order to perform non-face-to-face identity authentication, and directly uses the learned neural network to identify objects required for authentication included in the ID card image 50. It can be recognized and output as a result.

For example, the text included in the name, resident registration number, and address information of the user 10 included in the ID card image 50 can be recognized using a neural network.

First, the learned neural network can output location information about the area where the object to be recognized exists as a recognition result.

As an example, information about the border of the ID card can be output to the entire area of the ID card image 50 as coordinates on the image. Alternatively, the value for the area where texts such as the owner's name and resident registration number exist as objects in the ID card image 50 can be detected and output.

At this time, the area can be defined as a rectangular bounding box. Specifically, the bounding box can be defined as a four-dimensional vector value of the x- and y-axis coordinates of the center point and the width w and height h of the border.

Additionally, a neural network can predict and output the value of the object to be recognized.

Specifically, classification results can be output according to the domain to which the object belongs. For example, in the case of a resident registration number, the classification result can be output as one of the numbers 0 to 9, or in the case of letters, the corresponding character can be output.

Additionally, the ID image 50 may include a printed facial photo for identification of the same person, and facial feature information extracted from the facial photo may also be generated as output.

In addition, anti-counterfeiting devices such as coatings and holograms are applied to ID cards, and when photographing ID cards in non-face-to-face situations, they can cause reflections or blocking phenomena, impeding recognition of objects. Therefore, it is also possible for the neural network to determine whether the captured ID image can be used by determining the presence or absence of these recognition deterioration factors.

The ID card image 50 includes various objects required for identity authentication, and therefore, the more accurately the neural network recognizes the included objects, the more reliable the identity authentication becomes.

Specifically, for identity authentication, it is possible to check whether the ID card has been stolen by comparing the object information recognized by the neural network with the information received from the issuing agency that issued the ID card. It is also possible to check the possibility of forgery or alteration of the ID card by storing and comparing past authentication results for more accurate authentication.

Accordingly, the learning server 100 can store and manage the results recognized using the neural network in a separate database 110.

However, since the recognition results are objects included in the ID card and contain the user's personal information, they are stored in the database 110 built within a separate closed network that cannot be accessed from the outside even within the system that provides non-face-to-face services. Access or modification may be denied.

Specifically, the database 110 may be composed of a first database 112 that stores personal information in the ID card and a second database 114 that stores non-personal information.

In this embodiment, the learning server 100 intends to use the existing recognition results stored in the database 110 for re-learning when the structure is updated according to a request for weight reduction or improvement in recognition rate after distribution of the application for ID recognition. do.

However, as mentioned above, since past recognition results include personal information and non-personal information, it is necessary to devise a learning method that can minimize the use of personal information for learning and improve the performance of the updated neural network. .

Therefore, the learning method according to this embodiment proposes an efficient learning method of a neural network using non-personal information in a database managed through a closed network.

Hereinafter, the learning method according to this embodiment will be described in more detail with reference to FIG. 2.

First, personal information and non-personal information in the ID card extracted from the neural network of the user 10 are stored in separate databases (S100).

The neural network may be pre-trained to output object recognition results for the ID card image 50.

Specifically, the neural network can output recognition results for location information of objects included in the ID card image 50 and classification results for objects, respectively.

In other words, the results output from the neural network may consist of personal information required to identify the ID holder and non-personal information unrelated to identification. At this time, the location information of the objects is non-personal information and may be a value that defines the area where the objects exist within the ID card image that is unnecessary for identification of the owner.

In addition, as a device included in the ID card to prevent forgery and falsification of the ID card, the recognition results of light reflection due to holograms, coatings, etc. or whether shadows are created due to occlusion during filming are unrelated to personal information and can therefore be included as non-personal information.

On the other hand, the name, address, and resident registration number used to identify the owner of the ID card are personal information, and facial feature information extracted from the identification photo printed on the ID card can also be used as personal information.

Therefore, the learning server divides the information in the ID card extracted from the neural network into personal information and non-personal information and stores them in each database.

Non-personal information corresponding to the ID card image 50 from the first database 112 and the ID card image 50 from the second database 114 are extracted from the database as training data of the neural network (S200).

In general, a neural network performs supervised learning using a training data set in which the regions and classifications of images and objects included in the images are pre-labeled as training data for learning.

In this embodiment, the neural network extracts the ID image and the label information labeled for the ID image from the database and uses it as learning data, but only values corresponding to non-personal information among the label information are extracted from the second database 114 and used.

In other words, the recognition result corresponding to personal information is not used for learning, but only the ID card image 50 used for recognition is extracted from the database 110 and used to reduce the scope of use of personal information.

Referring to FIG. 3, specifically, the neural network 30 uses a recognition result corresponding to personal information (70-1) as a classification value for the meaning of the object and a value for the object's presence area according to regression analysis. Results corresponding to non-personal information (70-2) can be output.

At this time, in this embodiment, the neural network 30 may be implemented as a CNN (Convolution Neural Network) model composed of a stacked structure of layers that perform convolution operations.

The convolution layers within the CNN model perform a convolution operation on the feature values in the input image data according to the set filters, and can generate a feature map according to the dimension determined as the output. The output final feature map can be output as a classification result and regression analysis result according to this embodiment while passing through each fully connected layer.

At this time, in addition to the CNN model described above, various algorithms such as R-CNN (Region with CNN), RNN (Recurrent Neural Network), and LSTM (Long Short-Term Memory) can be applied to the neural network depending on the properties of the object to be recognized. The implementation method is not limited to this.

Hereinafter, the results output from the ID card image 50 according to this embodiment will be described in more detail with reference to FIG. 4.

First, non-personal information 70-2 is location information of objects included in the ID card image 50 and may be coordinate values defined as a bounding box.

Accordingly, the border area information of the ID card and the area information of the text portion such as name or resident registration number within the ID card image 50 can be output as non-personal information (70-2).

In addition, it is also possible to determine whether a shadow or a hologram is present as recognition deterioration factors that affect identity authentication and output them as recognition results.

On the other hand, personal information 70-1 may be a classification result value regarding the meaning of objects included in the ID card image 50.

For example, text classification results in which the last name among the names is recognized as 'Hong' can be directly output, and the neural network 30 can output the numeric values of the resident registration number as classification results in the numeric domain.

Among the above output values, the value for the area where the object corresponding to non-personal information (70-2) exists is coordinate information of the bounding box and does not directly affect identity authentication.

However, the actual neural network 30 can classify the meaning of objects located within the area on the premise of accurate judgment of the area in which the objects exist in the image, so it provides non-personal information about the area (70-2) and Personal information (70-1) as a classification value has a correlation with each other during the output process and can be output from a common feature map.

Therefore, in this embodiment, the neural network 30 of the learning server 100 extracts the ID card image 50 and the label information about the area value of the object labeled with the ID card image 50 from the second database 114. I want to use it for learning.

In other words, instead of updating the internal parameters of the neural network 30 that directly affect the classification of objects, we want to indirectly improve classification performance by updating the parameters for extracting the presence area, which is the premise for classifying objects.

Therefore, the learning server updates the operation parameters for extracting non-personal information of the neural network using the extracted ID image 50 and non-personal information (S300).

Referring to FIG. 5, the first database 112 that specifically stores personal information can manage information by segmenting it according to the characteristics of the personal information.

The ID image 50 taken for ID recognition may also be considered personal information. Therefore, the first database 112 consists of an image database 112-1 that stores the ID card image 50 and a personal information database 112-2 that stores a personal information label defining an object classification value recognized from a neural network. It can be configured separately.

In this embodiment, the learning server 100 extracts values for the object area as a non-personal information label from the ID card image 50 and the second database 114 stored in the image database 112-1 as learning data and uses a neural network ( 30).

At this time, the second database 114 stores and manages non-personal information using a key value for referencing the ID card image 50 stored in the first database 112 (preferably the image database 112-1). By doing so, data extracted from both databases can be provided to the neural network 30 in pairs using the key value.

The neural network 30 receives the extracted data pair as learning data, and updates the parameters inside the neural network 30 using the error between the output result and the label information.

Specifically, the learning process will be explained in more detail through the configuration of the neural network 30.

Referring to FIG. 6, the neural network 30 for ID recognition is a layer layer 35 that performs a convolution operation on the input ID card and outputs classification results and regression results for the extracted feature map 38. It may be composed of a fully connected layer layer 36.

However, in this embodiment, the data input for learning of the neural network 30 is a result of region regression as shown in FIG. 5, and the label for the area of the object and the ID image 50 are input as a pair, so the object classification result There is no comparison target for calculating the error.

Accordingly, the learning server 100 may redefine the loss function that defines the error between the recognition result and the correct answer for learning.

Referring to FIG. 7, in this embodiment, for the non-personal information 70-2 output as a result of regression analysis among the recognition results for learning of the neural network 30, the label extracted from the second database 114 is used as the correct answer ( With Ground-Truth, learning can be performed by defining the difference from the correct value as the loss function of the neural network.

In other words, two types of recognition results corresponding to personal information and non-personal information can be output from the neural network. In this embodiment, for the non-personal information (70-2), the correct answer value is extracted from the second database 114 and used as learning data. By inputting , you can update the layers inside the neural network by calculating the difference between recognition results and back-propagating the error.

In other words, as in this embodiment, weakly supervised learning is performed in which the correct answer value is not set for some of the recognition results among the output, so the classification result corresponding to personal information (70-1) among the recognition results (70) For the value, the correct answer is not defined because the label is not extracted from the personal information database 112-2.

Therefore, the neural network 30 is trained by freezing the error for the output value corresponding to personal information 70-1 on the loss function side.

Specifically, by setting the loss of personal information (70-1) to a constant value (0), it affects non-personal information (70-1) without updating the fully connected layer related to the output of personal information (70-1). The parameters of the internal layers of the neural network 30 that affect can be selected and updated.

That is, instead of updating the internal parameters of the neural network 30 that directly affect the classification of objects, the classification performance can be indirectly improved by updating the parameters for domain regression, which is the premise for classifying objects.

Through the above process, the learning server 100 can efficiently learn the neural network 30 that can recognize various information included in the ID card.

In addition, the neural network that can recognize various information contained in the ID card can perform additional learning with only some recognition results to improve prediction performance when updating the neural network model.

Hereinafter, the configuration of the learning server 100 that performs the neural network learning method according to an embodiment of the present invention will be described.

Referring to FIG. 8, the learning server 100 may include a database 110, a learning data extraction unit 120, and a learning unit 130.

First, the database 110 stores recognition results using the neural network 30 that performs object recognition on the ID card image 50.

The database 110 can be implemented inside a closed network to prevent leakage of personal information included in recognition results, and in this embodiment, personal information and non-personal information can be stored separately according to data characteristics.

At this time, the first database 112 storing personal information and the second database 114 storing non-personal information may be physically independent, and the learning server 100 refers to files in each database to extract learning data. You can manage by defining key values for .

The first database 112 is an image database 112-1 that stores the ID card image 50 and a personal information database 112-2 that stores classification results of objects present in the ID card extracted from a neural network. It can be composed of:

Therefore, the learning server 100 classifies and stores the recognition results of various objects extracted from the ID card image 50 according to the criteria for whether or not they correspond to personal information. However, since the ID card image 50 itself is personal information, the first database 112 ) can be stored and managed in the image database 112-1.

The value defining the presence area on the ID card image 50 of the object as non-personal information is unrelated to user identification and may be stored and managed in the second database 114.

Next, the learning data extraction unit 120 extracts the non-personal information corresponding to the ID card image 50 from the first database 112 and the ID card image 50 from the second database 114 among the databases using the learning data of the neural network. do.

That is, the label for the area where the object exists in the ID card image 50 is extracted as learning data from the second database 114 using the key value.

Next, the learning unit 130 trains the neural network 30 by using the non-personal information label of the second database 114 among the extracted learning data as the correct value of the recognition result of the neural network 30.

Among the recognition results of the neural network 30, the correct answer value for personal information (70-1) shown as a classification result is not extracted and therefore not separately defined, but the correct answer value corresponding to non-personal information (70-2) is stored in the second database ( 114), learning can be performed by defining the extracted non-personal information label as the correct value and the difference between the recognition results as the loss function of the neural network 30.

In addition, the learning unit 130 manages the version of the neural network 30 to increase learning efficiency, and can perform learning using the recognition results of the previous version for prior learning before distribution according to the neural network update cycle. there is.

The present invention can reduce the problem of use and leakage of personal information by extracting and learning only the portion corresponding to non-personal information among the ID recognition results stored in a closed network.

In addition, the present invention can improve overall recognition performance by increasing the prediction probability of non-personal information related to object recognition corresponding to personal information in the ID card.

Furthermore, various embodiments described herein may be implemented in a recording medium readable by a computer or similar device, for example, using software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), and field programmable gate arrays (FPGAs). It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and other electrical units for performing functions. In some cases, as described herein, The described embodiments may be implemented as a control module itself.

According to software implementation, embodiments such as procedures and functions described in this specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code can be implemented as a software application written in an appropriate programming language. Software code is stored in the memory module and can be executed by the control module.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications, changes, and substitutions can be made by those skilled in the art without departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments and the attached drawings. . The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

Claims (12)

In the neural network learning method for ID recognition,
Storing personal information and non-personal information in the ID card extracted from the neural network in separate databases;
As training data of the neural network, extracting an identification card image from a first database among the databases and non-personal information corresponding to the identification card image from a second database among the databases; and
A step of updating operation parameters for extraction of non-personal information of a neural network using the extracted ID image and non-personal information,
The second database stores the non-personal information corresponding to a key value for referencing the ID card image stored in the first database,
A neural network learning method for ID recognition, characterized in that the key value is managed in the first database and the second database, respectively. According to claim 1,
A neural network learning method for ID recognition, characterized in that the non-personal information is a prediction result value for the presence or absence of a location or deterioration element in the ID card image of personal information that affects the extraction of personal information by the neural network. According to claim 1,
A neural network learning method for ID recognition, characterized in that the personal information is a classification result value of objects existing in the ID card. According to claim 3,
A neural network learning method for ID recognition, wherein the non-personal information is a result of a regression analysis of the area of the object existing in the ID card. According to claim 3,
The updating step freezes the loss term for the extraction result of the personal information in the loss function that defines the error of the extraction result of the neural network,
A neural network learning method for ID recognition, characterized in that the calculation parameters for extracting non-personal information are updated using the loss function. According to claim 5,
The updating step updates parameters for regression analysis among the layers of the neural network,
A neural network learning method for ID recognition, characterized in that the neural network outputs the classification result using updated parameters. In the neural network learning server for ID recognition,
A database that separately stores personal information and non-personal information in the ID card extracted from a neural network;
a learning data extraction unit that extracts an identification card image from a first database among the databases and non-personal information corresponding to the identification card image from a second database as the learning data of the neural network; and
A learning unit that updates operation parameters for extracting non-personal information of a neural network using the extracted ID image and non-personal information,
The second database stores the non-personal information corresponding to a key value for referencing the ID card image stored in the first database,
A neural network learning server for ID recognition, characterized in that the key value is managed in the first database and the second database, respectively. According to claim 7,
The first database consists of an image database that stores the ID card image and a personal information database that stores classification results of objects present in the ID card extracted from the neural network,
A neural network learning server for ID recognition, characterized in that the learning data extractor extracts an ID card image from the image database. According to claim 8,
A neural network learning server for ID recognition, characterized in that the non-personal information is a prediction result value for the presence or absence of a location or deterioration element in the ID card image of personal information that affects the extraction of personal information by the neural network. According to clause 9,
The learning unit fixes a loss term for the extraction result of the personal information in a loss function that defines an error in the extraction result of the neural network,
A neural network learning server for ID recognition, characterized in that it updates operation parameters for extracting non-personal information using the loss function. According to claim 10,
The learning unit updates parameters for regression analysis among the layers of the neural network,
A neural network learning server for ID recognition, wherein the neural network outputs the classification result using updated parameters. A computer-readable recording medium storing a program for performing the neural network learning method for ID card recognition according to any one of claims 1 to 7.

Images