KR102640356B1 - Control method of system for non-face-to-face identification using artificial intelligence model leareed with face template data - Google Patents

Abstract

The present invention includes the steps of generating, by the electronic device, first facial template data from a facial image of an identification card in an identification card image generated by photographing an identification card; generating, by the electronic device, second facial template data from a user facial image in a user image generated by photographing the user; The server inputs the first facial template data and the second facial template data into a first artificial intelligence model trained to determine a facial match value between the faces represented by each of the two facial images, so that the face represented by the ID card facial image and obtaining a facial match value between faces represented by the user's facial image; obtaining, by the server, the facial liveness value of the user's facial image by inputting the second facial template data to a second artificial intelligence model trained to determine the facial liveness value of the facial image; and performing, by the server, user authentication to determine whether the identity of the identification card is the user based on the facial match value and the facial liveness value.

Description

Control method of a non-face-to-face identity authentication system using an artificial intelligence model learned with facial template data { CONTROL METHOD OF SYSTEM FOR NON-FACE-TO-FACE IDENTIFICATION USING ARTIFICIAL INTELLIGENCE MODEL LEAREED WITH FACE TEMPLATE DATA }

The present invention relates to a control method of a system including an electronic device and a server that performs non-face-to-face identity authentication using facial recognition and liveness. More specifically, the present invention relates to a facial match value between an identification card facial image and a user's facial image. The present invention relates to a control method of a system including an electronic device and a server that performs user identity authentication by determining whether the user's identity is that of the user based on the facial liveness value of the user's facial image.

In general, financial companies such as banks, securities companies, insurance companies, and fintech-related companies use face-to-face identity verification to verify the identity of customers who request financial services. However, this face-to-face identity authentication method has the inconvenience of requiring customers to visit a financial company, and there is the problem of not being able to use financial services outside of the business hours of the financial company. In addition, the face-to-face identity authentication method not only increases the burden of customer verification work for financial companies, but also becomes a factor in spreading infectious diseases such as viruses when they are prevalent, which prevents customers from visiting financial companies. There is a limit to reluctance.

Accordingly, non-face-to-face identity authentication methods have recently been in the spotlight. Figures 1 and 2 are conceptual diagrams to explain the conventional non-face-to-face identity authentication method and its problems. The conventional non-face-to-face identity authentication methods are ① taking a copy of the real name verification certificate or a copy of the ID card, ② comparing the ID card and photo through a video call with the customer, ③ confirmation when delivering the access medium (cash card, security card, OTP delivery), ④ existing The customer's identity is confirmed through two or more of five authentication methods: ⑤ utilization of customer account (1-way authentication), ⑤ certificate from another organization (e.g. public certificate), and other equivalent methods.

These conventional non-face-to-face identity authentication methods require the financial company to make a video call with the customer, deliver a cash card, security card, or OTP to the customer, or require the customer to check the deposit details in his or her account and provide an authentication number. The process for customers to receive financial services such as opening an account is complicated, as it requires entering and authenticating 1 won or preparing a certificate from another organization. In addition, when using the 1-won authentication method, there is a disadvantage that financial companies incur costs for 1-won authentication.

In addition, the conventional non-face-to-face identity authentication method has the inconvenience of requiring customers who are reluctant to expose unnecessary personal information beyond that required for identity verification to cover the last digits of their resident registration number with a note, etc. Non-face-to-face identity authentication methods using ID cards have the risk of ID information being leaked or ID cards being lost, and if the ID card is lost or not in your possession, you will not be able to authenticate yourself.

Conventional non-face-to-face identity authentication methods also have the problem of lower identity verification accuracy compared to face-to-face methods. In the conventional non-face-to-face identity authentication method, it is difficult to secure the real-time accuracy of the photo submitted by the customer, and if a customer tries to use financial services by forging or altering his/her ID card, it is difficult to detect it compared to the face-to-face method. Recently, there has been an increase in fraudulent activities such as forging or altering an ID card or ID photo, using a Daepo bankbook, entering an account number, verifying 1 won, and then completing the authentication with a Daepo phone to open a new account or get a loan approved. there is. There is a problem that customers may feel inconvenience and discomfort when requiring a highly secure authentication method from customers in order to increase the verification accuracy of identity authentication.

Korean Patent Publication No. 10-2022-0081476

The problem to be solved by the present invention is an electronic device and server that performs user identity authentication to determine whether the identity of the ID card is the user based on the facial match value between the facial image of the ID card and the user's facial image and the facial liveness value of the user's facial image. The purpose is to provide a control method for a system including.

The objects of the present invention are not limited to the objects mentioned above, and other objects and advantages of the present invention that are not mentioned can be understood through the following description and will be more clearly understood by examples of the present invention. Additionally, it will be readily apparent that the objects and advantages of the present invention can be realized by the means and combinations thereof indicated in the patent claims.

In the control method of a system including an electronic device and a server that performs non-face-to-face identity authentication using facial recognition and liveness according to one aspect of the present invention to solve the above-described problem, the electronic device Generating first facial template data from the ID card facial image in the ID card image generated by photographing; generating, by the electronic device, second facial template data from a user facial image in a user image generated by photographing the user; The server inputs the first facial template data and the second facial template data into a first artificial intelligence model trained to determine a facial match value between the faces represented by each of the two facial images, so that the face represented by the ID card facial image and obtaining a facial match value between faces represented by the user's facial image; obtaining, by the server, the facial liveness value of the user's facial image by inputting the second facial template data to a second artificial intelligence model trained to determine the facial liveness value of the facial image; and performing, by the server, user authentication to determine whether the identity of the identification card is the user based on the facial match value and the facial liveness value.

Preferably, generating the first facial template data includes: extracting, by the electronic device, a plurality of first facial feature points from the ID card facial image to generate first facial feature point data; and generating, by the electronic device, the first facial template data by encrypting and tokenizing the first facial feature point data.

Preferably, generating the second facial template data includes: extracting, by the electronic device, a plurality of second facial feature points from the user's facial image to generate second facial feature point data; and generating, by the electronic device, the second facial template data by encrypting and tokenizing the second facial feature point data.

Preferably, performing the user identity authentication includes: determining, by the server, whether the face match value exceeds a reference face match value; determining, by the server, whether the facial liveness value exceeds a reference facial liveness value; and determining, by the server, that the identity of the ID card is the user when the face match value exceeds the reference face match value and the face liveness value exceeds the reference face liveness value. can do.

The control method of the above-described system includes the steps of the electronic device or the server recognizing characters from the ID card image to generate ID information; requesting, by the server, to confirm the authenticity of the identification card based on the identification information; and receiving, by the server, identification information indicating authenticity of the identification card in response to the confirmation request.

Preferably, the step of performing the user identity authentication includes the server determining whether the identity of the identification card is the user based on the identity information of the identification card in addition to the facial match value and the facial liveness value, and The step of determining whether the identity of the identification card is that of the user includes: the server determining that the face match value exceeds the reference face match value, the face liveness value exceeds the reference face liveness value, and the identification card If the authenticity information indicates that the identification card is real, determining that the identity of the identification card is the user.

In the control method of a server that performs non-face-to-face identity authentication using facial recognition and liveness according to one aspect of the present invention to solve the above-described problem, the server uses an ID card in an ID image generated by photographing an ID card. Receiving first facial template data generated from a facial image; Receiving, by the server, second facial template data generated from a user facial image in a user image generated by photographing the user; The server inputs the first facial template data and the second facial template data into a first artificial intelligence model trained to determine a facial match value between the faces represented by each of the two facial images, thereby Obtaining a facial match value between a first face and a second face represented by the user's facial image; obtaining, by the server, the facial liveness value of the user's facial image by inputting the second facial template data to a second artificial intelligence model trained to determine the facial liveness value of the facial image; and performing, by the server, user authentication to determine whether the identity of the identification card is the user based on the facial match value and the facial liveness value.

Other specific details of the invention are included in the detailed description and drawings.

A control method of a system including an electronic device and a server according to the present invention includes determining whether the identity of the identification card is the user based on the facial match value of the facial image of the identification card and the user's facial image and the facial liveness value of the user's facial image. By performing authentication, quick and accurate authentication through identification is possible.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a configuration diagram of a system according to an embodiment of the present disclosure.
Figure 2 is a configuration diagram of an electronic device and a server included in a system according to an embodiment of the present disclosure.
Figure 3 is a flowchart of a user identity authentication process according to an embodiment of the present disclosure.
Figure 4 is an example diagram of an electronic device acquiring an identification card image according to an embodiment of the present disclosure.
5 and 6 are exemplary diagrams of how an electronic device acquires a user image according to an embodiment of the present disclosure.
FIG. 7 is a diagram illustrating a process in which a server acquires a facial liveness value for a facial image using an artificial intelligence model according to an embodiment of the present disclosure.
FIG. 8 is a diagram illustrating a process in which a server according to an embodiment of the present disclosure obtains a facial match value between the faces represented by each of the two facial images using an artificial intelligence model.

The advantages and features of the present invention and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. The present embodiments are merely provided to ensure that the disclosure of the present invention is complete and to provide a general understanding of the technical field to which the present invention pertains. It is provided to fully inform the skilled person of the scope of the present invention, and the present invention is only defined by the scope of the claims.

The terminology used herein is for describing embodiments and is not intended to limit the invention. As used herein, singular forms also include plural forms, unless specifically stated otherwise in the context. As used in the specification, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other elements in addition to the mentioned elements. Like reference numerals refer to like elements throughout the specification, and “and/or” includes each and every combination of one or more of the referenced elements. Although “first”, “second”, etc. are used to describe various components, these components are of course not limited by these terms. These terms are merely used to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may also be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless clearly specifically defined.

As used in the specification, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and the “unit” or “module” performs certain roles. However, “part” or “module” is not limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium and may be configured to run on one or more processors. Thus, as an example, a “part” or “module” refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within components and “parts” or “modules” can be combined into smaller components and “parts” or “modules” or into additional components and “parts” or “modules”. Could be further separated.

Spatially relative terms such as “below”, “beneath”, “lower”, “above”, “upper”, etc. are used as a single term as shown in the drawing. It can be used to easily describe the correlation between a component and other components. Spatially relative terms should be understood as terms that include different directions of components during use or operation in addition to the directions shown in the drawings. For example, if you flip a component shown in a drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. You can. Accordingly, the illustrative term “down” may include both downward and upward directions. Components can also be oriented in other directions, so spatially relative terms can be interpreted according to orientation.

In this specification, a computer refers to all types of hardware devices including at least one processor, and depending on the embodiment, it may be understood as encompassing software configurations that operate on the hardware device. For example, a computer can be understood to include, but is not limited to, a smartphone, tablet PC, desktop, laptop, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

1 is a configuration diagram of a system according to an embodiment of the present disclosure.

As shown in FIG. 1, a system according to one aspect of the present invention may include an electronic device 100 and a server 200.

The electronic device 100 and the server 200 transmit and receive information about the generated value, information about the judgment result, data, and signals, and can perform user identity authentication for the user.

Here, user identity authentication may be a process of determining whether the identity indicated by the ID card is the user.

The electronic device 100 may be implemented as a smartphone, tablet PC, TV, console device, set-top box, or other control device.

The server 200 performs user identity authentication for the user based on information about the generated value, information about the judgment result, data, and signals received from the electronic device 100, and reports the user identity authentication result to the electronic device ( 100).

Figure 2 is a configuration diagram of an electronic device and a server included in a system according to an embodiment of the present disclosure.

Referring to FIG. 2 , the electronic device 100 may include a memory 110, a camera 120, a display 130, a processor 140, and a communication unit 150.

The memory 110 can store various programs and data necessary for the operation of the electronic device 100. The memory 110 may be implemented as a non-volatile memory 110, a volatile memory 110, a flash-memory 110, a hard disk drive (HDD), or a solid state drive (SSD).

The processor 140 can control the overall operation of the electronic device 100 using various programs stored in the memory 110. The processor 140 may be comprised of RAM, ROM, a graphics processing unit, a main CPU, first to n interfaces, and a bus. At this time, RAM, ROM, graphics processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the electronic device 100 is booted, the O/S is stored in RAM, and various application data selected by the user may be stored in RAM.

ROM stores a set of instructions for booting the system. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 110 to RAM according to the command stored in the ROM, executes the O/S, and boots the system. When booting is complete, the main CPU copies various application programs stored in the memory 110 to RAM and executes the application programs copied to RAM to perform various operations.

The graphics processing unit uses a calculation unit (not shown) and a rendering unit (not shown) to create a screen containing various objects such as items, images, and text. Here, the calculation unit may be configured to calculate attribute values such as coordinate values, shape, size, color, etc. for each object to be displayed according to the layout of the screen using a control command received from the input unit. Additionally, the rendering unit may be configured to generate screens of various layouts including objects based on attribute values calculated by the calculation unit. The screen generated by this rendering unit may be displayed within the display area of the display 130.

The main CPU accesses the memory 110 and performs booting using the OS stored in the memory 110. And, the main CPU performs various operations using various programs, contents, data, etc. stored in the memory 110.

The first to n interfaces are connected to the various components described above. One of the first to n interfaces may be a network interface connected to an external device through a network.

Meanwhile, the processor 140 may include one or more cores (not shown) and a graphics processing unit (not shown) and/or a connection path (e.g., bus, etc.) for transmitting and receiving signals with other components. You can.

Processor 140 according to one embodiment performs the method described in connection with the present invention by executing one or more instructions stored in memory 110.

For example, the processor 140 acquires new training data by executing one or more instructions stored in the memory 110, performs a test on the acquired new training data using the learned model, and performs a test on the acquired new training data, and the test result is, Extract first training data in which the labeled information is obtained with an accuracy higher than a predetermined first reference value, delete the extracted first training data from the new training data, and learn using the new training data from which the extracted training data was deleted. The model can be trained again.

Meanwhile, the processor 140 includes random access memory (RAM) (not shown) and read-only memory (ROM) that temporarily and/or permanently store signals (or data) processed within the processor 140. , not shown) may be further included. Additionally, the processor 140 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

The memory 110 may store programs (one or more instructions) for processing and controlling the processor 140. Programs stored in the memory 110 may be divided into a plurality of modules according to their functions.

The camera 120 is configured to photograph at least one subject, and can photograph a user to create a user image including the user's face, and photograph an ID card to create an ID image.

The display 130 is a configuration for visually outputting various information and can be implemented as LCD (Liquid Crystal Display), PDP (Plasma Display Panel), OLED (Organic Light Emitting Diodes), TOLED (Transparent OLED), Micro LED, etc. However, it is not limited to this and may include various types of displays known in the art.

In an embodiment, the display 130 may be implemented in the form of a touch screen capable of detecting a user's touch operation, and may also be implemented as a flexible display that can be folded or bent.

The communication unit 150 may communicate with the server 200. In particular, the communication unit 150 may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a low-energy Bluetooth chip (BLE chip). At this time, the Wi-Fi chip, Bluetooth chip, and NFC chip communicate in the LAN method, WiFi method, Bluetooth method, and NFC method, respectively. When using a Wi-Fi chip or Bluetooth chip, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. A wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), and 5G (5th Generation).

Referring to FIG. 2 , the server 200 may include a memory 210, a processor 220, and a communication unit 230.

The memory 210 can store various programs and data necessary for the operation of the server 200. The memory 210 may be implemented as a non-volatile memory 210, a volatile memory 210, a flash memory 210 (flash-memory), a hard disk drive (HDD), or a solid state drive (SSD).

The processor 220 can control the overall operation of the server 200 using various programs stored in the memory 210. The processor 220 may be comprised of RAM, ROM, a graphics processing unit, a main CPU, first to n interfaces, and a bus. At this time, RAM, ROM, graphics processing unit, main CPU, first to n interfaces, etc. may be connected to each other through a bus.

RAM stores O/S and application programs. Specifically, when the server 200 is booted, the O/S is stored in RAM, and various application data selected by the user may be stored in RAM.

ROM stores a set of instructions for booting the system. When a turn-on command is input and power is supplied, the main CPU copies the O/S stored in the memory 210 to RAM according to the command stored in the ROM, executes the O/S, and boots the system. When booting is complete, the main CPU copies various application programs stored in the memory 210 to RAM and executes the application programs copied to RAM to perform various operations.

The main CPU accesses the memory 210 and performs booting using the OS stored in the memory 210. And, the main CPU performs various operations using various programs, contents, data, etc. stored in the memory 210.

The first to n interfaces are connected to the various components described above. One of the first to n interfaces may be a network interface connected to an external device through a network.

Meanwhile, furthermore, the processor 220 can control an artificial intelligence model. In this case, of course, the processor 220 may include a graphics processor (eg, GPU) for controlling the artificial intelligence model.

Meanwhile, the artificial intelligence model according to the present invention may be a model based on supervised learning or unsupervised learning. Furthermore, the artificial intelligence model according to the present invention may include SVM (support vector machine), decision tree, neural network, etc., and methodologies applied thereto.

In one embodiment, the artificial intelligence model according to the present invention may be an artificial intelligence model based on convolutional deep neural networks (CNN) that is learned by inputting learning data. However, it is not limited to this, and of course, various artificial intelligence models can be applied to the present invention. For example, models such as Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Bidirectional Recurrent Deep Neural Network (BRDNN) can be used as artificial intelligence models, but are not limited to these.

At this time, Convolutional deep Neural Networks (CNN) are a type of multilayer perceptrons designed to use minimal preprocessing. A convolutional neural network consists of one or several convolutional layers and general artificial neural network layers on top of them, and additionally utilizes weights and pooling layers. Thanks to this structure, the convolutional neural network can fully utilize input data with a two-dimensional structure. Additionally, convolutional neural networks can be trained via standard back propagation. Convolutional neural networks have the advantage of being easier to train and using fewer parameters than other feedforward artificial neural network techniques.

Additionally, Deep Neural Networks (DNN) are Artificial Neural Networks (ANN) consisting of multiple hidden layers between an input layer and an output layer.

At this time, the structure of the deep neural network may be composed of a perceptron. A perceptron consists of several inputs, one processor, and one output. The processor multiplies multiple input values by their respective weights and then adds up all of the input values multiplied by the weights. Next, the processor substitutes the combined value into the activation function and outputs one output value. If you want a specific value to come out as the output value of the activation function, you can modify the weight multiplied by each input value and recalculate the output value using the modified weight. At this time, each perceptron can use a different activation function. Additionally, each perceptron accepts the outputs from the previous layer as input and then obtains the output using an activation function. The obtained output is passed to the input of the next layer. By going through the above-described process, you can finally obtain several output values.

Recurrent Neural Network (RNN) refers to a neural network in which the connections between units that make up the artificial neural network form a directed cycle. Unlike a forward neural network, a recurrent neural network can utilize the memory inside the neural network to process arbitrary input.

Deep Belief Networks (DBN) is a generative graphical model used in machine learning. In deep learning, it refers to a deep neural network composed of multiple layers of latent variables. It has the characteristic that there is a connection between layers, but no connection between units within the layer.

Due to the nature of the deep trust neural network as a generative model, it can be used for prior learning, and after learning the initial weights through prior learning, the weights can be fine-tuned through backpropagation or other discriminant algorithms. This characteristic is very useful when there is little training data, because the smaller the training data, the stronger the influence of the initial value of the weights on the resulting model. The pre-learned weight initial value becomes closer to the optimal weight compared to the arbitrarily set initial weight value, which enables improved performance and speed of the fine-tuning stage.

The content regarding the above-described artificial intelligence and its learning method is described for illustrative purposes, and the artificial intelligence and its learning method used in the above-described embodiments are not limited. For example, all types of artificial intelligence technologies and learning methods that can be applied by a person skilled in the art to solve the same problem can be used to implement the system according to the disclosed embodiments.

Meanwhile, the processor 220 may include one or more cores (not shown) and a graphics processing unit (not shown) and/or a connection path (e.g., bus, etc.) for transmitting and receiving signals with other components. You can.

Processor 220 according to one embodiment performs the method described in connection with the present invention by executing one or more instructions stored in memory 210.

For example, the processor 220 acquires new training data by executing one or more instructions stored in the memory 210, uses a learned model, performs a test on the acquired new training data, and produces a test result, Extract first training data in which the labeled information is obtained with an accuracy higher than a predetermined first reference value, delete the extracted first training data from the new training data, and learn using the new training data from which the extracted training data was deleted. The model can be trained again.

Meanwhile, the processor 220 includes RAM (Random Access Memory, not shown) and ROM (Read-Only Memory) that temporarily and/or permanently store signals (or data) processed within the processor 220. , not shown) may be further included. Additionally, the processor 220 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

The memory 210 may store programs (one or more instructions) for processing and controlling the processor 220. Programs stored in the memory 210 may be divided into a plurality of modules according to their functions.

The communication unit 230 may perform communication with the electronic device 100. In particular, the communication unit 230 may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, a wireless communication chip, an NFC chip, and a low-energy Bluetooth chip (BLE chip). At this time, the Wi-Fi chip, Bluetooth chip, and NFC chip communicate in the LAN method, WiFi method, Bluetooth method, and NFC method, respectively. When using a Wi-Fi chip or Bluetooth chip, various connection information such as SSID and session key are first transmitted and received, and various information can be transmitted and received after establishing a communication connection using this. A wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), and 5G (5th Generation).

Figure 3 is a flowchart of a user identity authentication process according to an embodiment of the present disclosure.

As shown in FIG. 3, the control method of a system that performs non-face-to-face identity authentication using facial recognition and liveness according to one aspect of the present invention includes the server 200 confirming the authenticity of the ID card (S1) ), the server 200 determines whether the user's facial image is lively (S2), and the server 200 determines whether the face represented by the ID card image matches the face represented by the user image (S3) , the server 200 may perform user identity authentication (S4) based on whether the liveness determination result matches the face.

Figure 4 is an example diagram of an electronic device acquiring an identification card image according to an embodiment of the present disclosure.

Describing step S1, the electronic device 100 may output an ID photographing UI on the display that allows the user to photograph an identification card (ID), allowing the user to photograph an identification card (ID).

At this time, the electronic device 100 can obtain an ID card image through an auto-scan function that obtains the image as an ID card image when the ID card is recognized from an image captured by a camera.

According to another embodiment, the electronic device 100 displays a photographing button on the display, and when the photographing button is selected and input by the user, the electronic device 100 may obtain an identification card image using a camera at the time when the photographing button is selected and input.

Thereafter, the electronic device 100 may generate ID information by recognizing characters (ID-C) from the ID image (ID-I) generated by photographing the identification card (ID).

To this end, the electronic device 100 can recognize the character (ID-C) using a character reading module.

Here, the ID information may include one or more of ID card type information, ID card unique information (for example, resident registration number), ID card validity date information, ID card issue date information, name information, and address information of the identity indicated by the ID card.

Meanwhile, the electronic device 100 may recognize the outline of the ID card from the ID card image (ID-I) and blind some of the above-described ID information in the ID card image (ID-I).

Here, the ID may be any one of a physical resident registration card, driver's license, or passport.

Meanwhile, the identification card may be one of a mobile driver's license, mobile resident registration card, or electronic passport. At this time, the electronic device 100 can obtain ID information from a mobile driver's license, mobile resident registration card, or electronic passport through an electronic method without taking a photo of the ID card.

The electronic device 100 may transmit the generated ID information to the server 200.

Meanwhile, on behalf of the electronic device 100, the server 200 may generate ID information through the above-described character recognition.

To this end, the electronic device 100 transmits an identification card image (ID-I) generated by photographing an identification card (ID) to the server 200, and the server 200 receives the text (ID -C) can be recognized to generate ID information.

To this end, the server 200 can recognize the character (ID-C) using a character reading module.

The server 200 may request verification of the authenticity of the ID card from the verification agency server based on the ID information.

To this end, the server 200 may transmit an authenticity confirmation request signal along with ID information to the verification agency server.

Thereafter, the server 200 may receive ID authenticity information indicating the authenticity of the ID card from the verification agency server in response to a request to confirm the authenticity of the ID card.

Here, the verification agency server may be a server operated by the organization that issued the ID card to provide verification results regarding the authenticity of the ID card.

The server 200 may transmit the received ID authenticity information to the electronic device 100.

If the received ID authenticity information indicates that the ID card is real, the electronic device 100 may output a message indicating that the ID card is fake.

Additionally, if the received ID authenticity information indicates that the ID card is genuine, the electronic device 100 may obtain or generate images, data, and values required in the process of determining liveness for a subsequent user's facial image.

Conversely, if the received ID authenticity information indicates that the ID card is fake, the electronic device 100 may output a message indicating that the ID card is fake and a message requesting to take a photo of the real ID card.

Additionally, the electronic device 100 may re-perform the identification card photographing and identification information generation and transmit the identification information to the server 200.

5 and 6 are exemplary diagrams of how an electronic device acquires a user image according to an embodiment of the present disclosure.

Describing step S2, the electronic device 100 outputs a user photography UI on the display that allows the user to take a picture of himself, allowing the user U to take a picture of himself.

At this time, the electronic device 100 may determine whether passive liveness exists by photographing the user in a random pose and a random expression without inducing the user to take a specific pose and a specific expression.

Thereafter, the electronic device 100 extracts a plurality of second facial feature points from the user facial image (U-FI) in the user image (U-I) generated by photographing the user, and generates a second facial feature point composed of a plurality of second facial feature points. Feature point data can be generated.

Here, the plurality of second facial feature points may be information about facial features indicated by the user facial image (U-FI). For example, the plurality of second facial feature points include coordinate information for each detailed part of the face (corners of the eyes, corners of the mouth, tip of the nose, etc.), color values of the unit image of the position corresponding to each coordinate information, and unit of the position corresponding to each coordinate information. It may include the exposure value of the image, the change value of each coordinate information, the depth value of the image corresponding to each coordinate information (distance between camera and object), the change value of the depth value, the change value of the color value, and the change value of the exposure value. .

To this end, the electronic device 100 can extract more than 10,000 facial feature points from the user facial image (U-FI) using a pre-learned 3D facial geometric model.

The electronic device 100 may generate second facial template data by encrypting and tokenizing the second facial feature point data.

Thereafter, the electronic device 100 may transmit the second facial template data to the server 200.

FIG. 7 is a diagram illustrating a process in which a server acquires a facial liveness value for a facial image using an artificial intelligence model according to an embodiment of the present disclosure.

Referring further to FIG. 7, when the server 200 receives the second facial template data (U-FT) from the electronic device 100, the server 200 stores the second facial template data (U-FT) in the second artificial intelligence model (M2). ), the facial liveness value of the user's facial image (U-FI) can be obtained.

Here, the liveness of an image indicates whether the image was obtained from an actual object, and the liveness value may be a value indicating the probability that the image was obtained from an actual object.

Accordingly, the facial liveness value may be a value indicating the probability that the facial image corresponding to the facial template data was obtained from an actual subject (user).

For example, the closer the facial liveness value is to 100%, the higher the probability that the facial image corresponding to the facial template data was obtained from an actual subject (user).

Conversely, the closer the facial liveness value is to 0%, the more likely it is that the facial image corresponding to the facial template data was obtained from a photo, video, or mask of the user's facial shape in advance, rather than from the actual target (user). It can indicate high.

Meanwhile, the second artificial intelligence model (M2) may be an artificial intelligence model trained to determine the facial liveness value of the facial image.

To this end, the server 200 may be equipped with an artificial intelligence training module that trains a second artificial intelligence model that outputs facial liveness values.

Specifically, the server 200 may train the second artificial intelligence model (M2) using facial template data generated from facial images and learning data consisting of facial liveness values predetermined for the facial template data.

Through this, the server 200 inputs the second facial template data (U-FT) as input data to the second artificial intelligence model (M2), and the facial liveness value as output data of the second artificial intelligence model (M2). can be obtained.

If the acquired facial liveness value exceeds the reference facial liveness value, the server 200 determines that the user facial image (U-FI) is an image taken from an actual user, and determines that the user facial image (U-FI) has liveness. It can be judged to have .

Conversely, if the acquired facial liveness value does not exceed the reference facial liveness value, the server 200 determines that the user facial image (U-FI) is not an image taken from an actual user and determines that the user facial image (U-FI) is not an image taken from an actual user. ) can be judged to have no liveness.

The server 200 may transmit a result of determining whether the user's facial image (U-FI) is lively according to the facial liveness value to the electronic device 100.

If the determination result of whether the user's facial image (U-FI) has liveness indicates that the electronic device 100 has liveness, the electronic device 100 may output a message indicating that the user's facial image (U-FI) has liveness. there is.

In addition, if the result of determining whether the user's facial image (U-FI) has liveness indicates that it has liveness, the electronic device 100 determines whether the face represented by the subsequent ID card image matches the face represented by the user image. In the process, required images, data, and values can be acquired or generated.

Conversely, if the result of determining whether the user's facial image (U-FI) has liveness indicates that the electronic device 100 does not have liveness, the electronic device 100 sends a message indicating that the user's facial image (U-FI) does not have liveness. can be output.

In addition, if the determination result of whether the user's facial image (U-FI) has liveness indicates that it does not have liveness, the electronic device 100 outputs a message requesting to re-photograph the user's face, and outputs a message requesting to re-photograph the user's facial image (U-FI). The user image (U-I) including -FI) may be rephotographed, and the second facial template data (U-FT) may be generated again and transmitted to the server 200.

Meanwhile, referring back to FIG. 4 and explaining step S3, the electronic device 100 may print the facial image of the ID card in the ID image (ID-I) of the ID card (ID) confirmed to be genuine as a result of checking the authenticity of the ID card described above. A plurality of first facial feature points may be extracted from (ID-FI), and first facial feature point data consisting of a plurality of first facial feature points may be generated.

Here, the plurality of first facial feature points may be information about facial features indicated by the ID facial image (ID-FI). For example, the plurality of first facial feature points include coordinate information for each detailed part of the face (corners of the eyes, corners of the mouth, tip of the nose, etc.), color values of the unit image of the position corresponding to each coordinate information, and unit of the position corresponding to each coordinate information. It may include the exposure value of the image, the change value of each coordinate information, the depth value of the image corresponding to each coordinate information (distance between camera and object), the change value of the depth value, the change value of the color value, and the change value of the exposure value. .

To this end, the electronic device 100 can extract more than 10,000 facial feature points from the identification card facial image (ID-FI) using a pre-learned 3D facial geometric model.

The electronic device 100 may generate first facial template data by encrypting and tokenizing the first facial feature point data.

Thereafter, the electronic device 100 may transmit the first facial template data to the server 200.

FIG. 8 is a diagram illustrating a process in which a server according to an embodiment of the present disclosure obtains a facial match value between the faces represented by each of the two facial images using an artificial intelligence model.

When the server 200 receives the first facial template data (ID-FT) from the electronic device 100, the second facial template data determined to have liveness with the received first facial template data (ID-FT) (U-FT) is input into the first artificial intelligence model (M1) to obtain a facial match value between the face represented by the ID card face image (ID-FI) and the second face represented by the user face image (U-FI). You can.

Here, face matching indicates whether the faces represented by the two facial images are the same face, and the facial matching value may be a value representing the degree to which the faces represented by the two facial images are the same.

For example, the closer the face matching value is to 100%, the higher the probability that the faces represented by the two facial images are the same face.

Conversely, the closer the face matching value is to 0%, the higher the probability that the face represented by the two facial images is not the same.

Meanwhile, the first artificial intelligence model (M1) may be an artificial intelligence model trained to determine the facial match value of the facial image.

To this end, the server 200 may be equipped with an artificial intelligence training module that trains a first artificial intelligence model that outputs facial match values.

Specifically, the server 200 may train the first artificial intelligence model (M1) using facial template data generated from each of the two facial images and learning data consisting of predetermined facial matching values for the two facial images.

Through this, the server 200 inputs the first facial template data (ID-FT) and the second facial template data (U-FT) as input data to the first artificial intelligence model (M1), and the first artificial intelligence model (M1) The face matching value can be obtained using the output data of (M1).

If the obtained face match value exceeds the reference face match value, the server 200 determines that the face represented by the identification card face image (ID-FI) and the face represented by the user face image (U-FI) are the same, that is, they match. You can.

Conversely, if the obtained face match value does not exceed the reference face match value, the server 200 determines that the face represented by the identification card face image (ID-FI) and the face represented by the user face image (U-FI) are not the same, that is, It can be judged as inconsistent.

The server 200 may transmit to the electronic device 100 a determination result regarding whether the face represented by the identification card face image (ID-FI) matches the face represented by the user face image (U-FI) according to the face match value. there is.

The electronic device 100 may output a message as a result of determining whether the face represented by the identification card face image (ID-FI) matches the face represented by the user face image (U-FI) according to the face match value.

Describing step S4, the server 200 may perform user identity authentication to determine whether the identity of the ID card is the user based on the facial match value and the facial liveness value.

Additionally, the server 200 may perform user identity authentication to determine whether the identity of the ID card is the user based on the authenticity information of the ID card.

Specifically, in step S1, the server 200 confirms that the photographed ID card is an ID card issued by an ID issuing agency and is not a counterfeit ID card, in step S2, it is determined that the user's facial image has liveness, and in step S3, the ID card is confirmed to be a fake ID card. If it is determined that the face in the facial image matches the face in the user's facial image, it is determined that the identity of the ID card is the user, and user identity authentication can be performed.

At this time, the server 200 omits step S1, and in step S2, if it is determined that the user's facial image has liveness, and if it is determined that the face of the ID card's facial image and the face of the user's facial image match in step S3, the ID card User identity authentication may be performed by determining that the identity is that of the user.

The server 200 transmits the user identity authentication result to the electronic device 100, and the electronic device 100 may output the received user identity authentication result as a message.

Additionally, the server 200 may transmit the result of the user identity authentication to the authentication request server. The authentication request server may be a server operated by an entity that has requested user identity authentication. For example, the authentication request server may be a financial company server.

Meanwhile, the electronic device 100 according to another embodiment may acquire location information of a user attempting to authenticate the user and transmit the acquired location information of the user to the server 200.

Thereafter, the server 100 according to another embodiment may check country information corresponding to the location of the address indicated by the address information among the ID information.

Thereafter, the server 100 according to another embodiment may determine whether the user's location indicated by the location information is included in the country indicated by the country information.

To this end, the server 100 according to another embodiment may obtain reference data in advance such as area information for each country and address information for each country.

The server 100 according to another embodiment may perform the above-described facial match determination (step S3) and user identity authentication (step S4) only when the user's location indicated by the location information is within the country indicated by the country information. there is.

Accordingly, the server 100 according to another embodiment may determine whether the user's location indicated by the location information is within the country indicated by the country information after confirming the authenticity of the ID card (step S1).

Thereafter, if the user's location indicated by the location information is not included in the country indicated by the country information, the server 100 according to another embodiment determines whether the face matches described above (step S3) and authenticates the user's identity (step S4). It may not be performed.

Conversely, when the user's location indicated by the location information is within the country indicated by the country information, the server 100 according to another embodiment performs the above-described facial match determination (step S3) and user identity authentication (step S4). can do.

Meanwhile, the server 100 according to another embodiment may correct one or more of the reference face match value and the reference face liveness value based on the separation distance between the location of the address indicated by the address information and the user's location indicated by the location information. You can.

Specifically, the server 100 according to another embodiment may correct one or more of the reference face matching value and the reference face liveness value to increase as the separation distance increases.

Through this, when the location of the address is far from the location of the user, security can be improved by increasing the difficulty of authentication.

The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or a combination thereof. The software module may be RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, hard disk, removable disk, CD-ROM, or It may reside on any type of computer-readable recording medium well known in the art to which the present invention pertains.

Additionally, different embodiments of the present invention may complement or be combined with each other.

The components of the present invention may be implemented as a program (or application) and stored in a medium in order to be executed in conjunction with a hardware computer. Components of the invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as combinations of data structures, processes, routines or other programming constructs, such as C, C++, , it can be implemented in a programming or scripting language such as Java, assembler, Python, etc. Functional aspects may be implemented as algorithms running on one or more processors.

Above, embodiments of the present invention have been described with reference to the attached drawings, but those skilled in the art will understand that the present invention can be implemented in other specific forms without changing its technical idea or essential features. You will be able to understand it. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive.

100: electronic device
200: server

Claims (1)

In a method of controlling a system including an electronic device and a server,
generating, by the electronic device, first facial template data from a facial image of an ID card in an ID image generated by photographing an ID card;
generating, by the electronic device, second facial template data from a user facial image in a user image generated by photographing the user;
The server inputs the first facial template data and the second facial template data into a first artificial intelligence model trained to determine a facial match value between the faces represented by each of the two facial images, so that the face represented by the ID card facial image and obtaining a facial match value between faces represented by the user's facial image;
obtaining, by the server, the facial liveness value of the user's facial image by inputting the second facial template data to a second artificial intelligence model trained to determine the facial liveness value of the facial image; and
A step of performing, by the server, user identity authentication to determine whether the identity of the identification card is the user based on the facial match value and the facial liveness value,
The step of generating the first facial template data includes,
extracting, by the electronic device, a plurality of first facial feature points from the facial image of the ID card to generate first facial feature point data; and
A step of generating, by the electronic device, the first facial template data by encrypting and tokenizing the first facial feature point data,
The plurality of first facial feature points are
Includes first coordinate information for each facial detail part of the face represented by the ID card facial image, color values of unit images corresponding to each of the first coordinate information, and exposure values of unit images corresponding to each of the first coordinate information; ,
The step of generating the second facial template data is,
Extracting, by the electronic device, a plurality of second facial feature points from the user's facial image to generate second facial feature point data; and
A step of generating, by the electronic device, the second facial template data by encrypting and tokenizing the second facial feature point data,
The plurality of second facial feature points are
Second coordinate information for each facial detail part of the face represented by the user's facial image, color values of unit images corresponding to each of the second coordinate information, exposure values of unit images corresponding to each of the second coordinate information, and the first coordinate information. 2 Contains depth values of unit images corresponding to each piece of coordinate information,
The depth value is
It is the distance between the camera and the object that is the facial detail part of the face represented by the user's facial image,
The control method of the system includes, prior to generating the first facial template data,
The server training the first artificial intelligence model using facial template data generated from two facial images and learning data consisting of predetermined facial matching values for the two facial images; and
The server further includes training the second artificial intelligence model using facial template data generated from facial images and learning data consisting of facial liveness values predetermined for the facial template data,
The step of performing the user identity authentication is,
determining, by the server, whether the face match value exceeds a reference face match value;
determining, by the server, whether the facial liveness value exceeds a reference facial liveness value; and
and determining, by the server, that the identity of the ID card is the user when the face match value exceeds the reference face match value and the face liveness value exceeds the reference face liveness value. ,
The control method of the system includes, after the step of training the first artificial intelligence model and the step of training the second artificial intelligence model, before the step of transmitting the following ID authenticity information to the electronic device,
generating, by the electronic device or the server, identification information by recognizing characters from the identification image;
requesting, by the server, to confirm the authenticity of the ID card to a verification agency server based on the ID information; and
It further includes the step of receiving, by the server, identification information indicating authenticity of the identification card in response to the verification request from the verification agency server,
The verification agency server is
It is a server operated by the organization that issued the ID card to provide verification results regarding the authenticity of the ID card,
The control method of the system includes the steps of receiving the identity card authenticity information and before the step of generating the first facial template data,
It further includes the step of the server transmitting the received identity card authenticity information to the electronic device,
The steps for performing the user identity authentication are:
A method of controlling a system comprising: transmitting, by the server, a result of the user identity authentication to the electronic device.