KR102307671B1 - Method for face verification - Google Patents

Abstract

A computer program stored on a computer readable storage medium, wherein the computer program, when executed on one or more processors, is configured to perform the following operations for performing face authentication, the operations comprising: converting a first training data set to liveness; ) inputting the liveness model for performing the test to train the liveness model; performing the liveness check on an object included in image data using the learned liveness model; and performing face authentication by inputting image data that has passed the liveness test into a face authentication model for performing face authentication.

Description

METHOD FOR FACE VERIFICATION

The present disclosure relates to an information processing method using a computer, and more particularly, to a method for performing face authentication.

In the user authentication system, the computing device may determine whether to allow access to the corresponding computing device based on the user's body characteristic information. The user body characteristic information may be used for authentication of the computing device. The body characteristic information may include, for example, user's fingerprint information, user's iris information, and user's facial feature information.

Recently, interest in face authentication technology for authenticating using user's facial feature information is increasing. However, the security of face authentication may be lowered due to the development of face spoofing technology. The face spoofing technique may be a technique for accessing a computing device with a forged face of a user.

In order to prevent face spoofing, there is an increasing demand for a liveness test that extracts a user's facial features and determines whether an input face is a forged face based on the extracted features.

Korean Patent Application Laid-Open No. 10-2019-0063582 discloses a method for estimating a driver's gaze gaze area through transfer learning.

The present disclosure has been made in response to the above background art, and is intended to provide a method for performing face authentication.

According to an embodiment of the present disclosure for realizing the above-described problems, a computer program including instructions stored in a computer-readable storage medium to cause the computer to perform the following operations is disclosed. The operations may include: inputting a first training data set into a liveness model for performing a liveness test to train the liveness model; performing a liveness check on an object included in image data using the learned liveness model; and performing face authentication by inputting image data that has passed the liveness check into a face authentication model for performing face authentication.

In an alternative embodiment, the first training data set includes a first training data subset including image data including at least one living object or image data included in the first training data subset using an image acquisition device. It may include at least one of the second learning data subset including at least one image data captured by using the .

In an alternative embodiment, the liveness model comprises: a first submodel for extracting features of the input image data; a second submodel for obtaining a feature map by inputting an output value of the first submodel; or a third sub-model for obtaining a depth map by inputting an output value of the first sub-model; may include at least one of

In an alternative embodiment, the first submodel is a model including at least one convolutional neural network, and an output value of the first submodel may be a value determined based on an output value obtained from the at least one convolutional neural network. have.

In an alternative embodiment, the operation of training the liveness model by inputting the first training data set into the liveness model for performing the liveness check includes: a projection map ( and training the third submodel to reduce a difference between the projection map) and the output value of the third submodel.

In an alternative embodiment, the projection map may include a map in which the image data included in the first training data subset is converted into 3D image data, and then the converted 3D image data is projected onto a two-dimensional plane.

In an alternative embodiment, the operation of training the liveness model by inputting the first training data set into the liveness model for performing the liveness check comprises: an output value of the second submodel and an output value of the learned third submodel; The method may include training the second submodel in a direction to reduce a difference between the first map, which is an intersection, and a blood flow feature map including facial color information extracted from the input image data.

In an alternative embodiment, the training of the second submodel includes: acquiring a first map that is an intersection of an output value of the second submodel and an output value of the learned third submodel; inputting the calculated first map into a face alignment model to obtain a second map on which face alignment is performed; and learning the second submodel in a direction to reduce a difference between the second map and the blood flow feature map.

In an alternative embodiment, the blood flow feature map is a map expressed in black and white based on whether the blood flow is a part, and is a map in at least one of a red value, a green value, and a black value for each pixel. It may include a black-and-white value for each pixel calculated based on it.

In an alternative embodiment, performing the liveness check on the object included in the image data using the learned liveness model includes inputting an output of the learned first submodel to the learned second submodel. ; and performing the liveness check based on an output of the learned second submodel.

In an alternative embodiment, performing the liveness check based on the output of the second learned submodel may include: pixels having a feature value greater than or equal to a predetermined feature value in a feature map obtained using the learned second submodel. If the region including the ? is greater than or equal to the threshold size, determining that the liveness check has been passed may be included.

In an alternative embodiment, the face authentication model may include a model trained using a second training data set including at least one object.

In an alternative embodiment, the second training data set is one of first image data that is a target face image, second image data that is an image in which noise is included in the first image data, or third image data that is an image of a human face different from the target face. It may include at least one.

In an alternative embodiment, training the face authentication model includes: training to reduce a difference between the first image data and the second image data; and learning to increase a difference between the first image data and the third image data.

According to another embodiment of the present disclosure, a method for performing face authentication is disclosed. The method comprises: training the liveness model by inputting a first training data set into a liveness model for performing a liveness check; performing the liveness check on the object included in the image data using the learned liveness model; and performing face authentication by inputting image data that has passed the liveness check into a face authentication model for performing face authentication.

A computing device for performing face authentication is disclosed according to another embodiment of the present disclosure. one or more processors; and a memory storing instructions executable by the one or more processors; including, wherein the one or more processors train the liveness model by inputting a first training data set to a liveness model for performing a liveness check, and using the learned liveness model, A liveness test may be performed on the object, and image data that has passed the liveness test may be input to a face authentication model for performing face authentication to perform face authentication.

According to an embodiment of the present disclosure, a computing device for performing face authentication may be provided.

BRIEF DESCRIPTION OF THE DRAWINGS So that the above-mentioned features of the present disclosure may be understood in detail, with a more specific description, and with reference to the following embodiments, some of the embodiments are shown in the accompanying drawings. Also, like reference numerals in the drawings are intended to refer to the same or similar functions throughout the various aspects. However, it should be noted that the accompanying drawings only show certain typical embodiments of the present disclosure and are not to be considered as limiting the scope of the present disclosure, and other embodiments having the same effect may be fully appreciated. Take note.
1 is a block diagram of a computing device for performing face authentication according to an embodiment of the present disclosure.
2 is a schematic diagram exemplarily illustrating a neural network for a liveness check model and/or a face authentication model, according to an embodiment of the present disclosure.
3 is an exemplary diagram for explaining a liveness check operation.
4 is an exemplary diagram for explaining an operation for learning a third sub-model.
5 is an exemplary diagram for explaining an operation for learning the second sub-model.
6 is a flowchart for performing face authentication.
7 is a block diagram illustrating a module for performing face authentication.
8 is a simplified, general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Various embodiments are now described with reference to the drawings. In this specification, various descriptions are presented to provide an understanding of the present disclosure. However, it is apparent that these embodiments may be practiced without these specific descriptions.

The terms “component,” “module,” “system,” and the like, as used herein, refer to a computer-related entity, hardware, firmware, software, a combination of software and hardware, or execution of software. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and the computing device may be a component. One or more components may reside within a processor and/or thread of execution. A component may be localized within one computer. A component may be distributed between two or more computers. In addition, these components can execute from various computer readable media having various data structures stored therein. Components may communicate via a network such as the Internet with another system, for example via a signal having one or more data packets (eg, data and/or signals from one component interacting with another component in a local system, distributed system, etc.) may communicate via local and/or remote processes depending on the data being transmitted).

In addition, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from context, "X employs A or B" is intended to mean one of the natural implicit substitutions. That is, X employs A; X employs B; or when X employs both A and B, "X employs A or B" may apply to either of these cases. It should also be understood that the term “and/or” as used herein refers to and includes all possible combinations of one or more of the listed related items.

Also, the terms "comprises" and/or "comprising" should be understood to mean that the feature and/or element in question is present. However, it should be understood that the terms "comprises" and/or "comprising" do not exclude the presence or addition of one or more other features, elements and/or groups thereof. Also, unless otherwise specified or unless the context is clear as to designating a singular form, the singular in the specification and claims should generally be construed to mean "one or more."

In addition, the term “at least one of A or B” should be interpreted as meaning “includes only A”, “includes only B”, and “in the case of a combination of A and B”.

Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be implemented in electronic hardware, computer software, or combinations of both. It should be recognized that they can be implemented with To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application. However, such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

Descriptions of the presented embodiments are provided to enable those skilled in the art to use or practice the present invention. Various modifications to these embodiments will be apparent to those skilled in the art of the present disclosure. The generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present invention is not limited to the embodiments presented herein. This invention is to be construed in its widest scope consistent with the principles and novel features presented herein.

1 is a block diagram of a computing device for performing face authentication according to an embodiment of the present disclosure.

The configuration of the computing device 100 shown in FIG. 1 is only a simplified example. In an embodiment of the present disclosure, the computing device 100 may include other components for performing the computing environment of the computing device 100 , and only some of the disclosed components may configure the computing device 100 .

The computing device 100 may include a processor 110 , a memory 130 , and a network unit 150 .

The processor 110 may include one or more cores, and a central processing unit (CPU) of a computing device, a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU). unit) and the like, and may include a processor for data analysis and deep learning. The processor 110 may read a computer program stored in the memory 130 and perform data processing for machine learning according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the processor 110 may perform an operation for learning the neural network. The processor 110 for learning of the neural network, such as processing input data for learning in deep learning (DL), extracting features from the input data, calculating an error, updating the weight of the neural network using backpropagation calculations can be performed. At least one of a CPU, a GPGPU, and a TPU of the processor 110 may process learning of a network function. For example, the CPU and the GPGPU can process learning of a network function and data classification using the network function together. In addition, in an embodiment of the present disclosure, learning of a network function and data classification using the network function may be processed by using the processors of a plurality of computing devices together. In addition, the computer program executed in the computing device according to an embodiment of the present disclosure may be a CPU, GPGPU, or TPU executable program.

According to an embodiment of the present disclosure, the processor 110 may perform face authentication by performing the following operations. The processor 110 may perform a liveness check to prevent face spoofing. The liveness check may be to check whether the object to be checked is a living object. For example, the liveness test is to check whether a face included in an image captured by a camera is a real face of a person or a forged face. Liveness checks can be used, for example, to distinguish between non-living objects (e.g., photographs, paper, images, and models used as a means of forgery) and living objects (living people) in the course of performing one-of-a-kind authentication. have. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. According to an embodiment of the present disclosure, the liveness model may include a model for checking whether an object to be examined is a living object. Here, the model may include a machine learning model and a deep learning model. Specifically, the liveness model may include at least one convolutional neural network. According to an embodiment of the present disclosure, the training data set may be a data set used to train a model. The training data set may include at least one training data. The processor 110 may train the model using the training data set. The processor 110 may allow the model to acquire knowledge (eg, regularity, pattern information) included in the training data set by training the model using the training data set. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the first training data set may include a training data set for training a liveness model. The processor 110 may input the first training data set to the liveness model in order to train the liveness model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the first training data set is an image of a first training data subset including image data including at least one living object or image data included in the first training data subset It may include at least one of the second training data subset including at least one image data captured using the acquisition device. By dividing the first training data set into a first training data subset and a second training data subset, the processor 110 generates image data (forged The difference in liveness of image data) can be trained on the model. For example, the processor 110 may train a neural network that outputs a depth map. In this case, the processor 110 may train the neural network to output a depth map of a normal human face with respect to the first training data subset. On the other hand, the processor 110 may train the neural network to output the value of the depth map as 0 for the second training data subset. An image obtained by photographing an image including a human face is a fake image and may not have a three-dimensional effect because it is not a real human face. In this case, the processor 110 may train the neural network to output all depth maps as 0. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the first training data set may include at least one of a first training data subset or a second training data subset. According to an embodiment of the present disclosure, the first subset of learning data may include image data including living organisms. The living object is a living organism whose heart does not stop beating, and may be an object in which blood flows continuously under the skin. Therefore, the color of blood flow can appear clearly on the skin of a living object. On the other hand, in a non-living object, blood flow color may not appear clearly on the skin. For example, a dead person may not show blood color on their skin. As another example, the non-living object may also include a statue of a human face. In addition, the non-living object may include a picture of a person, a video, and the like. According to an embodiment of the present disclosure, the object may include an object included in an image. The object may include any material object with a certain shape included in the image. The object may be distinguished from the background. For example, the object may include a person, an animal, a car, and the like. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the second training data set may include at least one image data obtained by photographing the image data included in the first training data subset using an image acquisition device. The image acquisition device may include a camera and the like as a photographing device for acquiring a picture, a video, or the like. The second training data subset may be an image data set for attempting spoofing. In the process of performing face authentication, there may be spoofing that attempts face authentication with an image (eg, photo or video) including a human face, rather than attempting authentication with a human face. Spoofing can be an act that can create security concerns by tricking the computing device performing face authentication. Therefore, in order to prevent spoofing, there may be a need for the processor 110 to distinguish an image for attempting spoofing from an actual human face using a liveness model. Accordingly, the processor 110 configures the second training data subset as an image data set for attempting spoofing, thereby causing the liveness model to be applied to the face of a non-living person (eg, a photo containing a human face). It can also be trained to extract features for The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. The liveness model includes: a first sub-model for extracting features of input image data; a second submodel for obtaining a feature map by inputting an output value of the first submodel; or a third sub-model for obtaining a depth map by inputting an output value of the first sub-model; may include at least one of A feature may be a value including a feature of the image. The feature may be an output of a neural network. The feature may be a value obtained through a convolution operation. The processor 110 may acquire features of the image by using the learned neural network. The processor 110 may acquire features for each batch that are at least a part of the image. In addition, the processor 110 may acquire a feature for each pixel included in the image. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the first sub-model may be a model for extracting features of image data. According to an embodiment of the present disclosure, the first sub-model may be a model including at least one convolutional neural network. The output value of the first sub-model may include a value determined based on an output value obtained from at least one convolutional neural network. That is, the processor 110 may determine the output value of the first sub-model based on the output value of each of the at least one convolutional neural network included in the first sub-model. The processor 110 may determine the output value of the first sub-model by concatenating the output values of each of the at least one convolutional neural network. Each of the at least one convolutional neural network may have a different number of layers, a different number of nodes, and/or a connection relationship between different nodes. Through this, the processor 110 may obtain an output value of the first sub-model in which various output values are combined. That is, by combining values obtained through a convolutional neural network having each feature, information present in an image may be more accurately extracted. The above-described first sub-model is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the second sub-model may be a model for obtaining a feature map. The processor 110 may perform a liveness check using the output value of the second sub-model. The processor 110 may determine that the liveness check has been passed when an area of pixels having a feature value for each pixel in a feature map that is an output value of the second submodel is equal to or greater than a predetermined reference size. The threshold size may be a reference value determined based on whether blood flows in the image and can be determined as a living object. According to an embodiment of the present disclosure, the feature map may include a map indicating features of the image. The feature map may be a map in which features are expressed in a matrix. The feature map may include feature values for each arrangement of images or for each pixel. Through this, the processor 110 may acquire a specific part identified from other parts of the image. For example, when a human face is included in the image, the processor 110 may determine a portion where the human face is located through the feature map. In addition, the processor 110 may find out the part where the eyes, nose, and mouth of the person are located through the feature map. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the third sub-model may be a model for obtaining a depth map. The processor 110 may obtain the depth map by using the third submodel. Through this, the processor 110 may acquire face depth information, such as the height of the human nose, the height of the lips, the height of the eyebrows, and the like. According to an embodiment of the present disclosure, the depth map may be a map including depth information in an image. The depth map may be a map obtained by projecting a 3D image into a 2D image. The depth map may be represented as a black and white image. The depth map may have a lighter color as the height increases and a darker color as the height decreases. Also, the depth map may be expressed on the image as black as the object is further away and whiter as the object is closer. The foregoing is merely an example, and the present disclosure is not limited thereto.

Hereinafter, a learning process of the third sub-model will be described in detail with reference to FIG. 4 .

According to an embodiment of the present disclosure, the processor 110 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. Specifically, the processor 110 learns the third sub-model to reduce the difference between the output values of the projection map 350 and the third sub-model 270 for image data included in the first training data subset 310 . can do it According to an embodiment of the present disclosure, the projection map may include a map obtained by projecting a 3D image into a 2D image. The projection map may include a map projected from a 3D image to a 2D image including depth information. The projection map may include a map in which the 3D image data 330 is projected onto the projection map 350 . The projection map may include a map in which image data included in the first subset of learning data is converted into 3D image data, and then the converted 3D image data is projected onto a two-dimensional plane. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may obtain the projection map 350 for the image data included in the first training data subset 310 . The processor 110 may convert image data included in the first training data subset 310 into 3D image data 330 . In this case, the converted 3D image data 330 may be a 3D mesh of a human face. The processor 110 may acquire 3D image data using a 2D to 3D conversion model. 2D to 3D conversion model may include Framing U-net. The processor 110 may train the model to output a 3D face landmark by inputting a 2D image to the 2D to 3D transformation model. The training data set used to train the 2D to 3D transformation model may be a 2D image as the training input data and 3D face landmark data as the label data. The processor 110 may acquire the projection map 350 by projecting the 3D image data 330 in two dimensions. The processor 110 may obtain a projection map by projecting the 3D image data 330 in two dimensions by using a transformation matrix. The transformation matrix may be a transformation matrix in consideration of at least one of translation, rotation, or size transformation. Like the depth map, the projection map may be a map including depth information (or 3D structure information) in an image. Accordingly, in the learning process of the third sub-model 270 , the projection map 350 may be used as label data (correct answer data) in the supervised learning method. The processor 110 may obtain an output value 230 of the first sub-model by inputting the first subset of training data 310 to the first sub-model 210 . The processor 110 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the third sub-model 270 . The processor 110 may acquire a loss value 1 370 that is a difference between the acquired depth map 273 and the projection map 350 . The processor 110 may update the weight of the third sub-model 270 in the direction of reducing the loss value 1 370 in a backpropagation method. Through this, the processor 110 may obtain a depth map having depth information of a human face with high accuracy using the learned third sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may obtain the projection map 340 having Depth=0 for the image data included in the second training data subset 320 . The second training data set may include at least one image data obtained by photographing the image data included in the first training data subset using an image acquisition device. The second training data subset may be an image data set for attempting spoofing. Accordingly, since the images included in the second training data subset are photos and/or videos obtained by capturing a two-dimensional image through an image acquisition device, they may all have the same depth. For example, when an ID photo of a human face is taken through a camera, since the ID picture is a two-dimensional plane, the depths of all human faces may be equal to 0. The processor 110 may obtain an output value 230 of the first sub-model by inputting the second subset of training data 320 to the first sub-model 210 . The processor 110 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the third sub-model 270 . The processor 110 may acquire a loss value 1 370 that is a difference between the acquired depth map 273 and the projection map 340 having Depth=0. The processor 110 may update the weight of the third sub-model 270 in the direction of reducing the loss value 1 370 in a backpropagation method. Accordingly, the processor 110 may train the third sub-model to obtain a depth map having Depth=0 for the image included in the second training data subset 320 . Through this, when an image for attempting spoofing is received, the processor 110 may obtain a depth map having Depth=0 by using the third sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

Hereinafter, a method for learning the second sub-model will be described in detail with reference to FIG. 5 .

According to an embodiment of the present disclosure, the processor 110 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. The processor 110 includes a first map 410 that is an intersection of an output value of the second sub-model and an output value of the learned third sub-model and a blood flow feature map 450 including facial color information extracted from the input image data. The second sub-model may be trained in a direction to reduce the difference. The first map may include an intersection of a feature map 253 that is an output of the second sub-model and a depth map 273 that is an output of the third sub-model 271 . The processor 110 may acquire the blood flow color existing in the face region by acquiring the first map. A feature value for each pixel existing in the feature map may have a high feature value when blood flows through the corresponding pixel. In the depth map, pixels corresponding to the face region may have high values. For example, the processor 110 may obtain the intersection of pixels corresponding to the face region in the depth map when the feature value for each pixel present in the feature map is 0.5 or more. Through this, the processor 110 may determine the degree of blood flow color only for the pixel corresponding to the face. The foregoing is merely an example, and the present disclosure is not limited thereto.

(수학식1)을 이용하여 혈류 피처맵을 획득할 수 있다. S는 픽셀별 흑백값을 포함할 수 있다. R은 픽셀별 레드값, G는 픽셀별 그린값, B는 픽셀별 블랙값일 수 있다. a는 픽셀별 레드값, 그린값, 블랙값의 표준 편차에 기초하여 결정된 값일 수 있다. a는

(수학식2)에 기초하여 결정된 값일 수 있다. X는

와 같이 픽셀별 레드값과 픽셀별 그린값에 기초하여 결정된 값일 수 있다. Y는

와 같이 픽셀별 레드값, 픽셀별 그린값, 픽셀별 블랙값에 기초하여 결정된 값을 수 있다. 프로세서(110)는 제 1 맵(410)과 혈류 피처맵(253)과의 차이인 손실값2(470)를 감소시키는 방향으로 제 2 서브 모델(250)에 포함된 가중치를 역전파 방식으로 업데이트 할 수 있다. 프로세서(110)는 학습된 제 3 서브 모델에 포함된 가중치를 고정시키고 제 2 서브 모델에 포함된 가중치를 업데이트할 수도 있다. 프로세서(110)는 제 2 서브 모델의 학습 과정에서 제 3 서브 모델의 출력인 깊이맵도 포함시킴으로써, 얼굴 영역 부분에 대한 혈류색을 추출할 수 있다. 따라서 얼굴 영역에 대해서만 라이브니스 검사를 수행함으로써 라이브니스 모델의 정확도를 높일 수 있다. 전술한 사항은 예시일 뿐 본 개시는 이에 제한되지 않는다.According to an embodiment of the present disclosure, the processor 110 may obtain an output value 230 of the first sub-model by inputting the first subset of training data to the first sub-model. The processor 110 may obtain the feature map 253 by inputting the output value 230 of the first sub-model to the second sub-model 250 . In addition, the processor 110 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the learned third sub-model 271 . The processor 110 may obtain a first map 410 that is an intersection of the feature map 253 and the depth map 273 . The processor 110 may acquire the blood flow feature map 253 based on the first training data subset 310 . According to an embodiment of the present disclosure, the blood flow feature map is a map expressed in black and white based on whether or not the blood flow flows, and a black and white value for each pixel calculated based on at least one of a red value, a green value, and a black value for each pixel may include. The processor 110 is

(Equation 1) can be used to obtain a blood flow feature map. S may include a black and white value for each pixel. R may be a red value for each pixel, G may be a green value for each pixel, and B may be a black value for each pixel. a may be a value determined based on a standard deviation of a red value, a green value, and a black value for each pixel. a is

It may be a value determined based on (Equation 2). X is

It may be a value determined based on the red value for each pixel and the green value for each pixel as shown in FIG. Y is

A value determined based on a red value for each pixel, a green value for each pixel, and a black value for each pixel as shown in FIG. The processor 110 updates the weights included in the second sub-model 250 in a backpropagation method in a direction to reduce the loss value 2 470, which is a difference between the first map 410 and the blood flow feature map 253. can do. The processor 110 may fix the weight included in the learned third sub-model and update the weight included in the second sub-model. The processor 110 may extract the blood flow color for the face region by also including the depth map that is the output of the third sub-model in the learning process of the second sub-model. Therefore, it is possible to increase the accuracy of the liveness model by performing the liveness check only on the face region. The foregoing is merely an example, and the present disclosure is not limited thereto.

The description of the process of obtaining the black-and-white value for each pixel based on at least one of the red value, the green value, and the black value for each pixel is described in the paper Robust pulse-rate from chrominance-based rPPG (published date: 2013, by Gerard de Haan, Vincent Jeanne).

According to another embodiment of the present disclosure, the processor 110 may train the second sub-model. The processor 110 may obtain the first map 410 that is the intersection of the output value of the second sub-model 250 and the output value of the third sub-model 271 . The processor 110 may obtain a second map on which face alignment is performed by inputting the first map 410 into the face alignment model 420 . The face alignment may be an act of aligning the face so that the face faces the front when the face does not face the front. The second map 430 may be a map arranged so that the face included in the image faces the front. Hereinafter, the face alignment process will be described. The processor 110 may extract at least one fiducial point from the face included in the 2D image. The processor 110 may extract a reference point from the face using a first support vector regressor (SVR). The reference point is a reference point for facial alignment, and may include, for example, the center of the eye, the tip of the nose, the position of the lips, and the like. The processor 110 may extract at least one fixed point from the face-aligned 2D image by using the second SVR. The processor 110 may convert the 2D image into an aligned 3D image based on the reference point and the fixed point. Specifically, the processor 110 may convert the 2D image into an aligned 3D image through affine transformation based on the reference point and the fixed point. The foregoing is merely an example, and the present disclosure is not limited thereto.

A description of the process of performing face alignment is provided in the paper DeepFace: Closing the Gap to Human-Level Performance in Face Verification (published in 2014, by Yaniv Taigman, Ming Yang, Marc'), which is incorporated herein by reference in its entirety. Aurelio Ranzato, Lior Wolf).

According to an embodiment of the present disclosure, the processor 110 may train the second submodel in a direction to reduce a difference between the second map and the blood flow feature map 253 . The processor 110 may obtain a loss value 2 470 that is a difference between the second map 430 and the blood flow feature map 253 . The processor 110 may update the weight included in the second sub-model 250 in order to reduce the loss value 2 470 . The processor 110 may fix the weight included in the third sub-model 271 and update the weight included in the second sub-model 250 in order to reduce the loss value 2 470 . The foregoing is merely an example, and the present disclosure is not limited thereto.

Hereinafter, a process of performing the liveness check using the learned liveness model will be described in detail with reference to FIG. 3 .

According to an embodiment of the present disclosure, the processor 110 may perform a liveness check on an object included in image data using the learned liveness model. The processor 110 may input the output of the learned first sub-model to the learned second sub-model. The processor 110 may perform a liveness check based on the output of the learned second sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 determines that an area including pixels having a feature value greater than or equal to a predetermined feature value in the feature map 253 obtained using the learned second sub-model 250 is critical. If it is larger than the size ( 260 ), it may be determined that the liveness test has been passed ( 285 ). If the liveness check is passed ( 285 ), the processor 110 may perform a face authentication process ( 290 ). According to an embodiment of the present disclosure, the predetermined feature value may be a value determined based on a criterion for selecting a pixel through which blood flows in an image. Accordingly, the processor 110 may acquire a region through which blood flows in the image by acquiring pixels having a feature value equal to or greater than a predetermined feature value. According to an embodiment of the present disclosure, the threshold size may be a reference value determined based on whether the blood flow in the image can be determined as a living object. The processor 110 may determine a living object when an area including pixels having a feature value greater than or equal to a predetermined feature value in the image is greater than or equal to a threshold size. That is, the processor 110 may determine that the object included in the image is a living object. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, when an area including pixels having a feature value greater than or equal to a predetermined feature value in the feature map 253 obtained using the learned second sub-model 251 is less than a threshold size, the processor 110 may determine that the liveness check failed 280 . That is, when the area including pixels having a feature value greater than or equal to the predetermined feature value in the feature map 253 is less than the threshold size, it may mean that the blood flow is not flowing enough to be determined as a living object in the object included in the input image. . Accordingly, the processor 110 may determine the input image as an image for attempting spoofing and determine the liveness check failure 280 . The foregoing is merely an example, and the present disclosure is not limited thereto.

Hereinafter, the face authentication process will be described in detail.

According to an embodiment of the present disclosure, the face authentication model may include a model trained using the second training data set including at least one object. According to an embodiment of the present disclosure, face authentication may include identifying a person's face in image data and confirming that the person is a specific person. Face authentication may include actions to prevent spoofing. Specifically, the face authentication may be performed by comparing a person's facial features included in image data with data stored in a face database. According to an embodiment of the present disclosure, the face authentication model may be a model for identifying a person's face included in image data and confirming that the person is a specific person. The face authentication model may include a model that outputs features from a person's face to prevent spoofing. The processor 110 may compare a feature value that is an output of the face authentication model with a feature value of a specific person's face stored in the database to determine whether they are the same person. The model may include a machine learning model and a deep learning model. According to an embodiment of the present disclosure, the second training data set may be a training data set for training a face authentication model. The second training data set may include at least one of first image data that is a target face image, second image data that is an image in which noise is included in the target face, or third image data that is an image of a human face different from the target face. The reason why the second training data set includes the first image data, the second image data, and the third image data may be to train the face authentication model using a triplet loss technique. The target face image may be an image including a face of a specific person. The second image data may be an image in which noise is included in the first image data (target image data). The processor 110 may add noise to the first image data by arbitrarily changing the RGB values for each pixel of the first image data. Accordingly, the processor 110 may acquire an image in which noise is added to the target face image. Through this, a slightly deformed image may be obtained from the target face image of the processor 110 . The reason for generating the second image data is, for example, that RGB values may vary depending on lighting and image acquisition devices even for the same human face. The third image data may be a face image of a person different from the target face. The third image may be an image used in a face authentication model learning process to more accurately distinguish a specific person's face from another's face. The processor 110 may train the face authentication model in a direction that maximizes the difference between the first image data and the third image data. Through this, the processor 110 can clearly distinguish the faces of each of the plurality of people without confusing them by using the face authentication model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the processor 110 may train a face authentication model. The processor 110 may learn to reduce a difference between the first image data and the second image data. Specifically, the processor 110 may update the weight included in the face authentication model to reduce a loss value that is a difference between the first image data and the second image data. The processor 110 may learn to increase the difference between the first image data and the third image data. Specifically, the processor 110 may update the weight included in the face authentication model to increase the loss value that is the difference between the first image data and the third image data. Through this, the processor 110 may obtain a face authentication model that has a high recognition rate for the same person and distinguishes a specific person from another person with high accuracy.

According to the present disclosure, by using a second sub-model for extracting blood blood color information of a face from an image and a third sub model for extracting depth information from an image, both blood flow color and stereoscopic information (depth information) are used to improve liveness. inspection can be performed. Therefore, the anti-spoofing effect can be obtained by performing a liveness check with high accuracy before performing face authentication. In addition, in the face authentication step, the amount of computation, memory usage, power, etc. of the computing device can be saved by selectively performing face authentication only on images that have passed the liveness check.

2 is a schematic diagram exemplarily illustrating a neural network for a liveness check model and/or a face authentication model, according to an embodiment of the present disclosure.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is configured by including at least one or more nodes. Nodes (or neurons) constituting the neural networks may be interconnected by one or more links.

In the neural network, one or more nodes connected through a link may relatively form a relationship between an input node and an output node. The concepts of an input node and an output node are relative, and any node in an output node relationship with respect to one node may be in an input node relationship in a relationship with another node, and vice versa. As described above, an input node-to-output node relationship may be created around a link. One or more output nodes may be connected to one input node through a link, and vice versa.

In the relationship between the input node and the output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting the input node and the output node may have a weight. The weight may be variable, and may be changed by the user or algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node sets values input to input nodes connected to the output node and links corresponding to the respective input nodes. An output node value may be determined based on the weight.

As described above, in a neural network, one or more nodes are interconnected through one or more links to form an input node and an output node relationship within the neural network. The characteristics of the neural network may be determined according to the number of nodes and links in the neural network, the correlation between the nodes and the links, and the value of a weight assigned to each of the links. For example, when the same number of nodes and links exist and there are two neural networks having different weight values between the links, the two neural networks may be recognized as different from each other.

A neural network may include one or more nodes. Some of the nodes constituting the neural network may configure one layer based on distances from the initial input node. For example, a set of nodes having a distance n from the initial input node may constitute n layers. The distance from the initial input node may be defined by the minimum number of links that must be traversed to reach the corresponding node from the initial input node. However, the definition of such a layer is arbitrary for description, and the order of the layer in the neural network may be defined in a different way from the above. For example, a layer of nodes may be defined by a distance from the final output node.

The initial input node may refer to one or more nodes to which data is directly input without going through a link in a relationship with other nodes among nodes in the neural network. Alternatively, in a relationship between nodes based on a link in a neural network, it may mean nodes that other input nodes connected by a link do not have. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes among nodes in the neural network. In addition, the hidden node may mean nodes constituting the neural network other than the first input node and the last output node. The neural network according to an embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be the same as the number of nodes in the output layer, and the number of nodes decreases and then increases again as progresses from the input layer to the hidden layer. can In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be less than the number of nodes in the output layer, and the number of nodes decreases as the number of nodes progresses from the input layer to the hidden layer. have. In addition, the neural network according to another embodiment of the present disclosure may be a neural network in which the number of nodes in the input layer may be greater than the number of nodes in the output layer, and the number of nodes increases as the input layer progresses to the hidden layer. can The neural network according to another embodiment of the present disclosure may be a neural network in a combined form of the aforementioned neural networks.

A deep neural network (DNN) may refer to a neural network including a plurality of hidden layers in addition to an input layer and an output layer. Deep neural networks can be used to identify the latent structures of data. In other words, it can identify the potential structure of photos, texts, videos, voices, and music (for example, what objects are in the photos, what the text and emotions are, what the texts and emotions are, etc.) . Deep neural networks include convolutional neural networks (CNNs), recurrent neural networks (RNNs), auto encoders, generative adversarial networks (GANs), and restricted Boltzmann machines (RBMs). boltzmann machine), a deep belief network (DBN), a Q network, a U network, a Siamese network, and the like. The description of the deep neural network described above is only an example, and the present disclosure is not limited thereto.

In an embodiment of the present disclosure, the network function may include an auto-encoder. The auto-encoder may be a kind of artificial neural network for outputting output data similar to input data. The auto encoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between the input/output layers. The number of nodes in each layer may be reduced from the number of nodes of the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically with reduction from the bottleneck layer to the output layer (symmetrically with the input layer). In this case, although the dimensionality reduction layer and the dimension reconstruction layer are illustrated as being symmetrical in the example of FIG. 2 , the present disclosure is not limited thereto, and the nodes of the dimension reduction layer and the dimension reconstruction layer may or may not be symmetrical. The auto-encoder can perform non-linear dimensionality reduction. The number of input layers and output layers may correspond to the number of sensors remaining after pre-processing of input data. In the auto-encoder structure, the number of nodes of the hidden layer included in the encoder may have a structure that decreases as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes between the encoder and the decoder) is too small, a sufficient amount of information may not be conveyed, so a certain number or more (e.g., more than half of the input layer, etc.) ) may be maintained.

The neural network may be trained by at least one of supervised learning, unsupervised learning, and semi-supervised learning. The training of the neural network is to minimize the error in the output. In the training of a neural network, iteratively inputs the training data to the neural network, calculates the output of the neural network and the target error for the training data, and calculates the error of the neural network from the output layer of the neural network to the input layer in the direction to reduce the error. It is a process of updating the weight of each node in the neural network by backpropagation in the direction. In the case of teacher learning, learning data in which the correct answer is labeled in each learning data is used (ie, labeled learning data), and in the case of comparative learning, the correct answer may not be labeled in each learning data. That is, for example, the learning data in the case of teacher learning regarding data classification may be data in which categories are labeled in each of the learning data. Labeled training data is input to the neural network, and an error can be calculated by comparing the output (category) of the neural network with the label of the training data. As another example, in the case of comparison learning about data classification, an error may be calculated by comparing the input training data with the neural network output. The calculated error is back propagated in the reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the back propagation. A change amount of the connection weight of each node to be updated may be determined according to a learning rate. The computation of the neural network on the input data and the backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of repetitions of the learning cycle of the neural network. For example, in the early stage of training of a neural network, a high learning rate can be used to enable the neural network to quickly acquire a certain level of performance, thereby increasing efficiency, and using a low learning rate at the end of learning can increase accuracy.

In the training of neural networks, in general, the training data may be a subset of real data (that is, data to be processed using the trained neural network), and thus the error on the training data is reduced, but the error on the real data is reduced. There may be increasing learning cycles. Overfitting is a phenomenon in which errors on actual data increase by over-learning on training data as described above. For example, a phenomenon in which a neural network that has learned a cat by showing a yellow cat does not recognize that it is a cat when it sees a cat other than yellow may be a type of overfitting. Overfitting can act as a cause of increasing errors in machine learning algorithms. In order to prevent such overfitting, various optimization methods can be used. In order to prevent overfitting, methods such as increasing training data, regularization, or dropout in which a part of nodes in the network are omitted in the process of learning, may be applied.

A computer-readable medium storing a data structure is disclosed according to an embodiment of the present disclosure.

The data structure may refer to the organization, management, and storage of data that enables efficient access and modification of data. A data structure may refer to an organization of data to solve a specific problem (eg, data retrieval, data storage, and data modification in the shortest time). A data structure may be defined as a physical or logical relationship between data elements designed to support a particular data processing function. The logical relationship between data elements may include a connection relationship between data elements that a user thinks. Physical relationships between data elements may include actual relationships between data elements physically stored on a computer-readable storage medium (eg, a hard disk). A data structure may specifically include a set of data, relationships between data, and functions or instructions applicable to data. Through an effectively designed data structure, a computing device can perform an operation while using the resources of the computing device to a minimum. Specifically, the computing device may increase the efficiency of operations, reads, insertions, deletions, comparisons, exchanges, and retrievals through effectively designed data structures.

A data structure may be classified into a linear data structure and a non-linear data structure according to the type of the data structure. The linear data structure may be a structure in which only one piece of data is connected after one piece of data. The linear data structure may include a list, a stack, a queue, and a deck. A list may mean a set of data in which an order exists internally. The list may include a linked list. The linked list may be a data structure in which data is linked in such a way that each data is linked in a line with a pointer. In a linked list, a pointer may contain information about a link with the next or previous data. A linked list may be expressed as a single linked list, a doubly linked list, or a circularly linked list according to a shape. A stack can be a data enumeration structure with limited access to data. A stack can be a linear data structure in which data can be processed (eg, inserted or deleted) at only one end of the data structure. The data stored in the stack may be a data structure LIFO-Last in First Out. A queue is a data listing structure that allows limited access to data. Unlike a stack, the queue may be a data structure that comes out later (FIFO-First in First Out) as data stored later. A deck can be a data structure that can process data at either end of the data structure.

The nonlinear data structure may be a structure in which a plurality of data is connected after one data. The nonlinear data structure may include a graph data structure. A graph data structure may be defined as a vertex and an edge, and the edge may include a line connecting two different vertices. A graph data structure may include a tree data structure. The tree data structure may be a data structure in which one path connects two different vertices among a plurality of vertices included in the tree. That is, it may be a data structure that does not form a loop in the graph data structure.

Throughout this specification, computational model, neural network, network function, and neural network may be used interchangeably. (Hereinafter, the neural network is unified and described.) The data structure may include a neural network. And the data structure including the neural network may be stored in a computer-readable medium. Data structures, including neural networks, may also include data input to the neural network, weights of the neural network, hyperparameters of the neural network, data obtained from the neural network, activation functions associated with each node or layer of the neural network, and loss functions for learning the neural network. have. A data structure comprising a neural network may include any of the components disclosed above. That is, the data structure including the neural network includes all or all of the data input to the neural network, the weights of the neural network, the hyperparameters of the neural network, the data acquired from the neural network, the activation function associated with each node or layer of the neural network, and the loss function for training the neural network. may be configured including any combination of In addition to the above-described configurations, a data structure including a neural network may include any other information that determines a characteristic of a neural network. In addition, the data structure may include all types of data used or generated in the operation process of the neural network, and is not limited to the above. Computer-readable media may include computer-readable recording media and/or computer-readable transmission media. A neural network may be composed of a set of interconnected computational units, which may generally be referred to as nodes. These nodes may also be referred to as neurons. A neural network is configured by including at least one or more nodes.

The data structure may include data input to the neural network. A data structure including data input to the neural network may be stored in a computer-readable medium. The data input to the neural network may include learning data input in a neural network learning process and/or input data input to the neural network in which learning is completed. Data input to the neural network may include pre-processing data and/or pre-processing target data. The preprocessing may include a data processing process for inputting data into the neural network. Accordingly, the data structure may include data to be pre-processed and data generated by pre-processing. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

The data structure may include data input to or output from the neural network. A data structure including data input to or output from the neural network may be stored in a computer-readable medium. The data structure stored in the computer-readable medium may include data input in an inference process of the neural network or output data output as a result of inference of the neural network. In addition, since the data structure may include data processed by a specific data processing method, it may include data before and after processing. Accordingly, the data structure may include data to be processed and data processed through a data processing method.

The data structure may include the weights of the neural network. (In this specification, a weight and a parameter may be used interchangeably.) And a data structure including a weight of a neural network may be stored in a computer-readable medium. The neural network may include a plurality of weights. The weight may be variable, and may be changed by the user or algorithm in order for the neural network to perform a desired function. For example, when one or more input nodes are interconnected to one output node by respective links, the output node sets values input to input nodes connected to the output node and links corresponding to the respective input nodes. An output node value may be determined based on the parameter. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

By way of example and not limitation, the weight may include a weight variable in a neural network learning process and/or a weight in which neural network learning is completed. The variable weight in the neural network learning process may include a weight at the start of the learning cycle and/or a variable weight during the learning cycle. The weight for which neural network learning is completed may include a weight for which a learning cycle is completed. Accordingly, the data structure including the weight of the neural network may include a data structure including the weight variable in the neural network learning process and/or the weight in which the neural network learning is completed. Therefore, it is assumed that the above-described weights and/or combinations of weights are included in the data structure including the weights of the neural network. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

The data structure including the weights of the neural network may be stored in a computer-readable storage medium (eg, memory, hard disk) after being serialized. Serialization can be the process of converting a data structure into a form that can be reconstructed and used later by storing it on the same or a different computing device. The computing device may serialize the data structure to send and receive data over the network. A data structure including weights of the serialized neural network may be reconstructed in the same computing device or in another computing device through deserialization. The data structure including the weights of the neural network is not limited to serialization. Furthermore, the data structure including the weights of the neural network is a data structure to increase the efficiency of computation while using the resources of the computing device to a minimum (e.g., B-Tree, Trie, m-way search tree, AVL tree, Red-Black Tree). The foregoing is merely an example, and the present disclosure is not limited thereto.

The data structure may include hyper-parameters of the neural network. In addition, the data structure including the hyperparameters of the neural network may be stored in a computer-readable medium. The hyperparameter may be a variable variable by a user. Hyperparameters are, for example, learning rate, cost function, number of iterations of the learning cycle, weight initialization (e.g., setting the range of weight values subject to weight initialization), Hidden Unit The number (eg, the number of hidden layers, the number of nodes of the hidden layer) may be included. The above-described data structure is merely an example, and the present disclosure is not limited thereto.

3 is an exemplary diagram for explaining a liveness check operation.

3 shows the first sub-model 210, the output value 230 of the first sub-model, the learned second sub-model 251, the feature map 253, and the liveness determination criterion.

According to an embodiment of the present disclosure, the computing device 100 may perform a liveness check on an object included in image data using the learned liveness model. The computing device 100 may input an output of the learned first sub-model to the learned second sub-model. The computing device 100 may perform a liveness check based on the output of the learned second sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 includes an area including pixels having a feature value greater than or equal to a predetermined feature value in the feature map 253 obtained using the learned second sub-model 250 . If it is equal to or greater than the threshold size ( 260 ), it may be determined that the liveness test has been passed ( 285 ). If the liveness check is passed ( 285 ), the computing device 100 may perform a face authentication process ( 290 ). According to an embodiment of the present disclosure, the predetermined feature value may be a value determined based on a criterion for selecting a pixel through which blood flows in an image. Accordingly, the computing device 100 may acquire a region through which blood flows in the image by acquiring pixels having a feature value equal to or greater than a predetermined feature value. According to an embodiment of the present disclosure, the threshold size may be a reference value determined based on whether the blood flow in the image can be determined as a living object. The computing device 100 may determine a living object when an area including pixels having a feature value greater than or equal to a predetermined feature value in the image is greater than or equal to a threshold size. That is, the computing device 100 may determine that the object included in the image is a living object. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, when an area including pixels having a feature value greater than or equal to a predetermined feature value in the feature map 253 obtained using the learned second sub-model 251 is less than a threshold size, computing The device 100 may determine that the liveness check failed 280 . That is, when the area including pixels having a feature value greater than or equal to the predetermined feature value in the feature map 253 is less than the threshold size, it may mean that the blood flow is not flowing enough to be determined as a living object in the object included in the input image. . Accordingly, the computing device 100 may determine the input image as an image for attempting spoofing and determine the liveness check failure 280 . The foregoing is merely an example, and the present disclosure is not limited thereto.

4 is an exemplary diagram for explaining an operation for learning a third sub-model.

4, the first sub-model 210, the output value 230 of the first sub-model, the depth map 273, the first training data subset 310, 3D image data 330, the projection map 350, A second training data subset 320, a projection map 340 with Depth=0, and a loss value 1 370 are shown.

According to an embodiment of the present disclosure, the computing device 100 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. Specifically, the computing device 100 generates the third sub-model to reduce the difference between the output values of the projection map 350 and the third sub-model 270 for the image data included in the first training data subset 310 . can learn According to an embodiment of the present disclosure, the projection map may include a map obtained by projecting a 3D image into a 2D image. The projection map may include a map projected from a 3D image to a 2D image including depth information. The projection map may include a map in which the 3D image data 330 is projected onto the projection map 350 . The projection map may include a map in which image data included in the first subset of learning data is converted into 3D image data, and then the converted 3D image data is projected onto a two-dimensional plane. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may acquire the projection map 350 for the image data included in the first training data subset 310 . The computing device 100 may convert image data included in the first training data subset 310 into 3D image data 330 . In this case, the converted 3D image data 330 may be a 3D mesh of a human face. The computing device 100 may acquire 3D image data using a 2D to 3D transformation model. 2D to 3D conversion model may include Framing U-net. The computing device 100 may train the model to output a 3D face landmark by inputting a 2D image to the 2D to 3D transformation model. The training data set used to train the 2D to 3D transformation model may be a 2D image as the training input data and 3D face landmark data as the label data. The computing device 100 may acquire the projection map 350 by projecting the 3D image data 330 in two dimensions. The computing device 100 may obtain a projection map by projecting the 3D image data 330 in two dimensions by using a transformation matrix. The transformation matrix may be a transformation matrix in consideration of at least one of translation, rotation, or size transformation. Like the depth map, the projection map may be a map including depth information (or 3D structure information) in an image. Accordingly, in the learning process of the third sub-model 270 , the projection map 350 may be used as label data (correct answer data) in the supervised learning method. The computing device 100 may obtain an output value 230 of the first sub-model by inputting the first subset of training data 310 to the first sub-model 210 . The computing device 100 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the third sub-model 270 . The computing device 100 may acquire a loss value 1 370 that is a difference between the acquired depth map 273 and the projection map 350 . The computing device 100 may update the weight of the third sub-model 270 in the direction of reducing the loss value 1 370 in a back-propagation method. Through this, the computing device 100 may obtain a depth map having depth information of a human face with high accuracy by using the learned third sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may obtain the projection map 340 having Depth=0 for the image data included in the second training data subset 320 . The second training data set may include at least one image data obtained by photographing the image data included in the first training data subset using an image acquisition device. The second training data subset may be an image data set for attempting spoofing. Accordingly, since the images included in the second training data subset are photos and/or videos obtained by capturing a two-dimensional photo or video through an image acquisition device, all of the images may have the same depth. For example, when an ID photo of a human face is taken through a camera, since the ID picture is a two-dimensional plane, the depths of all human faces may be equal to 0. The computing device 100 may obtain the output value 230 of the first sub-model by inputting the second subset of training data 320 into the first sub-model 210 . The computing device 100 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the third sub-model 270 . The computing device 100 may acquire a loss value 1 370 that is a difference between the acquired depth map 273 and the projection map 340 having Depth=0. The computing device 100 may update the weight of the third sub-model 270 in the direction of reducing the loss value 1 370 in a back-propagation method. Accordingly, the computing device 100 may train the third sub-model to obtain a depth map having Depth=0 with respect to the image included in the second training data subset 320 . Through this, when an image for attempting spoofing is received, the processor 110 may obtain a depth map having Depth=0 by using the third sub-model. The foregoing is merely an example, and the present disclosure is not limited thereto.

5 is an exemplary diagram for explaining an operation for learning the second sub-model.

In FIG. 4 , the first sub model 210 , the output value 230 of the first sub model, the depth map 273 , the first training data subset 310 , the second sub model 270 , and the third sub model learned A model 271 , a feature map 253 , a depth map 273 , a first map 410 , a second map 430 , a blood flow feature map 450 , and a loss value 2 470 are shown.

According to an embodiment of the present disclosure, the computing device 100 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test. The computing device 100 includes a first map 410 that is an intersection of an output value of the second sub-model and an output value of a learned third sub-model, and a blood flow feature map 450 including facial color information extracted from input image data, and The second sub-model can be trained in a direction to reduce the difference between . The first map may include an intersection of a feature map 253 that is an output of the second sub-model and a depth map 273 that is an output of the third sub-model 271 . The computing device 100 may acquire the blood flow color existing in the face region by acquiring the first map. A feature value for each pixel existing in the feature map may have a high feature value when blood flows through the corresponding pixel. In the depth map, pixels corresponding to the face region may have high values. For example, the computing device 100 may obtain the intersection of pixels corresponding to the face region in the depth map when the feature value for each pixel present in the feature map is 0.5 or more. Through this, the computing device 100 may determine the degree of blood flow color of only the pixel corresponding to the face. The foregoing is merely an example, and the present disclosure is not limited thereto.

(수학식1)을 이용하여 혈류 피처맵을 획득할 수 있다. S는 픽셀별 흑백값을 포함할 수 있다. R은 픽셀별 레드값, G는 픽셀별 그린값, B는 픽셀별 블랙값일 수 있다. a는 픽셀별 레드값, 그린값, 블랙값의 표준 편차에 기초하여 결정된 값일 수 있다. a는

(수학식2)에 기초하여 결정된 값일 수 있다. X는

와 같이 픽셀별 레드값과 픽셀별 그린값에 기초하여 결정된 값일 수 있다. Y는

와 같이 픽셀별 레드값, 픽셀별 그린값, 픽셀별 블랙값에 기초하여 결정된 값을 수 있다. 컴퓨팅 장치(100)는 제 1 맵(410)과 혈류 피처맵(253)과의 차이인 손실값2(470)를 감소시키는 방향으로 제 2 서브 모델(250)에 포함된 가중치를 역전파 방식으로 업데이트 할 수 있다. 컴퓨팅 장치(100)는 학습된 제 3 서브 모델에 포함된 가중치를 고정시키고 제 2 서브 모델에 포함된 가중치를 업데이트할 수도 있다. 컴퓨팅 장치(100)는 제 2 서브 모델의 학습 과정에서 제 3 서브 모델의 출력인 깊이맵도 포함시킴으로써, 얼굴 영역 부분에 대한 혈류색을 추출할 수 있다. 따라서 얼굴 영역에 대해서만 라이브니스 검사를 수행함으로써 라이브니스 모델의 정확도를 높일 수 있다. 전술한 사항은 예시일 뿐 본 개시는 이에 제한되지 않는다.According to an embodiment of the present disclosure, the computing device 100 may obtain the output value 230 of the first sub-model by inputting the first subset of training data to the first sub-model. The computing device 100 may obtain the feature map 253 by inputting the output value 230 of the first sub-model to the second sub-model 250 . Also, the computing device 100 may obtain the depth map 273 by inputting the output value 230 of the first sub-model to the learned third sub-model 271 . The computing device 100 may obtain a first map 410 that is an intersection of the feature map 253 and the depth map 273 . The computing device 100 may acquire the blood flow feature map 253 based on the first training data subset 310 . According to an embodiment of the present disclosure, the blood flow feature map is a map expressed in black and white based on whether or not the blood flow flows, and a black and white value for each pixel calculated based on at least one of a red value, a green value, and a black value for each pixel may include. Computing device 100

(Equation 1) can be used to obtain a blood flow feature map. S may include a black and white value for each pixel. R may be a red value for each pixel, G may be a green value for each pixel, and B may be a black value for each pixel. a may be a value determined based on a standard deviation of a red value, a green value, and a black value for each pixel. a is

It may be a value determined based on (Equation 2). X is

It may be a value determined based on the red value for each pixel and the green value for each pixel as shown in FIG. Y is

A value determined based on a red value for each pixel, a green value for each pixel, and a black value for each pixel as shown in FIG. The computing device 100 reverse-propagates the weights included in the second sub-model 250 in the direction of reducing the loss value 2 470, which is the difference between the first map 410 and the blood flow feature map 253. can be updated The computing device 100 may fix the weight included in the learned third sub-model and update the weight included in the second sub-model. The computing device 100 may extract the blood flow color for the face region by also including the depth map that is the output of the third sub-model in the learning process of the second sub-model. Therefore, it is possible to increase the accuracy of the liveness model by performing the liveness check only on the face region. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to another embodiment of the present disclosure, the computing device 100 may train the second sub-model. The computing device 100 may obtain the first map 410 that is the intersection of the output value of the second sub-model 250 and the output value of the third sub-model 271 . The computing device 100 may obtain a second map on which face alignment is performed by inputting the first map 410 into the face alignment model 420 . The face alignment may be an act of aligning the face so that the face faces the front when the face does not face the front. The second map 430 may be a map arranged so that the face included in the image faces the front. Hereinafter, the face alignment process will be described. The computing device 100 may extract at least one fiducial point from a face included in the 2D image. The computing device 100 may extract a reference point from the face using a first support vector regressor (SVR). The reference point is a reference point for facial alignment, and may include, for example, the center of the eye, the tip of the nose, the position of the lips, and the like. The computing device 100 may extract at least one fixed point from the face-aligned 2D image by using the second SVR. The computing device 100 may convert the 2D image into an aligned 3D image based on the reference point and the fixed point. In more detail, the computing device 100 may convert the 2D image into an aligned 3D image through affine transformation based on the reference point and the fixed point. The foregoing is merely an example, and the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the computing device 100 may train the second submodel in a direction to reduce a difference between the second map and the blood flow feature map 253 . The computing device 100 may obtain a loss value 2 470 that is a difference between the second map 430 and the blood flow feature map 253 . The computing device 100 may update the weight included in the second sub-model 250 in order to reduce the loss value 2 470 . In order to reduce the loss value 2 470 , the computing device 100 may fix the weight included in the third sub-model 271 and update the weight included in the second sub-model 250 . The foregoing is merely an example, and the present disclosure is not limited thereto.

6 is a flowchart for performing face authentication.

According to an embodiment of the present disclosure, the computing device 100 may train the liveness model by inputting the first training data set to the liveness model for performing the liveness test ( 510 ).

According to an embodiment of the present disclosure, the first training data set is an image of a first training data subset including image data including at least one living object or image data included in the first training data subset It may include at least one of the second training data subset including at least one image data captured using the acquisition device.

According to an embodiment of the present disclosure, the liveness model includes: a first sub-model for extracting features of input image data; a second submodel for obtaining a feature map by inputting an output value of the first submodel; or a third sub-model for obtaining a depth map by inputting an output value of the first sub-model; may include at least one of

According to an embodiment of the present disclosure, the first submodel is a model including at least one convolutional neural network, and an output value of the first submodel is based on an output value obtained from the at least one convolutional neural network. It may be a determined value.

According to an embodiment of the present disclosure, the projection map may include a map in which the image data included in the first training data subset is converted into 3D image data, and then the converted 3D image data is projected onto a two-dimensional plane. .

According to an embodiment of the present disclosure, the blood flow feature map is a map expressed in black and white based on whether the blood flow is a part, and is calculated based on at least one of a red value, a green value, and a black value for each pixel. It can contain black and white values.

According to an embodiment of the present disclosure, the computing device 100 may perform a liveness check 520 on an object included in image data using the learned liveness model.

According to an embodiment of the present disclosure, the computing device 100 may perform face authentication by inputting image data that has passed the liveness test into a face authentication model for performing face authentication ( 530 ).

According to an embodiment of the present disclosure, the face authentication model may include a model trained using the second training data set including at least one object.

According to an embodiment of the present disclosure, the second training data set includes first image data that is a target face image, second image data that is an image in which noise is included in the first image data, or third image data that is a human face image different from the target face. It may include at least one of image data.

7 is a block diagram illustrating a module for performing face authentication.

A method for performing face authentication according to an embodiment of the present disclosure may be implemented by the following module. a module 610 for training a liveness model by inputting the first training data set into a liveness model for performing a liveness check; a module 620 for performing a liveness check on an object included in image data using the learned liveness model; and a module 630 for performing face authentication by inputting image data that has passed the liveness test into a face authentication model for performing face authentication.

In an alternative embodiment for performing face authentication, the module 610 for training a liveness model by inputting a first training data set into a liveness model for performing a liveness check comprises: the first training data subset and a module for training the third sub-model to reduce a difference between the output value of the projection map and the third sub-model for the image data included in the .

In an alternative embodiment for performing face authentication, the module 610 for training the liveness model by inputting the first training data set to the liveness model for performing the liveness check includes: the output value of the second submodel and a module for learning the second sub-model in a direction to reduce the difference between the first map, which is the intersection of the output values of the learned third sub-model, and the blood flow feature map including the complexion information of the face extracted from the input image data. can do.

In an alternative embodiment for performing face authentication, the module for training the second submodel comprises: a module for obtaining a first map that is an intersection of an output value of the second submodel and an output value of the learned third submodel; a module for inputting the calculated first map into the face alignment model to obtain a second map on which face alignment is performed; and a module for training the second submodel in a direction for reducing a difference between the second map and the blood flow feature map.

In an alternative embodiment for performing face authentication, the module 620 for performing a liveness check on an object included in the image data using the learned liveness model is configured to: input to the learned second submodel; and a module for performing the liveness check based on an output of the learned second submodel.

In an alternative embodiment for performing face authentication, the module for performing a liveness check based on the output of the learned second submodel includes: a predetermined feature in a feature map obtained using the learned second submodel and a module for determining that the liveness check has passed when the region including the pixel having the feature value equal to or greater than the threshold size is greater than or equal to the threshold size.

In an alternative embodiment for performing face authentication, the module for training the face authentication model includes: a module for training to reduce a difference between the first image data and the second image data; and a module for learning to increase a difference between the first image data and the third image data.

According to an embodiment of the present disclosure, a module for performing face authentication may be implemented by means, circuitry or logic for implementing a computing device. Those skilled in the art will further appreciate that the various illustrative logical blocks, configurations, modules, circuits, means, logics, and algorithm steps described in connection with the embodiments disclosed herein may be combined with electronic hardware, computer software, or combinations of both. It should be recognized that it can be implemented as To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, configurations, means, logics, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

8 is a simplified, general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has been described above generally as being capable of being implemented by a computing device, those skilled in the art will appreciate that the present disclosure is a combination of hardware and software and/or in combination with computer-executable instructions and/or other program modules that may be executed on one or more computers. It will be appreciated that it can be implemented as a combination.

Generally, program modules include routines, programs, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In addition, those skilled in the art will appreciate that the methods of the present disclosure are suitable for single-processor or multiprocessor computer systems, minicomputers, mainframe computers as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, etc. (each of which is It will be appreciated that other computer system configurations may be implemented, including those that may operate in connection with one or more associated devices.

The described embodiments of the present disclosure may also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Any medium accessible by a computer can be a computer readable medium, and such computer readable media includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-transitory media. including removable media. By way of example, and not limitation, computer-readable media may include computer-readable storage media and computer-readable transmission media. Computer-readable storage media includes volatile and non-volatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. includes media. A computer-readable storage medium may be RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital video disk (DVD) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device. device, or any other medium that can be accessed by a computer and used to store the desired information.

Computer readable transmission media typically embodies computer readable instructions, data structures, program modules or other data, etc. in a modulated data signal such as a carrier wave or other transport mechanism, and Includes any information delivery medium. The term modulated data signal means a signal in which one or more of the characteristics of the signal is set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

An example environment 1100 implementing various aspects of the disclosure is shown including a computer 1102 , the computer 1102 including a processing unit 1104 , a system memory 1106 , and a system bus 1108 . do. The system bus 1108 couples system components, including but not limited to system memory 1106 , to the processing device 1104 . The processing device 1104 may be any of a variety of commercially available processors. Dual processor and other multiprocessor architectures may also be used as processing unit 1104 .

The system bus 1108 may be any of several types of bus structures that may further be interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. System memory 1106 includes read only memory (ROM) 1110 and random access memory (RAM) 1112 . A basic input/output system (BIOS) is stored in non-volatile memory 1110, such as ROM, EPROM, EEPROM, etc., which is the basic input/output system (BIOS) that helps transfer information between components within computer 1102, such as during startup. contains routines. RAM 1112 may also include high-speed RAM, such as static RAM, for caching data.

The computer 1102 may also be configured for external use within an internal hard disk drive (HDD) 1114 (eg, EIDE, SATA) - this internal hard disk drive 1114 may also be configured for external use within a suitable chassis (not shown). Yes-, magnetic floppy disk drive (FDD) 1116 (eg, for reading from or writing to removable diskette 1118), and optical disk drive 1120 (eg, CD-ROM) for reading from, or writing to, disk 1122, or other high capacity optical media such as DVD. The hard disk drive 1114 , the magnetic disk drive 1116 , and the optical disk drive 1120 are connected to the system bus 1108 by the hard disk drive interface 1124 , the magnetic disk drive interface 1126 , and the optical drive interface 1128 , respectively. ) can be connected to The interface 1124 for external drive implementation includes at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer-readable media provide non-volatile storage of data, data structures, computer-executable instructions, and the like. In the case of computer 1102, drives and media correspond to storing any data in a suitable digital format. Although the description of computer readable media above refers to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will use zip drives, magnetic cassettes, flash memory cards, cartridges, etc. It will be appreciated that other tangible computer-readable media such as etc. may also be used in the exemplary operating environment and any such media may include computer-executable instructions for performing the methods of the present disclosure.

A number of program modules may be stored in the drive and RAM 1112 , including an operating system 1130 , one or more application programs 1132 , other program modules 1134 , and program data 1136 . All or portions of the operating system, applications, modules, and/or data may also be cached in RAM 1112 . It will be appreciated that the present disclosure may be implemented in various commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer 1102 via one or more wired/wireless input devices, for example, a pointing device such as a keyboard 1138 and a mouse 1140 . Other input devices (not shown) may include a microphone, IR remote control, joystick, game pad, stylus pen, touch screen, and the like. Although these and other input devices are often connected to the processing unit 1104 through an input device interface 1142 that is connected to the system bus 1108, parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, It may be connected by other interfaces, etc.

A monitor 1144 or other type of display device is also coupled to the system bus 1108 via an interface, such as a video adapter 1146 . In addition to the monitor 1144, the computer typically includes other peripheral output devices (not shown), such as speakers, printers, and the like.

Computer 1102 may operate in a networked environment using logical connections to one or more remote computers, such as remote computer(s) 1148 via wired and/or wireless communications. Remote computer(s) 1148 may be workstations, computing device computers, routers, personal computers, portable computers, microprocessor-based entertainment devices, peer devices, or other common network nodes, and are typically connected to computer 1102 . Although it includes many or all of the components described for it, only memory storage device 1150 is shown for simplicity. The logical connections shown include wired/wireless connections to a local area network (LAN) 1152 and/or a larger network, eg, a wide area network (WAN) 1154 . Such LAN and WAN networking environments are common in offices and companies, and facilitate enterprise-wide computer networks, such as intranets, all of which can be connected to a worldwide computer network, for example, the Internet.

When used in a LAN networking environment, the computer 1102 is connected to the local network 1152 through a wired and/or wireless communication network interface or adapter 1156 . Adapter 1156 may facilitate wired or wireless communication to LAN 1152 , which also includes a wireless access point installed therein for communicating with wireless adapter 1156 . When used in a WAN networking environment, the computer 1102 may include a modem 1158, be connected to a communication computing device on the WAN 1154, or establish communications over the WAN 1154, such as over the Internet. have other means. A modem 1158 , which may be internal or external and a wired or wireless device, is coupled to the system bus 1108 via a serial port interface 1142 . In a networked environment, program modules described for computer 1102 , or portions thereof, may be stored in remote memory/storage device 1150 . It will be appreciated that the network connections shown are exemplary and other means of establishing a communication link between the computers may be used.

Computer 1102 may be associated with any wireless device or object that is deployed and operates in wireless communication, for example, printers, scanners, desktop and/or portable computers, portable data assistants (PDAs), communications satellites, wireless detectable tags. It operates to communicate with any device or place, and phone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network or may simply be an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) makes it possible to connect to the Internet, etc. without a wire. Wi-Fi is a wireless technology such as cell phones that allows these devices, eg, computers, to transmit and receive data indoors and outdoors, ie anywhere within range of a base station. Wi-Fi networks use a radio technology called IEEE 802.11 (a, b, g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks may operate in unlicensed 2.4 and 5 GHz radio bands, for example, at 11 Mbps (802.11a) or 54 Mbps (802.11b) data rates, or in products that include both bands (dual band). .

One of ordinary skill in the art of this disclosure will understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, instructions, information, signals, bits, symbols and chips that may be referenced in the above description are voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields particles or particles, or any combination thereof.

Those of ordinary skill in the art of the present disclosure will recognize that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein include electronic hardware, (convenience For this purpose, it will be understood that it may be implemented by various forms of program or design code (referred to herein as software) or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. A person skilled in the art of the present disclosure may implement the described functionality in various ways for each specific application, but such implementation decisions should not be interpreted as a departure from the scope of the present disclosure.

The various embodiments presented herein may be implemented as methods, apparatus, or articles of manufacture using standard programming and/or engineering techniques. The term article of manufacture includes a computer program, carrier, or media accessible from any computer-readable storage device. For example, computer-readable storage media include magnetic storage devices (eg, hard disks, floppy disks, magnetic strips, etc.), optical disks (eg, CDs, DVDs, etc.), smart cards, and flash drives. memory devices (eg, EEPROMs, cards, sticks, key drives, etc.). Also, various storage media presented herein include one or more devices and/or other machine-readable media for storing information.

It is understood that the specific order or hierarchy of steps in the presented processes is an example of exemplary approaches. Based on design priorities, it is understood that the specific order or hierarchy of steps in the processes may be rearranged within the scope of the present disclosure. The appended method claims present elements of the various steps in a sample order, but are not meant to be limited to the specific order or hierarchy presented.

The description of the presented embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the embodiments presented herein, but is to be construed in the widest scope consistent with the principles and novel features presented herein.

Claims (16)

A computer program stored in a computer readable storage medium, wherein the computer program, when executed on one or more processors, performs the following operations for performing face authentication, the operations comprising:
inputting a first training data set into a liveness model for performing a liveness test to train the liveness model;
performing the liveness check on an object included in image data using the learned liveness model; and
performing face authentication by inputting the image data that has passed the liveness test into a face authentication model for performing face authentication;
including,
The face authentication model is
A model trained using a second training data set including at least first image data that is a target face image and third image data that is a human face image different from the target face,
The operation of learning the face authentication model includes:
learning to increase a difference between the first image data and the third image data;
comprising,
A computer program stored in a computer-readable storage medium.
The method of claim 1,
The first training data set is
a first subset of training data comprising image data comprising at least one living object; or
A second training data subset including at least one image data obtained by photographing the image data included in the first training data subset using an image acquisition device
comprising at least one of
A computer program stored in a computer-readable storage medium.
The method of claim 1,
The liveness model is
a first sub-model for extracting features of the input image data;
a second submodel for obtaining a feature map by inputting an output value of the first submodel; or
a third sub-model for obtaining a depth map by inputting an output value of the first sub-model;
comprising at least one of
A computer program stored in a computer-readable storage medium.
4. The method of claim 3,
The first sub-model is
A model comprising at least one convolutional neural network, comprising:
The output value of the first submodel is,
A value determined based on an output value obtained from at least one convolutional neural network,
A computer program stored in a computer-readable storage medium.
The method of claim 1,
The operation of learning the liveness model by inputting the first training data set into a liveness model for performing a liveness check comprises:
training the third sub-model to reduce a difference between an output value of a projection map for image data included in the first training data subset and an output value of the third sub-model;
comprising,
A computer program stored in a computer-readable storage medium.
6. The method of claim 5,
The projection map is
After converting the image data included in the first learning data subset into 3D image data, it is a map in which the converted 3D image data is projected onto a two-dimensional plane,
A computer program stored in a computer-readable storage medium.
The method of claim 1,
The operation of learning the liveness model by inputting the first training data set into a liveness model for performing a liveness check comprises:
The second sub-model in a direction to reduce the difference between the first map, which is the intersection of the output value of the second sub-model and the output value of the learned third sub-model, and the blood flow feature map including the complexion information of the face extracted from the input image data the operation of learning;
comprising,
A computer program stored in a computer-readable storage medium.
8. The method of claim 7,
The operation of learning the second sub-model is,
obtaining a first map that is an intersection of an output value of the second sub-model and an output value of the learned third sub-model;
inputting the obtained first map into a face alignment model to obtain a second map on which face alignment is performed; and
training a second submodel in a direction to reduce a difference between the second map and the blood flow feature map;
comprising,
A computer program stored in a computer-readable storage medium.
8. The method of claim 7,
The blood flow feature map is
As a map expressed in black and white based on whether the blood flow is a part,
A black-and-white value for each pixel calculated based on at least one of a red value, a green value, and a black value for each pixel
comprising,
A computer program stored in a computer-readable storage medium.
The method of claim 1,
The operation of performing the liveness check on an object included in image data using the learned liveness model includes:
inputting an output of the learned first submodel to the learned second submodel; and
performing the liveness check based on an output of the learned second submodel;
A computer program stored in a computer-readable storage medium.
11. The method of claim 10,
The operation of performing the liveness check based on the output of the learned second submodel includes:
determining that the liveness test has been passed when an area including pixels having a feature value greater than or equal to a predetermined feature value in the feature map obtained using the learned second submodel is greater than or equal to a threshold size;
comprising,
A computer program stored in a computer-readable storage medium.
delete The method of claim 1,
The second training data set is
Further comprising second image data, which is an image in which noise is included in the first image data,
A computer program stored in a computer-readable storage medium.
14. The method of claim 13,
The operation of learning the face authentication model includes:
learning to reduce a difference between the first image data and the second image data;
further comprising,
A computer program stored in a computer-readable storage medium.
A method for performing face authentication, comprising:
training the liveness model by inputting a first training data set into a liveness model for performing a liveness check;
performing the liveness check on an object included in image data using the learned liveness model; and
performing face authentication by inputting image data that has passed the liveness test into a face authentication model for performing face authentication;
including,
The face authentication model is
A model trained using a second training data set including at least first image data that is a target face image and third image data that is a human face image different from the target face,
The operation of learning the face authentication model includes:
learning to increase a difference between the first image data and the third image data;
A method for performing face authentication.
The computing device for performing face authentication is
one or more processors; and
a memory storing instructions executable by the one or more processors;
including,
The one or more processors,
training the liveness model by inputting a first training data set into a liveness model for performing a liveness check,
performing the liveness check on the object included in the image data using the learned liveness model, and
Face authentication is performed by inputting image data that has passed the liveness test into a face authentication model for performing face authentication,
The face authentication model is
A model trained using a second training data set including at least first image data that is a target face image and third image data that is a face image of a person different from the target face,
The operation of learning the face authentication model is,
learning to increase a difference between the first image data and the third image data;
computing device.

Images