TWI807851B - A feature disentanglement system, method and computer-readable medium thereof for domain generalized face anti-spoofing - Google Patents

Abstract

The present invention provides a feature disentanglement system, method and computer-readable medium thereof for domain generalized face anti-spoofing, including a liveness encoder, a content encoder, a domain encoder, a liveness classifier, a content decoder and a domain classifier, wherein a liveness loss, a liveness confusion loss, a content loss, a content confusion loss, a domain loss and a domain confusion loss are calculated to update the above encoders and the above classifiers until the above encoders and the above classifiers reach convergence, and then complete the training of its neural network. Therefore, the present invention can accurately extract the in liveness features of the human face from an image after training. When performing in liveness judgment on the face in the image, it will not be disturbed by domain or environmental factors, thereby achieving the effect of domain generalization face anti-spoofing.

Description

A feature analysis system, method and computer-readable medium for face anti-counterfeiting with domain generalization

The present invention relates to a face anti-counterfeiting feature analysis technology, in particular to a field generalized face anti-counterfeiting feature analysis system, method and computer readable medium.

With the gradual maturity of the face recognition system, more and more systems or devices (such as mobile phones and building access control) are based on face recognition for verification, but this also faces new security issues. For example, malicious attackers try to deceive the face recognition model through printed faces or faces displayed on the screen. Therefore, face anti-spoofing has become an important technology to improve information security.

However, existing face anti-counterfeiting models often affect the recognition effect due to differences in fields. For example, a face anti-counterfeiting model is trained using face data captured by a camera, but when the model is applied to face recognition on monitor images, it may be affected by the light of the data, shooting angle, Differences in fields such as the focal length or resolution of devices affect the recognition effect. This problem is called field differences.

Therefore, how to avoid the problem of domain differences, and realize domain generalized face anti-counterfeiting, so as to accurately achieve the effect of domain generalized face anti-counterfeiting, has become an urgent problem to be solved in the industry.

In order to solve the aforementioned conventional technical problems or provide related effects, the present invention provides a feature analysis system for face anti-counterfeiting generalized in the field, which includes: a living body feature extraction module, which receives image data of a human face image to judge whether there is a living body, so as to extract the living body features from the image data; The living body classification result of the image data; and a processing module, which is communicatively connected to the living body classifier module to receive the living body classification result of the image data, and then the processing module judges whether the face image in the image data is a living body according to the living body classification result of the image data.

The present invention further provides a feature analysis method for anti-counterfeiting of a generalized human face, which includes: a living body feature extraction module receives an image data of a human face image for judging whether there is a living body, so as to extract living body features from the image data; after receiving the living body features of the image data, the living body classifier module calculates the living body classification result of the image data according to the living body characteristics of the image data; Whether the face image in the image is live.

In one embodiment, the face anti-counterfeiting feature analysis system and method of generalization in the field further includes: a contour feature extraction module communicating with the living body classifier module, a face contour reconstruction module communicating with the contour feature extraction module, a domain feature extraction module communicating with the living body classifier module, and a domain classifier module communicating with the living body feature extraction module, the contour feature extraction module and the domain feature extraction module.

In one embodiment, the processing module further calculates a living body classification loss, a living body confusion loss, a contour reconstruction loss, a contour confusion loss, a domain classification loss, and a domain confusion loss, so that the processing module can update the living body feature extraction module, the contour feature extraction module, the domain feature extraction module, and the living body classifier module according to at least one of the living body classification loss, the living body confusion loss, the contour reconstruction loss, the contour confusion loss, the domain classification loss, and the domain confusion loss , the face contour reconstruction group and the domain classifier module.

In one embodiment, the living body feature extraction module receives at least one training image data to extract the living body features of the training image data from the training image data, and then the living body classifier module and the domain classifier module calculate a first living body classification result and a first domain classification result according to the living body features of the training image data, so that the processing module calculates the living body classification loss and the living body confusion loss according to the first living body classification result and the first domain classification result.

In one embodiment, the contour feature extraction module receives at least one training image data to extract the contour features of the face image from the training image data, and then the facial contour reconstruction group reconstructs a facial contour according to the contour features, so that the processing module calculates the contour reconstruction loss based on the reconstructed facial contour.

In one embodiment, the living body classifier module and the domain classifier module respectively calculate a second living body classification result and a second domain classification result according to the contour feature, so that the processing module calculates the contour confusion loss according to the second living body classification result and the second domain classification result.

In one embodiment, the domain feature extraction module receives at least one training image data to extract domain features from the training image data, and then the living body classifier module and the domain classifier module respectively calculate a third living body classification result and a third domain classification result according to the domain features, so that the processing module calculates the domain classification loss and the domain confusion loss according to the third domain classification result and the third living body classification result respectively.

The present invention further provides a computer-readable medium, which is applied to a computer or computing device with a processor and/or memory, and the computer or the computing device executes a target program and the computer-readable medium through the processor and/or memory, and is used to perform the feature analysis method for face anti-counterfeiting as described above when the computer-readable medium is executed.

As can be seen from the above, the domain generalized face anti-counterfeiting feature analysis system, method and computer-readable medium of the present invention update the above-mentioned modules through the calculated loss functions (i.e., living body classification loss, living body confusion loss, contour reconstruction loss, contour confusion loss, domain classification loss, and domain confusion loss), and complete the training of the neural network until the modules reach convergence. Therefore, the feature analysis system of the domain generalized face anti-counterfeiting of the present invention can accurately extract the living features of the domain generalized human face from an image after training, so that the liveness judgment of the human face in the image will not be disturbed by the domain or environmental factors, so the present invention can be applied to any field to achieve the effect of domain generalization.

1: Feature analysis system for face anti-counterfeiting in domain generalization

11: Living body feature extraction module

12: Contour feature extraction module

13: Domain feature extraction module

14: Living classifier module

15: Facial contour reconstruction group

16: Domain classifier module

17: Processing module

S21A~S23A and S21B~S23B: steps

S31A~S33A and S31B~S33B: steps

S41A~S43A and S41B~S43B: steps

S51~S511: steps

Fig. 1 and Fig. 1-1 are the feature analysis system of the generalized face anti-counterfeiting in the field of the present invention.

FIG. 2A is a schematic diagram of the calculation flow of the living body classification loss in the present invention.

FIG. 2B is a schematic diagram of the calculation flow of the living body confusion loss of the present invention.

FIG. 3A is a schematic diagram of the calculation flow of the contour reconstruction loss in the present invention.

FIG. 3B is a schematic diagram of the calculation flow of the contour confusion loss of the present invention.

FIG. 4A is a schematic diagram of the calculation flow of the domain classification loss of the present invention.

FIG. 4B is a schematic diagram of the calculation flow of the domain confusion loss of the present invention.

Fig. 5 is a schematic flow chart of the training method of the neural network of the module in the feature analysis system of face anti-counterfeiting generalized in the field of the present invention.

The implementation of the present invention is described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification.

It should be noted that the structures, proportions, and sizes shown in the drawings attached to this specification are only used to match the content disclosed in the specification for the understanding and reading of those who are familiar with this technology, and are not used to limit the conditions for the implementation of the present invention, so they have no technical significance. At the same time, references in this specification such as "a", "first", "second", Terms such as "above" and "below" are only used for clarity of description, and are not used to limit the applicable scope of the present invention. Changes or adjustments in their relative relationships are deemed to be within the applicable scope of the present invention if there is no substantial change in the technical content.

Fig. 1 is the feature analysis system 1 of the anti-counterfeiting face of the field generalization of the present invention, comprising: a living body feature extraction module 11, a contour feature extraction module 12, a field feature extraction module 13, a living body classifier module 14, a facial contour reconstruction group 15 and a field classifier module 16, wherein the living body feature extraction module 11, contour feature extraction module 12, field feature extraction module 13, living body classifier module 14, Both the contour reconstruction group 15 and the domain classifier module 16 are constructed by an artificial neural network (ANN for short) (or called a neural network, a neural network (NN for short)). In one embodiment, the face anti-counterfeiting feature analysis system 1 for domain generalization further includes a processing module 17 (as shown in FIG. 1-1 ) that is communicatively connected to the above-mentioned modules, and is used to calculate the loss function (Loss Function) of the above-mentioned modules.

In one embodiment, the face anti-counterfeiting feature analysis system 1 of domain generalization can be established in the same (or different) servers (such as general-purpose server, file server, storage unit type server, etc.) and computers with appropriate calculation mechanism, wherein each module in the domain generalization face anti-counterfeiting feature analysis system 1 can be software, hardware or firmware; if it is hardware, it can be a processing unit, processor, computer or server with data processing and computing capabilities; if it is software or Firmware may include instructions executable by a processing unit, processor, computer or server, and may be installed on the same hardware device or distributed across multiple different hardware devices.

The living body feature extraction module 11 is a neural network with an input dimension of H×W×3 and an output dimension of D, wherein H and W are respectively the height and width of the image data received by the feature analysis system 1, and 3 represents the three colors of the color image channel, and D is the vector length (or feature length) of the living body feature. In this embodiment, the neural network used by the living body feature extraction module 11 is a ResNet 18 minus the last layer, and the feature length D is 512. Specifically, the face anti-counterfeiting feature analysis system 1 of domain generalization receives an input image data (including a face image), and the living body feature extraction module 11 extracts a living body feature of length D from the complex image data as information for judging whether the face in the image data is a living body.

In this embodiment, in order to make the living body features extracted from the image data not contain irrelevant domain information, a living body classification loss and a living body confusion loss are calculated through the processing module 17, so as to train the living body feature extraction module 11 according to the living body classification loss and the living body confusion loss. The calculation methods of the living body classification loss and the living body confusion loss are as follows:

1. Live classification loss:

As shown in Figure 2A, the calculation process of the living body classification loss includes:

In step S21A, a training image data is input, and the training image data includes a face image.

In step S22A, after the living body features are extracted from the training image data by the living body feature extraction module 11, a first living body classification result is calculated by the living body classifier module 14 according to the living body features of the training image data. In detail, the living body classifier module 14 is a kind of neural network, and the input dimension is the living body feature of D, and the output is the predicted value of the same dimension as the living body label y _i . Taking the living body classifier module 14 as a fully connected layer as an example, _the first living body classification result predicted by the living body classifier module 14 is θ _L × EL ( xi _,j )+b _L , where θ _L and b _L are the parameters of the living body classifier module 14.

In step S23A, the processing module 17 uses the loss function (1) of the living body classification loss to calculate the difference between the first living body classification result and the living body label of the training image data, so as to obtain the living body classification loss. In detail, the living body classification loss is to ensure that the living body features contain important information in the input image that can be used to judge the living body. Among them, the equation of the loss function (1) of the living body classification loss is as follows:

Among them, x _{i, j} are the _input training image data; E _L is the living body feature extraction module 11; CL is the living body classifier module 14; y _i is _the living body label of the training image data; The gap between the obtained first living body classification result and the living body label.

2. Live body confusion loss:

As shown in Figure 2B, the calculation process of the living body confusion loss includes:

In step S21B, a training image data is input.

In step S22B, the living body feature extraction module 11 extracts living body features from the training image data, and then the domain classifier module 16 calculates a first domain classification result based on the living body features of the training image data. In detail, the domain classifier module 16 is a kind of neural network _, and the input dimension is the living body feature of D, and the output is the predicted value of the same dimension as the domain label mi . Taking the domain classifier module 16 as a fully connected layer as an example, the first domain classification result predicted by the domain classifier module 16 _is θ _D × EL ( xi _,j )+b _D , where θ _D and b _D are parameters of the domain classifier module 16.

In step S23B, the processing module 17 uses the loss function (2) of the living body confusion loss to calculate the difference between the classification result of the first domain and the uniform distribution, so as to obtain the living body confusion loss. In detail, the living body confusion loss is to ensure that the living body features of the training image data do not include the domain information of the training image data (such as the light when shooting, the focal length and resolution of the equipment used, etc.), so the classification result of the first field identified by the domain classifier module 16 should be close to a uniform distribution. Among them, the equation of the loss function (2) of the living body confusion loss is as follows:

為活體混淆損失，即活體特徵經由領域分類器模組16計算後所得到之第一領域分類結果與均勻分佈之間的差距。 Among them, x _{i, j} is the input training image data; E _L is the living body feature extraction module 11; C _D is the domain classifier module 16; S is the number of training image data sets; N is the number of training image data input at one time;

is the living body confusion loss, that is, the gap between the first domain classification result and the uniform distribution obtained after the living body features are calculated by the domain classifier module 16 .

In one embodiment, the first living body classification result and the first field classification result may be a probability value between 0 and 1, or a numerical value expressed as a percentage, and the numerical presentation of the living body classification result is not limited here.

The contour feature extraction module 12 is also a neural network with an input dimension of H×W×3 and an output dimension of D. In this embodiment, the neural network used by the living body feature extraction module 11 is a ResNet 18 minus the last layer, and the feature length D is 512. In other words, the feature analysis system 1 for face anti-counterfeiting with domain generalization receives an input image data, and uses contour features The extraction module 12 extracts contour features of length D from the complex image data as information for reconstructing the facial contour in the image data.

In this embodiment, in order to prevent the contour features extracted from the image data from including irrelevant living body information and domain information, a contour reconstruction loss and a contour confusion loss are calculated through the processing module 17, wherein the calculation methods of the contour reconstruction loss and contour confusion loss are as follows:

1. Contour reconstruction loss:

As shown in Figure 3A, the calculation process of contour reconstruction loss includes:

In step S31A, a training image data is input.

In step S32A, the contour feature extraction module 12 extracts contour features from the training image data, and then the facial contour reconstruction group 15 reconstructs a facial contour according to the contour features of the training image data.

In detail, the facial contour reconstruction group 15 is a kind of neural network, and the input dimension is the contour feature of D, and the dimension of the output prediction value is the same as the contour dimension of the contour Φ( xi _,j ) generated by the pre-trained facial contour generation module Φ. Taking the facial contour reconstruction group 15 as a fully connected layer and the contour dimension as H×W as an example, the facial contour reconstructed by the facial contour reconstruction group 15 is θ _C × E _c ( xi _,j )+b _C , where θ _C and b _C are the parameters of the domain classifier module 16, the dimension of θ _C is H×W×D, and the dimension of b _C is H×W, so the reconstructed contour is the predicted value of size H×W.

In step S33A, the processing module 17 uses the loss function (3) of the contour reconstruction loss to calculate the difference between the reconstructed facial contour and the facial contour of the training image data to obtain the contour reconstruction loss. In detail, the contour reconstruction loss is to ensure that the contour features contain the input The important information of the facial contour can be reconstructed from the training image data of , among which, the equation of the loss function (3) of the contour reconstruction loss is as follows:

Among them, x _{i, j} are the input training image data; E _C is the contour feature extraction module 12; D _C is the facial contour reconstruction group 15; The gap between the facial contours extracted by the facial contour generation model and the facial _contours extracted by the pre-trained facial contour generation module .

In one embodiment, the pre-trained facial contour generation module is a neural network with an input dimension of D and an output dimension of H×W×3 established by PRNet.

2. Contour confusion loss:

As shown in Figure 3B, the calculation process of the contour confusion loss includes:

In step S31B, a training image data is input.

In step S32B, after the contour feature extraction module 12 extracts the contour features from the training image data, the living body classifier module 14 and the field classifier module 16 respectively calculate a second living body classification result and a second field classification result according to the contour features of the training image data. Specifically, taking the living body classifier module 14 and the domain classifier module 16 as an example of a fully connected layer, the second living body classification result predicted by the living body classifier module 14 is θ _L × EC ( xi _{,j )} +b _L , and the second domain classification result predicted by the domain classifier module 16 is θ _D × _EC ( xi ,j ) + b _D , wherein θ _L and b _L are the parameters _of the living body classifier module 14, and θ _D and b _D are parameters of the domain classifier module 16 .

In step S33B, the processing module 17 uses the loss function (4) of the contour confusion loss to calculate the difference between the second living body classification result, the second field classification result and the uniform distribution, so as to obtain the contour confusion loss. In detail, the contour confusion loss is to ensure that the contour features do not include the living body information or domain information in the input image, so the second living body classification results and the second domain classification results should be close to uniform distribution. Among them, the equation of the loss function (4) of the contour confusion loss is as follows:

為輪廓混淆損失，即輪廓特徵經由活體分類器模組14及領域分類器模組16計算後所得到之第二活體分類結果及第二領域分類結果相較於均勻分佈之間的差距之總和。 Wherein, x _{i, j} is the input training image data; E _C is the contour feature extraction module 12; CL is the living body classifier module 14; CD is the field classifier module 16; S is the number of training image data sets; N is the number _of training image data input at _one time;

Contour confusion loss, that is, the sum of the difference between the second living body classification result and the second domain classification result obtained after the contour features are calculated by the living body classifier module 14 and the domain classifier module 16 compared with the uniform distribution.

In one embodiment, the second living body classification result and the second field classification result may be a probability value between 0 and 1, or a numerical value expressed as a percentage, and the numerical presentation of the living body classification result is not limited here.

The domain feature extraction module 13 is also a neural network with an input dimension of H×W×3 and an output dimension of D. In this embodiment, the neural network used by the domain feature extraction module 13 is ResNet 18 minus the last layer, and the feature length D is 512. In other words, the feature analysis system 1 for face anti-counterfeiting with domain generalization receives an input image data, and the domain feature extraction The extraction module 13 extracts domain features of length D from the complex image data as domain information used to distinguish the image data (such as the light when shooting, the focal length and resolution of the equipment used, etc.).

In this embodiment, in order to prevent the domain features extracted from the image data from including irrelevant living body information, a domain classification loss and a domain confusion loss are calculated through the processing module 17, wherein the calculation methods of the domain classification loss and domain confusion loss are as follows:

1. Domain classification loss:

As shown in Figure 4A, the calculation process of domain classification loss includes:

In step S41A, a training image data is input.

In step S42A, after domain features are extracted from the training image data by the domain feature extraction module 13, a third domain classification result is calculated by the domain classifier module 16 according to the domain features of the training image data. Specifically, taking the domain classifier module 16 as an example of a fully connected layer, _the third domain classification result predicted by the domain classifier module 16 is θ _D × ED ( xi _,j )+b _D , where θ _D and b _D are parameters of the domain classifier module 16.

In step S43A, the processing module 17 uses the loss function (5) of the domain classification loss to calculate the distance between the third domain classification result and the domain label of the training image data to obtain the domain classification loss. In detail, the domain classification loss is to ensure that the collar features contain important information in the input image that can be used for domain judgment. Among them, the equation of the loss function (5) of the domain classification loss is as follows:

Among them , x _i,j is the input training image data; E _D is the domain feature extraction module 13; CD is the domain classifier module 16; m _i is the domain label of the training image data; S is _the number of training image data sets _;

2. Domain confusion loss:

As shown in Figure 4B, the calculation process of domain confusion loss includes:

In step S41B, a training image data is input.

In step S42B, the domain feature extraction module 13 extracts domain features from the training image data, and then the living body classifier module 14 calculates a third living body classification result according to the domain features of the training image data. Specifically, taking the living body classifier module 14 as a fully connected layer as an example, the third living body classification result predicted by the living body classifier module 14 is θ _L × ED ( xi _,j )+b _L , where θ _L and b _L are parameters of the _living body classifier module 14.

In step S43B, the processing module 17 uses the loss function (6) of the domain confusion loss to calculate the distance between the third living body classification result and the uniform distribution, so as to obtain the domain confusion loss. In detail, the domain confusion loss is to ensure that the domain features of the training image data do not include the living body information of the training image data, so the third living body classification result identified by the living body classifier module 14 should be close to a uniform distribution. Among them, the equation of the loss function (6) of the domain confusion loss is as follows:

為領域混淆損失，即領域特徵經由活體分類器模組14計算後所得到之第三活體分類結果與均勻分佈之間的差距。 Among _them , x _{i, j} is the input training image data; E _D is the domain feature extraction module 13; CL is the living body classifier module 14; S is the number of training image data sets; N is the number of training image data input at one time;

is the domain confusion loss, that is, the gap between the third living body classification result and the uniform distribution after the domain features are calculated by the living body classifier module 14 .

In one embodiment, the third domain classification result and the third living body classification result may be a probability value between 0 and 1, or a numerical value expressed as a percentage, and the numerical presentation method of the living body classification result is not limited here.

In another embodiment, after obtaining the losses of various loss functions (1)-(6) (i.e., living body classification loss, living body confusion loss, contour reconstruction loss, contour confusion loss, domain classification loss, and domain confusion loss) through the input training image data, the processing module 17 uses neural network-like backpropagation (Backpropagation, BP) and combines gradient descent method (Gradient descent) to calculate the gradients of various loss functions (1)-(6), so as to use each loss function (1)-(6) (6) The gradient feedback optimization method is used to update the weights in the living body feature extraction module 11, contour feature extraction module 12, domain feature extraction module 13, living body classifier module 14, facial contour reconstruction module 15 and domain classifier module 16, and then optimize each loss function (1)~(6), that is, obtain a smaller loss function. Furthermore, the weights of each loss function (1)~(6) can be adjusted according to the training image data and the design of the neural network.

In detail, when the living body feature extraction module 11, contour feature extraction module 12, domain feature extraction module 13, living body classifier module 14, facial contour reconstruction module 15 and domain classifier module 16 are updated, the loss functions used respectively are as follows:

1. Living body feature extraction module 11: living body classification loss and living body confusion loss

2. Live Classifier Module 14: Live Classification Loss

3. Contour Feature Extraction Module 12: Contour Reconstruction Loss and Contour Confusion Loss

4. Facial Contour Remodeling Group 15: Contour Reconstruction Loss

5. Domain Feature Extraction Module 13: Domain Classification Loss and Domain Confusion Loss

6. Domain Classifier Module 16: Domain Classification Loss

Next, continuously input multiple training image data into the living body feature extraction module 11, contour feature extraction module 12, domain feature extraction module 13, living body classifier module 14, facial contour reconstruction module 15 and domain classifier module 16, so as to carry out the training process as described in the above-mentioned embodiment on these modules, until the loss functions (1)~(6) reach convergence, thereby completing the living body feature extraction module 11, contour feature extraction module 12, and domain feature extraction module 13. The neural network training of the living body classifier module 14, the facial contour reconstruction group 15 and the domain classifier module 16. On the other hand, it is usually used to observe the downward trend of the loss function as a reference for whether the model converges during training.

In one embodiment, it is judged whether the neural network training of the living body feature extraction module 11, the outline feature extraction module 12, the domain feature extraction module 13, the living body classifier module 14, the facial contour reconstruction group 15, and the domain classifier module 16 has reached convergence, and is judged by the sum of the loss functions (1)-(6). Specifically, since the size of each loss function (1)~(6) will oscillate back and forth during implementation, it is necessary to represent the changes of each loss function (1)~(6) with the number of updates in a line graph, and smooth the line, and the processing module 17 calculates whether the drop value of each loss function (1)~(6) within a certain number of updates (such as: 500 times) is less than a threshold value, so as to judge whether the training of the neural network of these modules has reached the goal when the drop value is less than a threshold value. convergence, and the threshold can be set according to the training accuracy requirements, so there is no limit to the threshold.

In one embodiment, the quality of the facial contour reconstructed by the processing module 17 from the facial contour reconstruction module 15, the accuracy of the classification results obtained by the living body classifier module 14 and/or the domain classifier module 16 can also be used to determine whether the training of the neural network of the living body feature extraction module 11, the contour feature extraction module 12, the domain feature extraction module 13, the living body classifier module 14, the facial contour reconstruction group 15 and the domain classifier module 16 is completed.

FIG. 5 is a schematic flow chart of the neural network training method of the modules in the face anti-counterfeiting feature analysis system generalized in the field of the present invention, and it is explained with reference to FIGS. 1 to 4-2. In addition, the similarities between this embodiment and the above embodiment will not be repeated, and the training method flow includes the following steps S51 to S511:

In step S51, a training image data is input, which includes a face image and domain information other than the face image (such as the light when shooting, the focal length and resolution of the equipment used, etc.).

In step S52, the living body feature extraction module 11 extracts the living body features related to the face image from the training image data.

In step S53, the living body classifier module 14 calculates a first living body classification result according to the living body features, and then calculates the difference between the first living body classification result and the living body label of the training image data to obtain the living body classification loss.

In step S54, the domain classifier module 16 calculates a first domain classification result according to the living body features of the training image data, and then calculates the distance between the first domain classification result and the uniform distribution to obtain the living body confusion loss.

In step S55, the contour feature extraction module 12 extracts contour features related to the face image from the training image data.

In step S56, the facial contour reconstruction group 15 reconstructs a facial contour according to the contour features of the training image data, and then calculates the difference between the reconstructed facial contour and the facial contour of the training image data to obtain the contour reconstruction loss.

In step S57, the living body classifier module 14 and the domain classifier module 16 respectively calculate a second living body classification result and a second domain according to the contour features of the training image data Classification results, and then calculate the second living body classification results, the gap between the second domain classification results and the uniform distribution to obtain the contour confusion loss.

In step S58, the domain feature extraction module 13 extracts domain features from complex image data.

In step S59, the domain classifier module 16 calculates a third domain classification result according to the domain characteristics of the training image data, and then calculates the distance between the third domain classification result and the domain label of the training image data to obtain the domain classification loss.

In step S510 , the living body classifier module 14 calculates a third living body classification result according to the domain characteristics of the training image data, and then calculates the distance between the third living body classification result and the uniform distribution to obtain the domain confusion loss.

In step S511, the weights in the living body feature extraction module 11, the contour feature extraction module 12, the domain feature extraction module 13, the living body classifier module 14, the facial contour reconstruction group 15 and the domain classifier module 16 are updated according to at least one of the living body classification loss, the living body confusion loss, the contour reconstruction loss, the contour confusion loss, the domain classification loss and the domain confusion loss, until the above-mentioned modules reach convergence.

In addition, the present invention also discloses a computer-readable medium, which is applied to a computing device or computer having a processor (for example, CPU, GPU, etc.) and/or memory, and stores instructions, and the computing device or computer can be used to execute the computer-readable medium through the processor and/or memory, so as to execute the above-mentioned method and steps when executing the computer-readable medium.

The following is an application embodiment of the face anti-counterfeiting feature analysis system 1 generalized in the field of the present invention, and it is explained with reference to FIG. 1 .

In this embodiment, the modules in the domain generalized face anti-counterfeiting feature analysis system 1 of the present invention have been trained in the above-mentioned embodiments, so that after receiving an image data to be judged as living, the living body feature extraction module 11 extracts the living body features from the image data, and after training, the domain generalized face anti-counterfeiting feature analysis system 1 can separate the contour features and domain features that have nothing to do with the living body, so the living body features will not be affected by the contour of the face and the difference in the field.

Furthermore, the living body classifier module 14 receives the living body features extracted by the living body feature extraction module 11 to calculate a living body classification result, which can determine whether the image is a living body according to the living body classification result. Specifically, a threshold can be taken, and if the result of the living body classification is higher than the threshold, the processing module 17 determines that the image data is abnormal, that is, there is no living body in the image data; if it is lower than the threshold, the processing module determines that the image data is normal, that is, there is a living body in the image data.

In one embodiment, the living body classification result and the threshold value can be a probability value between 0 and 1, or a value expressed as a percentage, and the threshold value can be set according to requirements, so there is no limitation on the numerical presentation method of the living body classification result and the threshold value, and the size of the threshold value.

To sum up, the feature analysis system, method and computer-readable medium of the domain-generalized face anti-counterfeiting of the present invention update modules such as the living body feature extraction module, the contour feature extraction module, the domain feature extraction module, the living body classifier module, the facial contour reconstruction group, and the domain classifier module through the calculated loss functions (i.e., the living body classification loss, the living body confusion loss, the contour reconstruction loss, the contour confusion loss, the domain classification loss, and the domain confusion loss), until the aforementioned modules reach convergence, and then Complete the training of the neural network of these modules. Therefore, the generalized human face anti-counterfeiting feature analysis system of the present invention can accurately extract the living body features of the human face with generalized domain from an image after training, so that when the human face in the image is judged live, it will not It will be interfered by field or environmental factors, so the present invention can be applied in any field to achieve the effect of field generalized face anti-counterfeiting.

Furthermore, compared to the prior art using a single feature extractor to extract and judge live features of the face, its application is limited to the field used in training, and the live features extracted by the single feature extractor include irrelevant face contour features and domain features (such as the light when shooting, the focal length and resolution of the equipment used, etc.), which in turn affects the judging effect of face anti-counterfeiting. However, the present invention uses the contour feature extraction module and the domain feature extraction module respectively, which can separate the facial contour information and domain features that are not related to the living body features, so that the living body features will not be interfered by the differences in the face contour and domain differences, and can focus on the interpretation of the living body features of the human face.

Therefore, the present invention can be applied in different fields to judge the living body features of human faces, so as to prevent interested people from using non-living human face photos or videos for verification, and then achieve the purpose of face anti-counterfeiting in generalized fields.

The above-mentioned embodiments are only illustrative to illustrate the principles and effects of the present invention, and are not intended to limit the present invention. Anyone skilled in the art can modify and change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the rights of the present invention should be listed in the scope of the patent application.

1: Feature analysis system for face anti-counterfeiting in domain generalization

11: Living body feature extraction module

12: Contour feature extraction module

13: Domain feature extraction module

14: Living classifier module

15: Facial contour reconstruction group

16: Domain classifier module

Claims (13)

A feature analysis system for face anti-counterfeiting with generalization in the field includes: a living body feature extraction module, which receives image data of a human face image for judging whether there is a living body, so as to extract living body features from the image data; a living body classifier module, which is connected to the living body feature extraction module by communication, to receive the living body features of the image data, and then the living body classifier module calculates the living body classification result of the image data based on the living body features of the image data; Communication connection with the living body feature extraction module; and a processing module, which is communication connected with the living body classifier module to receive the living body classification result of the image data, and then the processing module judges whether the face image in the image data is a living body according to the living body classification result of the image data, wherein the living body feature extraction module receives at least one training image data, so as to extract the living body features of the training image data from the training image data, and then the living body classifier module and the field classifier module according to the training A first living body classification result and a first domain classification result are respectively calculated from the living body features of the image data, so that the processing module calculates a living body classification loss and a living body confusion loss respectively according to the first living body classification result and the first domain classification result. The face anti-counterfeiting feature analysis system for domain generalization as described in claim item 1 further includes: a contour feature extraction module communicating with the living body classifier module, a face contour reconstruction module communicating with the contour feature extraction module, a domain feature extraction module communicating with the living body classifier module, and a domain classifier module communicating with the contour feature extraction module and the domain feature extraction module, wherein the processing module calculates a contour reconstruction loss, a contour confusion loss, and a domain Classification loss and one domain confusion loss. The feature analysis system for face anti-counterfeiting with domain generalization as described in Claim 2, wherein the processing module updates the living body feature extraction module based on the living body classification loss and the living body confusion loss, updates the living body classifier module based on the living body classification loss, updates the contour feature extraction module based on the contour reconstruction loss and the contour confusion loss, updates the face contour reconstruction group according to the contour reconstruction loss, updates the domain feature extraction module according to the domain classification loss and the domain confusion loss, and updates the domain classification loss based on the domain classification loss. Domain classifier mod. The feature analysis system for face anti-counterfeiting with domain generalization as described in Claim 2, wherein the contour feature extraction module receives at least one training image data to extract the contour features of the face image from the training image data, and then the facial contour reconstruction group reconstructs a facial contour according to the contour features, so that the processing module calculates the contour reconstruction loss based on the reconstructed facial contour. The feature analysis system for face anti-counterfeiting with domain generalization as described in Claim 4, wherein the living body classifier module and the domain classifier module respectively calculate a second living body classification result and a second domain classification result based on the contour feature, so that the processing module calculates the contour confusion loss based on the second living body classification result and the second domain classification result. The feature analysis system for face anti-counterfeiting with domain generalization as described in Claim 2, wherein the domain feature extraction module receives at least one training image data to extract domain features from the training image data, and then the living body classifier module and the domain classifier module respectively calculate a third living body classification result and a third domain classification result according to the domain features, so that the processing module calculates the domain classification loss and the domain confusion loss based on the third domain classification result and the third living body classification result respectively. A feature analysis method for face anti-counterfeiting with domain generalization, comprising: The living body feature extraction module receives the image data of a human face image to determine whether it has a living body, so as to extract the living body features from the image data; after receiving the living body features of the image data, the living body classifier module calculates the living body classification result of the image data according to the living body features of the image data; The group receives at least one training image data, so as to extract the living body features of the training image data from the training image data; the living body classifier module and a domain classifier module respectively calculate a first living body classification result and a first domain classification result according to the living body features of the training image data; The face anti-counterfeiting feature analysis method for domain generalization as described in Claim 7 further includes calculating a contour reconstruction loss, a contour confusion loss, a domain classification loss, and a domain confusion loss by the processing module. The face anti-counterfeiting feature analysis method of domain generalization as described in claim item 8 further includes the processing module updating the living body feature extraction module based on the living body classification loss and the living body confusion loss, updating the living body classifier module based on the living body classification loss, updating a contour feature extraction module based on the contour reconstruction loss and the contour confusion loss, updating a facial contour reconstruction group based on the contour reconstruction loss, updating a domain feature extraction module based on the domain classification loss and the domain confusion loss, and updating the domain classification loss. Domain classifier mod. The feature analysis method for face anti-counterfeiting with domain generalization as described in claim 9 further includes: receiving at least one training image data by the contour feature extraction module, so as to extract the contour features of the face image from the training image data; and reconstructing a facial contour by the facial contour reconstruction group according to the contour features, so that the processing module calculates the contour reconstruction loss based on the reconstructed facial contour. The feature analysis method for face anti-counterfeiting with domain generalization as described in Claim 10 further includes: calculating a second living body classification result and a second domain classification result by the living body classifier module and the domain classifier module according to the contour feature; and calculating the contour confusion loss by the processing module based on the second living body classification result and the second domain classification result. The feature analysis method for face anti-counterfeiting with domain generalization as described in Claim 9 further includes: the domain feature extraction module receives at least one training image data to extract domain features from the training image data; the living body classifier module and the domain classifier module respectively calculate a third living body classification result and a third domain classification result according to the domain features; and the processing module calculates the domain classification loss and the domain confusion loss based on the third domain classification result and the third living body classification result respectively. A computer-readable medium, used in a computing device or a computer, stores instructions to execute the feature analysis method for generalized face anti-counterfeiting described in any one of claims 7 to 12.

Images