CN114495245B - Face counterfeit image identification method, device, equipment and medium - Google Patents
Face counterfeit image identification method, device, equipment and medium Download PDFInfo
- Publication number
- CN114495245B CN114495245B CN202210363659.8A CN202210363659A CN114495245B CN 114495245 B CN114495245 B CN 114495245B CN 202210363659 A CN202210363659 A CN 202210363659A CN 114495245 B CN114495245 B CN 114495245B
- Authority
- CN
- China
- Prior art keywords
- model
- forged
- teacher
- teacher model
- output result
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present disclosure relates to a method, apparatus, device, and medium for discriminating a face-forged image. Wherein, the method comprises the following steps: acquiring a first sample image set and a second sample image set, wherein false faces in the first sample image set are obtained by counterfeiting based on a first counterfeiting mode, and false faces in the second sample image set are obtained by counterfeiting based on a second counterfeiting mode; training and generating a first teacher model for identifying a first counterfeiting mode based on the first sample image set; training and generating a second teacher model for identifying a second counterfeiting mode based on the second sample image set; and fusing the first teacher model and the second teacher model to train the student models, and generating a target face forged image identification model for identifying the first forged mode and the second forged mode. According to the technical scheme provided by the embodiment of the disclosure, the generated target face forged image identification model is suitable for identifying the face images generated by different forging methods, and has better generalization and expansibility and higher efficiency.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a medium for identifying a face counterfeit image.
Background
With the continuous development of image processing technology, the identification of face forged images is more accurate, and the identification of the existing face forged images mainly aims at extracting features from specific domains, namely extracting the features from time domains, space domains and frequency domains to identify the authenticity of the faces.
At present, if the characteristics of only one domain are used, the shortage of characteristic information causes that the single-domain methods are difficult to deal with face forgery identification of all classes. If the characteristic information of a plurality of domains is used, the redundancy of the information can cause the multi-domain model to excessively fit the trained sample, and further new data to be tested cannot be well processed. Therefore, it is necessary to propose a new feature extraction method.
Disclosure of Invention
To solve the above technical problems or to at least partially solve the above technical problems, the present disclosure provides a face counterfeit image authentication method, apparatus, device, and medium.
In a first aspect, the present disclosure provides a method for identifying a face forged image, including:
acquiring a first sample image set and a second sample image set, wherein false faces in the first sample image set are obtained by counterfeiting based on a first counterfeiting mode, and false faces in the second sample image set are obtained by counterfeiting based on a second counterfeiting mode;
Training and generating a first teacher model for identifying the first counterfeiting mode based on the first sample image set;
training and generating a second teacher model for identifying the second counterfeiting mode based on a second sample image set;
and based on the first teacher model and the second teacher model, fusing the capability of identifying a forged face of the two teacher models respectively to train a student model, and generating a target forged face image identification model for identifying the first forged mode and the second forged mode.
In some embodiments, based on the first teacher model and the second teacher model, the ability of two teacher models to identify a type of fake face respectively is fused to train student models, and a target fake face image identification model for identifying the first fake manner and the second fake manner is generated, including:
respectively inputting a preset third sample image set into a first teacher model, a second teacher model and a student model to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model;
Distilling the knowledge of the first teacher model and the second teacher model based on the first output result, the second output result, the third output result, and the third sample image set;
and after distillation processing, skipping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute the steps until the evaluation index of the student model tends to be stable, and stopping training to obtain the target forged face image identification model.
In some embodiments, said distilling the knowledge of the first teacher model and the second teacher model based on the first output result, the second output result, the third output result, and the third sample image set comprises:
determining a cross-entropy loss for the student model based on the third sample image set and the third output result;
determining a first relative entropy loss between the student model and the first teacher model based on the first output result and the third output result;
determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result;
Determining a distillation loss based on the cross entropy loss, the first relative entropy loss, and the second relative entropy loss;
adjusting parameters of the student model based on the distillation loss to reduce the distillation loss.
In some embodiments, after the capability of two teacher models to identify a fake face is fused to train a student model based on the first teacher model and the second teacher model, and a target fake face image identification model for identifying the first fake manner and the second fake manner is generated, the method further includes:
and calibrating the confidence coefficient of the identification model of the target face forged image.
In some embodiments, the calibrating the confidence of the target face-counterfeit image identification model includes:
inputting a preset fourth sample image into the first teacher model, the second teacher model and the target forged face image identification model respectively to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true and a third probability value that the target face forged face image is identified as true;
Determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image authentication model and the first teacher model based on the first confidence coefficient to obtain a first weighted value;
determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value;
correcting the distillation loss of the target face forged image identification model based on the cross entropy loss among the first weighted value, the second weighted value, the third probability value and the forged probability value corresponding to the fourth sample image to obtain the corrected distillation loss;
and adjusting parameters of the target face forged image identification model based on the corrected distillation loss, skipping to the step of inputting a preset fourth sample image into the first teacher model, the second teacher model and the target face forged image identification model respectively after parameter adjustment, continuing to execute the steps until the variation degree of the output result of the target face forged image identification is smaller than a preset threshold value, and stopping calibration.
In a second aspect, the present disclosure provides a face counterfeit image identification apparatus, including:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a first sample image set and a second sample image set, the first sample image set is forged based on a first forging mode, and the second sample image set is forged based on a second forging mode;
a first teacher model generation unit configured to generate a first teacher model for identifying the first forgery mode based on the first sample image set training;
a second teacher model generation unit configured to generate a second teacher model for identifying the second forgery manner based on second sample image training;
and the student model generation unit is used for training a student model based on the first teacher model and the second teacher model and generating a target face forged image identification model for identifying the first forged mode and the second forged mode.
In some embodiments, the student model generation unit is specifically configured to:
respectively inputting a preset third sample image into a first teacher model, a second teacher model and a student model to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model;
Distilling the knowledge of the first teacher model and the knowledge of the second teacher model based on the first output result, the second output result, the third output result and a sample label corresponding to the third sample image;
and after the distillation treatment, skipping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute the steps until the evaluation index of the output result of the student model tends to be stable, stopping training, and obtaining the target face forged image identification model.
In some embodiments, the student model generation unit includes:
a distillation processing subunit for determining cross-entropy loss for the student model based on the third sample image set and the third output result;
determining a first relative entropy loss between the student model and the first teacher model based on the first output result and the third output result;
determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result;
Determining a distillation loss based on the cross entropy loss, the first relative entropy loss, and the second relative entropy loss;
adjusting parameters of the student model based on the distillation loss to reduce the distillation loss.
In some embodiments, the apparatus further comprises:
and the confidence coefficient calibration unit is used for calibrating the confidence coefficient of the target face forged image identification model.
In some embodiments, the confidence calibration unit is specifically configured to:
inputting a preset fourth sample image into the first teacher model, the second teacher model and the target forged face image identification model respectively to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true and a third probability value that the target face forged face image is identified as true;
determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image identification model and the first teacher model based on the first confidence coefficient to obtain a first weighting value;
determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value;
Correcting the distillation loss of the target face forged image identification model based on the cross entropy loss among the first weighted value, the second weighted value, the third probability value and the forged probability value corresponding to the fourth sample image to obtain the corrected distillation loss;
and adjusting parameters of the target face forged image identification model based on the corrected distillation loss, and skipping to the step of respectively inputting a preset fourth sample image into the first teacher model, the second teacher model and the target face forged image identification model after parameter adjustment to continue to execute until the variation degree of the output result of the target face forged image identification is smaller than a preset threshold value, and stopping calibration.
In a third aspect, the present disclosure provides a computer device comprising:
a processor and a memory;
the processor is used for executing the steps of the method for identifying a face forged image according to any one of the first aspect by calling the program or the instructions stored in the memory.
In a fourth aspect, the present disclosure provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a program or instructions for causing a computer to perform the steps of the method according to any one of the first aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
the embodiment of the disclosure provides a method, a device, equipment and a medium for identifying a face forged image, wherein the method comprises the following steps: after a first sample image set forged based on a first forging mode and a second sample image set forged based on a second forging mode are obtained, a first teacher model used for identifying the first forging mode is generated based on the first sample image set training, a second teacher model used for identifying the second forging mode is generated based on the second sample image set training, then a student model is trained based on the first teacher model and the second teacher model, and a target face forging image identification model used for identifying the first forging mode and the second forging mode is generated. Therefore, the generated target face forged image identification model is suitable for identifying the face images generated by different forging methods, and has better generalization and expansibility and higher efficiency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a flowchart of a method for identifying a face forged image according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of generating a face-forged image in a different manner according to an embodiment of the present disclosure;
FIG. 3 is a schematic diagram of a method for generating a teacher model according to an embodiment of the disclosure;
FIG. 4 is a schematic diagram illustrating training of a first teacher model according to an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of student model confidence calibration provided by an embodiment of the present disclosure;
fig. 6 is a block diagram of a device for generating an image identification model according to an embodiment of the present disclosure;
fig. 7 is a hardware structure diagram of a computer device provided by the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.
Fig. 1 is a flowchart of a method for identifying a face forged image according to an embodiment of the present disclosure, where the method is suitable for identifying a face forged image and may be executed by a computer device in an embodiment of the present disclosure, where the computer device may include an electronic device or a server. Electronic devices may include, but are not limited to, mobile terminals such as notebook computers and the like, and stationary terminals such as desktop computers and the like. The server may be a cloud server or a server cluster or other devices with storage and computing functions. As shown in fig. 1, the method comprises the steps of:
s110, a first sample image set and a second sample image set are obtained, false faces in the first sample image set are obtained through counterfeiting based on a first counterfeiting mode, and false faces in the second sample image set are obtained through counterfeiting based on a second counterfeiting mode.
In an exemplary embodiment, the first counterfeit manner may be exemplarily understood as a counterfeit manner of replacing the identity information of the object in the image, for example, the clothing of the person in the person image a may be replaced in the person image B by using computer technology so that the person in the person image B has the clothing information of the person in the person image a.
The second forgery mode may be exemplarily understood as a forgery mode of replacing motion information of an object in an image, for example, an expressive motion of a person in a person image B may be replaced in the person image a by computer technology so that the expressive motion information of the person in the person image B is provided in the person image a.
It should be noted that the above description is only an example and not a limitation of the first counterfeiting manner and the second counterfeiting manner, and in other embodiments, both the first counterfeiting manner and the second counterfeiting manner can be understood as other counterfeiting manners.
Fig. 2 is a schematic diagram of generating a face forged image in a different manner according to an embodiment of the present disclosure, where in the embodiment of the present disclosure, a first sample image set and a second sample image set can be obtained through the manner shown in the figure. Illustratively, the computer device acquires two real images of a person image A and a person image B for counterfeiting, the person image A can be used as a source image, the other person image B can be used as a target image, the action information of the person image A is kept unchanged, and the dress of the person image B is replaced on the person image A, namely the dress can be regarded as counterfeiting for replacing the identity information, so that a first sample image set is obtained. When the second sample image set is obtained, the character image B can be used as a source image, another character image a can be used as a target image, the identity information of the character image B is kept unchanged, and the expression of the character image a is replaced on the character image B, so that the character image can be regarded as the forgery of the more appropriate action information.
In the disclosed embodiment, the first sample image set and the second sample image set may be obtained from public data sets in a specific face forgery database, and in other embodiments, the first sample image set and the second sample image set may also be obtained from other databases, which is not limited herein.
Thus, the first sample image set of the replacement identification information and the second sample image set of the replacement operation information can be acquired.
And S120, training and generating a first teacher model for identifying a first counterfeiting mode based on the first sample image set.
Fig. 3 is a schematic diagram of a generation teacher model according to an embodiment of the present disclosure, in fig. 3, a complete data set may include an identity information replacement sub-data set and an action information replacement sub-data set, where the identity information replacement sub-data set includes data of an image in which identity information of an object is replaced, the action information replacement sub-data set includes data of an action information image in which the object is replaced, and the dual-teacher module includes a first teacher model and a second teacher model.
As shown in fig. 3, after the complete data set is obtained, the identity information replacement sub-data set in the complete data set may be used to train the first teacher model, so that the first teacher model may identify a first falsification manner, and the action information replacement sub-data set in the complete data set may be used to train the second teacher model, so that the second teacher model may identify a second falsification manner. The first teacher model may be regarded as a combination of EfficientNet-B3 and the feature Transformor encoder, and may output a probability of whether an image is a real image or a forged image, where for example, if the output probability is 0.1, the probability of the image being a forged image is 0.1, and the probability of the real image is 0.9.
Illustratively, fig. 4 is a schematic diagram of a first teacher model training provided in an embodiment of the present disclosure, as shown in fig. 4, when training the first teacher model, a 224 × 224 first sample image 201 is used as an input, a 1536 × 7 matrix 203 is obtained at an output end thereof through EfficientNet-B3202, 1536 7 matrices 204 are obtained by cutting on a first dimension (i.e., 1536), and then the obtained 7 × 7 matrix 204 is visualized and subjected to a feature linearization expansion process, and a result of the process is used as an input of a feature Transformor encoder 205, and the first sample image 201 is output as a predicted forgery probability 207 through the feature Transformor encoder 205 and a multi-layer perceptron 206.
In the continuous training process of the first teacher model, if the probability of the first teacher model output to each picture in the first sample image does not change greatly any more and is close to the attribute of the image (namely, the image is a real image or a false image), it can be determined that the first teacher model is generated and can be used for identifying the first counterfeiting mode.
Thus, the first teacher model can be acquired on the basis of the first sample image training.
And S130, training and generating a second teacher model for identifying a second counterfeiting mode based on the second sample image.
And the second teacher model and the first teacher model have the same structure.
The method for generating the second teacher model for identifying the second counterfeiting mode based on the training of the second sample image is the same as the method for generating the first teacher model for identifying the first counterfeiting mode based on the training of the first sample image, so that the method has the same or corresponding beneficial effects, and is not repeated herein for avoiding repetition.
And S140, based on the first teacher model and the second teacher model, fusing the capability of identifying a fake face of the two teacher models respectively to train the student models, and generating a target fake face image identification model for identifying the first fake mode and the second fake mode.
The student model and the teacher model adopt backbones with the same structure and different parameters, the backbones and the student model are only different in the number of layers of the characteristic Transformor encoder and the number of interfaces on each layer, for example, the characteristic Transformor encoder of the teacher model is a 1-layer encoder, the number of interfaces is 6, the data volume borne by the student model is larger, a 2-layer encoder is adopted, and the number of interfaces on each layer is 12. In the disclosed embodiment, the student model can be regarded as being learned by the first teacher model and the second teacher model.
In some embodiments, based on the first teacher model and the second teacher model, the capability of the two teacher models to identify a fake face is fused to train the student models, and a target fake face image identification model for identifying the first fake manner and the second fake manner is generated, including:
and S11, inputting a preset third sample image set into the first teacher model, the second teacher model and the student model respectively to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model.
And S12, distilling the knowledge of the first teacher model and the knowledge of the second teacher model based on the first output result, the second output result, the third output result and the sample labels corresponding to the third sample image set.
And S13, after the distillation treatment, jumping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute the steps until the evaluation index of the output result of the student model tends to be stable, stopping training, and obtaining the target face counterfeiting identification model.
The preset third sample image set may be an image forged based on the first forging manner or an image forged based on the second forging manner.
The sample label may indicate the attribute of the image itself, for example, in the embodiment of the present disclosure, the sample label may be 0 or 1, where 0 indicates that the image is a real face image, and 1 indicates that the image is a fake face image.
In the embodiment of the present disclosure, the fact that the degree of change of the output result of the student model is smaller than the preset threshold indicates that the identification of the student model on the preset third sample image is close to the attribute of the preset third sample image set and the fluctuation is small, and the student model can identify the first counterfeiting mode and the second counterfeiting mode.
In some embodiments, distilling the knowledge of the first teacher model and the second teacher model based on the first output result, the second output result, the third output result, and the sample labels corresponding to the third sample image set includes:
and S21, determining the cross entropy loss of the student model based on the sample label and the third output result.
And S22, determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result.
S23, determining the distillation loss based on the cross entropy loss, the first relative entropy loss and the second relative entropy loss.
And S24, adjusting parameters of the student model based on the distillation loss to reduce the distillation loss.
The cross entropy loss can indicate the loss of the identification result of the student model on the third sample image set relative to the attribute of the third sample image, and in the embodiment of the disclosure, the smaller the cross entropy loss is, the stronger the identification capability of the student model on the third sample image is.
The relative entropy loss can indicate the similarity between the authentication result of the student model and the authentication result of the teacher model, and the smaller the relative entropy loss is, the more the student model inherits the authentication capability of the teacher model corresponding to the counterfeit manner.
Specifically, after the computer device obtains the sample label, the first output result, the second output result and the third output result, the cross entropy loss of the student model can be determined according to the sample label and the third output result, the first relative entropy loss of the student model and the first teacher model can be determined according to the first output result and the third output result, the second relative entropy loss between the student model and the second teacher model can be determined according to the second output result and the third output result, and the distillation loss is determined according to the cross entropy loss, the first relative entropy loss and the second relative entropy loss. In the continuous training process of the student model, the distillation loss is continuously changed, and the parameters of the student model can be adjusted according to the dynamically changed distillation loss, so that the parameters influencing the distillation loss are reduced, namely the cross entropy loss, the first relative entropy loss and the second relative entropy loss, and the distillation loss is further reduced.
Illustratively, distillation loss can be defined as:
wherein, the first and the second end of the pipe are connected with each other,
the cross entropy loss of both in brackets is indicated,
the relative entropy loss of the two in parentheses is indicated,
in order to be the label of the sample,
in the form of a third output result,
in order to be the first output result,
in order to be the second output result,
are lost to distillation.
Therefore, the discrimination capability of the student model for the image forged by the first forging mode and the image forged by the second forging mode can be improved.
At present, some identification methods of face forged images use independence constraints among different domains to try to solve the over-fitting problem. But in practice these methods of adding independence constraints increase the consumption of computational resources and the runtime of the model can grow significantly.
In the method for generating an image identification model provided by the embodiment of the disclosure, after acquiring a first sample image set forged based on a first forging mode and a second sample image set forged based on a second forging mode, a computer device may train and generate a first teacher model for identifying the first forging mode based on the first sample image set, train and generate a second teacher model for identifying the second forging mode based on the second sample image set, and train and generate a target forged face image identification model for identifying the first forging mode and the second forging mode based on the first teacher model and the second teacher model.
Therefore, the generated target forged face image identification model is suitable for identifying face images generated by different forging methods, independence constraint is not required to be added, and calculation cost is saved.
In other embodiments, if a new counterfeiting method appears in the future, a new teacher model can be directly added, and training is continued on the basis of the current student model, so that the identification capability of the student model on various types of forged faces, including the faces generated by the novel counterfeiting method, is continuously enhanced, and is not limited herein.
In another embodiment of the present disclosure, in order to describe the task of face forgery identification more accurately and comprehensively, the present disclosure further provides a confidence calibration method for an identification model of a target forged face image.
In the embodiment of the present disclosure, after training a target forged face image authentication model based on a first teacher model and a second teacher model, and generating a target forged face image authentication model for authenticating a first forged mode and a second forged mode, the method further includes:
and calibrating the confidence coefficient of the identification model of the target forged face image.
In the embodiment of the present disclosure, a label redefinition method may be used to calibrate the confidence of the target face-forged image identification model, and in other embodiments, other methods may also be used to calibrate the confidence of the target face-forged image identification model, which is not limited herein.
In some embodiments, the process of calibrating the confidence of the identification model of the target face forged image comprises:
and S31, respectively inputting a preset fourth sample image set into the first teacher model, the second teacher model and the target face forged image identification model to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true, and a third probability value that the target face forged image identification model is identified as true.
And S32, determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image identification model and the first teacher model based on the first confidence coefficient to obtain a first weighted value.
And S33, determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value.
And S34, correcting the distillation loss of the target face counterfeit image identification model based on the cross entropy loss between the first weighted value, the second weighted value and the label probability value corresponding to the fourth sample image to obtain the corrected distillation loss.
And S35, adjusting parameters of the target face forged image identification model based on the corrected distillation loss, and skipping to the step of respectively inputting a preset fourth sample image set into the first teacher model, the second teacher model and the target face forged image identification model after parameter adjustment to continue execution until the variation degree of the output result of the target face forged image identification model is smaller than a preset threshold value, and stopping calibration.
The preset fourth sample image set comprises an image set forged based on the first forging mode or an image set forged based on the second forging mode.
The first probability value may be a first output result, the second probability value may be a second output result, and the third probability value may be a third output result.
The confidence is the absolute value of the difference between the probability value that the model identifies as true and the probability value that identifies as false, and in the disclosed embodiment, the sum of the probability value that the model identifies as true and the probability value that the model identifies as false is 1.
The label probability value is the probability value that the preset fourth sample image set is true.
Fig. 5 is a schematic diagram of a confidence calibration of a target face-forged image identification model according to an embodiment of the present disclosure, in fig. 5, a preset fourth sample image set may be an image with object identity information replaced or an image with object motion information replaced, y1 is a first weighted value of a first teacher model, y2 is a second weighted value of a second teacher model, s is a third probability value of the target face-forged image model being identified as true, and t is a label probability value.
Specifically, as shown in fig. 5, in the process of calibrating the confidence level of the target face forged image authentication model, a fourth sample image set may be introduced, when the student model, the first teacher model, and the second teacher model authenticate a preset fourth sample image set, a first probability value output by the first teacher model, a second probability value output by the second teacher model, and a third probability value output by the student model may be obtained, based on the first probability value, the confidence level of the first teacher model may be determined, based on the second probability value, the confidence level of the second teacher model may be determined, and the confidence level of the first teacher model is used as a weight of the first relative entropy loss, that is, y1, and similarly, the confidence level of the second teacher model may be used as a weight of the second relative entropy loss, that is, y 2. And determining the cross entropy loss between the third probability value and the label probability value t corresponding to the fourth sample image set, further adjusting the parameters of the student model based on the following distillation loss expression, skipping to the step of inputting the preset fourth sample image set into the first teacher model, the second teacher model and the target face forged image identification model respectively after the parameters are adjusted, continuing to execute the steps until the variation degree of the output result of the target face forged image identification model is smaller than a preset threshold value, and stopping calibration.
Wherein, the first and the second end of the pipe are connected with each other,
in order to be the first degree of confidence,
in order to be the second degree of confidence,
is the cross entropy loss between the third probability value and the label probability value corresponding to the fourth sample image set, is
，
Corrected for distillation losses.
Therefore, the confidence coefficient distribution condition of the target face counterfeit image identification model can be effectively optimized, overfitting is prevented from occurring, and further generalization is effectively improved, meanwhile, the label redefinition optimizes the original simple classification problem into the regression problem with fidelity as the target, and the task of face counterfeit identification is more accurately and comprehensively described.
Corresponding to the method for identifying the face forged image provided by the embodiment of the disclosure, the embodiment of the disclosure also provides a device for identifying the face forged image. Fig. 6 is a block diagram of a structure of an apparatus for generating an image authentication model according to an embodiment of the present disclosure, and as shown in fig. 6, the apparatus for identifying a face-forged image includes:
an obtaining unit 301, configured to obtain a first sample image set and a second sample image set, where the first sample image set is obtained by counterfeiting based on a first counterfeiting manner, and the second sample image set is obtained by counterfeiting based on a second counterfeiting manner;
a first teacher model generation unit 302 for training and generating a first teacher model for identifying a first forgery manner based on the first sample image set;
A second teacher model generation unit 303 configured to generate a second teacher model for identifying a second forgery manner based on the second sample image training;
a student model generation unit 304, configured to train a student model based on the first teacher model and the second teacher model, and generate a target face counterfeit image identification model for identifying the first counterfeit manner and the second counterfeit manner.
In some embodiments, the student model generation unit is specifically configured to:
respectively inputting a preset third sample image into the first teacher model, the second teacher model and the student model to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model;
distilling the knowledge of the first teacher model and the knowledge of the second teacher model based on the first output result, the second output result, the third output result and the sample label corresponding to the third sample image;
and after the distillation treatment, skipping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute until the evaluation index of the output result of the student model tends to be stable, stopping training, and obtaining the target face forged image identification model.
In some embodiments, the student model generation unit includes:
a distillation processing subunit, configured to determine cross entropy loss of the student model based on the third sample image set and the third output result;
determining a first relative entropy loss between the student model and the first teacher model based on the first output result and the third output result;
determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result;
determining a distillation loss based on the cross entropy loss, the first relative entropy loss, and the second relative entropy loss;
parameters of the student model are adjusted based on the distillation loss to reduce the distillation loss.
In some embodiments, the apparatus further comprises:
and the confidence coefficient calibration unit is used for calibrating the confidence coefficient of the identification model of the target face forged image.
In some embodiments, the confidence calibration unit is specifically configured to:
respectively inputting a preset fourth sample image into the first teacher model, the second teacher model and the target forged face image identification model to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true and a third probability value that the target face forged image is identified as true;
Determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image identification model and the first teacher model based on the first confidence coefficient to obtain a first weighted value;
determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value;
correcting the distillation loss of the target face forged image identification model based on the first weighted value, the second weighted value and the cross entropy loss between the third probability value and the forged probability value corresponding to the fourth sample image to obtain the corrected distillation loss;
and adjusting parameters of the target face forged image identification model based on the corrected distillation loss, skipping to the step of respectively inputting a preset fourth sample image into the first teacher model, the second teacher model and the target face forged image identification model after the parameters are adjusted, continuing to execute the steps until the variation degree of the output result of the target face forged image identification is smaller than a preset threshold value, and stopping calibration.
The face-forged image identification device disclosed in the above embodiments can execute the face-forged image identification method disclosed in each of the above embodiments, has the same or corresponding beneficial effects, and is not described herein again to avoid repetition.
Fig. 7 is a schematic hardware structure diagram of a computer device provided in the present disclosure. As shown in fig. 7, the computer device includes one or more processors 401 and memory 402.
The processor 401 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the computer device to perform desired functions.
In one example, the computer device may further include: an input device 403 and an output device 404, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
The input device 403 may also include, for example, a keyboard, a mouse, and the like.
The output device 404 may output various information to the outside, including the determined distance information, direction information, and the like. The output devices 404 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the computer device relevant to the present disclosure are shown in fig. 7, omitting components such as buses, input/output interfaces, and the like. In addition, the computer device may include any other suitable components, depending on the particular application.
Embodiments of the present invention further provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the processor executes the method for identifying a face counterfeit image according to the embodiments of the present invention.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable medium may be embodied in the computer device; or may exist separately without being assembled into the computer device.
In an embodiment of the present invention, program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A method for identifying a counterfeit image of a human face, the method comprising:
acquiring a first sample image set and a second sample image set, wherein false faces in the first sample image set are obtained by counterfeiting based on a first counterfeiting mode, and false faces in the second sample image set are obtained by counterfeiting based on a second counterfeiting mode;
training and generating a first teacher model for identifying the first counterfeiting mode based on the first sample image set;
training and generating a second teacher model for identifying the second counterfeiting mode based on a second sample image set;
based on the first teacher model and the second teacher model, fusing the capability of identifying a forged face of the two teacher models respectively to train student models, and generating a target forged face image identification model for identifying the first forged mode and the second forged mode;
Calibrating the confidence coefficient of the identification model of the target forged face image;
the method for calibrating the confidence of the identification model of the target face forged image comprises the following steps:
inputting a preset fourth sample image into the first teacher model, the second teacher model and the target forged face image identification model respectively to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true and a third probability value that the target face forged face image is identified as true;
determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image identification model and the first teacher model based on the first confidence coefficient to obtain a first weighting value;
determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value;
correcting the distillation loss of the target face forged image identification model based on the cross entropy loss between the first weighted value, the second weighted value, the third probability value and the forged probability value corresponding to the fourth sample image to obtain the corrected distillation loss;
And adjusting parameters of the target face forged image identification model based on the corrected distillation loss, and skipping to the step of respectively inputting a preset fourth sample image into the first teacher model, the second teacher model and the target face forged image identification model after parameter adjustment to continue to execute until the variation degree of the output result of the target face forged image identification is smaller than a preset threshold value, and stopping calibration.
2. The method of claim 1, wherein the fusing the ability of two teacher models to identify a fake face respectively based on the first teacher model and the second teacher model to train student models to generate a target fake face image identification model for identifying the first fake manner and the second fake manner comprises:
respectively inputting a preset third sample image set into a first teacher model, a second teacher model and a student model to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model;
distilling the knowledge of the first teacher model and the second teacher model based on the first output result, the second output result, the third output result, and the third sample image set;
And after distillation processing, skipping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute the steps until the evaluation index of the student model tends to be stable, and stopping training to obtain the target forged face image identification model.
3. The method of claim 2, wherein the distilling the knowledge of the first instructor model and the second instructor model based on the first output result, the second output result, the third output result, and the third sample image set comprises:
determining a cross-entropy loss for the student model based on the third sample image set and the third output result;
determining a first relative entropy loss between the student model and the first teacher model based on the first output result and the third output result;
determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result;
determining a distillation loss based on the cross entropy loss, the first relative entropy loss, and the second relative entropy loss;
Adjusting parameters of the student model based on the distillation loss to reduce the distillation loss.
4. A face-forged-image discrimination apparatus, characterized by comprising:
the image processing device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a first sample image set and a second sample image set, false faces in the first sample image set are forged based on a first forging mode, and false faces in the second sample image set are forged based on a second forging mode;
a first teacher model generation unit configured to generate a first teacher model for identifying the first forgery mode based on the first sample image set training;
a second teacher model generation unit configured to generate a second teacher model for identifying the second falsification manner based on a second sample image set training;
a student model generation unit configured to fuse, based on the first teacher model and the second teacher model, capabilities of the two teacher models to identify a type of fake face, respectively, to train a student model, and generate a target fake face image identification model for identifying the first fake manner and the second fake manner;
the confidence coefficient calibration unit is used for calibrating the confidence coefficient of the identification model of the target face forged image;
The confidence calibration unit is specifically configured to:
inputting a preset fourth sample image into the first teacher model, the second teacher model and the target forged face image identification model respectively to obtain a first probability value that the first teacher model is identified as true, a second probability value that the second teacher model is identified as true and a third probability value that the target face forged face image is identified as true;
determining a first confidence coefficient of the first teacher model based on the first probability value, and performing weighting processing on a third relative entropy loss between the target face forged image identification model and the first teacher model based on the first confidence coefficient to obtain a first weighting value;
determining a second confidence coefficient of the second teacher model based on the second probability value, and performing weighting processing on a fourth relative entropy loss between the target face forged image identification model and the second teacher model based on the second confidence coefficient to obtain a second weighted value;
correcting the distillation loss of the target face forged image identification model based on the cross entropy loss between the first weighted value, the second weighted value, the third probability value and the forged probability value corresponding to the fourth sample image to obtain the corrected distillation loss;
And adjusting parameters of the target face forged image identification model based on the corrected distillation loss, and skipping to the step of respectively inputting a preset fourth sample image into the first teacher model, the second teacher model and the target face forged image identification model after parameter adjustment to continue to execute until the variation degree of the output result of the target face forged image identification is smaller than a preset threshold value, and stopping calibration.
5. The apparatus according to claim 4, wherein the student model generation unit is specifically configured to:
respectively inputting a preset third sample image set into a first teacher model, a second teacher model and a student model to obtain a first output result of the first teacher model, a second output result of the second teacher model and a third output result of the student model;
distilling the knowledge of the first teacher model and the second teacher model based on the first output result, the second output result, the third output result, and the third sample image set;
and after distillation processing, skipping to the step of respectively inputting a preset third sample image set into the first teacher model, the second teacher model and the student model, and continuing to execute the steps until the evaluation index of the student model tends to be stable, and stopping training to obtain the target forged face image identification model.
6. The apparatus according to claim 5, wherein the student model generation unit comprises:
a distillation processing subunit for determining cross-entropy loss for the student model based on the third sample image set and the third output result;
determining a first relative entropy loss between the student model and the first teacher model based on the first output result and the third output result;
determining a second relative entropy loss between the student model and the second teacher model based on the second output result and the third output result;
determining a distillation loss based on the cross entropy loss, the first relative entropy loss, and the second relative entropy loss;
adjusting parameters of the student model based on the distillation loss to reduce the distillation loss.
7. A computer device, comprising:
a processor and a memory;
the processor is used for executing the steps of the human face forged image identification method according to any one of claims 1-3 by calling the program or the instructions stored in the memory.
8. A computer-readable storage medium, characterized in that it stores a program or instructions for causing a computer to carry out the steps of the method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210363659.8A CN114495245B (en) | 2022-04-08 | 2022-04-08 | Face counterfeit image identification method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210363659.8A CN114495245B (en) | 2022-04-08 | 2022-04-08 | Face counterfeit image identification method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114495245A CN114495245A (en) | 2022-05-13 |
| CN114495245B true CN114495245B (en) | 2022-07-29 |
Family
ID=81488380
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210363659.8A Active CN114495245B (en) | 2022-04-08 | 2022-04-08 | Face counterfeit image identification method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114495245B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116797493B (en) * | 2023-08-02 | 2024-01-26 | 北京中科闻歌科技股份有限公司 | Image denoising processing system |
| CN116740794B (en) * | 2023-08-15 | 2023-11-28 | 中国科学技术大学 | Face fake image identification method, system, equipment and storage medium |
| CN117133039B (en) * | 2023-09-01 | 2024-03-15 | 中国科学院自动化研究所 | Image fake identification model training method, image fake identification device and electronic equipment |
| CN117275068B (en) * | 2023-09-21 | 2024-05-17 | 北京中科闻歌科技股份有限公司 | Method and system for training human face fake detection in test stage containing uncertainty guidance |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418343A (en) * | 2020-12-08 | 2021-02-26 | 中山大学 | Multi-teacher self-adaptive joint knowledge distillation |
| CN114170655A (en) * | 2021-11-29 | 2022-03-11 | 西安电子科技大学 | Knowledge distillation-based face counterfeiting cue migration method |
| CN114170332A (en) * | 2021-11-27 | 2022-03-11 | 北京工业大学 | Image recognition model compression method based on anti-distillation technology |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013052025A1 (en) * | 2011-10-03 | 2013-04-11 | Hewlett-Packard Development Company, L.P. | Region selection for counterfeit determinations |
| CN112381209B (en) * | 2020-11-13 | 2023-12-22 | 平安科技（深圳）有限公司 | Model compression method, system, terminal and storage medium |
| CN113936275A (en) * | 2021-10-14 | 2022-01-14 | 上海交通大学 | Unsupervised domain adaptive semantic segmentation method based on region feature alignment |
-
2022
- 2022-04-08 CN CN202210363659.8A patent/CN114495245B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418343A (en) * | 2020-12-08 | 2021-02-26 | 中山大学 | Multi-teacher self-adaptive joint knowledge distillation |
| CN114170332A (en) * | 2021-11-27 | 2022-03-11 | 北京工业大学 | Image recognition model compression method based on anti-distillation technology |
| CN114170655A (en) * | 2021-11-29 | 2022-03-11 | 西安电子科技大学 | Knowledge distillation-based face counterfeiting cue migration method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114495245A (en) | 2022-05-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114495245B (en) | Face counterfeit image identification method, device, equipment and medium | |
| US11893099B2 (en) | Systems and methods for dynamic passphrases | |
| EP3723008A1 (en) | Method for protecting a machine learning model against extraction | |
| Wang et al. | Ponzi scheme detection via oversampling-based long short-term memory for smart contracts | |
| Wang et al. | Expression of Concern: Facial feature discovery for ethnicity recognition | |
| CN109840413B (en) | Phishing website detection method and device | |
| CN112464117A (en) | Request processing method and device, computer equipment and storage medium | |
| Zhou et al. | Semi-supervised fabric defect detection based on image reconstruction and density estimation | |
| Peng et al. | An improvement to linear regression classification for face recognition | |
| CN112231592A (en) | Network community discovery method, device, equipment and storage medium based on graph | |
| JP2023514120A (en) | Image Authenticity Verification Using Decoding Neural Networks | |
| Nezami et al. | Pick-object-attack: Type-specific adversarial attack for object detection | |
| KR102228737B1 (en) | Method And Apparatus for Detecting Fake News by Using Artificial Intelligence | |
| CN114663957A (en) | Face detection method, and training method and device of face detection model | |
| CN114548300B (en) | Method and device for explaining service processing result of service processing model | |
| KR102153790B1 (en) | Computing apparatus, method and computer readable storage medium for inspecting false offerings | |
| WO2023192808A1 (en) | Authentication of age, gender, and other biometric data from live images of users | |
| CN116502705A (en) | Knowledge distillation method and computer equipment for dual-purpose data set inside and outside domain | |
| CN113723111B (en) | Small sample intention recognition method, device, equipment and storage medium | |
| CN115130545A (en) | Data processing method, electronic device, program product, and medium | |
| WO2023272594A1 (en) | Image forgery detection via pixel-metadata consistency analysis | |
| CN115293222A (en) | Systems, methods, and computer program products for determining uncertainty from deep learning classification models | |
| KR102463070B1 (en) | Content agency management system and operation method thereof | |
| US11997137B2 (en) | Webpage phishing detection using deep reinforcement learning | |
| CN116843432B (en) | Anti-fraud method and device based on address text information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |