CN116188439A - False face-changing image detection method and device based on identity recognition probability distribution - Google Patents
False face-changing image detection method and device based on identity recognition probability distribution Download PDFInfo
- Publication number
- CN116188439A CN116188439A CN202310202602.4A CN202310202602A CN116188439A CN 116188439 A CN116188439 A CN 116188439A CN 202310202602 A CN202310202602 A CN 202310202602A CN 116188439 A CN116188439 A CN 116188439A
- Authority
- CN
- China
- Prior art keywords
- identity
- image
- probability distribution
- identity recognition
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a false face image detection method and device based on identity recognition probability distribution, which are used for constructing a real image and a corresponding identity label, and carrying out smoothing operation on the identity label to construct a smooth identity label; carrying out multiple rounds of supervision learning based on the identity tag on the identity recognition model by adopting the real image so as to optimize parameters; copying the identity recognition model optimized by the current parameters to serve as a copy model, and shielding a pixel area with the largest contribution in a prediction result of the copy model based on the real image to construct a shielding image; carrying out multiple rounds of supervised learning based on the smooth identity tag by adopting an occlusion image to optimize the parameters; and comparing the maximum predicted probability value output by the identity recognition model with a threshold value until training is finished, and detecting that the face image is forged when the maximum predicted probability value is smaller than the threshold value. The method improves the detection accuracy and generalization of the fake face-changing image while defending attack.
Description
Technical Field
The invention belongs to the technical field of deep fake detection, and particularly relates to a fake face-changing image detection method and device based on identity recognition probability distribution.
Background
In recent years, with the development of deep generation technologies such as a generation countermeasure network (GAN), the generation effect of a face-changing image becomes more realistic and the generation process becomes simpler, so that the threat to public opinion security becomes more serious, and research into a reliable fake face-changing image detection method is highly demanded.
In the existing fake face-change image detection method, fake face-change image detection is mostly modeled as an authenticity classification problem, a large number of fake face-change images and real face images are collected to train a classification depth neural network to be used for detecting authenticity of an image to be detected, for example, a deep fake image detection method combining multi-scale features is disclosed in patent document CN115100128A, and further, a face fake detection method is disclosed in patent document CN 115240243A. The method is easy to excessively fit on the face-changing fake image in the training set, and the detection accuracy of the image generated by the face-changing method which is not contained in the training set is reduced, so that the method is difficult to cope with the face-changing generation technology which is updated and iterated rapidly.
Another type of method improves generalization of the detection method in different face changing technologies by researching and extracting general fake marks caused by the face changing fake process, for example, detecting mixed boundary marks of faces and the like, but the marks are easily damaged by operations such as image compression, blurring and the like, so that the method is difficult to be used for detecting face changing images with reduced quality through network propagation.
In addition, existing counterfeit face image detection studies rarely defend against grey box attack scenarios where face makers know the detection method and try to bypass, and therefore there is a risk of bypassing by face makers.
Disclosure of Invention
In view of the above, the present invention aims to provide a method and a device for detecting a fake face-changing image based on an identification probability distribution, which improve the detection accuracy of the fake face-changing image while defending attacks.
In order to achieve the above object, an embodiment of the present invention provides a fake face-change image detection method based on an identification probability distribution, including the following steps:
step 1, constructing a real image and a corresponding identity tag, and performing smoothing operation on the identity tag to construct a smoothed identity tag;
step 2, performing multiple rounds of supervised learning based on the identity tag on the identity recognition model by adopting a real image to optimize parameters;
step 3, copying the identity recognition model optimized by the current parameters as a copy model, and shielding a pixel area with the largest contribution in a prediction result of the copy model based on a real image to construct a shielding image;
step 4, carrying out multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters;
and 5, repeating the step 3 and the step 4 until training is finished, and extracting a final parameter-optimized identity recognition model for fake face-changing image detection, wherein the method comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
Preferably, the smoothing operation refers to combining the unique hot label of the identity label with the uniform distribution, and the smoothed identity label vector is expressed as:
wherein y is s,c And c=target represents that the category C is the identity tag corresponding to the real image.
Preferably, the label smoothed super parameter α takes a value of 0.5.
Preferably, the identity recognition model is used for predicting and outputting the identity recognition probability distribution based on the input image, and when the current identity recognition model is subjected to multiple rounds of supervised learning based on the identity tag by adopting the real image, the identity recognition probability distribution and the unique thermal code of the identity tag are encoded by the predicted and outputted identity recognition modelCross entropy of codes as a loss function l i Expressed as:
wherein z is i Representing the identification probability distribution vector output by the ith image in the identification model,
representing z i Corresponding identity label y of middle real image i Probability value, z of (2) i,c Representing z i The probability value of the identity tag C, C representing the total number of identity tag classes, exp () representing the e-exponential function, log () representing the logarithmic function.
Preferably, the masking of the pixel region that contributes most in the prediction result of the copy model based on the real image to construct the mask image includes:
and (3) calculating the identity recognition probability distribution of the real image in the copy model, extracting probability values corresponding to the identity tags from the identity recognition probability distribution, carrying out counter propagation to calculate gradients, selecting N target pixel points with the largest gradient value, extracting rectangular areas taking each target pixel point as the center, and setting the pixel values of the N rectangular areas to be zero to realize shielding, so as to obtain a shielding image.
Preferably, when the occlusion image is adopted to perform multiple rounds of supervised learning based on the smoothed identity label on the identity recognition model optimized by the current parameters, the cross entropy of the predicted output identity recognition probability distribution and the smoothed identity label vector is used as a loss function l i ' expressed as:
wherein z is i Representing the identification probability distribution vector, z of the ith image output in the identification model i,c Representing z i Probability value, z of medium identity tag c i,j Representing z i The probability value of the identity tag j, C represents the total number of identity tag categories, exp () represents the e exponential function, log () represents the logarithmic function, y s,c Representing the smoothed identity tag vector value corresponding to category c.
Preferably, the identification model comprises a feature extraction unit for extracting image features and an identification unit for predicting identification probability distribution;
the feature extraction unit adopts an ArcFace model, and the identification unit adopts a fully-connected network.
In order to achieve the above object, an embodiment provides a false face image detection device based on identity recognition probability distribution, which includes a tag processing module, a pre-optimization module, a shielding image construction module, a re-optimization module and a detection module;
the label processing module is used for constructing a real image and a corresponding identity label, and smoothing the identity label to construct a smooth identity label;
the pre-optimization module is used for performing multiple rounds of supervision learning based on the identity tag on the identity recognition model by adopting the real image so as to optimize parameters;
the occlusion image construction module is used for copying the identity recognition model optimized by the current parameters as a copy model, and constructing an occlusion image by occluding a pixel area with the greatest contribution in a classification result of the copy model based on the real image;
the re-optimization module is used for performing multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters;
the detection module is used for extracting a final parameter-optimized identity recognition model for fake face-changing image detection, and comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
To achieve the above object, an embodiment of the present invention further provides a false face image detection device based on an identification probability distribution, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the false face image detection method based on the identification probability distribution when executing the computer program.
Compared with the prior art, the invention has the beneficial effects that at least the following steps are included:
the identity characteristic of the face-changing fake image is utilized to fuse the common essential characteristic of two different identities participating in face-changing, only the real image is needed to train an identity recognition model, and the conventional fake face-changing image is not needed to train an authenticity classification model, so that generalization of different face-changing methods is good; meanwhile, the identity characteristics are not easily damaged by image operations such as compression, so that the generalization of the invention to different quality images is good; meanwhile, the training strategy is designed to enlarge the attention area of the identity recognition model, so that the robustness to the gray box attack is enhanced.
The invention particularly aims at the scene of the protection character set which is known to need to detect the fake image, the identity recognition model is trained by utilizing the real face of the protection character, and the authenticity detection is carried out according to the difference of the identity recognition probability distribution of the face-changing fake image and the real image; aiming at the gray box attack scene that the fake image generator knows the detection method of the invention but cannot acquire specific model parameters, the invention utilizes the property that gray box attack depends on similar model attention areas, and expands the area of attention of the identity recognition model by shielding the area with the largest contribution to model classification and then training the strategy, thereby reducing the success rate of gray box attack; at the same time, using smooth labels in the loss calculation of the occluded image prevents overfitting.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a fake face-image detection method based on an identification probability distribution, which is provided by an embodiment;
FIG. 2 is a diagram of a training process for an identification model provided by an embodiment;
FIG. 3 is a flowchart of detection of a fake face-change image using an identification model according to an embodiment;
fig. 4 is a schematic structural diagram of a fake face-image detection device based on an identification probability distribution according to an embodiment.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the detailed description is presented by way of example only and is not intended to limit the scope of the invention.
The embodiment provides a false face-changing image detection method based on identity recognition probability distribution, which has the core innovation point that the identity characteristics of false face-changing images are fused with the common characteristics of two different identities participating in face-changing to detect, so that the face-changing detection generalization exceeding the existing method is achieved, an identity recognition model training strategy based on the smooth combination of attention-based image shielding and labels is designed for gray box attack scene defense, and a better defense effect is achieved under the gray box scene which is known by a face-changing generator and is attempted to bypass by the detection method.
The identity fusion common characteristic of the fake face-changing image is represented by the probability distribution output by the identity recognition model, so that the fake face-changing image and the real face image are distinguished, namely, the uncertainty of the identity recognition model on the identity of the fake face-changing image is higher than that of the real face image, and therefore the maximum probability value of the probability distribution is lower than that of the real face image. The training of the identity recognition model designs two training strategies of image shielding and label smoothing based on attention so as to detect fake face-changing images which are attacked by the gray box.
As shown in fig. 1, the false face image detection method based on the identification probability distribution provided by the embodiment includes the following steps:
and 1, constructing a real image and a corresponding identity label, and performing smoothing operation on the identity label to construct a smoothed identity label.
In the embodiment, a real image of a fake image to be detected and a corresponding identity tag are obtained, specifically taking a fake face-changing image data set CelebDF as an example, wherein the real image comprises 59 real videos with different identities and a fake video obtained by changing faces of any two identities. For each identity, 10 frames are randomly selected from frames acquired by all real videos of the identity, and 590 real face images are obtained as sample data of an identity recognition model.
The label smoothing operation is to combine the single thermal code label and the uniform distribution, and has the function of enabling the identity recognition model to pay attention to the whole face area as uniformly as possible, avoiding the fitting of the identity recognition model to the residual face after shielding, and the smoothed identity label vector is expressed as:
wherein y is s,c The smoothed identity tag vector value corresponding to category c, α, preferably 0.5, represents the label smoothing hyper-parameter, c represents the total number of identity tag categories, in this case 59, c=y represents the identity tag y corresponding to the real image for category c.
And 2, performing multiple rounds of supervised learning based on the identity tags on the identity recognition model by adopting the real image so as to optimize parameters.
In an embodiment, the identification model f is composed of a feature extraction unit for image feature extraction and an identification unit for predicting the identification probability distribution, and the identification model f needs to be parameter optimized before being applied.
The feature extraction unit encodes an input face image into identity feature vectors, and the extracted identity feature vectors have the properties of short feature distances of the faces with the same identity and long feature distances of the faces with different identities after pretraining on a large number of real face data sets; the identification unit is used for identifying the characters in the image to be detected, wherein the input dimension is the same as the output feature dimension of the feature extraction unit, and the output dimension is the same as the number of the characters in the determined protection character set needing to detect the forged image.
Specifically, the feature extraction unit takes an open source model ArcFace as an example, the network structure of the feature extraction unit is ir_se50, and the ArcFace loss function is used for pre-training on a large-scale face recognition data set, and pre-training parameters are publicly available. Inputting 112×112 aligned face image, and outputting 512-dimensional identity feature vector; the identification unit adopts the FC layer to connect the output end at the last layer of ArcFace, inputs 512-dimensional identity feature vectors, and the output dimension is the identity identification probability distribution of the identity tag class number, and in this example, the output dimension is 59.
Before training the identity recognition model by using the real image, carrying out data processing on the real image, wherein the data processing comprises the following steps: identifying a face region by using the MTCNN, carrying out affine transformation on the image according to the coordinates of the face feature points to align the face, cutting the face region at a certain scaling scale, and scaling the face region to be the input size of Arcface (112 multiplied by 112); the parameters of the FC layer are randomly initialized by loading the pre-training parameters of the ArcFace.
When the identity recognition model is pretrained for a plurality of times (for example, 10 times), the Arcface parameter is fixed, only the FC layer parameter is updated, and the cross entropy loss l is calculated by using the unique thermal code coding of the identity label of the real image and the identity recognition probability distribution of the prediction output i ：
Wherein z is i Representing that the ith training sample outputs 59-dimensional identity recognition probability distribution vectors through the FC layer,
representing z i Corresponding identity label y of middle real image i Probability value, z of (2) i,c Representing z i The probability value of the identity tag C in (a), C represents the total number of identity tag categories, i.e. 59, exp () represents the e exponential function and log () represents the logarithmic function.
And 3, copying the identity recognition model optimized by the current parameters as a copy model, and shielding a pixel area with the largest contribution in a prediction result of the copy model based on the real image to construct an shielding image.
In an embodiment, as shown in fig. 2, after the identification model is subjected to multiple rounds of parameter optimization, the identification model optimized by the current parameter is copied as a copy model f ', and the copy model f' is used for calculating an occlusion region according to a gradient. Specifically, inputting the real image to the copy model f', and based on the pixel region where the real image contributes most in the prediction result of the copy model, performing occlusion to construct an occlusion image, including:
calculating the output identity recognition probability distribution of the real image in the FC layer of the copy model, and extracting the identity tag y from the identity recognition probability distribution i Corresponding probability value
After counter propagation is carried out to calculate gradients, N target pixel points with the largest gradient value are selected, rectangular areas taking each target pixel point as the center are extracted, and pixel values of the N rectangular areas are set to be zero to realize shielding, so that a shielding image is obtained. Wherein N is preferably 5, and when the rectangular area is extracted, two integers can be randomly selected from 25-35 as the length and width to obtain a rectangular area.
And 4, performing multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters.
In an embodiment, when a current parameter-optimized identity recognition model is subjected to multiple rounds (for example, 5 rounds) of supervised learning based on smooth identity labels by using occlusion images, cross entropy loss l of predicted output identity recognition probability distribution and smooth identity label vector is calculated i ' and back-propagating to update parameters of the identification model, namely, optimizing parameters of the FC layer and parameters of a higher layer of the ArcFace model, in this embodiment, fixing parameters of the 20 th convolution module and the lower layer of the ArcFace, updating parameters above the 20 th convolution module, and according to actual conditionsThe situation may vary. Wherein cross entropy loss l i ' is expressed as:
wherein z is i,j Representing z i Probability value of identity tag j.
And 5, repeating the step 3 and the step 4 until training is finished, and extracting a final parameter-optimized identity recognition model for fake face-changing image detection, wherein the method comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
After the step 4 is carried out on the identity recognition model by utilizing the shielding image for training for multiple times, updating the copy model by using the parameters of the new identity recognition model, namely extracting the identity recognition model optimized by the parameters in the step 4 as the copy model, continuing the steps 3 and 4 for a plurality of times until the iteration is finished, wherein the iteration finishing condition can be a loss function l of the identity recognition model i 'convergence'.
After training, extracting a final parameter-optimized identity recognition model for fake face-changing image detection, and based on the characteristic that the face-changing image fuses two different identities for face-changing, the uncertainty of the identity recognition model on the identity of the face-changing image is higher than that of a real face image, so that the maximum probability value of the face-changing image is lower than that of the real face image, and the fake face-changing image is recognized according to the maximum probability value in the identity recognition probability distribution, as shown in fig. 3, and the method comprises the following steps:
firstly, inputting an image to be detected into a trained identity recognition model, obtaining a logits vector output by an FC layer, and calculating the identity recognition probability distribution through Softmax, wherein the formula is as follows:
wherein x represents an image to be measured, and p (x) c Representing identity of input imageThe probability of tag c, exp () represents the e exponential function, f (x) represents the logits vector output by the identity model, f (x) c And f (x) j The values at subscript c and subscript j in the logits vector are represented, respectively.
Then, the maximum probability value p is identified from the identification probability distribution output from the identification model max Expressed by the formula:
and finally, comparing the maximum predicted probability value with a threshold value, and detecting that the face image is forged when the maximum predicted probability value is smaller than the threshold value, or else, detecting that the face image is a real image. The threshold value is obtained through testing, and specifically comprises the following steps: and collecting real and fake test samples, inputting the trained identity recognition model, obtaining the maximum probability value in the identity recognition probability distribution, and determining the optimal threshold value of the true and false classification according to the maximum probability value and the real label of the test sample.
In order to test the effect of the present invention, a test was performed on a partial face-change forgery data set with reference to the procedure of the above specific embodiment example, and the closer the AUC is to 1.0, the better the detection effect is, using AUC (Area Under Curve) as an evaluation index. Table 1 shows the test results on three datasets, faceforensis++, celeb-DF, deeperForensics-1.0, where faceforensis++ contains only three of the face counterfeits: deepFakes, faceSwap, faceShifter, does not contain expression activation counterfeits, and faceforensics++ is compressed to varying degrees.
TABLE 1 results of authenticity test
As can be seen from the test results in Table 1, the detection results of the invention are all over 95%, 98.9% and 97.9% are achieved under different compression rates of faceforensis++ face-changing fake data sets, and the detection performance is high.
In order to test the implementation effect of the invention, grey box attack defense effect test is carried out on a CelebDF face-changing fake data set, and attack success rate ASR (Attack Success Rate) is used as an evaluation index, and the closer ASR is to 0, the better the defense effect is.
Specifically, the gray box attack refers to that an attacker knows a detection method, but cannot acquire specific parameters and/or training data sets of an identification model for detection, so that a face-changing fake image is modified by using other available models and/or data so as to be identified as a real image by the detection method. As an example, the gray box attack implementation method of the simulation of the present example is that after the face-changing counterfeit image is generated, an attacker obtains a public pre-training face identity feature extraction model, for example ArcFace, inputs the face-changing counterfeit image and the real image of the target person of face-changing counterfeit into ArcFace respectively, so that two 512-dimensional identity feature vectors can be obtained, and adds an anti-noise on the face-changing counterfeit image by using a projection gradient descent method, so that the cosine distance between the identity feature vector obtained by the face-changing counterfeit image after noise addition through ArcFace and the identity feature vector of the target real image is reduced. Table 2 shows ASR after the above attack on the fake image of the CelebDF dataset, and shows, for comparison, ASR that directly uses the ArcFace fixed, training only the identification model of the FC layer obtained in step 2 without taking the training strategies described in steps 3 to 4 of the present invention.
Table 2 ash box attack resistance test results
| Method | ASR(％) | AUC |
| The training strategy described in steps (3-3) to (3-5) is not taken | 97.11％ | 0.9933 |
| The complete steps of the invention | 39.07％ | 0.9617 |
From the test results in table 2, it can be seen that the training strategies described in steps 3 to 4 of the present invention reduced ASR from 97% to 39% while maintaining a high AUC and a good defense effect.
Based on the same inventive concept, the false face image detection device based on the identity recognition probability distribution, as shown in fig. 4, comprises a label processing module, a pre-optimization module, a shielding image construction module, a re-optimization module and a detection module;
the label processing module is used for constructing a real image and a corresponding identity label, and smoothing the identity label to construct a smooth identity label; the pre-optimization module is used for performing multiple rounds of supervision learning based on the identity tag on the identity recognition model by adopting the real image so as to optimize parameters; the occlusion image construction module is used for copying the identity recognition model optimized by the current parameters as a copy model, and constructing an occlusion image by occluding a pixel area with the greatest contribution in a classification result of the copy model based on the real image; the re-optimization module is used for performing multiple rounds of supervision learning based on the smooth identity tag by adopting the shielding image to optimize the parameters; the detection module is used for extracting a final parameter-optimized identity recognition model for fake face-change image detection, and comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
It should be noted that, when the fake face-image detection device provided in the foregoing embodiment performs fake face-image detection, the division of the foregoing functional modules should be used for illustration, and the foregoing functional allocation may be performed by different functional modules according to needs, that is, the internal structure of the terminal or the server is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the foregoing embodiment of the present invention provides a false face image detection device and a false face image detection method, which belong to the same concept, and detailed implementation procedures of the false face image detection device and the false face image detection method are detailed in the false face image detection method, and are not described herein.
According to the fake face-change image detection device provided by the embodiment, the identity characteristic is applied to fake face-change image detection, the common characteristic that two different identities for face change are fused with each other by using the face-change image, the fake face-change image and the real face image are distinguished through probability distribution output by the identity recognition model, and the design of resisting grey box attack is performed on the training strategy of the identity recognition model. The invention acquires the real image of the character set needing to detect the fake image and the corresponding identity tag as a training set; acquiring a pre-trained face identity feature extraction model, and adding a full connection layer to serve as an identity recognition model; training an identity recognition model by adopting a strategy of combining image shielding based on attention with label smoothing; inputting the image to be detected for authenticity into an identity recognition model to obtain identity recognition probability distribution; the fake face-changing image is identified according to the maximum probability value in the identity identification probability distribution, and the uncertainty of the identity identification model on the identity of the face-changing image is higher than that of the real face image because the face-changing image is fused with two different identities for face-changing, so that the maximum probability value of the face-changing image is lower than that of the real face image. In the training step, only real images are used, and training is carried out without depending on the existing forged face-changing images, so that generalization of different face-changing methods is good; meanwhile, the identity characteristics are not easily damaged by image operations such as compression, so that generalization of images with different qualities is good; meanwhile, the training strategy is designed to enlarge the attention area of the identity recognition model, so that the robustness to the gray box attack is enhanced.
Based on the same inventive concept, the embodiment also provides a false face image detection device based on the identity recognition probability distribution, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to realize the false face image detection method, and the false face image detection device comprises the following steps:
step 1, constructing a real image and a corresponding identity tag, and performing smoothing operation on the identity tag to construct a smoothed identity tag;
step 2, performing multiple rounds of supervised learning based on the identity tag on the identity recognition model by adopting a real image to optimize parameters;
step 3, copying the identity recognition model optimized by the current parameters as a copy model, and shielding a pixel area with the largest contribution in a prediction result of the copy model based on a real image to construct a shielding image;
step 4, carrying out multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters;
and 5, repeating the step 3 and the step 4 until training is finished, and extracting a final parameter-optimized identity recognition model for fake face-changing image detection, wherein the method comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
In practical applications, the memory may be a volatile memory at the near end, such as a RAM, or a nonvolatile memory, such as a ROM, a FLASH, a floppy disk, a mechanical hard disk, or a remote storage cloud. The processor may be a Central Processing Unit (CPU), a Microprocessor (MPU), a Digital Signal Processor (DSP), or a Field Programmable Gate Array (FPGA), i.e. the steps of the fake face image detection method based on the identification probability distribution may be implemented by these processors.
The foregoing detailed description of the preferred embodiments and advantages of the invention will be appreciated that the foregoing description is merely illustrative of the presently preferred embodiments of the invention, and that no changes, additions, substitutions and equivalents of those embodiments are intended to be included within the scope of the invention.
Claims (9)
1. A fake face-changing image detection method based on identity recognition probability distribution is characterized by comprising the following steps:
step 1, constructing a real image and a corresponding identity tag, and performing smoothing operation on the identity tag to construct a smoothed identity tag;
step 2, performing multiple rounds of supervised learning based on the identity tag on the identity recognition model by adopting a real image to optimize parameters;
step 3, copying the identity recognition model optimized by the current parameters as a copy model, and shielding a pixel area with the largest contribution in a prediction result of the copy model based on a real image to construct a shielding image;
step 4, carrying out multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters;
and 5, repeating the step 3 and the step 4 until training is finished, and extracting a final parameter-optimized identity recognition model for fake face-changing image detection, wherein the method comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
2. The false face image detection method based on the identity recognition probability distribution according to claim 1, wherein the smoothing operation is to combine a single thermal code label of the identity label with uniform distribution, and the smoothed identity label vector is expressed as:
wherein y is s,c And c=target represents that the category C is the identity tag corresponding to the real image.
3. The false face image detection method based on the identity recognition probability distribution according to claim 2, wherein the label smoothed super-parameter alpha takes a value of 0.5.
4. The false face image detection method based on the identification probability distribution according to claim 1, wherein the identification model is used for predicting and outputting the identification probability distribution based on the input image, and when the current identification model is subjected to multiple rounds of supervised learning based on the identification label by adopting the real image, the cross entropy of the identification probability distribution outputted by prediction and the single thermal code coding of the identification label is used as a loss function l i Expressed as:
wherein z is i Representing the identification probability distribution vector, z of the ith image output in the identification model i,yi Representing z i Corresponding identity label y of middle real image i Probability value, z of (2) i,c Representing z i The probability value of the identity tag C, C representing the total number of identity tag classes, exp () representing the e-exponential function, log () representing the logarithmic function.
5. The false face image detection method based on the identity recognition probability distribution according to claim 1, wherein the masking of the pixel region which contributes most to the prediction result of the copy model based on the real image to construct a mask image includes:
and (3) calculating the identity recognition probability distribution of the real image in the copy model, extracting probability values corresponding to the identity tags from the identity recognition probability distribution, carrying out counter propagation to calculate gradients, selecting N target pixel points with the largest gradient value, extracting rectangular areas taking each target pixel point as the center, and setting the pixel values of the N rectangular areas to be zero to realize shielding, so as to obtain a shielding image.
6. The false face image detection method based on identity recognition probability distribution according to claim 1, wherein when the mask image is adopted to perform multiple rounds of supervised learning based on the smooth identity tag on the identity recognition model optimized by the current parameters, cross entropy of the predicted output identity recognition probability distribution and the smooth identity tag vector is taken as a loss function l i ' expressed as:
wherein z is i Representing the identification probability distribution vector, z of the ith image output in the identification model i,c Representing z i Probability value, z of medium identity tag c i,j Representing z i The probability value of the identity tag j, C represents the total number of identity tag categories, exp () represents the e exponential function, log () represents the logarithmic function, y s,c Representing the smoothed identity tag vector value corresponding to category c.
7. A false face image detection method based on an identification probability distribution according to claim 1, wherein the identification model includes a feature extraction unit for image feature extraction and an identification unit for predicting the identification probability distribution.
8. The false face image detection device based on the identity recognition probability distribution is characterized by comprising a label processing module, a pre-optimization module, a shielding image construction module, a re-optimization module and a detection module;
the label processing module is used for constructing a real image and a corresponding identity label, and smoothing the identity label to construct a smooth identity label;
the pre-optimization module is used for performing multiple rounds of supervision learning based on the identity tag on the identity recognition model by adopting the real image so as to optimize parameters;
the occlusion image construction module is used for copying the identity recognition model optimized by the current parameters as a copy model, and constructing an occlusion image by occluding a pixel area with the greatest contribution in a classification result of the copy model based on the real image;
the re-optimization module is used for performing multiple rounds of supervised learning based on the smooth identity tag by adopting the shielding image to optimize the parameters;
the detection module is used for extracting a final parameter-optimized identity recognition model for fake face-changing image detection, and comprises the following steps: and comparing the maximum predicted probability value output by the identity recognition model with a threshold value, and detecting the false face-changing image when the maximum predicted probability value is smaller than the threshold value.
9. A false face image detection device based on an identification probability distribution, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the false face image detection method based on an identification probability distribution according to any one of claims 1-7 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310202602.4A CN116188439A (en) | 2023-03-06 | 2023-03-06 | False face-changing image detection method and device based on identity recognition probability distribution |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310202602.4A CN116188439A (en) | 2023-03-06 | 2023-03-06 | False face-changing image detection method and device based on identity recognition probability distribution |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116188439A true CN116188439A (en) | 2023-05-30 |
Family
ID=86444222
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310202602.4A Pending CN116188439A (en) | 2023-03-06 | 2023-03-06 | False face-changing image detection method and device based on identity recognition probability distribution |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116188439A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117611923A (en) * | 2024-01-08 | 2024-02-27 | 北京锐融天下科技股份有限公司 | Identification method and system for identity document authenticity |
-
2023
- 2023-03-06 CN CN202310202602.4A patent/CN116188439A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117611923A (en) * | 2024-01-08 | 2024-02-27 | 北京锐融天下科技股份有限公司 | Identification method and system for identity document authenticity |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230022943A1 (en) | Method and system for defending against adversarial sample in image classification, and data processing terminal | |
| Zhu et al. | AR-Net: Adaptive attention and residual refinement network for copy-move forgery detection | |
| CN111753881B (en) | Concept sensitivity-based quantitative recognition defending method against attacks | |
| CN110941794B (en) | Challenge attack defense method based on general inverse disturbance defense matrix | |
| CN109902018B (en) | Method for acquiring test case of intelligent driving system | |
| CN111160313B (en) | Face representation attack detection method based on LBP-VAE anomaly detection model | |
| CN113269228B (en) | Method, device and system for training graph network classification model and electronic equipment | |
| CN105654056A (en) | Human face identifying method and device | |
| CN114693607B (en) | Tamper video detection method and tamper video detection system based on multi-domain block feature marker point registration | |
| CN113283590A (en) | Defense method for backdoor attack | |
| CN111222583B (en) | Image steganalysis method based on countermeasure training and critical path extraction | |
| Suratkar et al. | Deep-fake video detection approaches using convolutional–recurrent neural networks | |
| CN117113163A (en) | Malicious code classification method based on bidirectional time domain convolution network and feature fusion | |
| CN114724218A (en) | Video detection method, device, equipment and medium | |
| CN114758113A (en) | Confrontation sample defense training method, classification prediction method and device, and electronic equipment | |
| CN116188439A (en) | False face-changing image detection method and device based on identity recognition probability distribution | |
| CN116343301A (en) | Personnel information intelligent verification system based on face recognition | |
| Liu et al. | Defend Against Adversarial Samples by Using Perceptual Hash. | |
| CN116978096A (en) | Face challenge attack method based on generation challenge network | |
| CN116824695A (en) | Pedestrian re-identification non-local defense method based on feature denoising | |
| Hirofumi et al. | Did You Use My GAN to Generate Fake? Post-hoc Attribution of GAN Generated Images via Latent Recovery | |
| CN116260565A (en) | Chip electromagnetic side channel analysis method, system and storage medium | |
| Girish et al. | Inter-frame video forgery detection using UFS-MSRC algorithm and LSTM network | |
| CN113283520B (en) | Feature enhancement-based depth model privacy protection method and device for membership inference attack | |
| CN113723215A (en) | Training method of living body detection network, living body detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |