CN113723295B - Face counterfeiting detection method based on image domain frequency domain double-flow network - Google Patents
Face counterfeiting detection method based on image domain frequency domain double-flow network Download PDFInfo
- Publication number
- CN113723295B CN113723295B CN202111009733.8A CN202111009733A CN113723295B CN 113723295 B CN113723295 B CN 113723295B CN 202111009733 A CN202111009733 A CN 202111009733A CN 113723295 B CN113723295 B CN 113723295B
- Authority
- CN
- China
- Prior art keywords
- image
- network
- model
- labels
- frequency domain
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 39
- 238000012549 training Methods 0.000 claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000001228 spectrum Methods 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 15
- 238000012360 testing method Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 4
- 238000013527 convolutional neural network Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000001902 propagating effect Effects 0.000 claims description 3
- 239000011248 coating agent Substances 0.000 claims description 2
- 238000000576 coating method Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of computer vision, in particular to a face counterfeiting detection method based on an image domain frequency domain double-flow network. The detection method comprises two stages of model training and model inference; in the model training stage, a server with high computing performance is utilized to train a network model, network parameters are optimized by reducing a network loss function until the network converges, and a frequency domain double-flow network model based on an image domain is obtained; in the model deducing stage, the network model obtained in the model training stage is utilized to judge whether the new image is a face fake image. Compared with the method for converting the face fake detection into the two-classification problem, the method realizes hierarchical classification, utilizes the supervision information of fake image diversity, and can achieve better detection precision.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a face counterfeiting detection method based on an image domain frequency domain double-flow network.
Background
With the development of advanced face synthesis algorithms, various realistic fake faces are generated, and the application of the face synthesis algorithms to social media is attracting high attention. The malicious use of fake faces can have a great adverse effect on individuals and society, and therefore detection of fake faces is very important.
The task of face-counterfeit detection presents a number of challenges, particularly the diversity of counterfeit algorithms, and face-counterfeit images that are streamed over the internet tend to be of low quality and difficult to detect.
Early studies attempted to use hand-made features or simply modify existing neural networks. The MesoNet designs a shallow neural network consisting of two initial modules and two classical convolution layers, but a simple shallow network cannot achieve good accuracy in realistic face falsification data.
Some methods make use of the biological features of the face, such as training images obtained from the internet that typically do not include a photograph of a closed eye, can lead to a lack of true blinking of a counterfeit face. And detecting the blink phenomenon through the neural network model, thereby detecting the fake human face. However, such detection can be avoided by deliberately combining the images of the closed eye during training.
The FaceX-ray uses its own generated data to locate the region of the face change, but the method cannot achieve good accuracy and effect in the detection of low quality counterfeit images.
Most of face fake detection algorithms at present only use the information of image domain to convert the task into two classification tasks to judge the authenticity of the image, and cannot use the supervision information of various fake algorithms. Detectable elements such as artifacts present in the counterfeit image can be compressed in the low quality image and difficult to detect, while artifacts in the frequency domain are still present in the low quality image.
Disclosure of Invention
The invention aims to provide a face counterfeiting detection method based on an image domain frequency domain double-flow network aiming at the defects of face counterfeiting detection at present.
The method is based on a human face counterfeiting detection framework of an image domain frequency domain double-flow network, and meanwhile, hierarchical supervision is carried out by utilizing supervision information of different counterfeiting algorithms, so that the counterfeiting detection precision can be ensured in images with different definition.
The method solves the problems that the traditional face fake detection algorithm has poor detection effect on low-quality images, and the detection by using image domain artifacts is unstable.
The technical scheme adopted for solving the technical problems is as follows:
a face fake detection method based on an image domain frequency domain double-flow network comprises two stages of model training and model inference;
in the model training stage, a server with high computing performance is utilized to train a network model, network parameters are optimized by reducing a network loss function until the network converges, and a frequency domain double-flow network model based on an image domain is obtained;
in the model deducing stage, the network model obtained in the model training stage is utilized to judge whether the new image is a face fake image.
Preferably, in the model training phase, the method specifically comprises the following steps:
firstly, preparing image data, namely, performing Discrete Cosine Transform (DCT) on images in a training set to obtain spectrum images of the images, performing image data enhancement processing on the images before and after processing, and simultaneously, re-integrating corresponding image data labels to enable each image to have two-stage labels;
then, training a network model, namely sending the original image and the spectrum image subjected to data enhancement processing into an image domain frequency domain double-flow network in pairs, and training by adopting a hierarchical supervision method; and calculating a loss function to obtain a gradient, and then performing back propagation to obtain a trained network model.
Preferably, with respect to the Discrete Cosine Transform (DCT), in particular using the type ii 2D-DCT, the two-dimensional DCT transform of the input I (N dimensions) is defined as:
D＝C N ·I·(C N ) T ，
wherein:
d is an output, and D is an output,
C N is the coefficient of the transformation matrix and,
where j=0, α j =1; when j > 0, alpha j ＝2；
The upper left corner of the spectrum image after DCT transformation contains image low-frequency information, and the lower right corner contains image high-frequency information.
Preferably, the image data enhancement includes performing horizontal inversion, vertical inversion, and application of Mix Up and Random spectroscopy on the original image and the spectrum image;
wherein Mix up is the virtual data of the linear interpolation of two random samples and the labels thereof in the training set, which is used as training data; random erase is a block of area in a Random erase image.
Preferably, the re-integration of the image tag includes:
firstly, converting binary labels of images into multi-class labels according to fine-granularity label information; one type for real images and multiple types for counterfeit images, wherein specific counterfeit methods are considered, such as FaceSwap, face2Face, deepfake; furthermore, there is a hierarchy in the multi-class labels described above, because the entire dataset is divided into true and false classes;
meanwhile, various fake images belong to the whole fake category;
the data set labels are re-integrated into the hierarchical structure of the data set labels, so that the labels of coarse classification and fine classification levels of each image are realized, wherein the coarse classification labels are first-level labels, and the fine classification labels are second-level labels.
Preferably, for the model inference phase, specific steps include:
firstly, the test data image is subjected to Discrete Cosine Transform (DCT) to obtain an image spectrogram,
and then, transmitting the test data image and the obtained corresponding image spectrogram into a trained model to obtain an inference result, obtaining a two-classification result and a multi-classification result of the test image, and carrying out weighted average fusion on the two results to obtain a final model inference result.
Preferably, the network model training includes image data forward propagation and gradient backward propagation; specific:
the image data is transmitted forward, and all the original images subjected to data enhancement and the spectrum images subjected to data enhancement in the training set are sent into a network in pairs for training;
the network comprises a double-flow characteristic fusion module and a hierarchical supervision module;
the overall network structure consists of two branches, namely an image domain branch and a frequency domain branch, wherein the image domain branch is taken as a main stream, and the frequency domain branch is taken as an auxiliary information stream and comprises a convolution layer, a pooling layer, a full connection layer+relu and a full connection layer; carrying out concat fusion on the frequency domain feature map and the image domain feature map at a specific layer of the network; the reasonable fusion utilization of the information of the image domain and the frequency domain is realized;
a hierarchical supervision module: modifying a network structure based on a hierarchical label, extracting image features based on a convolutional neural network in a hierarchical manner, adding a coarse classification head to a shallow part of the network to obtain coarse classification output, and adding a fine classification head to a deep part of the network to obtain fine classification output;
gradient back propagation: calculating a binary cross entropy loss by using the coarse classification output and the first-stage label, and calculating a multi-classification cross entropy loss by using the fine classification output and the second-stage label; and calculating the gradient according to the loss function, reversely propagating the gradient to update the network parameters, accelerating by utilizing the GPU, and stopping training until the error of the network is reduced to be within a set threshold value or the number of network iterations meets the requirement, so as to obtain a trained network model.
Preferably, the loss function used for cross entropy loss is:
wherein:
n is the number of all pictures and,
h is the number of all of the levels,
α h is the weight of the h-th level,
CE is the cross entropy loss.
According to the face counterfeiting detection method based on the image domain frequency domain double-flow network, the frequency domain information of the image is obtained through DCT transformation, and fusion of the image domain information and the frequency domain information is achieved in the network through the double-flow network, so that the network can simultaneously utilize the information of the two domains. Meanwhile, the hierarchical label is realized on the image label, hierarchical supervision is utilized and coarse classification loss and fine classification loss are weighted and averaged when the network is trained, supervision information of different fake algorithms can be fully utilized, and the model detection performance is further improved.
Compared with the prior art, the invention has the beneficial effects that:
compared with the method for converting the face fake detection into the two-classification problem, the method realizes the hierarchical classification, utilizes the supervision information of the fake image diversity, and can achieve better detection precision;
compared with a method for carrying out face counterfeiting detection by using only the fake image domain artifact, the method realizes that the double-flow network fuses the image frequency domain information and the image domain information in the network, and can effectively solve the problem that the fake image domain artifact with low definition is compressed and difficult to detect.
Drawings
Fig. 1 is a schematic diagram of a face forgery detection flow based on an image domain frequency domain dual-flow network of the present invention.
FIG. 2 is a schematic diagram of tag re-integration according to the present invention.
Fig. 3 is a schematic view of the hierarchical supervision of the present invention.
Detailed Description
Representative embodiments based on the drawings will now be further refined. It should be understood that the following description is not intended to limit the embodiments to one preferred embodiment. On the contrary, it is intended to cover alternatives, modifications, and equivalents as may be included within the spirit and scope of the embodiments as defined by the appended claims.
A face fake detection method based on an image domain frequency domain double-flow network comprises two stages of model training and model inference;
in the model training stage, a server with high computing performance is utilized to train a network model, network parameters are optimized by reducing a network loss function until the network converges, and a frequency domain double-flow network model based on an image domain is obtained;
in the model deducing stage, the network model obtained in the model training stage is utilized to judge whether the new image is a face fake image.
In the model training stage, the method specifically comprises the following steps:
firstly, preparing image data, namely obtaining a spectrum image of an image by Discrete Cosine Transform (DCT) of the image in a training set, carrying out image data enhancement processing on the image before and after processing, and simultaneously, re-integrating corresponding image data labels to enable each image to have two-stage labels;
regarding Discrete Cosine Transform (DCT), in particular using type ii 2D-DCT, the two-dimensional DCT transform of input I (N dimensions) is defined as:
D＝C N ·I·(C N ) T ，
wherein:
d is an output, and D is an output,
C N is the coefficient of the transformation matrix and,
where j=0, α j =1; when j > 0, alpha j ＝2；
The upper left corner of the spectrum image after DCT transformation contains image low-frequency information, and the lower right corner contains image high-frequency information.
The image data enhancement comprises the steps of horizontally overturning, vertically overturning and applying Mix Up and Random coating to an original image and a frequency spectrum image;
wherein Mix up is the virtual data of the linear interpolation of two random samples and the labels thereof in the training set, which is used as training data; random erase is a block of area in a Random erase image.
The re-integration of the image tag includes:
firstly, converting binary labels of images into multi-class labels according to fine-granularity label information; one type for real images and multiple types for counterfeit images, wherein specific counterfeit methods are considered, such as FaceSwap, face2Face, deepfake; furthermore, there is a hierarchy in the multi-class labels described above, because the entire dataset is divided into true and false classes;
meanwhile, various fake images belong to the whole fake category;
the data set labels are re-integrated into the hierarchical structure of the data set labels, so that the labels of coarse classification and fine classification levels of each image are realized, wherein the coarse classification labels are first-level labels, and the fine classification labels are second-level labels.
Then, after image data preparation, training a network model, and sending the original image and the spectrum image subjected to data enhancement processing into an image domain frequency domain double-flow network in pairs, and training by adopting a hierarchical supervision method; and calculating a loss function to obtain a gradient, and then performing back propagation to obtain a trained network model.
Network model training includes image data forward propagation and gradient backward propagation; specific:
the image data is transmitted forward, and all the original images subjected to data enhancement and the spectrum images subjected to data enhancement in the training set are sent into a network in pairs for training;
the network comprises a double-flow characteristic fusion module and a hierarchical supervision module;
the overall network structure consists of two branches, namely an image domain branch and a frequency domain branch, wherein the image domain branch is taken as a main stream, and the frequency domain branch is taken as an auxiliary information stream and comprises a convolution layer, a pooling layer, a full connection layer+relu and a full connection layer; carrying out concat fusion on the frequency domain feature map and the image domain feature map at a specific layer of the network; the reasonable fusion utilization of the information of the image domain and the frequency domain is realized; the concat is a simple and effective feature fusion method, and is a combination of features in the channel dimension. And finally, reasonably fusing and utilizing the information of the image domain and the frequency domain.
A hierarchical supervision module: modifying a network structure based on a hierarchical label, extracting image features based on a convolutional neural network in a hierarchical manner, adding a coarse classification head to a shallow part of the network to obtain coarse classification output, and adding a fine classification head to a deep part of the network to obtain fine classification output; specifically, HRNet is taken as an example. The original HRNet includes four phases, corresponding to two levels of labels, we select the outputs of the third and fourth phases to calculate the classification loss. In the coarse classification, we obtain a classification result by using the output of the third stage, and in the fine classification, we obtain a fine classification result by using the output of the fourth stage. Wherein both outputs are processed by a classification head, which is global averaging pooling, deriving feature vectors from the feature map.
Gradient back propagation: calculating a binary cross entropy loss by using the coarse classification output and the first-stage label, and calculating a multi-classification cross entropy loss by using the fine classification output and the second-stage label; and calculating the gradient according to the loss function, reversely propagating the gradient to update the network parameters, accelerating by utilizing the GPU, and stopping training until the error of the network is reduced to be within a set threshold value or the number of network iterations meets the requirement, so as to obtain a trained network model.
Here, by SGD optimizing the network, we set the reference learning rate to 0.002 and the momentum to 0.9 using the multi-step learning rate scheduler. Specifically, HRNet is taken as an example. We calculate the binary cross entropy loss using the output of the third stage and the first stage label (true or false), respectively, and in a fine classification we calculate the multi-class cross entropy loss using the output of the fourth stage and the output of the second stage label.
The loss function used for cross entropy loss is:
wherein:
n is the number of all pictures and,
h is the number of all of the levels,
α h is the weight of the h-th level,
CE is the cross entropy loss.
The gradient is a vector formed by summing partial derivatives of all variables of the loss function, and the direction indicated by the gradient is the direction in which the function value at each point is reduced most. And updating parameters in the HRNet by gradient back propagation until the error of the network is reduced to be within a set threshold value or the number of network iterations meets the requirement, and stopping training to obtain a trained HRNet network model.
For the model inference phase, specifically including:
firstly, the test data image is subjected to Discrete Cosine Transform (DCT) to obtain an image spectrogram,
and then, transmitting the test data image and the obtained corresponding image spectrogram into a trained model to obtain an inference result, obtaining a two-classification result and a multi-classification result of the test image, and carrying out weighted average fusion on the two results to obtain a final model inference result.
Specific examples based on the above principle are as follows:
as shown in fig. 1, a face forgery detection method based on an image domain frequency domain double-flow network is provided.
Using faceforensis++ data set, 1000 videos of each quality are selected, 720 videos are selected as training sets, 140 videos are selected as verification sets, 140 videos are selected as test sets, video frames are extracted, and face parts of people are intercepted as image data.
According to figure 1, a double-flow network for face counterfeiting detection is built, the frequency spectrum image obtained through DCT and the corresponding original image are sent into the double-flow network, the image domain feature image and the frequency domain feature image are fused at a specific layer of the network, and information of two domains of the image is fully utilized.
The label is re-integrated according to the figure 2, hierarchical supervision in the network is realized according to the form in the figure 3, cross entropy loss is calculated according to the two layers of labels and the two corresponding outputs, iterative updating of parameters is carried out according to a gradient back propagation method, and acceleration is carried out by utilizing the GPU until the error of the network is reduced to be within a set threshold value or the number of network iterations meets the requirement, and training is stopped.
The method is specifically realized by taking HRNet as an example, adding a coarse classification head after a stage three, adding a fine classification head after a stage four, and carrying out concat fusion on an image domain feature map and a frequency domain feature map on the stage three and the stage four respectively by feature fusion. The network is optimized by SGD. We set the baseline learning rate to 0.002 and the momentum to 0.9 using the multi-step learning rate scheduler.
By adopting the method for face detection, compared with the method for converting face counterfeiting detection into two classification problems, the method realizes hierarchical classification, utilizes the supervision information of the diversity of the counterfeiting images, and can achieve better detection precision; compared with a method for carrying out face counterfeiting detection by using only the fake image domain artifact, the method realizes that the double-flow network fuses the image frequency domain information and the image domain information in the network, and can effectively solve the problem that the fake image domain artifact with low definition is compressed and difficult to detect.
The above-described embodiments are intended to illustrate the present invention, not to limit the present invention, and any modifications and variations of the present invention fall within the spirit of the present invention and the scope of the appended claims.
Claims (5)
1. A face fake detection method based on an image domain frequency domain double-flow network is characterized by comprising the following steps of: the detection method comprises two stages of model training and model inference;
in the model training stage, a server with high computing performance is utilized to train a network model, network parameters are optimized by reducing a network loss function until the network converges, and a frequency domain double-flow network model based on an image domain is obtained;
in the model deducing stage, judging whether the new image is a face fake image or not by utilizing the network model obtained in the model training stage;
in the model training stage, the method specifically comprises the following steps:
firstly, preparing image data, namely performing discrete cosine transform on images in a training set to obtain frequency spectrum images of the images, performing image data enhancement processing on the images before and after the discrete cosine transform processing, and simultaneously, re-integrating corresponding image data labels to enable each image to have two-stage labels;
then, training a network model, namely sending the original image and the spectrum image subjected to data enhancement processing into an image domain frequency domain double-flow network in pairs, and training by adopting a hierarchical supervision method; calculating a loss function to obtain a gradient, and then performing back propagation to obtain a trained network model;
the re-integration of the image tag includes:
firstly, converting binary labels of images into multi-class labels according to fine-granularity label information; one for true images and multiple for counterfeit images, wherein specific counterfeiting methods are considered, including FaceSwap, face2Face, deepfake; furthermore, there is a hierarchy in the multi-class labels described above, because the entire dataset is divided into true and false classes;
meanwhile, various fake images belong to the whole fake category;
the data set labels are re-integrated into the hierarchical structure of the data set labels, so that the labels of coarse classification and fine classification levels of each image are realized, wherein the coarse classification labels are first-stage labels, and the fine classification labels are second-stage labels;
the network model training comprises image data forward propagation and gradient backward propagation; specific:
the image data is transmitted forward, and all the original images subjected to data enhancement and the spectrum images subjected to data enhancement in the training set are sent into a network in pairs for training;
the network comprises a double-flow characteristic fusion module and a hierarchical supervision module;
the overall network structure consists of two branches, namely an image domain branch and a frequency domain branch, wherein the image domain branch is taken as a main stream, and the frequency domain branch is taken as an auxiliary information stream and comprises a convolution layer, a pooling layer, a full connection layer+relu and a full connection layer; carrying out concat fusion on the frequency domain feature map and the image domain feature map at a specific layer of the network; the reasonable fusion utilization of the information of the image domain and the frequency domain is realized;
a hierarchical supervision module: modifying a network structure based on a hierarchical label, extracting image features based on a convolutional neural network in a hierarchical manner, adding a coarse classification head to a shallow part of the network to obtain coarse classification output, and adding a fine classification head to a deep part of the network to obtain fine classification output;
gradient back propagation: calculating a binary cross entropy loss by using the coarse classification output and the first-stage label, and calculating a multi-classification cross entropy loss by using the fine classification output and the second-stage label; and calculating the gradient according to the loss function, reversely propagating the gradient to update the network parameters, accelerating by utilizing the GPU, and stopping training until the error of the network is reduced to be within a set threshold value or the number of network iterations meets the requirement, so as to obtain a trained network model.
2. The face forgery detection method based on the image domain frequency domain double-flow network according to claim 1, characterized by comprising the following steps: regarding discrete cosine transforms, in particular using type 2D-DCT, the two-dimensional DCT transform of input I is defined as:
D＝C N ·I·(C N ) T ，
wherein:
d is an output, and D is an output,
C N is the coefficient of the transformation matrix and,
where j=0, α j ＝1；j>At 0, alpha j ＝2；
N represents the dimension of the input data;
the upper left corner of the spectrum image after DCT transformation contains image low-frequency information, and the lower right corner contains image high-frequency information.
3. The face forgery detection method based on the image domain frequency domain double-flow network according to claim 1, characterized by comprising the following steps: the image data enhancement comprises the steps of horizontally overturning, vertically overturning and applying Mix Up and Random coating to an original image and a frequency spectrum image;
wherein Mix up is the virtual data of the linear interpolation of two random samples and the labels thereof in the training set, which is used as training data; random erase is a block of area in a Random erase image.
4. The face forgery detection method based on the image domain frequency domain double-flow network according to claim 1, characterized by comprising the following steps: for the model inference phase, specifically including:
firstly, the test data image is subjected to discrete cosine transform to obtain an image spectrogram,
and then, transmitting the test data image and the obtained corresponding image spectrogram into a trained model to obtain an inference result, obtaining a two-classification result and a multi-classification result of the test image, and carrying out weighted average fusion on the two results to obtain a final model inference result.
5. The face forgery detection method based on the image domain frequency domain double-flow network according to claim 1, characterized by comprising the following steps: the loss function used for cross entropy loss is:
wherein:
n is the number of all pictures and,
h is the number of all of the levels,
α h is the weight of the h-th level,
CE is the cross entropy loss.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111009733.8A CN113723295B (en) | 2021-08-31 | 2021-08-31 | Face counterfeiting detection method based on image domain frequency domain double-flow network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111009733.8A CN113723295B (en) | 2021-08-31 | 2021-08-31 | Face counterfeiting detection method based on image domain frequency domain double-flow network |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113723295A CN113723295A (en) | 2021-11-30 |
| CN113723295B true CN113723295B (en) | 2023-11-07 |
Family
ID=78679589
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111009733.8A Active CN113723295B (en) | 2021-08-31 | 2021-08-31 | Face counterfeiting detection method based on image domain frequency domain double-flow network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113723295B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114444566B (en) * | 2021-12-16 | 2024-07-23 | 厦门市美亚柏科信息股份有限公司 | Image forgery detection method and device and computer storage medium |
| CN114596608B (en) * | 2022-01-19 | 2023-03-28 | 中国科学院自动化研究所 | Double-stream video face counterfeiting detection method and system based on multiple clues |
| CN114596609B (en) * | 2022-01-19 | 2023-05-09 | 中国科学院自动化研究所 | Audio-visual falsification detection method and device |
| CN114841340B (en) * | 2022-04-22 | 2023-07-28 | 马上消费金融股份有限公司 | Identification method and device for depth counterfeiting algorithm, electronic equipment and storage medium |
| CN115311525B (en) * | 2022-10-08 | 2023-03-14 | 阿里巴巴（中国）有限公司 | Depth forgery detection method and corresponding device |
| CN117557562B (en) * | 2024-01-11 | 2024-03-22 | 齐鲁工业大学(山东省科学院) | Image tampering detection method and system based on double-flow network |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105320947A (en) * | 2015-11-04 | 2016-02-10 | 博宏信息技术有限公司 | Face in-vivo detection method based on illumination component |
| CN109583342A (en) * | 2018-11-21 | 2019-04-05 | 重庆邮电大学 | Human face in-vivo detection method based on transfer learning |
| CN110210570A (en) * | 2019-06-10 | 2019-09-06 | 上海延华大数据科技有限公司 | The more classification methods of diabetic retinopathy image based on deep learning |
| CN110263910A (en) * | 2018-03-12 | 2019-09-20 | 罗伯特·博世有限公司 | For storing the method and apparatus for efficiently running neural network |
| CN111783505A (en) * | 2019-05-10 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Method and device for identifying forged faces and computer-readable storage medium |
| CN112686331A (en) * | 2021-01-11 | 2021-04-20 | 中国科学技术大学 | Forged image recognition model training method and forged image recognition method |
| CN112784790A (en) * | 2021-01-29 | 2021-05-11 | 厦门大学 | Generalization false face detection method based on meta-learning |
| CN113011357A (en) * | 2021-03-26 | 2021-06-22 | 西安电子科技大学 | Depth fake face video positioning method based on space-time fusion |
| CN113177965A (en) * | 2021-04-09 | 2021-07-27 | 上海工程技术大学 | Coal rock full-component extraction method based on improved U-net network and application thereof |
-
2021
- 2021-08-31 CN CN202111009733.8A patent/CN113723295B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105320947A (en) * | 2015-11-04 | 2016-02-10 | 博宏信息技术有限公司 | Face in-vivo detection method based on illumination component |
| CN110263910A (en) * | 2018-03-12 | 2019-09-20 | 罗伯特·博世有限公司 | For storing the method and apparatus for efficiently running neural network |
| CN109583342A (en) * | 2018-11-21 | 2019-04-05 | 重庆邮电大学 | Human face in-vivo detection method based on transfer learning |
| CN111783505A (en) * | 2019-05-10 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Method and device for identifying forged faces and computer-readable storage medium |
| CN110210570A (en) * | 2019-06-10 | 2019-09-06 | 上海延华大数据科技有限公司 | The more classification methods of diabetic retinopathy image based on deep learning |
| CN112686331A (en) * | 2021-01-11 | 2021-04-20 | 中国科学技术大学 | Forged image recognition model training method and forged image recognition method |
| CN112784790A (en) * | 2021-01-29 | 2021-05-11 | 厦门大学 | Generalization false face detection method based on meta-learning |
| CN113011357A (en) * | 2021-03-26 | 2021-06-22 | 西安电子科技大学 | Depth fake face video positioning method based on space-time fusion |
| CN113177965A (en) * | 2021-04-09 | 2021-07-27 | 上海工程技术大学 | Coal rock full-component extraction method based on improved U-net network and application thereof |
Non-Patent Citations (3)
| Title |
|---|
| Leveraging Frequency Analysis for Deep Fake Image Recognition;Joel Frank 等;《Proceedings of the 37 th International Conference on Machine Learning》;第1-12页 * |
| 基于Inception3D 网络的眼部与口部区域协同视频换脸伪造检测;韩语晨 等;《信号处理》;第37卷(第4期);第567-577页 * |
| 采用超复数小波生成对抗网络的活体人脸检测算法;李策 等;《西安交通大学学报》;第55卷(第5期);第113-122页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113723295A (en) | 2021-11-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113723295B (en) | Face counterfeiting detection method based on image domain frequency domain double-flow network | |
| Lee et al. | Detecting handcrafted facial image manipulations and GAN-generated facial images using Shallow-FakeFaceNet | |
| Guo et al. | Fake face detection via adaptive manipulation traces extraction network | |
| Chen et al. | A robust GAN-generated face detection method based on dual-color spaces and an improved Xception | |
| Chang et al. | Deepfake face image detection based on improved VGG convolutional neural network | |
| CN113313657B (en) | Unsupervised learning method and system for low-illumination image enhancement | |
| He et al. | Beyond the spectrum: Detecting deepfakes via re-synthesis | |
| CN113762138B (en) | Identification method, device, computer equipment and storage medium for fake face pictures | |
| Wang et al. | HidingGAN: High capacity information hiding with generative adversarial network | |
| Yu et al. | Detecting deepfake-forged contents with separable convolutional neural network and image segmentation | |
| Peng et al. | BDC-GAN: Bidirectional conversion between computer-generated and natural facial images for anti-forensics | |
| CN115131218A (en) | Image processing method, image processing device, computer readable medium and electronic equipment | |
| CN115035052B (en) | Fake face-changing image detection method and system based on identity difference quantification | |
| Guo et al. | Blind detection of glow-based facial forgery | |
| Wu et al. | Interactive two-stream network across modalities for deepfake detection | |
| CN118037641A (en) | Multi-scale image tampering detection and positioning method based on double-flow feature extraction | |
| El-Gayar et al. | A novel approach for detecting deep fake videos using graph neural network | |
| Hu et al. | Deep Learning‐Enabled Variational Optimization Method for Image Dehazing in Maritime Intelligent Transportation Systems | |
| Ren et al. | Student behavior detection based on YOLOv4-Bi | |
| Raza et al. | Holisticdfd: Infusing spatiotemporal transformer embeddings for deepfake detection | |
| CN111191549A (en) | Two-stage face anti-counterfeiting detection method | |
| Zhang et al. | Thinking in patch: Towards generalizable forgery detection with patch transformation | |
| Tariang et al. | Synthetic Image Verification in the Era of Generative Artificial Intelligence: What Works and What Isn’t There yet | |
| CN114743148A (en) | Multi-scale feature fusion tampering video detection method, system, medium, and device | |
| Singh et al. | Deepfake images, videos generation, and detection techniques using deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |