CN116524557A - Face counterfeiting detection model optimization method, device and system based on federal learning - Google Patents
Face counterfeiting detection model optimization method, device and system based on federal learning Download PDFInfo
- Publication number
- CN116524557A CN116524557A CN202310138757.6A CN202310138757A CN116524557A CN 116524557 A CN116524557 A CN 116524557A CN 202310138757 A CN202310138757 A CN 202310138757A CN 116524557 A CN116524557 A CN 116524557A
- Authority
- CN
- China
- Prior art keywords
- face
- basic parameter
- parameter set
- detection model
- clients
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 280
- 238000000034 method Methods 0.000 title claims abstract description 92
- 238000005457 optimization Methods 0.000 title claims abstract description 54
- 238000012549 training Methods 0.000 claims abstract description 155
- 230000008569 process Effects 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000004364 calculation method Methods 0.000 claims description 29
- 238000000605 extraction Methods 0.000 claims description 26
- 230000004927 fusion Effects 0.000 claims description 25
- 238000011156 evaluation Methods 0.000 claims description 13
- 238000011176 pooling Methods 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 9
- 238000005520 cutting process Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 2
- 230000006870 function Effects 0.000 description 28
- 230000004913 activation Effects 0.000 description 12
- 238000004590 computer program Methods 0.000 description 11
- 238000003860 storage Methods 0.000 description 8
- 238000005242 forging Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 5
- 238000012935 Averaging Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 230000003213 activating effect Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/40—Spoof detection, e.g. liveness detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Collating Specific Patterns (AREA)
Abstract
The invention provides a face counterfeiting detection model optimization method, device and system based on federal learning, and relates to the technical field of image processing, wherein the method comprises the following steps: updating the last updated basic parameter set according to the last iteration training basic parameter set of the plurality of clients, obtaining the last updated basic parameter set, and sending the last updated basic parameter set to the clients so that the clients optimize the face counterfeiting detection model according to the last updated basic parameter set and the last iteration training personalized layer parameter set on the face counterfeiting training data set in the local database, and taking the parameters of the basic network layer of the optimized face counterfeiting detection model as the current iteration training basic parameter set; and iteratively executing the training process, and if the preset termination condition is met, acquiring an optimal face counterfeiting detection model. The invention ensures the face information safety of each client and improves the generalization and detection performance of the face counterfeiting detection model.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a face counterfeiting detection model optimization method, device and system based on federal learning.
Background
With the rapid development of deep learning technology, the effect of the face deep forging technology is approaching to that of a real face. Deep forgery techniques based on the generation of a countermeasure network (Generative Adversarial Networks, GAN) can easily generate high quality forgery images that are difficult for the human eye to discern. The deep forging technology is a double-edged sword, which not only provides a new development space for the fields of artistic creation, intelligent medical treatment and the like, but also brings serious challenges for information security. The most common counterfeiting generation method, such as face exchange, can be used for spoofing face recognition systems to obtain unauthorized face verification, so that personal property security, information security and the like are threatened.
The detection and identification technology for deep face counterfeiting such as face exchange and replay is one of important research subjects in the field of digital image evidence obtaining and network security. The current mainstream depth falsification detection methods can be largely classified into methods based on image pixel level artifacts and methods based on specific cues. The method based on image pixel level artifacts regards false detection as a true-false classification problem, and trains a deep neural network to distinguish false marks of the face change. The method based on specific clues carries out various types of detection on the human face, including blink detection, head gesture, facial expression movement pattern and the like. However, the current counterfeiting detection method generally trains the model based on face data in a specific range, and although good detection accuracy can be obtained, the deep counterfeiting and the deep counterfeiting detection technology are in continuous evolution iteration and have a tendency of eliminating each other. Therefore, when the current model encounters new fake data, the effect of the model is often greatly reduced, and the generalization of the model is poor. One of the reasons for poor generalization ability of the model is that the current fake data set has small data size, uneven data quality and poor diversity. The generalization and robustness of the deep learning-based forgery detection model need a lot of quality data, but most teams or companies are not enough to support, except a few huge companies can meet the condition. In addition, the face images or videos in different databases relate to the portrait rights of the person, and the use of the different databases should be more careful than the use of other computer vision databases, and the information security should be taken into account, so that a sufficient counterfeit data set cannot be obtained across the databases.
Therefore, in the prior art, due to the limitations of information security and data quality, the quantity of the forged data sets is small and the quality is poor, so that the generalization capability of the forged detection model is poor.
Disclosure of Invention
The invention provides a face counterfeiting detection model optimization method, device and system based on federal learning, which are used for solving the defects of poor generalization capability of a counterfeiting detection model caused by small quantity and poor quality of counterfeiting data sets due to the limitation of information security and data quality in the prior art, and improving generalization of the face counterfeiting detection model while improving the information security.
The invention provides a face counterfeiting detection model optimization method based on federal learning, which comprises the following steps:
for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
transmitting the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
Iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
According to the face fake detection model optimization method based on federal learning, the updating of the last updated basic parameter set according to the basic parameter sets after a plurality of last iteration training comprises the following steps:
performing fusion calculation on the basic parameter set after the last iteration training in the plurality of clients;
and updating the basic parameter set updated last time according to the fusion calculation result.
According to the face counterfeiting detection model optimization method based on federal learning provided by the invention, the fusion calculation is carried out on the basic parameter set after the last iteration training in the plurality of clients, and the method comprises the following steps:
obtaining model evaluation indexes of face fake detection models and the sample number of the face fake training data set after the last iteration training of each client;
Determining a weight coefficient of the basic parameter set after the last iteration training of each client according to the model evaluation index and the sample number;
and according to the weight coefficient, carrying out fusion calculation on the plurality of basic parameter sets after the last iteration training.
According to the face counterfeiting detection model optimization method based on federal learning provided by the invention, the fusion calculation is carried out on the basic parameter set after the last iteration training in the plurality of clients, and the method comprises the following steps:
calculating a weighted average value of the plurality of basic parameter sets after the last iteration training;
and obtaining the fusion calculation result according to the weighted average calculation result.
According to the face counterfeiting detection model optimization method based on federal learning, the method further comprises the following steps:
under the condition that a face counterfeiting detection instruction is received, analyzing a face image to be detected and a client identification from the face counterfeiting detection instruction;
and carrying out face counterfeiting detection on the face image to be detected based on an optimal face counterfeiting detection model in the target client corresponding to the client identifier, so as to obtain a counterfeiting detection result of the face image to be detected.
According to the method for optimizing the face counterfeiting detection model based on federal learning, the face counterfeiting detection is performed on the face image to be detected based on the optimal face counterfeiting detection model in the target client corresponding to the client identifier, and the method comprises the following steps:
preprocessing the face image to be detected; the preprocessing comprises one or more of face recognition processing, alignment processing and cutting processing;
and carrying out face counterfeiting detection on the preprocessed face image to be detected based on the optimal face counterfeiting detection model in the target client.
According to the face fake detection model optimization method based on federal learning, the optimal face fake detection model is constructed and generated based on a basic network layer and a personalized network layer;
the base network layer is formed by stacking a first number of feature extraction modules;
the personalized network layer is formed by stacking a second number of feature extraction modules;
each feature extraction module includes one or more combinations of a convolutional layer, a pooling layer, and a residual layer.
The invention also provides a facial counterfeiting detection model optimizing device based on federal learning, which comprises:
The updating module is used for acquiring a basic parameter set after the last iteration training of the plurality of clients for the current iteration training, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iteration training to obtain the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
the sending module is used for sending the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
the optimization module is used for iteratively executing the iterative training process until the preset termination condition is met, and sending the basic parameter set updated last time to the plurality of clients so as to enable the plurality of clients to acquire an optimal face counterfeiting detection model according to the basic parameter set updated last time; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
The invention also provides a face counterfeiting detection model optimization system based on federal learning, which comprises the face counterfeiting detection model optimization device based on federal learning and a plurality of clients;
and the face fake detection model optimizing device is respectively connected with the plurality of clients in a communication way.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the face fake detection model optimization method based on federal learning when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a federal learning-based face-forgery-detection-model optimization method as described in any one of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a federal learning-based face falsification detection model optimization method as described in any one of the above.
According to the face fake detection model optimization method, device and system based on federal learning, the face fake detection models of all clients are jointly trained on the basis of guaranteeing data privacy through personalized federal learning, so that all clients can directly acquire shared network layer parameters trained by other clients through a rich and high-quality face fake training data set, and the shared network layer parameters trained by other clients and the shared network layer parameters trained by the clients and personalized network layer parameters are combined to train the face fake detection models, the trained face fake detection models can be suitable for face image detection in various databases, and the generalization, robustness and accuracy of the face fake detection models in all clients are improved; in the whole optimization process, the face counterfeiting training data set exchange is not needed between the clients, the face counterfeiting training samples in other clients cannot be snooped, and the face information security of the clients can be effectively ensured.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a face counterfeiting detection model optimization method based on federal learning;
FIG. 2 is a second flow chart of the face fake detection model optimization method based on federal learning;
FIG. 3 is a schematic diagram of a face counterfeiting detection model in the federal learning-based face counterfeiting detection model optimization method provided by the invention;
FIG. 4 is a schematic structural diagram of the face counterfeiting detection model optimizing device based on federal learning;
FIG. 5 is a schematic structural diagram of the federal learning-based face counterfeit detection model optimization system provided by the invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that the terms "first," "second," and the like herein are used for distinguishing between similar objects and not for describing a particular sequential or chronological order.
The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus/apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus/apparatus.
The main implementation body of the method is a face fake detection model optimizing device based on federal learning, and the device can be electronic equipment, a component in the electronic equipment, an integrated circuit or a chip. The electronic device may be a mobile electronic device or a non-mobile electronic device. The mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, etc., and the non-mobile electronic device may be a server, a personal computer, etc., which is not particularly limited in the present invention. The following describes a face forgery detection model optimization method based on federal learning in this embodiment, taking an execution subject as a server as an example.
The human face forgery detection model optimization method based on federal learning of the present invention is described below with reference to fig. 1 to 3.
As shown in fig. 1, a flow chart of the face fake detection model optimization method based on federal learning provided in this embodiment includes the specific steps of:
step 101, for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
102, sending the basic parameter set updated at the present time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
Step 103, iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face fake detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
The plurality of clients are clients participating in personalized federal learning training of the face fake detection model; the number of the clients is N, and the clients can be specifically set according to actual requirements.
The face fake detection model in each client comprises a basic network layer and a personalized network layer; the base network layer is a network layer shared by face fake detection models in a plurality of clients, and the personalized network layer is a network layer not shared by the face fake detection models in the plurality of clients.
The basic parameter set after the last iteration training is the basic parameter set obtained after the local face fake detection model is trained in the last iteration training process of each client.
The last updated basic parameter set is a basic parameter set obtained by updating the local basic parameter set according to the basic data sets transmitted by a plurality of clients in the last iterative training process of the server.
The face fake detection model is used for detecting whether the face image is a fake face or a real face.
Optionally, when the current iterative training is the initial iterative training, the server may initialize a basic parameter set inside the face fake detection model optimizing device, and use the initialized basic parameter set as the basic parameter set updated at the current time.
And when the current iterative training is the intermediate iterative training, the server can monitor the basic parameter set after the last iterative training sent by the N clients in real time, and after receiving the basic parameter set after the last iterative training sent by the N clients, update the basic parameter set after the last update in the server according to the basic parameter set after the N last iterative training to obtain the basic parameter set after the current update in the server.
The updating manner of the last updated basic parameter set includes, but is not limited to: the basic parameter set after each previous iteration training is sent to N clients, the N clients are received to verify the face fake detection model corresponding to the basic parameter set after each previous iteration training, the obtained total model evaluation index is summarized, and the basic parameter set with the optimal total model evaluation index is updated to the basic parameter set after the previous updating; or, carrying out fusion calculation on the N basic parameter sets after the last iteration training, and updating the basic parameter set after the last updating according to the fusion calculation result.
Optionally, after the basic parameter set updated at the current time is obtained, the basic parameter set updated at the current time is respectively sent to N clients; after receiving the basic parameter set updated at the present time, each client replaces the basic network layer parameters of the face fake detection model after the previous iteration training with the basic parameter set updated at the present time, optimizes the replaced face fake detection model according to the face fake training data set in the local database so as to optimize the parameters of the basic network layer and the parameters of the personalized network layer of the face fake detection model, takes the optimized face fake detection model as the face fake detection model after the present iteration training, and sends the basic parameter set in the face fake detection model after the present iteration training to the face fake detection model optimizing device so as to iteratively execute the next iteration training.
The face forging training data set may be a training data set formed by preprocessing a sample face image, so that the image data of N different face forging training data sets are preprocessed, such as face detection, alignment, cutting, etc., so that the data obtained by different databases are respectively used as the face forging training data sets of different clients; the size of the preprocessed image may be set according to practical requirements, for example, the size of the image is 224×224.
In some preferred embodiments, the face-forgery training data set in the client may be determined by the best accuracy obtained after combining.
And iteratively executing the iterative training process until a preset termination condition is met.
The preset termination condition includes receiving termination training requests sent by N clients or determining that face fake detection models in the N clients are all converged, or that the number of iterative training reaches a maximum preset training number r, which is not specifically limited in this embodiment.
Optionally, if the preset termination condition is met, the base parameter set updated last time is sent to the N clients. Under the condition that each client receives the basic parameter set updated last time, the face fake detection model can be updated directly according to the basic parameter set updated last time so as to obtain an optimal face fake detection model; the face fake detection model can be updated according to the basic parameter set updated last time, and then optimized again based on the face fake training data set in the local database, so that the optimal face fake detection model can be obtained.
The optimal face fake detection model is a personalized face fake detection model aiming at a plurality of different databases in each client, and can accurately identify face images in various different databases.
As shown in fig. 2, a second flow chart of the face counterfeit detection model optimization method based on federal learning specifically includes:
step 201, initializing a basic parameter set W in a server 0 To base parameter set W 0 Initial parameters of a basic network layer which are sent to N clients and serve as a face fake detection model of the client;
202, preprocessing such as face recognition, alignment, cutting and the like is carried out on image data in a local database of N different clients to form face forging training data sets of N different clients;
step 203, for each client, execute the followingThe steps are as follows: the current client is the i client, i is {1,2, …, N }, with positive integer K B Representing the number of parameters in the base network layer by a positive integer K P Representing the number of parameters in the personalized network layer, the weight matrix of the basic network layer is represented asThe vector value activation function corresponding to the weight matrix of the base network layer is +.>The weight matrix of the personalized network layer is denoted +.>The client i can perform optimization training on the loss function of the face fake detection model constructed by the minimized cross entropy loss function by using an optimization algorithm, such as random gradient descent (Stochastic Gradient Descent, SGD), so as to perform one round of training on the face fake detection model, and obtain optimized base layer weight- >And individualization layer weight->The training process of the face counterfeiting detection model is as follows:
the vector value activation function of the weight matrix of the base network layer isThe vector value activation function of the weight matrix of the personalized network layer is +.>The domain dimension and range of the vector value activation function are implicitly unique, thereby satisfying the forward transfer or inference operation, and for the input data point x of the current client i, the output expression of the face falsification detection model in the current client i is:
wherein, the liquid crystal display device comprises a liquid crystal display device,a face counterfeiting detection result output by the face counterfeiting detection model of any client;
the above can be expressed simply as:
using p j Label, x representing jth sample image of any client j Representing the jth sample image on any client, useThe loss function of the face fake detection model of any client is represented, specifically, the learning target is set to be minimized as a two-class cross entropy loss function:
thus, for the ith client, the empirical risk is used as the loss function formed by the corresponding bi-classification cross entropy loss function, expressed in particular as:
wherein n is i Is the number of training samples in the face fake training dataset of the ith client, (x) i,j ,y i,j ) Is the j sample image and label, j e {1, …, n i }。
Minimizing empirical risk functions using random gradient descent (SGD):
step 204, the client weighting the base layerThe basic parameter set is transmitted to the face fake detection model optimizing device side, and the face fake detection model optimizing device receives the basic parameter set which is transmitted by N clients and is subjected to iterative training>i epsilon {1,2, …, N }, calculating according to the basic parameter sets after a plurality of iterative training to obtain a new basic parameter set W 1 The new basic parameter set is sent to N clients again to carry out iterative training of the next round;
step 205, obtaining an optimal solution after the iterative training r rounds to obtain the final iterative training of the client iAnd->Forming an optimal face fake detection model; if the client i is required to detect the face image to be detected, inputting the face image to be detected into an optimal face counterfeiting detection model in the client i, and performing face counterfeiting detection on the face image to be detected to obtain a counterfeiting detection result of the face image to be detected.
In the embodiment, through personalized federal learning, on the basis of ensuring data privacy, the face fake detection model of each client is jointly trained, so that each client can directly acquire shared network layer parameters trained by other clients through a rich and high-quality face fake training data set, and the shared network layer parameters trained by other clients and the shared network layer parameters trained by the client and personalized network layer parameters are jointly trained on the face fake detection model, so that the trained face fake detection model can be suitable for face image detection in various databases, and the generalization, robustness and accuracy of the face fake detection model in each client are improved; in the whole optimization process, the face counterfeiting training data set exchange is not needed between the clients, the face counterfeiting training samples in other clients cannot be snooped, and the face information security of the clients can be effectively ensured.
In some embodiments, the updating the last updated basic parameter set according to the plurality of last iteration trained basic parameter sets in step 101 includes:
performing fusion calculation on the basic parameter set after the last iteration training in the plurality of clients;
and updating the basic parameter set updated last time according to the fusion calculation result.
Optionally, in order to make the face fake detection model of each client have good generalization energy, improve the face fake detection effect of the face fake detection model on a cross-data set or an unknown data set, in the current iterative training process, if the basic parameter set after the last iterative training in the plurality of clients is obtained, fusion calculation is performed on the basic parameter set after the last iterative training in the plurality of clients, so as to update the basic parameter set after the last update according to the fusion calculation result.
The manner of fusion calculation includes, but is not limited to, weighted addition or addition averaging, which is not specifically limited in this embodiment.
In some embodiments, the performing a fusion calculation on the base parameter set after the training of the last iteration in the plurality of clients in step 101 includes:
Obtaining model evaluation indexes of face fake detection models and the sample number of the face fake training data set after the last iteration training of each client;
determining a weight coefficient of the basic parameter set after the last iteration training of each client according to the model evaluation index and the sample number;
and according to the weight coefficient, carrying out fusion calculation on the plurality of basic parameter sets after the last iteration training.
The model evaluation index of the face counterfeiting detection model after the last iteration training is index data generated by performing model performance evaluation on the face counterfeiting detection model after the last iteration training by adopting a face counterfeiting verification data set for each client, including but not limited to one or more of accuracy, precision and recall rate.
Optionally, the step of performing fusion calculation on the plurality of basic parameter sets after the last iteration training specifically includes:
firstly, obtaining model evaluation indexes of face fake detection models in each client after the last iteration training and the number of samples of a face fake training data set participating in the face fake detection models;
and then, determining the weight coefficient of the basic parameter set after the last iteration training of each client according to the model evaluation index of the face counterfeiting detection model after the last iteration training in each client and the sample number of the face counterfeiting training data set.
Based on the weight coefficient, carrying out weighted addition on a plurality of basic parameter sets after the last iteration training, then carrying out averaging or direct weighted addition, and updating the basic parameter set after the last updating according to a weighted average result or a weighted addition result;
in this embodiment, in each iterative training process, the model evaluation index of the face counterfeiting detection model of each client and the number of samples of the face counterfeiting training data set are fully considered, so that the basic parameter set inside the face counterfeiting detection model optimizing device is dynamically updated and then sent to each client, so that each client trains the face counterfeiting detection model of the local end on the basis, the trained face counterfeiting detection model can be suitable for face image detection in various databases, and generalization, robustness and accuracy of the face counterfeiting detection model in each client are improved.
In some embodiments, the performing a fusion calculation on the base parameter set after the training of the last iteration in the plurality of clients in step 101 includes:
calculating a weighted average value of the plurality of basic parameter sets after the last iteration training;
And obtaining the fusion calculation result according to the weighted average calculation result.
Optionally, the step of performing fusion calculation on the plurality of basic parameter sets after the last iteration training further comprises:
under the condition that the basic parameter set after the last iterative training sent by a plurality of clients is obtained, the weighted average value of the basic parameter sets after the last iterative training is directly calculated; and updating the basic parameter set updated last time according to the weighted average value calculation result.
In this embodiment, in each iterative training process, the basic parameter set after the last iterative training of each client is combined, and the basic parameter set inside the face counterfeiting detection model optimizing device is dynamically updated and then issued to each client, so that each client trains the face counterfeiting detection model of the client on the basis, so that the trained face counterfeiting detection model can be suitable for face image detection in various databases, and generalization, robustness and accuracy of the face counterfeiting detection model in each client are improved.
In some embodiments, the method further comprises:
under the condition that a face counterfeiting detection instruction is received, analyzing a face image to be detected and a client identification from the face counterfeiting detection instruction;
And carrying out face counterfeiting detection on the face image to be detected based on an optimal face counterfeiting detection model in the target client corresponding to the client identifier, so as to obtain a counterfeiting detection result of the face image to be detected.
The human face fake detection instruction is an instruction for fake detection of a human face image to be detected; the face counterfeiting detection instruction at least comprises a face image to be detected and a client identifier for face counterfeiting detection, namely a client identifier.
Optionally, under the condition that a face counterfeiting detection instruction is received, analyzing the face counterfeiting detection instruction to obtain a face image to be detected and a client identifier;
inputting the face image to be detected, and performing face counterfeiting detection on the face image to be detected in an optimal face counterfeiting detection model in the target client corresponding to the client identifier so as to obtain a counterfeiting detection result of the face image to be detected.
In this embodiment, under the condition of receiving the face fake detection instruction, the optimal face fake detection model in the corresponding target client can be called in real time, so that not only can the diversified detection requirements of users be met, but also the detection accuracy can be improved.
In some embodiments, the performing face-forgery detection on the face image to be detected based on the optimal face-forgery detection model in the target client corresponding to the client identifier includes:
preprocessing the face image to be detected; the preprocessing comprises one or more of face recognition processing, alignment processing and cutting processing;
and carrying out face counterfeiting detection on the preprocessed face image to be detected based on the optimal face counterfeiting detection model in the target client.
Optionally, before the face image to be detected is subjected to face counterfeiting detection, the face image to be detected is further required to be preprocessed, so that interference caused by the size, the position and the like of the face image to be detected is avoided, and accuracy and effectiveness of face counterfeiting detection are ensured.
Optionally, face recognition processing is performed on the face image to be detected to ensure that the face image to be detected contains a face, alignment processing and cutting processing are performed on the face image to be detected to ensure that each face image to be detected is registered with the image template, and cutting processing is performed on the face image to be detected to cut each face image to be detected to a target size, such as 224×224.
In some embodiments, the optimal face-forgery detection model is generated based on a basic network layer and a personalized network layer construction;
the base network layer is formed by stacking a first number of feature extraction modules;
the personalized network layer is formed by stacking a second number of feature extraction modules;
each feature extraction module includes one or more combinations of a convolutional layer, a pooling layer, and a residual layer.
The face fake detection models in the clients share the same basic network layer and respectively have different personalized network layers, and the basic network layer and the personalized network layers jointly construct and generate a deep feed-forward neural network model of each client, namely the face fake detection model.
The first number and the second number may be set according to actual requirements, for example, the first number is 11, and the second data amount is 3.
Alternatively, the face falsification detection model may be constructed and generated based on a deep convolution Xception network framework as a basic framework.
The number of layers of the basic network layer and the personalized network layer of the face counterfeiting detection model and the number of convolution layers can be set according to actual requirements, for example, based on 36 convolution layers, a feature extraction basis of the face counterfeiting detection model is formed. In the pseudo-discrimination domain, the convolution basis follows a logistic regression layer. The full connection layer is inserted before the logistic regression layer.
The specific structure of the face-forgery-detection model will be specifically described below using 36 convolution layers as an example.
As shown in fig. 3, the face counterfeit detection model includes three data stream layers, namely an input data stream layer (i.e. Entry flow), an intermediate data stream layer (i.e. Middle flow), and an Exit data stream layer (i.e. Exit flow); the 36 convolutional layers are divided into 14 feature extraction modules, all of which have residual connections except for the first and last feature extraction. The Entry flow comprises 4 feature extraction modules, wherein the first layer of feature extraction modules is formed by constructing a convolutional network Conv with a convolution kernel of 32 multiplied by 3, a step size stride of 2 multiplied by 2 and an activation function of RELU function, and a convolutional network Conv with a convolution kernel of 64 multiplied by 3 and an activation function of RELU function, which are connected in sequence; the second layer feature extraction module is based on a residual network constructed by a convolutional network Conv with a convolutional kernel of 1 multiplied by 1 and a step size stride of 2 multiplied by 2, a separable convolutional network Separabloonv with a convolutional kernel of 128 multiplied by 3, an activation function of RELU function, a separable convolutional network Separabloonv with a convolutional kernel of 128 multiplied by 3, and a maximum pooling network MaxPooling with a pooling scale of 3 multiplied by 3 and a step size stride of 2 multiplied by 2; the third layer of feature extraction module is based on a residual network constructed by a convolution network Conv with a convolution kernel of 1 multiplied by 1 and a step size stride of 2 multiplied by 2, and two groups of separable convolution networks Separablonv with a convolution kernel of 256 multiplied by 3 and a maximum pooling network MaxPooling with a step size stride of 2 multiplied by 2, wherein an activation function connected in sequence is a RELU function; the fourth layer feature extraction module is based on a residual network constructed by a convolution network Conv with a convolution kernel of 1×1 and a step size stride of 2×2, and two groups of separable convolution networks SeparableConv with a convolution kernel of 728×3×3 and a maximum pooling network MaxPooling with a step size stride of 2×2, and sequentially connected activation functions as RELU functions.
The Middle flow comprises 8 feature extraction modules, wherein each feature extraction module is constructed and generated based on a linear residual network and three groups of separable convolution networks sepalablecon with a convolution kernel of 728×3×3, wherein an activation function connected in sequence is a RELU function.
The Exit flow comprises 2 feature extraction modules, wherein the first layer feature extraction module is formed by constructing a residual network based on a convolutional network Conv with a convolutional kernel of 1 multiplied by 1 and a step size stride of 2 multiplied by 2, and a separable convolutional network SeparableConv with a convolutional kernel of 728 multiplied by 3 and a step size stride of 3 multiplied by 3, an separable convolutional network SeparableConv with an activating function of RELU, a separable convolutional network SeparableConv with a convolutional kernel of 1024 multiplied by 3, and a max pooling network MaxPooling with a pooling scale of 3 multiplied by 3 and a step size stride of 2 multiplied by 2; the second-layer feature extraction module is constructed and generated based on a separable convolutional network sepalablecon with a convolution kernel of 1536×3×3 and an output activation function of RELU function, a separable convolutional network sepalablecon with a convolution kernel of 2048×3×3 and an output activation function of RELU function, and a global average pooling network globalaeragepooling.
It should be noted that the first 11 feature extraction modules may be used as a base network layer, and the last 3 feature extraction modules may be used as the personalized network layers.
In the embodiment, the feature extraction module is constructed based on one or more combinations of the convolution layer, the pooling layer and the residual layer, and then the face counterfeiting detection model is formed by stacking a plurality of groups of feature extraction modules, and the federal learning training is performed, so that each client side ensures that each client side has an optimal basic data set of all the clients integrated by the face counterfeiting detection model optimizing device after the training is finished on the premise that the face data and the features of other client sides are not acquired, and the optimal parameters of the personalized network layer at the local side can be obtained, so that the face counterfeiting detection model can extract richer and deep features and is applicable to various face image counterfeiting detection scenes, and the accuracy and the applicability of the face counterfeiting detection result are effectively improved.
In order to verify the effective direction of the face counterfeiting detection model optimization method based on personalized federal learning, the face counterfeiting detection model in the embodiment can be tested in a database faceforensics++, and the test result is shown in the first column of table 1. And the face fake detection model in the embodiment is tested on the database DFDC and Celeb-DF, and the test results are shown in the second and third columns of the table 1. And comparing the face fake detection model constructed on the basis of the Xreception model with the face fake detection model optimization method in the embodiment at the existing single client side. As can be seen from table 1, the face fake detection model optimization method in this embodiment has higher accuracy and better generalization capability for face fake detection.
Table 1 human face forgery detection model optimization method comparison results
| FaceForensics++ | DFDC | Celeb-DF | |
| Xception | 95.73 | 70.90 | 73.7 |
| This embodiment | 97.42 | 86.09 | 81.92 |
In summary, the face counterfeiting detection model optimization method based on personalized federal learning provided by the embodiment aims at protecting data privacy and safety, realizes common modeling of various databases on the basis of legal compliance, and improves generalization and robustness of the face counterfeiting detection model. The method specifically comprises the following steps: performing face detection, alignment, cutting and other processing on the database data; initializing a basic parameter set, sending the initialized basic parameter set to a plurality of clients, receiving the basic parameter set obtained by training sent by the clients after the clients train for one round, and updating the basic parameter set; each client receives the initialized basic parameter set sent by the face fake detection model optimizing device, inputs the preprocessed data for training, and sends the trained basic parameter set to the face fake detection model optimizing device after training. After the face counterfeiting detection model at each client is trained, the trained face counterfeiting detection model is used for carrying out counterfeiting detection on the face image to be detected. According to the embodiment, the fake sample can be detected from the real face picture, and meanwhile, the anti-interference capability of the face fake detection model on various post-processing operations and the generalization capability of a fake method which is not homologous with training data are effectively improved. The method has the following advantages:
The method has the advantages that the cross-database access of the face data is avoided, and the data privacy security is protected.
The personalized layer is introduced, so that adverse effects of statistical heterogeneity among different face fake databases can be eliminated, and accuracy of fake detection models can be improved.
And thirdly, generalization of the face counterfeiting detection model is improved, and detection effects on cross-data sets or unknown data sets are effectively improved.
Fourth, each client can join in new face fake training data set at any time to perform sustainable learning, and training of other existing models is not affected.
The face counterfeiting detection model optimizing device based on personalized federal learning provided by the invention is described below, and the face counterfeiting detection model optimizing device based on personalized federal learning described below and the face counterfeiting detection model optimizing method based on personalized federal learning described above can be correspondingly referred to each other.
As shown in fig. 4, the device for optimizing a face fake detection model based on federal learning provided in this embodiment includes an updating module 401, a sending module 402, and an optimizing module 403, where:
the updating module 401 is configured to obtain, for a current iteration training, a base parameter set after a last iteration training of a plurality of clients, and update, according to the plurality of base parameter sets after the last iteration training, the base parameter set after the last update to obtain a base parameter set after the current update; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
The sending module 402 is configured to send the updated basic parameter set to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
the optimization module 403 is configured to iterate the iterative training process until a preset termination condition is met, send a last updated basic parameter set to the plurality of clients, so that the plurality of clients obtain an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
In the embodiment, through personalized federal learning, on the basis of ensuring data privacy, the face counterfeiting detection model of each client is jointly trained, so that each client can directly acquire shared network layer parameters trained by other clients through a rich and high-quality face counterfeiting training data set, and the face counterfeiting detection model is trained by combining the shared network layer parameters trained by other clients and the shared network layer parameters trained by the client, so that the trained face counterfeiting detection model can be suitable for face image detection in various databases, and the generalization, robustness and accuracy of the face counterfeiting detection model in each client are improved; in the whole optimization process, the face counterfeiting training data set exchange is not needed between the clients, the face counterfeiting training samples in other clients cannot be snooped, and the face information security of the clients can be effectively ensured.
As shown in fig. 5, the present embodiment further provides a face counterfeit detection model optimization system based on federal learning, which includes a face counterfeit detection model optimization device 501 based on federal learning and a plurality of clients 502; the face fake detection model optimization device 501 is in communication connection with a plurality of clients 502 and is used for realizing a face fake detection model optimization method based on federal learning.
The face fake detection model optimizing device 501 may be configured and generated based on a server, and is configured to initialize a basic parameter set, send the initialized basic parameter set to the plurality of clients 502, receive the optimized basic parameter set sent by the plurality of clients 502 after the clients 502 train one round, and update the basic parameter set; the client 502 is configured to receive the initialized basic parameter set sent by the face fake detection model optimization device 501, input the preprocessed face fake training data for training, and send the optimized basic parameter set to the face fake detection model optimization device 501 after training. After training, the trained face fake detection model is used for fake detection, so that the accuracy of the data set detection is obtained.
The face fake detection model optimization device 501 includes a base network layer, and the client 502 includes a base network layer and a personalized network layer, where the base network layer and the personalized network layer are connected.
Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 601, communication interface (Communications Interface) 602, memory 603 and communication bus 604, wherein processor 601, communication interface 602, memory 603 complete the communication between each other through communication bus 604. The processor 601 may invoke logic instructions in the memory 603 to perform a face-forgery-detection-model optimization method based on personalized federal learning, the method comprising: for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients; transmitting the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present; iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
Further, the logic instructions in the memory 603 described above may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, where the computer program, when executed by a processor, can perform the face falsification detection model optimization method based on personalized federal learning provided by the above methods, where the method includes: for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients; transmitting the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present; iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the personalized federal learning based face forgery detection model optimization method provided by the above methods, the method comprising: for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients; transmitting the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present; iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the storage device and the processing device described above and the related description may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
Computer program code for carrying out operations of the present application may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the c-language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of remote computers, the remote computer may be connected to the user computer through any kind of network, including a local area network or a wide area network, or may be connected to an external computer (e.g., connected through the internet using an internet service provider).
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (9)
1. The human face counterfeiting detection model optimization method based on federal learning is characterized by comprising the following steps of:
for the current iterative training, acquiring a basic parameter set after the last iterative training of a plurality of clients, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iterative training to acquire the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
transmitting the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
Iteratively executing the iterative training process until a preset termination condition is met, and sending the last updated basic parameter set to the plurality of clients for the plurality of clients to acquire an optimal face counterfeiting detection model according to the last updated basic parameter set; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
2. The federal learning-based face forgery detection model optimization method according to claim 1, wherein updating the last updated basic parameter set according to a plurality of last iteration trained basic parameter sets comprises:
performing fusion calculation on the basic parameter set after the last iteration training in the plurality of clients;
and updating the basic parameter set updated last time according to the fusion calculation result.
3. The federal learning-based face forgery detection model optimization method according to claim 2, wherein the performing fusion calculation on the basic parameter set after the last iteration training in the plurality of clients includes:
Obtaining model evaluation indexes of face fake detection models and the sample number of the face fake training data set after the last iteration training of each client;
determining a weight coefficient of the basic parameter set after the last iteration training of each client according to the model evaluation index and the sample number;
and according to the weight coefficient, carrying out fusion calculation on the plurality of basic parameter sets after the last iteration training.
4. The federal learning-based face forgery detection model optimization method according to claim 2, wherein the performing fusion calculation on the basic parameter set after the last iteration training in the plurality of clients includes:
calculating a weighted average value of the plurality of basic parameter sets after the last iteration training;
and obtaining the fusion calculation result according to the weighted average calculation result.
5. A federally learned face falsification detection model optimization method according to any one of claims 1-4, wherein the method further comprises:
under the condition that a face counterfeiting detection instruction is received, analyzing a face image to be detected and a client identification from the face counterfeiting detection instruction;
And carrying out face counterfeiting detection on the face image to be detected based on an optimal face counterfeiting detection model in the target client corresponding to the client identifier, so as to obtain a counterfeiting detection result of the face image to be detected.
6. The federal learning-based face-forgery-detection model optimization method according to claim 5, wherein the performing face-forgery detection on the face image to be detected based on the optimal face-forgery-detection model in the target client corresponding to the client identifier includes:
preprocessing the face image to be detected; the preprocessing comprises one or more of face recognition processing, alignment processing and cutting processing;
and carrying out face counterfeiting detection on the preprocessed face image to be detected based on the optimal face counterfeiting detection model in the target client.
7. The face fake detection model optimizing method based on federal learning according to any one of claims 1 to 4, wherein the optimal face fake detection model is generated based on a basic network layer and a personalized network layer construction;
the base network layer is formed by stacking a first number of feature extraction modules;
The personalized network layer is formed by stacking a second number of feature extraction modules;
each feature extraction module includes one or more combinations of a convolutional layer, a pooling layer, and a residual layer.
8. Face fake detection model optimizing device based on federal study, characterized by comprising:
the updating module is used for acquiring a basic parameter set after the last iteration training of the plurality of clients for the current iteration training, and updating the basic parameter set after the last updating according to the basic parameter sets after the last iteration training to obtain the basic parameter set after the current updating; the basic parameter set comprises parameters of a basic network layer shared by face fake detection models in the plurality of clients;
the sending module is used for sending the basic parameter set updated at the current time to the plurality of clients; each client is used for optimizing the face fake detection model in each client on the face fake training data set in the local database of each client according to the basic parameter set updated at present and the personalized layer parameter trained at last iteration, and taking the parameters of the basic network layer of the optimized face fake detection model as the basic parameter set trained at present;
The optimization module is used for iteratively executing the iterative training process until the preset termination condition is met, and sending the basic parameter set updated last time to the plurality of clients so as to enable the plurality of clients to acquire an optimal face counterfeiting detection model according to the basic parameter set updated last time; the optimal face counterfeiting detection model is used for outputting a face counterfeiting detection result of the face image to be detected according to the face image to be detected.
9. A face counterfeiting detection model optimizing system based on federal learning, which is characterized by comprising the face counterfeiting detection model optimizing device based on federal learning and a plurality of clients according to claim 8;
and the face fake detection model optimizing device is respectively connected with the plurality of clients in a communication way.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310138757.6A CN116524557B (en) | 2023-02-10 | 2023-02-10 | Face counterfeiting detection model optimization method, device and system based on federal learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310138757.6A CN116524557B (en) | 2023-02-10 | 2023-02-10 | Face counterfeiting detection model optimization method, device and system based on federal learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116524557A true CN116524557A (en) | 2023-08-01 |
| CN116524557B CN116524557B (en) | 2024-06-14 |
Family
ID=87391015
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310138757.6A Active CN116524557B (en) | 2023-02-10 | 2023-02-10 | Face counterfeiting detection model optimization method, device and system based on federal learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116524557B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170124464A1 (en) * | 2015-10-28 | 2017-05-04 | Fractal Industries, Inc. | Rapid predictive analysis of very large data sets using the distributed computational graph |
| CN111461089A (en) * | 2020-06-17 | 2020-07-28 | 腾讯科技（深圳）有限公司 | Face detection method, and training method and device of face detection model |
| CN111666987A (en) * | 2020-05-22 | 2020-09-15 | 中国电子科技集团公司电子科学研究院 | Cross-domain data safety interconnection method and system based on federal learning |
| CN113642664A (en) * | 2021-08-24 | 2021-11-12 | 安徽大学 | Privacy protection image classification method based on federal learning |
| US20220164643A1 (en) * | 2019-08-26 | 2022-05-26 | Chenope, Inc. | System to detect, assess and counter disinformation |
| CN114639174A (en) * | 2021-12-10 | 2022-06-17 | 大连理工大学 | Privacy type deep forgery detection method under federal cooperation |
| CN115546864A (en) * | 2022-09-20 | 2022-12-30 | 广州大学 | Double-branch face counterfeiting detection method based on self-attention mechanism |
-
2023
- 2023-02-10 CN CN202310138757.6A patent/CN116524557B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170124464A1 (en) * | 2015-10-28 | 2017-05-04 | Fractal Industries, Inc. | Rapid predictive analysis of very large data sets using the distributed computational graph |
| US20220164643A1 (en) * | 2019-08-26 | 2022-05-26 | Chenope, Inc. | System to detect, assess and counter disinformation |
| CN111666987A (en) * | 2020-05-22 | 2020-09-15 | 中国电子科技集团公司电子科学研究院 | Cross-domain data safety interconnection method and system based on federal learning |
| CN111461089A (en) * | 2020-06-17 | 2020-07-28 | 腾讯科技（深圳）有限公司 | Face detection method, and training method and device of face detection model |
| CN113642664A (en) * | 2021-08-24 | 2021-11-12 | 安徽大学 | Privacy protection image classification method based on federal learning |
| CN114639174A (en) * | 2021-12-10 | 2022-06-17 | 大连理工大学 | Privacy type deep forgery detection method under federal cooperation |
| CN115546864A (en) * | 2022-09-20 | 2022-12-30 | 广州大学 | Double-branch face counterfeiting detection method based on self-attention mechanism |
Non-Patent Citations (2)
| Title |
|---|
| YUTING XU.ET.: "Visual-Semantic Transformer for Face Forgery Detection", 《2021 IEEE INTERNATIONAL JOINT CONFERENCE ON BIOMETRICS (IJCB)》, 31 December 2021 (2021-12-31), pages 1 - 7 * |
| 暴雨轩等: "深度伪造视频检测技术综述", 《计算机科学》, vol. 47, no. 9, 30 September 2020 (2020-09-30), pages 283 - 292 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116524557B (en) | 2024-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yang et al. | Security and accuracy of fingerprint-based biometrics: A review | |
| CN109902546B (en) | Face recognition method, face recognition device and computer readable medium | |
| CN105917353B (en) | Feature extraction and matching for biological identification and template renewal | |
| US10719693B2 (en) | Method and apparatus for outputting information of object relationship | |
| WO2022105118A1 (en) | Image-based health status identification method and apparatus, device and storage medium | |
| TW202026984A (en) | User identity verification method, device and system | |
| CN110084603A (en) | Method, detection method and the corresponding intrument of training fraudulent trading detection model | |
| CN111310705A (en) | Image recognition method and device, computer equipment and storage medium | |
| CN111401219B (en) | Palm key point detection method and device | |
| CN106295501A (en) | The degree of depth based on lip movement study personal identification method | |
| US10997609B1 (en) | Biometric based user identity verification | |
| CN111178130A (en) | Face recognition method, system and readable storage medium based on deep learning | |
| WO2022142903A1 (en) | Identity recognition method and apparatus, electronic device, and related product | |
| CN111243698A (en) | Data security sharing method, storage medium and computing device | |
| CN116664930A (en) | Personalized federal learning image classification method and system based on self-supervision contrast learning | |
| Alyasseri et al. | EEG channel selection for person identification using binary grey wolf optimizer | |
| WO2021086572A1 (en) | Signature verification | |
| Phoka et al. | Image based phishing detection using transfer learning | |
| CN116524557B (en) | Face counterfeiting detection model optimization method, device and system based on federal learning | |
| CN117037244A (en) | Face security detection method, device, computer equipment and storage medium | |
| WO2022178210A1 (en) | Clustered dynamic graph convolutional neural network (cnn) for biometric three-dimensional (3d) hand recognition | |
| CN114299569A (en) | Safe face authentication method based on eyeball motion | |
| CN114550224A (en) | Fingerprint image identification comparison method and device based on deep learning and electronic equipment | |
| Arora et al. | Cryptography and Tay-Grey wolf optimization based multimodal biometrics for effective security | |
| WO2022104340A1 (en) | Artificial intelligence for passive liveness detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |