CN114021738A - Distributed generation countermeasure model-based federal learning method - Google Patents

Distributed generation countermeasure model-based federal learning method Download PDF

Info

Publication number
CN114021738A
CN114021738A CN202111393636.3A CN202111393636A CN114021738A CN 114021738 A CN114021738 A CN 114021738A CN 202111393636 A CN202111393636 A CN 202111393636A CN 114021738 A CN114021738 A CN 114021738A
Authority
CN
China
Prior art keywords
model
data
generated
client
server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111393636.3A
Other languages
Chinese (zh)
Inventor
王建新
盛韬
刘渊
路博
陈龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan Sanxiang Bank Co Ltd
Original Assignee
Hunan Sanxiang Bank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan Sanxiang Bank Co Ltd filed Critical Hunan Sanxiang Bank Co Ltd
Priority to CN202111393636.3A priority Critical patent/CN114021738A/en
Publication of CN114021738A publication Critical patent/CN114021738A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a distributed generation countermeasure model-based federal learning method. The method comprises the following steps: training for generating a countermeasure model in a distributed environment, wherein a server is provided with a generation model, and a client is respectively provided with a discrimination model and a classification model; generating generation data by a generation model of the server, and sending the generation data to the client; the client updates the classification model and the discrimination model by using the local data and the generated data, and transmits the discrimination result of the generated data to the server; the server updates the generated model according to the judgment result and judges whether the generated data meets a preset termination condition; if so, skipping to the step of sending the generated data to the client and then; otherwise, the generative model is used for carrying out the federal learning training process. According to the method, more data which obey global distribution are added in the original algorithm process by utilizing a distributed generation countermeasure network, and the generalization and accuracy of the model trained by the Federal learning algorithm are increased.

Description

Distributed generation countermeasure model-based federal learning method
Technical Field
The invention relates to the technical field of federal learning, in particular to a distributed countermeasure model generation-based federal learning method.
Background
With the progress of machine learning and big data technology, the value of data is further improved, and more attention is paid to the protection of data privacy security. Due to competition of each organization and organization in the industry and the provision of legal provisions, data cannot be effectively circulated, so that data value is lost and cannot be effectively trained, and different data islands are formed.
Federal study is proposed by Google company for the first time, and the purpose is to solve at user local model update problem to android mobile phone, because Federal study has fine application to present "data island" problem, so received people's attention. Federal learning is used as an innovative mechanism, a global model with privacy attributes can be trained from scattered data sources, and the specific method is as follows: firstly, the server selects different mechanisms or organizations to participate in the training and sends the training model. The selected institution then trains the model with the local data. And finally, uploading the locally trained model parameters by the selected mechanism, and performing the next iteration after the server side aggregates the model parameters.
While federal learning provides a solution to the current data dilemma, it presents challenges to the training of federal learning due to problems such as data distribution under realistic conditions. For example, when federal learning is performed among financial or medical institutions, the entire distribution of data does not meet the condition of independent and same distribution because the institutions participating in training may come from different regions or have certain bias on different groups of people, so that the traditional machine learning method is difficult to greatly improve. In addition, different application scenes and models may exist for the same batch of data, and training by using the federal learning method requires other mechanisms or organizations to participate in training together, otherwise, the training effect cannot be greatly improved. Meanwhile, the basic purpose of training by using the federal learning is to increase the local data volume under the constraint of a privacy condition, improve the effect of a local model and global generalization, which is one of the most concerned problems of the federal learning participants.
Therefore, for the federal learning of a cross-data island, how to slow down the problem caused by the non-independent and same distribution of data among different organizations, provide a public data set which does not relate to privacy problems to carry out subsequent tasks, and meanwhile, the effect of training a model in the federal learning is improved, which is a problem that research is needed urgently.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides a distributed generation countermeasure model-based federal learning method. A distributed generation countermeasure method is utilized, a generation model capable of simulating global distribution is learned on the premise that data are not transmitted, so that data volume in the federal learning process is increased, problems caused by data heterogeneity of different clients are relieved, and the federal learning effect and generalization are improved. Moreover, after the federal learning training is finished in the mode, the learned generation model can be reserved so as to carry out subsequent downstream tasks, and the federal learning training with other clients is not needed again.
In one aspect, the invention provides a federated learning method based on a distributed generative confrontation model, which comprises the following steps:
step 1: the server maintains a generating model, selects the client sides participating in training and sends generating data to the client sides participating in training;
step 2: the client uses local data to train a classification model, simultaneously uses local real data and generated data to train a discrimination model, and uploads a discrimination result of the client on the generated data to the server;
and step 3: after receiving the judgment result of the client, the server updates and iterates the generation model by using the judgment result, and finally sends the new generation data to the selected client;
and 4, step 4: after the client-side finishes training, uploading the model to the server again until a server generated model meets a preset standard, stopping training of the generated model and a discrimination model, and uploading classification model parameters by the client-side each time;
and 5: and after aggregating the client model parameters, the server trains by using the screened generated data, and sends the trained classification model as a new round of model to the client.
And after the training process meets the step 4, repeating the step 4 and the step 5 until meeting the preset federal learning stop condition.
The distributed generative confrontation network is erected among clients in federal learning, the discriminant and the classification model are trained by using client data, the generative model in the server is updated, and then data which are in accordance with global distribution are generated by using the trained generative model to improve the federal learning effect, and the process is effectively verified through experiments.
Optionally, in step 2, the client determines the determination result of the generated data according to the following:
for each generated data, the judgment result of the data is the output result of the judgment model in all the selected clients minus the cross entropy loss of the output result of the classification model and the generated data label, wherein the largest judgment result is selected as the data.
Optionally, the process of the client outputting the result of determining the generated data according to the method is as follows:
for K clients participating in federated learning, the server classifies the model C and the generated data in t iterations
Figure BDA0003369651070000021
And sending the K to the client, wherein K belongs to K. The server obtains the generated data through the following iteration steps
Figure BDA0003369651070000022
The judgment result of (1):
a: the client K is any one of K clients and receives the generated data from the server
Figure BDA0003369651070000023
Then, training the classification model by using local data to obtain an updated classification model
Figure BDA0003369651070000024
Using local data (x, y) and generated data simultaneously
Figure BDA0003369651070000031
Training discriminant model
Figure BDA0003369651070000032
Obtaining an updated discriminant model
Figure BDA0003369651070000033
B: client k will generate data
Figure BDA0003369651070000034
Respectively input the classification model
Figure BDA0003369651070000035
And discrimination model
Figure BDA0003369651070000036
To obtain the classified classification result
Figure BDA0003369651070000037
And the result output by the discriminator
Figure BDA0003369651070000038
Calculating classification results
Figure BDA0003369651070000039
And generating data tags
Figure BDA00033696510700000310
Cross entropy loss of (A) to obtain a classification discrimination
Figure BDA00033696510700000311
Wherein
Figure BDA00033696510700000312
Finally, the client end judges the result
Figure BDA00033696510700000313
Uploading to a server;
c: the server receives the discrimination results from the K clients, and selects the maximum discrimination result uploaded by the client for the ith generation data (wherein i belongs to m)
Figure BDA00033696510700000314
As generated data
Figure BDA00033696510700000315
And a generated model G of the server is generated by using the result of discrimination for each generated datatUpdate to obtain Gt+1Then, carrying out the next iteration;
optionally, the screening process for the generated data in step 5 is as follows:
firstly, a server updates a generation model, and a batch of generation data is generated by using the generation model;
secondly, the server classifies the generated data by using the aggregated classification model, screens out the inconsistent classification result and the preset label of the generated data, and divides the rest into the generated data for updating in the current round;
finally, the server updates the aggregation model by using the screened data to complete the iteration of the current round;
optionally, the steps 4 and 5 may be replaced by other federal learning algorithm implementation processes, for example, when a federal distillation algorithm process is used, the step 4 is performed after the generated model meets preset standards, wherein the client uploads local task model parameters and distillation results, and the distillation results are output results of the client before the client uses the same batch of generated data to pass through the Softmax layer on the task model; in step 5, the server aggregates the client task model and the distillation result respectively, then uses the aggregation model to output the distillation result on the same generated data, and finally uses the aggregated client distillation result as a teacher to guide the server to update the aggregation model.
Optionally, the termination condition for the model generated in step 4 is based on: and training an initialized task model by using the generated data generated by the generated model, and terminating the training of the generated model when the task model achieves a certain result in the real test data set by using the generated data.
In a second aspect, the invention provides a federated learning method based on a distributed generation countermeasure model, which comprises the following steps when applied to a client:
step 1: the client receives the task model and the generated data sent by the server, trains and generates a data training discrimination model by using local real data, trains a classification model by using the local data, and finally uploads a generated data discrimination result;
step 2: when the server generates a model meeting a preset standard, the client stops the training of the discrimination model, receives the aggregated task model from the server, and uploads task model parameters each time in an iteration mode;
and uploading the local classification model to a server for cyclic updating until a preset condition is met.
In a third aspect, the invention provides a federated learning method based on a distributed generation countermeasure model, which when applied to a server, comprises the following steps:
step 1: the server selects the clients participating in training and sends the latest generated data to the selected clients;
step 2: the server receives the discrimination result sent by the client, updates the generation model by using the discrimination result, and then executes the step 1;
and step 3: when the generated model reaches a preset condition, stopping executing the step 1 and the step 2, performing aggregation after receiving the client model parameters, training the aggregation model by using the screened generated data, and finally selecting a client participating in training to send the trained aggregation model to the selected client;
and when the generated model does not reach the preset condition, executing the step 1 and the step 2 in a circulating manner, and after the generated model reaches the preset condition, executing the step 3 until the task model reaches the preset condition.
In a fourth aspect, the invention provides a generative model after training is completed, and the generative model can generate generation data with approximate global distribution to train a downstream task or a series of tasks meeting the requirements of a client.
The invention has the following beneficial effects: according to the federated learning method based on the distributed generative confrontation model, provided by the invention, more data which obey global distribution are added in the original algorithm process by utilizing the distributed generative confrontation network, and the generalization and accuracy of the model trained by the federated learning algorithm are increased. In a further optimization scheme of the invention, the generated data generated by the generated model is used as a public distillation data set, and the performance and stability of task model training can be improved by using a federal knowledge distillation algorithm. In the subsequent work of training completed by the method, the distributed generation model trained through federal learning can be used in subsequent tasks or downstream tasks, the data volume required by the tasks can be increased, and the data approximate to global distribution is provided to improve the generalization of the model.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious to those skilled in the art that other drawings can be obtained according to the drawings without any inventive exercise.
FIG. 1 is a schematic diagram of model communication provided by the method of the present invention;
FIG. 2 is a schematic flow diagram of the process of the present invention;
FIG. 3 is a schematic diagram of an embodiment of the present invention.
Detailed Description
The invention provides a distributed generation countermeasure model-based federal learning implementation method, which is used for solving the problem of data island. Federal learning refers to the calculation process that data owners can participate in model training and obtain models without sharing data. The method focuses on the problems caused by data heterogeneity and insufficient data volume in federal learning. In order that those skilled in the art may better understand the technical solution described in the present invention and make the above objects, features and advantages of the present invention comprehensible, it is described in further detail below with reference to the accompanying drawings, but the present invention may be implemented in various different ways as defined and covered by the claims.
Example 1:
the invention provides a distributed generation countermeasure network-based federal learning implementation method, which specifically comprises the following steps:
step 1: the server maintains a generating model, selects clients participating in training and sends generated data;
as shown in fig. 1, the server keeps communication with the selected client after selecting the client, and sends generated data, where the model is a shared model with a good effect on the task before starting training, such as a convolutional neural network model.
Step 2: the client side trains a classification model by using local data, trains a discrimination model by using local real data and generated data, and uploads a discrimination result of the generated data to the server;
after initializing the classification model, the client trains the model by using local real data to obtain a local classification model. And simultaneously, initializing discrimination model parameters locally, and training by using generated data generated by a server generation model and local real data to ensure that the local discrimination model judges the generated data as false and judges the real data as true.
Assuming a total of K clients, n is the total number of samples, nkIs at the same timeSample size, F, distribution on kth clientkAn objective function of the model is classified for the kth client. Classification model C for client kkThe objective function is:
Figure BDA0003369651070000051
Figure BDA0003369651070000052
wherein the content of the first and second substances,
Figure BDA0003369651070000055
as a loss function of the model, ωc kTo classify the model parameters, yiAs real data xiLabel of (a), yi' is the classification model prediction result. For the general model with the optimization target of a convex function, a gradient descent algorithm is used for searching the minimum value of the target function, namely the optimal solution:
Figure BDA0003369651070000053
wherein η is the learning rate, and the classification model in the kth client updates the model through the formula.
For the discriminant model in the client, the local real data (x, y) and the generated data sent by the server are used
Figure BDA0003369651070000054
And (6) updating. Wherein the kth client discriminates the model DkModel parameter ω ofd kUpdating is done by maximizing the random gradient in equation (4):
Figure BDA0003369651070000061
wherein, the model D is judgedkBy interleaving m dataInstead, maximize the real data xiWhile minimizing the generation of data
Figure BDA0003369651070000062
To output of (c).
The discrimination result of the data generated in the kth client is divided into two parts, namely a score result of the discrimination model and a score result of the classification model. First, the scoring result for the discriminant model
Figure BDA0003369651070000063
To generate data
Figure BDA0003369651070000064
Upper discriminant model DkThe direct result of the output, where i ∈ m. While the scoring results for the classification model are: each generation data
Figure BDA0003369651070000065
Using a classification model CkClassification condition and generated data setting tag of
Figure BDA0003369651070000066
Cross entropy loss of
Figure BDA0003369651070000067
As shown in equation (5).
Figure BDA0003369651070000068
Finally, each client generates a data discrimination result
Figure BDA0003369651070000069
And uploading to a server to complete local iteration.
And step 3: after receiving the client result, the server updates and iterates the generation model by using the judgment result, and finally sends the new generation data to the selected client;
wherein, ω iscAnd (4) judging the parameters of the model for the client, wherein n is the total amount of the samples. When the server performs the update in the t +1 th round, the update is performed by equation (6).
Figure BDA00033696510700000610
Figure BDA00033696510700000611
In a distributed generative countermeasure network, when client 1 and client 2 only exist in one category of data x1And x2For arbiter output D in client1(x1)＞＞D2(x1) Same principle as D2(x2)＞＞D1(x2). Thus selecting the largest discriminator score maxkDk(x) Updating the generative model, i.e. selecting the client distribution max with the most distributed category among the data distributions of all clientskpk(x) And (6) updating. For the update of the generative model, it can be deduced from equation (7) that the arbiter D of the kth clientk(x) Distributing p by client datak(x) And generator data distribution pg(x) Reach the optimum
Figure RE-GDA0003430929760000071
When the temperature of the water is higher than the set temperature,
Figure RE-GDA0003430929760000072
can be seen as a distribution p by all clientsmax(x) The best discriminant trained, so:
Figure BDA0003369651070000073
wherein alpha is a positive integer, and when p is provedg(x) Is equal to pmax(x) The objective function of the time-generated model reaches the minimum, and the training of the generated model is obtained through a distributed discrimination modelAnd (6) training.
Because only the generation model capable of generating the unlabeled data is trained in the process of training the distributed generation countermeasure network, the invention combines the generation model with the training mode of federal learning, and utilizes the task model in the process of federal learning training, so that the generation model deployed in the server can generate the labeled generation data, thereby improving the federal learning and effect. In the invention, the discrimination result of the discrimination model and the classification model is used for replacing the original discriminator Dk(x) When only one kind of data (x) exists in the client 1 and the client 2 as an output result of (1)1,y1) And (x)2,y2) The output of the arbiter in the client is D1(x1)＞＞D2(x1) Simultaneously corresponding to the classification result C of the classifierk(x1) And the original label y of the data1Cross entropy loss of (1), i.e. the distance of the classification result from the true label, crossEncopy [ C1(x1),y1]＞＞CrossEntropy[C2(x1),y1]. At this time, the server selects the maximum score of each piece of data in the K pieces of clients as the judgment result of the round for the m pieces of generated data respectively:
Figure BDA0003369651070000076
wherein the generative model of the server is modeled by minimizing the gradient in equation (10) versus parameter ωgUpdating:
Figure BDA0003369651070000077
and after the server finishes updating, sending the new round of generated data to the client for the next iteration.
And 4, step 4: after the client training is finished, uploading the parameters to the server again until a server generated model meets a preset standard, stopping training of the generated model and a judgment model, and only uploading classification model parameters by the client each time;
when the client uses the generated model to train the classification model independently, the server stops training the generated model after the test set reaches a certain precision, and the client updates the task model and uploads the task model parameters only through local data each time.
And 5: and after aggregating the client model parameters, the server trains by using the screened generated data, and sends the trained classification model as a new round of model to the client.
The task model is updated in a mode of aggregating new update parameters from the update parameters obtained from the client, and the overall objective of the task model is as follows:
Figure BDA0003369651070000081
Figure BDA0003369651070000082
firstly, the screening process of the server for the generated data is as follows: the server receives the parameters of all the clients and then executes the parameters of the aggregation model through a formula (12), the aggregation model generates a batch of unscreened generated data after the aggregation is finished, the aggregated model is used for classifying the generated data, and the classified result is consistent with the preset label of the generated data and is regarded as the available data. After screening, a new round of generated data is selected to train the aggregated model, and the model updating mode is the same as that in the client, as shown in formula (3).
And then, the server sends the parameters of the aggregation model trained by using the generated data to the selected client side to complete the aggregation updating of the round. And if the Federal learning preset stop condition is not met, repeating the steps 4 and 5 until the training is terminated.
The specific flow of the above steps is shown in fig. 2. In the embodiment, a distributed generation countermeasure model is fused with the federal learning training process, so that the effect of mutual promotion is achieved, the effect and the generalization of the final federal learning model are improved, and meanwhile, a generation model capable of generating label data is trained.
Example 2:
in this embodiment, the federal learning algorithm is adjusted in embodiment 1, and the algorithm in federal learning is replaced by a knowledge distillation algorithm, so that the effect and stability of the original algorithm are improved, and the specific implementation steps are as follows:
step 1: the server maintains a generating model, selects clients participating in training and sends generated data;
step 2: the client side trains a classification model by using local data, trains a discrimination model by using local real data and generated data, and only uploads a discrimination result of the generated data to the server;
and step 3: after receiving the client result, the server updates and iterates the generation model by using the judgment result, and finally sends the generation data after the new round of screening to the selected client;
in this embodiment, reference may be made to the related description in embodiment 1 for step 1 to step 3.
And 4, step 4: after the training of the client is finished, uploading the data to the server again until a server generation model meets a preset standard, stopping training of the generation model and the discrimination model, and changing the parameters of the classification model to be uploaded and the output result of the same batch of generated data on the task model before passing through a Softmax layer by the client each time;
and 5: and the server aggregates the client model parameters and then serves as a student network, the output result of the aggregation model in the previous layer of Softmax is obtained by using the same batch of screened generated data, and the aggregated client output result serves as a teacher to guide the aggregated student model to be updated in a knowledge distillation mode.
The server aggregates the output results of the client by the following formula:
Figure BDA0003369651070000083
wherein logskAnd obtaining an output result of the aggregation model in the previous layer of Softmax for the kth client by using the same batch of screened generated data, wherein K is the number of the clients. The aggregated model of the server generates logs 'for the same batch of generated data, so that the logs' are continuously close to the logs, namely, the knowledge contained in each client is learned.
In the embodiment, the data collection or public data set process in the federal knowledge distillation is replaced by the data generation mode, and compared with the data collection mode, the data generated by the method is closer to the overall real distribution result, so that the federal learning model can learn more overall generalization results.
Example 3
After completion of the federal learning training, the generative model may be used for training of downstream tasks. And inputting noise and random labels which accord with Gaussian distribution into the generation model, and generating corresponding label data for a downstream task by the generation model.
The invention takes the public data set FashionMNSIT as an example, public data are divided into 10 clients without overlapping according to Dirichlet Distributions, a parameter alpha is used for controlling the degree of non-independent and same distribution of the client data set, and the probability of the non-independent and same distribution of the client data set is smaller when the alpha is larger. The task model in the present invention uses a multilayer Convolutional Neural network model (CNN), the generated model is a Conditional Deep Convolutional Generative adaptive network (cDCGAN), and the discrimination model is selected as a Deep Convolutional Generative Adaptive Network (DCGAN) for practical application, and the specific structure is shown in fig. 3.
The method is widely applied to various application scenes of cross-island problems, for example, a certain disease has different expression symptoms among different medical institutions, and the comprehensive consideration of all symptoms of the disease is beneficial to improving the treatment effect; in addition, anti-fraud data also has different expression forms among different financial institutions, and due to the regulations of policies and laws, the data privacy of the financial institutions needs to meet the requirements of supervision departments, and establishing an effective anti-fraud model while meeting the regulations has important significance for the financial institutions. Aiming at the cross-island problem, a distributed countermeasure generating model-based federal learning method is used for establishing a federal model, dispersed financial or medical institution nodes can be connected, multi-party data is repeatedly utilized for establishing an effective federal learning model to solve practical problems, generated data is used for improving the performance and the generalization of the federal model, and finally the generating model is reserved to facilitate the development of subsequent research work.
An embodiment of the present invention further provides a storage medium, where a computer program is stored in the storage medium, and when the computer program is executed by a processor, the computer program implements part or all of the steps in each embodiment of the federated learning method based on the distributed generative confrontation model provided in the present invention. The storage medium may be a magnetic disk, an optical disk, a Read-only memory (ROM) or a Random Access Memory (RAM).
Those skilled in the art will readily appreciate that the techniques of the embodiments of the present invention may be implemented as software plus a required general purpose hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.
The above-described embodiments of the present invention should not be construed as limiting the scope of the present invention.

Claims (6)

1. A federated learning method based on a distributed generative confrontation model is characterized by comprising the following steps:
training for generating a countermeasure model in a distributed environment, wherein a server is provided with a generation model, and a client is respectively provided with a discrimination model and a classification model for executing a specific task;
generating generation data by a generation model of the server, and sending the generation data to the client;
the client updates the classification model and the discrimination model by using local data and the generated data, and transmits a discrimination result of the generated data to the server;
the server updates the generated model according to the judgment result;
the server judges whether the generated data generated by the updated generation model meets a preset termination condition;
if the generated data generated by the updated generated model does not meet the preset termination condition, skipping to the step of sending the generated data to the client and the subsequent step;
and if the generated data generated by the updated generated model meets the preset termination condition, performing a federal learning training process by using the generated model.
2. The method of claim 1, wherein the step of updating the classification model and the discrimination model by the client using the local data and the generated data and sending the discrimination result of the generated data to the server is performed by the client according to the following:
and the judgment result of each generated data is the cross entropy loss of the output result of the classification model and the generated data label subtracted from the output result of the judgment model in all the selected clients, wherein the largest judgment result is selected as the data.
3. The method of claim 2, wherein the client outputs the decision on the generated data, and the decision result is obtained by the following process:
for K clients participating in federated learning, the server classifies the model C and the generated data in t iterations
Figure FDA0003369651060000011
Sending the data to a client K, wherein K belongs to K, and obtaining generated data by the server through the following iteration steps
Figure FDA0003369651060000012
The judgment result of (1):
the client K is any one of K clients and receives the classification model C from the servertAnd generating data
Figure FDA0003369651060000013
Then, training the classification model by using local data to obtain an updated classification model
Figure FDA0003369651060000014
Using local data (x, y) and generated data simultaneously
Figure FDA0003369651060000015
Training discriminant model
Figure FDA0003369651060000016
Obtaining an updated discriminant model
Figure FDA0003369651060000017
Client k will generate data
Figure FDA0003369651060000018
Respectively input the classification model
Figure FDA0003369651060000019
And discrimination model
Figure FDA00033696510600000110
To obtain the classified classification result
Figure FDA00033696510600000111
And the result output by the discriminator
Figure FDA00033696510600000112
Calculating classification results
Figure FDA00033696510600000113
And generating data tags
Figure FDA00033696510600000114
Cross entropy loss of (A) to obtain a classification discrimination
Figure FDA00033696510600000115
Wherein
Figure FDA0003369651060000021
Finally, the client end judges the result
Figure FDA0003369651060000022
Uploading to a server;
the server receives the discrimination results from the K clients, and selects the maximum discrimination result uploaded by the client for the ith generated data, wherein i belongs to m
Figure FDA0003369651060000023
As generated data
Figure FDA0003369651060000024
And a generated model G of the server is generated by using the result of discrimination for each generated datatUpdate to obtain Gt+1And then the next iteration is performed.
4. The method of claim 3, wherein the preset termination condition is: training an initialized task model by using generated data generated by the generated model, and terminating the training of the generated model when the task model achieves a preset result in the real test data set by using the generated data.
5. The method of claim 1, wherein performing a federal learning training procedure using generative models comprises: and the server receives the client model parameters and then carries out aggregation, trains the aggregation model by using the screened generated data, and finally selects the client participating in the training to send the trained aggregation model to the selected client.
6. The method of claim 5, wherein the screening process for generating data is as follows:
the server updates the generative model and generates a batch of generative data by using the generative model;
the server classifies the generated data by using the aggregated classification model, screens out the inconsistent classification result and the preset label of the generated data, and divides the rest into the generated data for updating in the current round;
and the server updates the aggregation model by using the screened data to complete the iteration of the round.
CN202111393636.3A 2021-11-23 2021-11-23 Distributed generation countermeasure model-based federal learning method Pending CN114021738A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111393636.3A CN114021738A (en) 2021-11-23 2021-11-23 Distributed generation countermeasure model-based federal learning method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111393636.3A CN114021738A (en) 2021-11-23 2021-11-23 Distributed generation countermeasure model-based federal learning method

Publications (1)

Publication Number Publication Date
CN114021738A true CN114021738A (en) 2022-02-08

Family

ID=80065975

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111393636.3A Pending CN114021738A (en) 2021-11-23 2021-11-23 Distributed generation countermeasure model-based federal learning method

Country Status (1)

Country Link
CN (1) CN114021738A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114785559A (en) * 2022-03-29 2022-07-22 南京理工大学 Differential privacy federation learning method for resisting member reasoning attack
CN114913390A (en) * 2022-05-06 2022-08-16 东南大学 Method for improving personalized federal learning performance based on data augmentation of conditional GAN
CN116168789A (en) * 2023-04-26 2023-05-26 之江实验室 Multi-center medical data generation system and method

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114785559A (en) * 2022-03-29 2022-07-22 南京理工大学 Differential privacy federation learning method for resisting member reasoning attack
CN114913390A (en) * 2022-05-06 2022-08-16 东南大学 Method for improving personalized federal learning performance based on data augmentation of conditional GAN
CN116168789A (en) * 2023-04-26 2023-05-26 之江实验室 Multi-center medical data generation system and method

Similar Documents

Publication Publication Date Title
CN114021738A (en) Distributed generation countermeasure model-based federal learning method
WO2021155706A1 (en) Method and device for training business prediction model by using unbalanced positive and negative samples
US11816183B2 (en) Methods and systems for mining minority-class data samples for training a neural network
CN109583501B (en) Method, device, equipment and medium for generating image classification and classification recognition model
CN109376242B (en) Text classification method based on cyclic neural network variant and convolutional neural network
Qian et al. Hierarchical CVAE for fine-grained hate speech classification
CN112199608B (en) Social media rumor detection method based on network information propagation graph modeling
CN110516539A (en) Remote sensing image building extracting method, system, storage medium and equipment based on confrontation network
CN108664893A (en) A kind of method for detecting human face and storage medium
CN108334949A (en) A kind of tachytelic evolution method of optimization depth convolutional neural networks structure
CN107846392A (en) A kind of intrusion detection algorithm based on improvement coorinated training ADBN
CN113065974B (en) Link prediction method based on dynamic network representation learning
WO2022218139A1 (en) Personalized search method and search system combined with attention mechanism
Gu et al. Application of fuzzy decision tree algorithm based on mobile computing in sports fitness member management
CN112101473A (en) Smoke detection algorithm based on small sample learning
Knop et al. Generative models with kernel distance in data space
CN111797935A (en) Semi-supervised deep network picture classification method based on group intelligence
CN117033997A (en) Data segmentation method, device, electronic equipment and medium
KR102145777B1 (en) Method and apparatus of object control based on self-verification type analysis platform for emotion and weather using artificial intelligence
CN113626685A (en) Propagation uncertainty-oriented rumor detection method and device
Zhang et al. Research on classification method of network resources based on modified SVM algorithm
Mao et al. QoS trust rate prediction for Web services using PSO-based neural network
CN111078882A (en) Text emotion measuring method and device
Hou et al. Prediction of learners' academic performance using factorization machine and decision tree
CN112632291B (en) Generalized atlas characterization method with enhanced ontology concept

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination