CN117473558A - Self-adaptive DPWGAN training method and system based on federal learning - Google Patents
Self-adaptive DPWGAN training method and system based on federal learning Download PDFInfo
- Publication number
- CN117473558A CN117473558A CN202311785715.8A CN202311785715A CN117473558A CN 117473558 A CN117473558 A CN 117473558A CN 202311785715 A CN202311785715 A CN 202311785715A CN 117473558 A CN117473558 A CN 117473558A
- Authority
- CN
- China
- Prior art keywords
- dpwgan
- training
- adaptive
- client
- generator parameters
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000003044 adaptive effect Effects 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 8
- 230000002776 aggregation Effects 0.000 claims description 4
- 238000004220 aggregation Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000003698 anagen phase Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
- G06F21/6263—Protecting personal data, e.g. for financial or medical purposes during internet communication, e.g. revealing personal data from cookies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0475—Generative networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Bioethics (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a self-adaptive DPWGAN training method and system based on federal learning, wherein the method comprises the steps that a server broadcasts a discriminator, generator parameters and noise scale in an initialized WGAN to a client; the client performs DPWGAN training by using a local data set, and perturbs the trained generator parameters by using a differential privacy scheme of self-adaptive noise; uploading the disturbed generator parameters to a server; the server performs weighted average operation on the disturbed generator parameters to obtain an aggregated global model, and generates an acceptance Score and an FID value of the image according to the aggregated global model; and updating until the values of the acceptance Score and the FID reach the expected targets. The invention can solve the problem of data island and privacy protection in the traditional GAN training.
Description
Technical Field
The invention relates to the field of federal learning, in particular to a self-adaptive DPWGAN training method and system based on federal learning.
Background
With the advent of the big data age, machine learning has entered a significant growth phase. To accommodate different scenarios, many machine learning models have been developed, including Convolutional Neural Networks (CNNs), recurrent Neural Networks (RNNs), and generation of countermeasure networks (GANs). GAN and its various iterations are excellent in producing high quality "synthetic" samples that are very difficult to discern from real data. Noteworthy applications include generating images from text descriptions, converting still images to video, and enhancing image resolution.
Unfortunately, GAN training faces three major challenges. First, recent findings emphasize that even complex, opaque generative models, such as GAN, produce output in an unexplainable manner, are susceptible to privacy disclosure. Second, while the deep learning approach achieves significant results, acquiring large amounts of labeled data is still a necessary prerequisite for building robust classifiers. Furthermore, real world data typically exists in isolated form. Privacy and security issues prevent data sharing despite the large amount of data available between various users, participants, and data owners. Finally, current DPGAN training protocols employ a uniform noise scale, lacking in adaptation. Surveys indicate that different data owners have different privacy requirements, making traditional DPGAN incapable of adaptive training.
Therefore, in order to solve the problem of data islanding and privacy protection in the conventional GAN training, it is highly desirable to provide a method or system for self-adaptive DPWGAN training for privacy protection.
Disclosure of Invention
The invention aims to provide a self-adaptive DPWGAN training method and system based on federal learning, which can solve the problems of data island and privacy protection in the traditional GAN training.
In order to achieve the above object, the present invention provides the following solutions:
an adaptive DPWGAN training method based on federal learning, comprising:
broadcasting the discriminator, generator parameters and noise scale in the initialized WGAN to each client by the server;
the client performs DPWGAN training by using a local data set, perturbs the trained generator parameters by using a differential privacy scheme of self-adaptive noise, and uploads the perturbed generator parameters to the server;
the server performs weighted average operation on the disturbed generator parameters uploaded by all clients to obtain an aggregated global model; generating an acceptance Score and an FID value of the image according to the aggregated global model;
and when the InceptionScare and FID values of the generated image do not reach the expected targets, broadcasting generator parameters in the current aggregated global model to the client, returning to the client to perform DPWGAN training by using the local data set, disturbing the trained generator parameters by using a self-adaptive noise differential privacy scheme, and uploading the disturbed generator parameters to the server until the InceptionScare and FID values of the generated image reach the expected targets.
Optionally, the wasperstein distance is introduced as a loss function in the client's DPWGAN training with the local data set.
Optionally, differential privacy is introduced in the client's DPWGAN training with the local data set.
Optionally, the adaptive noise differential privacy scheme is to track the privacy consumption of the user by using moment accounting, and adaptively adjust the noise scale in the training process.
Optionally, the updating mode of the moment accounting is:
；
wherein,l is the sampled data quantity, +.>For the sampling rate, satisfy->N is the amount of data owned by each client, < >>Is the amplitude of noise>For the order in moment accounting, +.>For moment accounting concept in differential privacy, which is used to measure privacy consumption, exp () is an exponential function based on a natural constant e.
Optionally, the noise scale formula is:；
wherein,for the noise scale>To the extent of privacy disclosure->Is the minimum of the sampled data amounts.
An adaptive DPWGAN training system based on federal learning, comprising:
the initialization module is used for broadcasting the identifier, the generator parameters and the noise scale in the initialized WGAN to each client by the server;
the training module is used for carrying out DPWGAN training by the client by utilizing the local data set and disturbing the trained generator parameters by utilizing the differential privacy scheme of the self-adaptive noise; uploading the disturbed generator parameters to a server;
the parameter aggregation module is used for carrying out weighted average operation on the disturbed generator parameters uploaded by all the clients by the server to obtain an aggregated global model; generating an acceptance Score and an FID value of the image according to the aggregated global model;
and the iteration module is used for broadcasting generator parameters in the current aggregated global model to the client when the information Score and the FID value of the generated image do not reach the expected target, and returning to the training module until the information Score and the FID value of the generated image reach the expected target.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the invention provides a self-adaptive DPWGAN training method and a self-adaptive DPWGAN training system based on federal learning, and provides a federal learning method with data sharing and privacy. The client performs self-adaptive DPWGAN training, uploads the parameter information of the local training, the server performs aggregation update on the received parameter information, and the federal learning process does not need to depend on a trusted central aggregator, so that the problem of data islanding can be solved, and training data of each participant can be protected. The invention makes attacks such as inference attack in federal learning difficult to carry out through the differential privacy scheme of the self-adaptive noise, further strengthens the privacy protection of data, and can dynamically change the noise added in the differential privacy mechanism so as to improve the accuracy.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic flow chart of a federally learning-based adaptive DPWGAN training method according to the present invention.
Fig. 2 is a schematic diagram of the overall architecture of a federally learning-based adaptive DPWGAN training method according to the present invention.
Fig. 3 is a block diagram of a GAN network.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention aims to provide a self-adaptive DPWGAN training method and system based on federal learning, which not only can solve the problem of data islanding, but also can protect training data of each participant.
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
As shown in fig. 1 and fig. 2, the adaptive DPWGAN training method based on federal learning provided by the present invention includes:
s101, broadcasting a discriminator, generator parameters and noise scales in the initialized WGAN to each client by the server; as shown in fig. 2, the system comprises a central server and N clients, data are distributed in the N clients, and the clients and the server only transmit parameters and not transmit data, wherein the server adopts a global model, and the clients adopt a local model.
S102, a client performs DPWGAN training by using a local data set, perturbs the trained generator parameters by using a differential privacy scheme of self-adaptive noise, and uploads the perturbed generator parameters to a server;
the wasperstein distance is introduced as a loss function in the client's DPWGAN training with the local data set.
The loss function is:
；
wherein,representing the discriminator->Representative generator->Representing the desired calculation +.>For a true data distribution,for random noise distribution, ++>Minimum generator->For the maximum of the discriminant, x is the real data and z is the random noise.
Differential privacy is introduced in DPWGAN training by a client by utilizing a local data set, so that attacks such as inference attack in federal learning are difficult to carry out, and the privacy protection of data is further enhanced.
By introducing the self-adaptive noise mechanism, the noise added in the differential privacy mechanism can be dynamically changed, so that the accuracy is improved.
The differential privacy scheme of the self-adaptive noise is to track the privacy consumption of the user by utilizing moment accounting, and the noise scale is self-adaptively adjusted in the training process.
The updating mode of the moment accounting is as follows：。
Wherein,l is the sampled data quantity, +.>For the sampling rate, satisfy->N is the amount of data owned by each client, < >>Is the amplitude of noise>For the order in moment accounting, +.>For moment accounting concept in differential privacy, which is used to measure privacy consumption, exp () is an exponential function based on a natural constant e.
The noise scale formula is:。
wherein,for the noise scale>To the extent of privacy disclosure->Is the minimum of the sampled data amounts.
S103, the server performs weighted average operation on the disturbed generator parameters uploaded by all clients to obtain an aggregated global model; and generating an acceptance Score and an FID value of the image according to the aggregated global model.
And S104, when the information Score and the FID value of the generated image do not reach the expected target, broadcasting generator parameters in the current aggregated global model to the client, and returning to S102 until the information Score and the FID value of the generated image reach the expected target.
The GAN network is configured as shown in fig. 3, and mainly includes two parts: a generator and a arbiter. The working strategy of the GAN is that firstly, the generator network is utilized to generate the generated data, and then, the discriminator network is utilized to compare the generated data with the real data so as to calculate the corresponding loss function to train the network. The whole network adopts the idea of game countermeasure, the purpose of the generator is to generate data as real as possible, and the purpose of the discriminator is to improve the capability of discriminating real data and generating data, namely, scoring the real data in high score and scoring the generated data in low score.
Corresponding to the method provided by the above embodiment, the adaptive DPWGAN training system based on federal learning provided by the present invention includes:
the initialization module is used for broadcasting the identifier, the generator parameters and the noise scale in the initialized WGAN to each client by the server.
The training module is used for the client to carry out DPWGAN training by using the local data set, disturbing the trained generator parameters by using the differential privacy scheme of the self-adaptive noise, and uploading the disturbed generator parameters to the server.
The parameter aggregation module is used for carrying out weighted average operation on the disturbed generator parameters uploaded by all the clients by the server to obtain an aggregated global model; and generating an acceptance Score and an FID value of the image according to the aggregated global model.
And the iteration module is used for broadcasting generator parameters in the current aggregated global model to the client when the information Score and the FID value of the generated image do not reach the expected target, and returning to the training module until the information Score and the FID value of the generated image reach the expected target.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to assist in understanding the methods of the present invention and the core ideas thereof; also, it is within the scope of the present invention to be modified by those of ordinary skill in the art in light of the present teachings. In view of the foregoing, this description should not be construed as limiting the invention.
Claims (7)
1. An adaptive DPWGAN training method based on federal learning, comprising:
broadcasting the discriminator, generator parameters and noise scale in the initialized WGAN to each client by the server;
the client performs DPWGAN training by using a local data set, perturbs the trained generator parameters by using a differential privacy scheme of self-adaptive noise, and uploads the perturbed generator parameters to the server;
the server performs weighted average operation on the disturbed generator parameters uploaded by all clients to obtain an aggregated global model; generating an acceptance Score and an FID value of the image according to the aggregated global model;
and when the InceptionScare and FID values of the generated image do not reach the expected targets, broadcasting generator parameters in the current aggregated global model to the client, returning to the client to perform DPWGAN training by using the local data set, disturbing the trained generator parameters by using a self-adaptive noise differential privacy scheme, and uploading the disturbed generator parameters to the server until the InceptionScare and FID values of the generated image reach the expected targets.
2. The adaptive DPWGAN training method based on federal learning of claim 1, wherein the wasperstein distance is introduced as a loss function in the DPWGAN training of the client using the local data set.
3. The adaptive DPWGAN training method based on federal learning of claim 1, wherein differential privacy is introduced in the client performing DPWGAN training using a local data set.
4. The adaptive DPWGAN training method as claimed in claim 1, wherein the differential privacy scheme of the adaptive noise is to track the privacy consumption of the user by using moment accounting, and the noise scale is adaptively adjusted during the training process.
5. The adaptive DPWGAN training method based on federal learning of claim 4, wherein the moment accounting is updated in the following manner:
；
wherein,l is the sampled data quantity, +.>For the sampling rate, satisfy->N is the amount of data owned by each client, < >>Is the amplitude of noise>Order in accounting for moments，/>For moment accounting concept in differential privacy, which is used to measure privacy consumption, exp () is an exponential function based on a natural constant e.
6. The adaptive DPWGAN training method based on federal learning of claim 5, wherein the noise scale formula is:；
wherein,for the noise scale>To the extent of privacy disclosure->Is the minimum of the sampled data amounts.
7. An adaptive DPWGAN training system based on federal learning, comprising:
the initialization module is used for broadcasting the identifier, the generator parameters and the noise scale in the initialized WGAN to each client by the server;
the training module is used for carrying out DPWGAN training by the client by utilizing the local data set, disturbing the trained generator parameters by utilizing the differential privacy scheme of the self-adaptive noise, and uploading the disturbed generator parameters to the server;
the parameter aggregation module is used for carrying out weighted average operation on the disturbed generator parameters uploaded by all the clients by the server to obtain an aggregated global model; generating an acceptance Score and an FID value of the image according to the aggregated global model;
and the iteration module is used for broadcasting generator parameters in the current aggregated global model to the client when the information Score and the FID value of the generated image do not reach the expected target, and returning to the training module until the information Score and the FID value of the generated image reach the expected target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311785715.8A CN117473558A (en) | 2023-12-25 | 2023-12-25 | Self-adaptive DPWGAN training method and system based on federal learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311785715.8A CN117473558A (en) | 2023-12-25 | 2023-12-25 | Self-adaptive DPWGAN training method and system based on federal learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117473558A true CN117473558A (en) | 2024-01-30 |
Family
ID=89639868
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311785715.8A Pending CN117473558A (en) | 2023-12-25 | 2023-12-25 | Self-adaptive DPWGAN training method and system based on federal learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117473558A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115329388A (en) * | 2022-10-17 | 2022-11-11 | 南京信息工程大学 | Privacy enhancement method for federally generated countermeasure network |
| CN116167084A (en) * | 2023-02-24 | 2023-05-26 | 北京工业大学 | Federal learning model training privacy protection method and system based on hybrid strategy |
| CN117056785A (en) * | 2023-08-31 | 2023-11-14 | 西安电子科技大学 | Federal learning classification model training method based on self-adaptive model disturbance |
-
2023
- 2023-12-25 CN CN202311785715.8A patent/CN117473558A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115329388A (en) * | 2022-10-17 | 2022-11-11 | 南京信息工程大学 | Privacy enhancement method for federally generated countermeasure network |
| CN116167084A (en) * | 2023-02-24 | 2023-05-26 | 北京工业大学 | Federal learning model training privacy protection method and system based on hybrid strategy |
| CN117056785A (en) * | 2023-08-31 | 2023-11-14 | 西安电子科技大学 | Federal learning classification model training method based on self-adaptive model disturbance |
Non-Patent Citations (3)
| Title |
|---|
| JIAQI HUANG: "DPWGAN:High-Quality Load Profiles Synthesis With Differential Privacy Guarantees", IEEE TRANSACTION ON SMART GRID, vol. 14, no. 4, 20 December 2022 (2022-12-20), pages 3283 - 3295 * |
| 史丹青: "生成对抗网络入门指南", vol. 2, 30 June 2021, 机械工业出版社, pages: 203 * |
| 徐宗本: "数据智能研究前沿", 31 December 2019, 上海交通大学出版社, pages: 60 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tang et al. | Digital signal modulation classification with data augmentation using generative adversarial nets in cognitive radio networks | |
| CN110008696A (en) | A kind of user data Rebuilding Attack method towards the study of depth federation | |
| CN113762530B (en) | Precision feedback federal learning method for privacy protection | |
| CN111951149B (en) | Image information steganography method based on neural network | |
| WO2021026944A1 (en) | Adaptive transmission method for industrial wireless streaming media employing particle swarm and neural network | |
| CN116167084A (en) | Federal learning model training privacy protection method and system based on hybrid strategy | |
| CN116681144A (en) | Federal learning model aggregation method based on dynamic self-adaptive knowledge distillation | |
| CN111985566B (en) | Distributed power supply time sequence joint output typical scene generation method based on Copula function | |
| CN114564746B (en) | Federal learning method and system based on client weight evaluation | |
| CN114418109A (en) | Node selection and aggregation optimization system and method for federal learning under micro-service architecture | |
| CN114116707A (en) | Method and device for determining contribution degree of participants in joint learning | |
| CN115204416A (en) | Heterogeneous client-oriented joint learning method based on hierarchical sampling optimization | |
| CN112836822A (en) | Federal learning strategy optimization method and device based on width learning | |
| CN117875408B (en) | Federal learning method of pulse neural network for flaw detection | |
| CN115879542A (en) | Federal learning method oriented to non-independent same-distribution heterogeneous data | |
| CN117473558A (en) | Self-adaptive DPWGAN training method and system based on federal learning | |
| Yang et al. | Towards group fairness via semi-centralized adversarial training in federated learning | |
| Wang et al. | A survey of applications of deep learning in radio signal modulation recognition | |
| CN115510472B (en) | Multi-difference privacy protection method and system for cloud edge aggregation system | |
| Zi et al. | Steganography with convincing normal image from a joint generative adversarial framework | |
| CN111145096A (en) | Super-resolution image reconstruction method and system based on recursive extremely-deep network | |
| Bhatti et al. | A Robust Aggregation Approach for Heterogeneous Federated Learning | |
| CN114298319A (en) | Method and device for determining joint learning contribution value, electronic equipment and storage medium | |
| Le et al. | Generating high-fidelity cybersecurity data with generative adversarial networks | |
| Liu et al. | Enhancing Robustness Against Heterogeneity via Class-Difficulty Based Weights |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |