CN114021188A - Method and device for interactive security verification of federated learning protocol and electronic equipment - Google Patents
Method and device for interactive security verification of federated learning protocol and electronic equipment Download PDFInfo
- Publication number
- CN114021188A CN114021188A CN202111299904.5A CN202111299904A CN114021188A CN 114021188 A CN114021188 A CN 114021188A CN 202111299904 A CN202111299904 A CN 202111299904A CN 114021188 A CN114021188 A CN 114021188A
- Authority
- CN
- China
- Prior art keywords
- attack
- federal learning
- privacy
- current
- protocol
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 87
- 238000012795 verification Methods 0.000 title claims abstract description 84
- 230000002452 interceptive effect Effects 0.000 title claims abstract description 76
- 230000006399 behavior Effects 0.000 claims abstract description 92
- 230000008569 process Effects 0.000 claims abstract description 34
- 230000003993 interaction Effects 0.000 claims abstract description 31
- 238000004088 simulation Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 50
- 238000001514 detection method Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 6
- 238000011156 evaluation Methods 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000007123 defense Effects 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000002787 reinforcement Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012502 risk assessment Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Evolutionary Biology (AREA)
- Medical Informatics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application provides a method and a device for interactive security verification of a federated learning protocol and electronic equipment, wherein the method comprises the following steps: in the federal learning process, the privacy stealing attack behavior aiming at the current federal learning protocol is simulated; detecting the total attack income reached by the currently simulated privacy stealing attack behavior; and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior. According to the method provided by the scheme, the attack simulation is carried out on the current federal learning protocol, and the safety of the current federal learning protocol in the aspect of data interaction is determined according to the attack total income which can be achieved by the current simulated privacy stealing attack behavior, so that a foundation is laid for further improving the privacy safety of the federal learning participants.
Description
Technical Field
The present application relates to the field of network security technologies, and in particular, to a method and an apparatus for interactive security verification of a bang learning protocol, and an electronic device.
Background
Currently, federal learning provides a distributed deep learning protocol with privacy protection characteristics, and a large number of participants can be accommodated in the same specific deep learning task to be cooperatively trained.
In the prior art, in order to ensure the information security of each participant, privacy protection mechanisms such as multi-party security calculation, differential privacy and the like are generally introduced into a federal learning protocol to resist privacy stealing attacks.
However, the performance of these privacy protection mechanisms in practical applications cannot be verified at present, and the security of the current federal learning protocol in terms of data interaction cannot be determined.
Disclosure of Invention
The application provides a method and a device for interactive security verification of a federated learning protocol and electronic equipment, which are used for solving the defects that the prior art cannot determine the security of the current federated learning protocol in the aspect of data interaction and the like.
A first aspect of the present application provides a method for interactive security verification of a federated learning protocol, including:
in the federal learning process, the privacy stealing attack behavior aiming at the current federal learning protocol is simulated;
detecting the total attack income reached by the currently simulated privacy stealing attack behavior;
and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
Optionally, the simulating privacy stealing attack behavior aiming at the current federal learning protocol includes:
respectively establishing a shadow model and a privacy stealing attack model corresponding to the federal learning model to be attacked according to the application scene of the current federal learning protocol;
training the shadow model by using a preset shadow sample;
training the privacy stealing attack model by using the output result of the trained shadow model to obtain a target privacy stealing attack model;
and simulating privacy stealing attack behaviors based on the target privacy stealing attack model.
Optionally, the detecting the total attack revenue achieved by the currently simulated privacy stealing attack behavior includes:
monitoring attack profits obtained by the current simulated privacy stealing attack behaviors at different execution stages;
and determining the total attack profit achieved by the privacy stealing attack behavior according to the attack profits obtained by the privacy stealing attack behavior in different execution stages.
Optionally, the detecting the total attack revenue achieved by the currently simulated privacy stealing attack behavior includes:
acquiring a reconstructed sample obtained by the current simulated privacy stealing attack behavior;
and determining the total attack income reached by the current simulated privacy stealing attack behavior according to the similarity between the reconstructed sample and the actual training sample of the attacked federated learning model.
Optionally, the determining, according to the total attack revenue achieved by the privacy stealing attack behavior, an interactive security verification result of the current federal learning protocol includes:
calculating the interaction safety coefficient of the current federal learning protocol according to the attack total income achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol;
and determining the interactive safety verification result according to the interactive safety coefficient of the current federal learning protocol.
Optionally, the determining the interactive security verification result according to the interactive security factor of the current federal learning protocol includes:
and determining the current federal learned safety level according to a preset safety level division rule and the interaction safety factor of the current federal learned protocol.
Optionally, the method further includes:
according to the interactive security verification result of the current federal learning protocol, determining the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of a local federal learning model of each participant;
and generating an interactive security verification report of the current federal learning protocol according to the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of the local federal learning model of each participant.
A second aspect of the present application provides a federated learning protocol interactive security verification apparatus, including:
the simulation module is used for simulating privacy stealing attack behaviors aiming at the current federal learning protocol in the federal learning process;
the detection module is used for detecting the total attack income reached by the currently simulated privacy stealing attack behavior;
and the verification module is used for determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
Optionally, the simulation module is specifically configured to:
respectively establishing a shadow model and a privacy stealing attack model corresponding to the federal learning model to be attacked according to the application scene of the current federal learning protocol;
training the shadow model by using a preset shadow sample;
training the privacy stealing attack model by using the output result of the trained shadow model to obtain a target privacy stealing attack model;
and simulating privacy stealing attack behaviors based on the target privacy stealing attack model.
Optionally, the detection module is specifically configured to:
monitoring attack profits obtained by the current simulated privacy stealing attack behaviors at different execution stages;
and determining the total attack profit achieved by the privacy stealing attack behavior according to the attack profits obtained by the privacy stealing attack behavior in different execution stages.
Optionally, the detection module is specifically configured to:
acquiring a reconstructed sample obtained by the current simulated privacy stealing attack behavior;
and determining the total attack income reached by the current simulated privacy stealing attack behavior according to the similarity between the reconstructed sample and the actual training sample of the attacked federated learning model.
Optionally, the verification module is specifically configured to:
calculating the interaction safety coefficient of the current federal learning protocol according to the attack total income achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol;
and determining the interactive safety verification result according to the interactive safety coefficient of the current federal learning protocol.
Optionally, the verification module is specifically configured to:
and determining the current federal learned safety level according to a preset safety level division rule and the interaction safety factor of the current federal learned protocol.
Optionally, the verification module is further configured to:
according to the interactive security verification result of the current federal learning protocol, determining the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of a local federal learning model of each participant;
and generating an interactive security verification report of the current federal learning protocol according to the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of the local federal learning model of each participant.
A third aspect of the present application provides an electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the method as set forth in the first aspect above and in various possible designs of the first aspect.
A fourth aspect of the present application provides a computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, implement a method as set forth in the first aspect and various possible designs of the first aspect.
This application technical scheme has following advantage:
the application provides a method and a device for interactive security verification of a federated learning protocol and electronic equipment, wherein the method comprises the following steps: in the federal learning process, the privacy stealing attack behavior aiming at the current federal learning protocol is simulated; detecting the total attack income reached by the currently simulated privacy stealing attack behavior; and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior. According to the method provided by the scheme, the attack simulation is carried out on the current federal learning protocol, and the safety of the current federal learning protocol in the aspect of data interaction is determined according to the attack total income which can be achieved by the current simulated privacy stealing attack behavior, so that a foundation is laid for further improving the privacy safety of the federal learning participants.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art according to these drawings.
Fig. 1 is a schematic structural diagram of a federal learning protocol interactive security verification system based on an embodiment of the present application;
fig. 2 is a schematic flowchart of a federated learning protocol interactive security verification method provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of an exemplary electric power federal learning agreement interactive security verification system provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a federal learning protocol interactive security verification apparatus provided in an embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
With the above figures, there are shown specific embodiments of the present application, which will be described in more detail below. These drawings and written description are not intended to limit the scope of the disclosed concepts in any way, but rather to illustrate the concepts of the disclosure to those skilled in the art by reference to specific embodiments.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following examples, "plurality" means two or more unless specifically limited otherwise.
In the prior art, in order to ensure the information security of each participant, privacy protection mechanisms such as multi-party security calculation, differential privacy and the like are generally introduced into a federal learning protocol to resist privacy stealing attacks. However, the performance of these privacy protection mechanisms in practical applications cannot be verified at present, and the security of the current federal learning protocol in terms of data interaction cannot be determined.
Aiming at the problems, the federal learning protocol interactive security verification method, the apparatus and the electronic device provided by the embodiment of the application simulate the privacy stealing attack behavior aiming at the current federal learning protocol in the federal learning process; detecting the total attack income reached by the currently simulated privacy stealing attack behavior; and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior. According to the method provided by the scheme, the attack simulation is carried out on the current federal learning protocol, and the safety of the current federal learning protocol in the aspect of data interaction is determined according to the attack total income which can be achieved by the current simulated privacy stealing attack behavior, so that a foundation is laid for further improving the privacy safety of the federal learning participants.
The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
First, a structure of a federal learning protocol interactive security verification system on which the present application is based will be described:
the method, the device and the electronic equipment for verifying the interaction safety of the federal learning protocol are suitable for verifying the safety of the federal learning protocol in the aspect of data interaction. As shown in fig. 1, the schematic structural diagram of the federal learning protocol interactive security verification system according to the embodiment of the present application mainly includes a plurality of federal learning models and federal learning protocol interactive security verification apparatuses. Specifically, the federal learning protocol interactive security verification device simulates privacy stealing attack behaviors aiming at the current federal learning protocol in the process that the federal learning models participate in federal learning, and then determines the security of the current federal learning protocol in the aspect of data interaction according to the defense capacity of the federal learning models for the privacy stealing attack behaviors simulated by the federal learning models.
The embodiment of the application provides a federated learning protocol interaction security verification method, which is used for verifying the security of a federated learning protocol in the aspect of data interaction. The execution subject of the embodiment of the application is an electronic device, such as a server, a desktop computer, a notebook computer, a tablet computer, and other electronic devices that can be used for verifying the security of the federal learning protocol in terms of data interaction.
As shown in fig. 2, a schematic flow chart of a federal learning protocol interaction security verification method provided in an embodiment of the present application is shown, where the method includes:
Specifically, a special privacy stealing attack library can be established based on typical privacy stealing attacks with great threat in recent years and combined with a federal learning scene, so as to prepare for subsequent simulation of privacy stealing attack behaviors aiming at the current federal learning protocol.
The privacy stealing attack behaviors simulated by the embodiment of the application can be reasoning attacks, reverse attacks and the like, and the reasoning attacks mainly comprise member reasoning attacks and characteristic reasoning attacks. The membership inference attack refers to that for a given data record (namely a single training sample), an attacker judges whether the data record is in training samples of other clients on the premise of granting the access right of the part of the federal learning model. The characteristic inference attack is to determine whether training samples of other clients (participants) have a certain special attribute (such as gender, age, income and the like) given by the characteristic inference attack.
One of the representatives of the inverse attack is the model inversion attack. The reverse attack is that an attacker carries out reverse analysis on an output result and model parameters of the federated learning model by constructing an attack model, and regenerates unknown training samples under the condition that private training samples of a client are not disclosed to the outside, so that the aim of stealing the privacy of the client is fulfilled. The reverse attack indicates that information can flow not only from the training samples to the federated learning model, but also from the federated learning model back to the training samples.
The inference attack and the reverse attack are both targeted to private training samples of a client, and the main difference between the two major types of attacks is that the inference attack infers the privacy information of a certain client by judging whether a single training sample or a single special attribute exists, while the reverse attack steals the sensitive information in the training sample of the client by analyzing the statistical distribution of the training sample of the client.
It should be noted that attack revenue is an important index for an attacker to measure the privacy stealing attack effect, and the attack revenue can also be one of the dimensions for evaluating the federal learning interactive security. An ideal federal learning protocol should be able to defend against a variety of privacy stealing attacks so that the total attack revenue of the attacker is as small as possible.
Specifically, the attack effect that can be achieved by the current simulated privacy stealing attack behavior can be evaluated from the perspective of an attacker, and the total attack revenue that can be obtained by the attacker can be evaluated.
And step 203, determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
Specifically, the defense capability of the current federal learning protocol in the aspect of privacy stealing attack can be determined according to the total attack revenue achieved by the current simulated privacy stealing attack behavior. In order to obtain a multi-dimensional interactive security verification result, a plurality of privacy stealing attack behaviors can be simulated for the current federal learning protocol, and the interactive security verification result of the current federal learning protocol is determined according to the attack total income achieved by the different privacy stealing attack behaviors.
On the basis of the above embodiment, in order to improve the simulation effect of the privacy stealing attack behavior, as an implementable manner, in an embodiment, the simulating the privacy stealing attack behavior aiming at the current federal learning protocol includes:
step 2011, according to the application scenario of the current federal learning protocol, a shadow model and a privacy stealing attack model corresponding to the federal learning model to be attacked are respectively established;
step 2012, training the shadow model by using a preset shadow sample;
step 2013, training a privacy stealing attack model by using the output result of the trained shadow model to obtain a target privacy stealing attack model;
step 2014, simulating privacy stealing attack behavior based on the target privacy stealing attack model.
For example, taking member inference attack as an example, firstly, from the perspective of an attacker, an attack target (to-be-attacked federal learning model), an attack scenario and how to implement the attack, that is, an application scenario of a current federal learning protocol is evaluated, and then an attack is launched.
For member reasoning attack, the attacker aims to deduce whether a certain data record exists in a training sample of the federal learning model, so as to obtain the membership information corresponding to the data record. For example, an attacker may wish to know whether revenue data for a participant is for a property analysis model for a business; if so, the attacker can infer the privacy information of the participant. In the membership inference attack, the data present in the Federal learning model training sample is referred to as membership data Dm(member data), but the non-existing data is referred to as non-member data Dn(non-member data)。
The member reasoning attack essentially needs to generate a two-classifier MIAI.e. the attacker needs to judge a certain dataRecord D exists in Member data DmWhether it is non-member data Dn. In other words, the attacker needs to determine whether a certain data record D exists in the member data DmOf the cell.
The classic method for realizing member reasoning attack in the existing research work is that an attacker constructs a shadow model Ms(shadow model), which is a deep learning model consistent with the structure of the target federated learning model (the federated learning model to be attacked). The attacker trains the shadow model by using data which is the same as the target data (training data of the federal learning model to be attacked) in distribution but does not overlap with each other as training samples (also called shadow samples). On the basis, an attacker trains an attack model thereof by means of a shadow model, namely a two-classifier MIA. The attack model is also a deep learning model, and the target function used in the training process is as follows:
because the shadow sample and the target data are distributed the same, and the shadow model is consistent with the target federal learning structure, the shadow model trained by the attacker also has similar output results with the local federal learning model corresponding to the target. By deducing the output result of a certain data record in the shadow model, the possibility that the data record exists in the target training sample can be obtained:
Pr(d∈Dm)＝MIA(d)
specifically, the threshold ψ may be set in the embodiment of the present application. When probability value MIA(d)>Psi, indicating that the attacker considers the data record d to belong to the member data; otherwise, the data belongs to the non-member data.
For member reasoning attack, the intuitive evaluation indexes of the attack effect are judgment accuracy (precision) and recall (recall) of the attack model. Wherein, note DIIf the record set of the member data is judged to be the attacker, the accuracy rate refers to the data record set DIIn (D) indeed belong to a memberThe proportion of the data is recorded, and the recall rate refers to the percentage of all member data sets DmIn the middle, the attacker judges the proportion of correct records. The accuracy rate is high, and the investigation is accurate, namely, an attacker requires accurate judgment; and the recall ratio is strong, namely the attacker requires to reduce omission. The closer the accuracy rate and the recall rate are to 1, the better the member reasoning attack effect is.
On the basis of the foregoing embodiment, as an implementable manner, in an embodiment, detecting a total attack gain achieved by a currently simulated privacy stealing attack behavior includes:
step 2021, monitoring attack profits obtained by the current simulated privacy stealing attack behavior at different execution stages;
step 2022, determining the total attack revenue reached by the privacy stealing attack behavior according to the attack revenue obtained by the privacy stealing attack behavior in different execution stages.
It should be noted that, even the same attack means (such as member reasoning attack) may be different in actual operation for different attack scenarios, because the attack means is standing in the perspective of the attacker, and it is necessary to adopt an optimal attack strategy to exploit the privacy information of other participants as much as possible. The privacy stealing attack library constructed in the embodiment of the application comprises a plurality of attack strategies which respectively correspond to different attack scenes so as to facilitate comprehensive evaluation of the interaction security of the subsequent power federal learning protocol.
Specifically, in the embodiments of the present application, by referring to the idea of Reinforcement Learning (Reinforcement Learning), in order to maximize the ability of the Federal Learning protocol to resist privacy theft attacks, it may be assumed that an attacker is "rational", which can take specific attack actions based on the actual environment to obtain the maximum expected benefit (attack profit).
The method and the device abstract the process of an attacker starting privacy stealing attack into a Markov Decision Process (MDP). The markov decision process is a process that an attacker adopts a certain attack strategy, so that the state of the attacker is changed to obtain attack income and the attacker interacts and circulates with the environment (the federal learning protocol). Several basic concepts in the markov decision process are as follows:
(1) s { S0, S1, …, sN } represents a set of states of an attacker, which starts from an initial state and transitions to a new state according to a change in an attack policy;
(2) a ═ { a0, a1, …, aN } represents a set of attack strategies, and aN attacker can adjust the attack strategies according to different environments and obtained attack gains;
(3) p (s '| s, a) represents the probability of transition to state s' after an attacker takes attack strategy a in state s;
(4) g(s) represents the attack gain obtained by the attacker in the state s;
when an attacker takes an attack strategy a to start an attack from one state s to another state s', some reward or punishment should be given to the attacker to calculate the attack income G(s), and the attack total income G of the attacker can be obtained until the federal learning process is finished, wherein the calculation formula is as follows:
G＝G(s0)+γG(s1)+γ2G(s2)+…+γNG(sN)
wherein γ represents a predetermined constant coefficient, G(s)i) The attack gains of the attacker in different execution stages are shown, the optimization goal of the attacker is to obtain the maximum attack gains, namely, the optimal attack strategy a is selected each time in a mode of automatically adjusting attack parameters.
Similarly, in an embodiment, a reconstructed sample obtained by a currently simulated privacy stealing attack behavior can be obtained; and determining the total attack income reached by the current simulated privacy stealing attack behavior according to the similarity between the reconstructed sample and the actual training sample of the attacked federated learning model.
Specifically, the attack profit G is different according to different attack strategy calculation modes, specifically, the attack profit is an index for measuring the success rate of privacy stealing attack, and is directly related to the attack target of each attack mode. For member reasoning attack, the attack gain G is F1 value; for the model inversion attack, the attack profit is the structural similarity SSIM between the reconstructed sample y generated by the attacker and the actual training sample x of the client (attacked federated learning model). The calculation formula is as follows, where precision and recall are the accuracy and recall of the attack model, respectively, and μ and σ represent the mean and variance, respectively, c1And c2Are two fixed constants.
On the basis of the above embodiment, as an implementable manner, in an embodiment, determining an interactive security verification result of a current federal learning protocol according to an attack total revenue achieved by privacy stealing attack behavior includes:
step 2031, calculating an interaction safety factor of the current federal learning protocol according to the attack total income achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol;
step 2032, determining an interactive security verification result according to the interactive security coefficient of the current federal learning protocol.
Specifically, consideration can be given from the aspects of data privacy Y, security threat T, system vulnerability V, and the like with reference to evaluation indexes in risk analysis, wherein the data privacy Y, the security threat T, and the system vulnerability V can be determined by analyzing the attack total revenue achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol. The data privacy Y is an index for measuring the data sensitivity of the participants and can be specifically divided into categories such as property privacy, position privacy and physiological privacy, and the definition and assignment of the data privacy are determined according to different business application scenes and the privacy requirements of users; the security threat T is a potential safety hazard caused by privacy stealing attack on participants, and comprises attributes of a threat main body (attacker), an influence object (victim), occurrence probability, implementation difficulty and the like; the system vulnerability V is used for representing the threat degree of the Federal learning protocol under privacy stealing attack, and the more vulnerable the system is, the more easily the electric power Federal learning protocol is broken by attackers.
Specifically, the method and the device for determining the data privacy include the following steps of judging the possibility of an attack event according to a security threat and the difficulty level of the threat by using the vulnerability of a system, taking a leakage value of the data privacy as privacy loss according to the security threat, and defining an interaction safety coefficient ζ as follows:
ζ＝Risk(Y,T,V)＝Risk(Pr(T,V),Loss(T,Y))
risk is a security Risk calculation function of privacy stealing attack, Pr (T, V) represents the probability of occurrence of a security threat T caused by system vulnerability V, and loss (Y) represents the loss generated after the security threat T causes data privacy Y to be disclosed.
Specifically, in an embodiment, the current federal learned safety level may be determined according to a preset safety level classification rule and an interaction safety factor of the current federal learned protocol.
The security level classification rule may be set according to actual requirements, and the embodiment of the present application is not limited.
Specifically, in one embodiment, in federated learning, the assessment of its availability varies for different learning tasks. Specifically, for the most common classification detection task, the usability index is usually accuracy (accuracy). Accuracy is defined as the number of samples | X that are classified correctlycThe ratio of | to the total number of samples | X |. In actual use, the accuracy of the test sample is generally used as an evaluation criterion for the usability of the model. The higher the accuracy, the higher the model availability, and the better the learning result.
Specifically, in an embodiment, the privacy disclosure degree of each participant of the current federal learning protocol and the safety degree of the local federal learning model of each participant can be determined according to the interactive safety verification result of the current federal learning protocol; and generating an interactive security verification report of the current federal learning protocol according to the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of the local federal learning model of each participant.
Specifically, according to simulation attack experiment results in different federal learning scenes, privacy disclosure risk indexes are evaluated in multiple dimensions such as actual attack income of an attacker, data privacy disclosure degree of a participant, vulnerability (safety degree) of a federal learning model and the like, and an interactive safety verification report of a current federal learning protocol is automatically generated.
Wherein the interactive security verification report may include the following:
(1) data privacy list: the method comprises data privacy classification, belonged clients, privacy sensitivity degree and the like;
(2) list of security threats: the method comprises the steps of threat name, threat category, threat motivation, threat subject, threat object, occurrence frequency, implementation difficulty and the like;
(3) list of system vulnerabilities: vulnerability and severity, etc. including the federal learning process;
(4) list of existing security measures: the method comprises a protection and defense scheme, an implementation effect and the like adopted by a federal learning protocol;
(5) interactive security assessment program record: the method comprises the steps of model availability, interaction safety factor, attack income and attack strategy of an attacker, data transmission in an interaction process and the like;
(6) safety threat handling recommendation: the threat level is clarified for unacceptable privacy disclosure threats in the interactive security assessment results and appropriate processing recommendations are given based on their actual performance in the automated verification process.
The contents contained in the above (1) to (6) and not mentioned in the above embodiments may be determined by conventional data analysis, or may be determined by other means, and the embodiments of the present application are not limited specifically.
Specifically, as shown in fig. 3, an execution subject of the exemplary power federal learning protocol interactive security verification system provided in the embodiment of the present application is an automatic verification tool, a constructed privacy stealing attack library includes a plurality of attack strategies, which respectively correspond to different attack scenarios, and then, based on the attack strategies in the privacy stealing attack library, the attack scenarios, that is, the privacy stealing attack behaviors are simulated, and finally, a federal learning protocol security analysis report (an interactive security verification report of the federal learning protocol) is generated according to the attack effect of the simulated privacy stealing attack behaviors.
The federal learning protocol interactive security verification method provided by the embodiment of the application is executed in the process of federal learning, and can also simulate the process of federal learning training for verifying the security of the federal learning protocol in the aspect of data interaction. Unlike traditional deep learning modeling, federated learning can be viewed as a distributed model training scenario based on training samples. In the federal learning process, all training samples do not need to be collected together for model training, but a plurality of mutually independent clients use respective private training samples to train a local model and then return model parameters which need to be updated to a power big data center server; the electric power big data center server aggregates the model parameters returned by the clients to update the federal learning model parameters, and then feeds the latest model parameters back to each client.
In the process, each client side has the same and complete local model, direct communication is not involved among the client sides, and model parameters are exchanged only by relying on the power big data center server. After the federal learning process is finished, each client can also independently perform inference prediction tasks, such as: judging the category of the object in the picture, deducing the next predicted word of the input method, and the like.
In the process of federal learning, a process of training a local model once by using a private training sample by a client is defined as 'one-round training', and the specific training process of federal learning is as follows:
step S101. client Ci(i 1, 2.., n) (n is the total number of clients participating in the federal learning) downloads the federal learning model parameters from the big power data center server S
As initialization parameters of its local model
Namely, it is
Step S102. each client CiPerforming a round of training, i.e. using the private training samples DiTraining respective local model, and updating local model parameters by gradient descent algorithm
(t represents the current training round number) and uploading the training round number to the big electric power data center server S, wherein the formula of the gradient descent algorithm is as follows:
wherein, eta is the learning rate,
representing gradient solving, f is a local model.
S103, the electric power big data center server S aggregates the model parameters uploaded by each client by using an aggregation algorithm AGG specified in an electric power federal learning protocol, and calculates to obtain the next round of federal learning model parametersI.e. by
Step S104, the power big data center server S returns the updated Federal learning model parameters to each client, and the client updates the local model parameters according to the updated Federal learning model parameters, namely
S105, repeating the steps S102 to 104 when the training round number is smaller than the set total training round number T, namely T is less than or equal to T; otherwise, the federal learning process ends.
According to the federal learning protocol interactive security verification method provided by the embodiment of the application, privacy stealing attack behaviors aiming at the current federal learning protocol are simulated in the federal learning process; detecting the total attack income reached by the currently simulated privacy stealing attack behavior; and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior. The method is characterized in that the attack simulation is carried out on the current federal learning protocol, the safety of the current federal learning protocol in the aspect of data interaction is determined according to the attack total income which can be achieved by the current simulated privacy stealing attack behavior, and a foundation is laid for further improving the privacy safety of the federal learning participants. In addition, the method provided by the embodiment of the application carries out automatic and comprehensive evaluation and verification on the safety and the usability of the federal learning model, comprehensively considers the security loophole and the privacy threat of each link in the process of the federal learning protocol, verifies the practicability of the federal learning model through a specific application scene, and provides a detailed analysis report, thereby providing an important reference basis for the use of the federal learning protocol in the ground in the real world.
The embodiment of the application provides a federated learning protocol interactive security verification device, which is used for executing the federated learning protocol interactive security verification method provided by the embodiment.
Fig. 4 is a schematic structural diagram of an interactive security verification apparatus for federal learning protocol provided in an embodiment of the present application. The federal learning protocol interactive security verification apparatus 40 includes a simulation module 401, a detection module 402, and a verification module 403.
The system comprises a simulation module, a monitoring module and a monitoring module, wherein the simulation module is used for simulating privacy stealing attack behaviors aiming at a current federal learning protocol in the federal learning process; the detection module is used for detecting the total attack income reached by the currently simulated privacy stealing attack behavior; and the verification module is used for determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
Specifically, in an embodiment, the simulation module is specifically configured to:
respectively establishing a shadow model and a privacy stealing attack model corresponding to the federal learning model to be attacked according to the application scene of the current federal learning protocol;
training the shadow model by using a preset shadow sample;
training a privacy stealing attack model by using the output result of the trained shadow model to obtain a target privacy stealing attack model;
and simulating privacy stealing attack behaviors based on the target privacy stealing attack model.
Specifically, in an embodiment, the detection module is specifically configured to:
monitoring attack profits obtained by the current simulated privacy stealing attack behaviors at different execution stages;
and determining the total attack profit achieved by the privacy stealing attack behavior according to the attack profits obtained by the privacy stealing attack behavior in different execution stages.
Specifically, in an embodiment, the detection module is specifically configured to:
acquiring a reconstructed sample obtained by the current simulated privacy stealing attack behavior;
and determining the total attack income reached by the current simulated privacy stealing attack behavior according to the similarity between the reconstructed sample and the actual training sample of the attacked federated learning model.
Specifically, in an embodiment, the verification module is specifically configured to:
calculating the interaction safety coefficient of the current federal learning protocol according to the attack total income achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol;
and determining an interactive safety verification result according to the interactive safety factor of the current federal learning protocol.
Specifically, in an embodiment, the verification module is specifically configured to:
and determining the current federal learning safety level according to a preset safety level division rule and the interaction safety factor of the current federal learning protocol.
Specifically, in an embodiment, the verification module is further configured to:
according to the interactive security verification result of the current federal learning protocol, determining the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of a local federal learning model of each participant;
and generating an interactive security verification report of the current federal learning protocol according to the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of the local federal learning model of each participant.
With regard to the federal learning protocol interactive security verification apparatus in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment related to the method, and will not be described in detail here.
The federal learning protocol interactive security verification device provided in the embodiment of the application is used for executing the federal learning protocol interactive security verification method provided in the embodiment, and the implementation manner and the principle of the device are the same, and are not described again.
The embodiment of the application provides electronic equipment, which is used for executing the federal learning protocol interactive security verification method provided by the embodiment.
Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. The electronic device 50 includes: at least one processor 51 and memory 52;
the memory stores computer-executable instructions; the at least one processor executes the memory-stored computer-executable instructions to cause the at least one processor to perform the federal learned protocol interactive security verification method as provided by the above embodiments.
The electronic device provided in the embodiment of the present application is configured to execute the federal learning protocol interaction security verification method provided in the above embodiment, and an implementation manner and a principle thereof are the same and are not described again.
The embodiment of the present application provides a computer-readable storage medium, where a computer execution instruction is stored in the computer-readable storage medium, and when a processor executes the computer execution instruction, the method for interactive security verification of a federal learning protocol provided in any of the above embodiments is implemented.
The storage medium containing the computer executable instructions in the embodiment of the present application may be used to store the computer executable instructions of the federated learning protocol interaction security verification method provided in the foregoing embodiment, and an implementation manner and a principle thereof are the same, and are not described again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.
Claims (10)
1. A method for interactive security verification of a federated learning protocol is characterized by comprising the following steps:
in the federal learning process, the privacy stealing attack behavior aiming at the current federal learning protocol is simulated;
detecting the total attack income reached by the currently simulated privacy stealing attack behavior;
and determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
2. The method of claim 1, wherein simulating privacy stealing attack behavior against current federal learning protocols comprises:
respectively establishing a shadow model and a privacy stealing attack model corresponding to the federal learning model to be attacked according to the application scene of the current federal learning protocol;
training the shadow model by using a preset shadow sample;
training the privacy stealing attack model by using the output result of the trained shadow model to obtain a target privacy stealing attack model;
and simulating privacy stealing attack behaviors based on the target privacy stealing attack model.
3. The method of claim 1, wherein detecting the total attack gain achieved by the currently simulated privacy stealing attack behavior comprises:
monitoring attack profits obtained by the current simulated privacy stealing attack behaviors at different execution stages;
and determining the total attack profit achieved by the privacy stealing attack behavior according to the attack profits obtained by the privacy stealing attack behavior in different execution stages.
4. The method of claim 1, wherein detecting the total attack gain achieved by the currently simulated privacy stealing attack behavior comprises:
acquiring a reconstructed sample obtained by the current simulated privacy stealing attack behavior;
and determining the total attack income reached by the current simulated privacy stealing attack behavior according to the similarity between the reconstructed sample and the actual training sample of the attacked federated learning model.
5. The method of claim 1, wherein determining the interactive security verification result of the current federated learning protocol according to the total attack revenue achieved by the privacy stealing attack behavior comprises:
calculating the interaction safety coefficient of the current federal learning protocol according to the attack total income achieved by the privacy stealing attack behavior and the attribute information of the current federal learning protocol;
and determining the interactive safety verification result according to the interactive safety coefficient of the current federal learning protocol.
6. The method according to claim 5, wherein determining the interactive security verification result according to the interactive security factor of the current federal learning protocol comprises:
and determining the current federal learned safety level according to a preset safety level division rule and the interaction safety factor of the current federal learned protocol.
7. The method of claim 1, further comprising:
according to the interactive security verification result of the current federal learning protocol, determining the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of a local federal learning model of each participant;
and generating an interactive security verification report of the current federal learning protocol according to the privacy disclosure degree of each participant of the current federal learning protocol and the security degree of the local federal learning model of each participant.
8. The utility model provides a mutual safety verification device of bang's study protocol which characterized in that includes:
the simulation module is used for simulating privacy stealing attack behaviors aiming at the current federal learning protocol in the federal learning process;
the detection module is used for detecting the total attack income reached by the currently simulated privacy stealing attack behavior;
and the verification module is used for determining the interactive security verification result of the current federal learning protocol according to the total attack income achieved by the privacy stealing attack behavior.
9. An electronic device, comprising: at least one processor and memory;
the memory stores computer-executable instructions;
the at least one processor executing the computer-executable instructions stored by the memory causes the at least one processor to perform the method of any of claims 1-7.
10. A computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a processor, implement the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111299904.5A CN114021188A (en) | 2021-11-04 | 2021-11-04 | Method and device for interactive security verification of federated learning protocol and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111299904.5A CN114021188A (en) | 2021-11-04 | 2021-11-04 | Method and device for interactive security verification of federated learning protocol and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114021188A true CN114021188A (en) | 2022-02-08 |
Family
ID=80060711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111299904.5A Pending CN114021188A (en) | 2021-11-04 | 2021-11-04 | Method and device for interactive security verification of federated learning protocol and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114021188A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114553533A (en) * | 2022-02-22 | 2022-05-27 | 中国电子信息产业集团有限公司第六研究所 | Protocol vulnerability evaluating method, device and storage medium |
| CN115600250A (en) * | 2022-12-12 | 2023-01-13 | 阿里巴巴（中国）有限公司（Cn） | Data processing method, storage medium and electronic device |
| CN116010944A (en) * | 2023-03-24 | 2023-04-25 | 北京邮电大学 | Federal computing network protection method and related equipment |
-
2021
- 2021-11-04 CN CN202111299904.5A patent/CN114021188A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114553533A (en) * | 2022-02-22 | 2022-05-27 | 中国电子信息产业集团有限公司第六研究所 | Protocol vulnerability evaluating method, device and storage medium |
| CN114553533B (en) * | 2022-02-22 | 2024-03-01 | 中国电子信息产业集团有限公司第六研究所 | Protocol vulnerability evaluation method, device and storage medium |
| CN115600250A (en) * | 2022-12-12 | 2023-01-13 | 阿里巴巴（中国）有限公司（Cn） | Data processing method, storage medium and electronic device |
| CN115600250B (en) * | 2022-12-12 | 2023-03-21 | 阿里巴巴（中国）有限公司 | Data processing method, storage medium and electronic device |
| CN116010944A (en) * | 2023-03-24 | 2023-04-25 | 北京邮电大学 | Federal computing network protection method and related equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | The effect of sample size on the accuracy of species distribution models: considering both presences and pseudo‐absences or background sites | |
| CN109922032B (en) | Method, device, equipment and storage medium for determining risk of logging in account | |
| CN114021188A (en) | Method and device for interactive security verification of federated learning protocol and electronic equipment | |
| Mitchell et al. | Bayesian model selection with BAMM: effects of the model prior on the inferred number of diversification shifts | |
| CN112231570B (en) | Recommendation system support attack detection method, device, equipment and storage medium | |
| CN113947215A (en) | Federal learning management method and device, computer equipment and storage medium | |
| CN114819190A (en) | Model training method, device, system and storage medium based on federal learning | |
| CN115102705A (en) | Automatic network security detection method based on deep reinforcement learning | |
| CN114925854A (en) | Federal learning node selection method and system based on gradient similarity measurement | |
| CN116361759B (en) | Intelligent compliance control method based on quantitative authority guidance | |
| CN111957053A (en) | Game player matching method and device, storage medium and electronic equipment | |
| CN110213094B (en) | Method and device for establishing threat activity topological graph and storage equipment | |
| CN115834251A (en) | Hypergraph transform based threat hunting model establishing method | |
| CN116362894A (en) | Multi-objective learning method, multi-objective learning device, electronic equipment and computer readable storage medium | |
| CN113673811A (en) | Session-based online learning performance evaluation method and device | |
| CN112231571A (en) | Information data processing method, device, equipment and storage medium | |
| CN114417394A (en) | Block chain-based data storage method, device, equipment and readable storage medium | |
| CN114139601A (en) | Evaluation method and system for artificial intelligence algorithm model of power inspection scene | |
| CN110460569A (en) | The detection method and detection device of online access | |
| CN117150321B (en) | Equipment trust evaluation method and device, service equipment and storage medium | |
| CN117235584B (en) | Picture data classification method, device, electronic device and storage medium | |
| CN115865519B (en) | Data processing method and system suitable for network attack and defense virtual simulation | |
| CN116384502B (en) | Method, device, equipment and medium for calculating contribution of participant value in federal learning | |
| CN114844889B (en) | Video processing model updating method and device, electronic equipment and storage medium | |
| Lin et al. | An Evolutionary Game Theoretical Framework for Decision Fusion in the Presence of Byzantines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |