CN113850399A - Prediction confidence sequence-based federal learning member inference method - Google Patents

Prediction confidence sequence-based federal learning member inference method Download PDF

Info

Publication number
CN113850399A
CN113850399A CN202111200539.8A CN202111200539A CN113850399A CN 113850399 A CN113850399 A CN 113850399A CN 202111200539 A CN202111200539 A CN 202111200539A CN 113850399 A CN113850399 A CN 113850399A
Authority
CN
China
Prior art keywords
model
attacker
training
federal learning
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111200539.8A
Other languages
Chinese (zh)
Inventor
白跃彬
顾育豪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202111200539.8A priority Critical patent/CN113850399A/en
Publication of CN113850399A publication Critical patent/CN113850399A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
    • G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G06N5/041—Abduction

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Bioethics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer And Data Communications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides a prediction confidence sequence-based federal learning member inference method, and belongs to the field of artificial intelligence safety. The method aims at disclosing privacy risks existing in federal learning, and solves the technical problem of how attackers with different roles can deduce whether a given data sample is a member of a training set of a participant or not in a federal learning scene. The invention establishes an attack model to learn the difference by utilizing the difference of the prediction confidence sequence of the training data and the test data of the participants in the federal learning, thereby implementing member inference. Generating a prediction confidence sequence sample with a label through a shadow model and a data set owned by an attacker, and training an attack model; for a given target sample, a target model is used for generating a corresponding prediction confidence sequence, and member information of the target sample is deduced by a trained attack model. The member inference method provided by the invention supports the malicious party or the malicious server to steal the training data privacy of the normal party.

Description

Prediction confidence sequence-based federal learning member inference method
Technical Field
The invention relates to the field of artificial intelligence safety, in particular to a prediction confidence sequence-based federal learning member inference method.
Background
With the appearance of data islanding and the attention of personal privacy protection, the application mode of centralized learning is restricted, and the federal learning is used as a distributed machine learning framework, so that model training can be completed on the premise of not revealing user data, and attention is paid from the beginning of birth. The framework of federal learning includes two types of roles: a plurality of participants and an aggregation server. Each participant has a data set with complete data characteristics, and data samples owned by different participants have no intersection or little intersection, and can be combined to train a global model, and the specific training process is as follows: federal learning involves multiple iterations. In each iteration, the aggregation server sends the current global model to the participants, and the participants use the local data to train the global model, calculate new local model parameters and send the new local model parameters to the server. And the server aggregates the received local model parameters of the participants to form a new global model for the next iteration. In the iterative training process, the training data is always stored in the local of the participant, and the participant and the server only exchange model parameters, so that the data privacy of the participant is ensured. However, the model parameters exchanged in the communication process still have the problem of privacy disclosure, and various privacy attacks including member inference seriously damage the confidentiality of federal learning.
In federal learning, a member inference attack is a member of a training data set used to infer whether a given target data sample is a normal participant, and is essentially a binary problem, where a data sample is input and an inference tag of 1 or 0 is output, where 1 indicates that the data sample is a member of the training set of participants and 0 indicates that the sample is not a member. Existing member inference attacks for federal learning can be divided into two categories depending on the capabilities of the attacker: black box attacks and white box attacks. The black box attack means that an attacker can only submit data records to a target model and obtain an output result of the model, but cannot know other information of the target model, so that differences between training data and test data output are generally inferred by using the model. In the white-box attack, an attacker can obtain the structure, parameters and intermediate operation results of the model, and can deduce by using the gradient and intermediate result difference of training and testing data at each layer of the target model. However, existing member inference attacks cannot be applied to different application scenarios of federal learning. When an attacker is a participant in federal learning, the existing black box attacks are less effective because the aggregation algorithm of federal learning reduces the contribution of training data of each participant; the existing white-box attack can only take effect in the federal learning application containing a small number of participants and cannot be applied to scenes of dozens of participants.
Disclosure of Invention
The invention provides a prediction confidence sequence-based federal learning member inference method, and different attack methods are designed aiming at the role of an attacker in a federal learning system, which can be divided into local member inference based on the prediction confidence sequence and global member inference based on the prediction confidence sequence. Wherein local member inference based on the prediction confidence sequence is conducted by a malicious participant, the attacker being one of the federal learning participants; while the global membership inference based on the prediction confidence sequence is initiated by a malicious server, the attacker is a federated learned aggregation server. The member inference method provided by the invention supports different types of attackers to infer the privacy data of the normal participants in the federated learning environment, thereby revealing the privacy risk existing in federated learning.
The invention provides a prediction confidence sequence-based federal learning member inference method, which comprises the following steps:
(1) before the start of federal learning, an attacker divides the own data set into two parts: a member subset and a non-member subset;
(2) in each iteration of the federal learning, an attacker integrates member subsets into the training of the federal learning, saves the shadow model and the target model of the iteration of the current iteration and participates in the training of the global model at the same time;
(3) after the federal learning is finished, an attacker selects a member to deduce the structure of an attack model, a plurality of stored shadow models are utilized to generate a prediction confidence sequence for each sample of a member subset and a non-member subset, a corresponding label 1 or 0 is marked, and the generated data is used for training the attack model;
(4) and the attacker generates a prediction confidence sequence for a given target sample by using the stored multiple target models, and delivers the prediction confidence sequence to the member inference attack model to infer whether the target sample is a member of the training set of the normal participant.
Further, the specific content of step (2) differs according to the role of the attacker, and the method comprises the following steps: for local member inference, an attacker firstly saves a copy of a received global model as a shadow model and a target model of the iteration, then trains the global model by using member subsets, calculates new local model parameters, and uploads the new local model parameters to a server. For global member inference, an attacker firstly trains a global model by using a member subset, the calculated local model is used as a shadow model, a model uploaded by a target participant is used as a target model, and then the attacker aggregates the local model and all received participant models to form a new global model and sends the new global model to the participants.
Further, the step (3) of training the member inference attack model specifically includes the following steps:
(3.1) selecting a two-classification neural network model by an attacker, determining a specific structure and a hyper-parameter of the model, and using the model as a member inference attack model;
(3.2) for each sample (x, y) of the member subset, the attacker respectively calculates the prediction confidence coefficient of each shadow model pair (x, y) in the shadow model sequence to form a prediction confidence coefficient sequence of (x, y), and a label 1 is marked to be used as a training sample of the attack model; for each sample of the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model to the sample, and marks a label 0 as a training sample of the attack model;
and (3.3) training the member inference model determined in the step (3.1) by an attacker by using the attack model training sample generated in the step (3.2).
Further, in step (4), the attacker calculates a prediction confidence sequence of the target sample in the target model, and takes the prediction confidence sequence as an input of the member inference model trained in step (3.3), so as to obtain a member inference result of the target sample: 1 or 0.
The invention has the advantages that:
(1) the invention can use different attack methods aiming at different roles of attackers in the federal learning system.
(2) The invention can obtain good attack effect in the federal learning application of dozens of participants.
Drawings
FIG. 1 is a schematic flow diagram of a federated learning membership inference method based on a predictive confidence sequence;
FIG. 2 is a schematic flow diagram of a global member inference method based on a prediction confidence sequence;
FIG. 3 shows the local membership inference effect on CIFAR-10, CIFAR-100, Purchase100, and Texas100 data sets;
FIG. 4 shows the global membership inference effect on CIFAR-10, CIFAR-100, Purchase100, and Texas100 datasets;
FIG. 5 is a comparison of the inference effect of the local members of the Purchase100 dataset under different total numbers of participants.
Detailed Description
The following detailed description of embodiments of the invention is made with reference to the accompanying drawings and specific examples:
the invention mainly utilizes two characteristics of federal learning: (1) the deep learning model has different prediction confidence degrees on training data and test data; (2) federated learning involves multiple rounds of iterative training, and attackers may be exposed to multiple versions of the model. As federal learning iterates, the prediction confidence of the training and test data will exhibit different trends. Although the prediction confidence of both will continue to rise, the magnitude of the rise in training data is greater. The invention deduces whether a given target sample is a member of a training set of normal participants by establishing the difference of the prediction confidence sequences of the learning training and testing data of an attack model.
The invention provides a prediction confidence sequence-based federal learning member inference method, and different attack methods are designed aiming at the role of an attacker in a federal learning system, which can be divided into local member inference based on the prediction confidence sequence and global member inference based on the prediction confidence sequence. Wherein local member inference based on the prediction confidence sequence is implemented by a malicious participant, and an attacker is one of the federal learning participants, can infer whether a certain data sample is a member of the training set of other participants, but cannot explicitly indicate which participant is specific. While global member inference based on the prediction confidence sequence is initiated by a malicious server, an attacker is an aggregation server for federal learning, and member inference can be implemented aiming at a specific target participant. As shown in fig. 1, the member inference method provided by the present invention specifically includes the following steps:
step 1, before the start of federal learning, an attacker divides a data set owned by the attacker into two parts: a member subset and a non-member subset. If the attacker is a participant in the federal learning system, the local private data can be directly divided; if the attacker is the aggregation server, an auxiliary data set needs to be collected in advance, and the auxiliary data set requires the same distribution as the training set of the participants. Each sample (x, y) in the dataset contains a sample feature x and a label y.
Step 2, in each iteration of the federal learning, an attacker needs to integrate member subsets into the training of the federal learning and save the shadow model M of the iteration of the current roundsAnd an object model MtAnd simultaneously participate in the training of the global model. Different operations are required to be executed according to the role of an attacker in the federal learning system, which specifically comprises the following steps:
if the attacker is a participant, the attacker receives the global model sent by the server in each iteration. An attacker firstly saves a copy of the global model as a shadow model and a target model of the iteration of the current round; and then, training the global model by using the member subset, calculating new local model parameters, and uploading the new local model parameters to a server. And the server forms a new global model after aggregating the model parameters uploaded by the participants and enters the next iteration.
If the attacker is the server, the attacker will receive the model parameters sent by the participants in each iteration. As shown in fig. 2, an attacker first trains the current global model with the member subsets to calculate local model parameters, and uses the local model as a shadow model of the current iteration and uses a model uploaded by the target participant as a target model. And then, the attacker aggregates the local model and all the received participant models to form a new global model, and sends the new global model to the participants to enter the next iteration.
The step 2 is a cyclic iteration process, and if the federate learning training process includes r iterations, the shadow model sequence and the target model sequence finally saved by the attacker respectively include r models.
And 3, after the federal learning is finished, selecting a two-classification neural network model by an attacker, determining a specific structure and hyper-parameters of the model, using the model as a member to infer the attack model, and starting entering a training stage. For each sample (x, y) of the member subset, the attacker respectively calculates the prediction confidence of each shadow model pair (x, y) in the shadow model sequence, wherein the ith shadow model
Figure BDA0003304755260000042
Confidence of prediction for (x, y), ConfiThe calculation method of (2) is as follows:
Figure BDA0003304755260000041
wherein
Figure BDA0003304755260000043
As model MiDeducing the posterior probability that the sample feature x belongs to the true label y,
Figure BDA0003304755260000044
as model MiThe posterior probability that the sample feature x belongs to class k is inferred. After the calculation is finished, the attacker obtains a prediction confidence coefficient sequence { Conf) of (x, y)1,Conf2,...,ConfrAnd labeled with 1 as a training sample for the attack model ({ Conf)1,Conf2,...,Confr},1). And for each sample of the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model to the sample and marks 0. The attacker collects the member subset and the non-member subsetAnd (4) taking all the prediction confidence sequence sets with the labels generated by the set as a training data set of the attack model to train the attack model.
And 4, after the training of the attack model is finished, the attacker can enter an inference phase. Given a target sample (x)t,yt) The attacker calculates each target model M in the target model sequence by adopting the method in the step 3t iTo (x)t,yt) Form (x) of the prediction confidence oft,yt) The sequence is used as an input of an attack model. Attack model Final output (x)t,yt) Member inference results of (1): 1 or 0, 1 represents (x)t,yt) Is a member of the training set of normal participants, 0 then means (x)t,yt) Are not members.
In order to fully illustrate the member inference method proposed by the present invention, the attack performance is evaluated by the following two indexes: member inferred accuracy and F1 score. And selecting an equal amount of data samples from the training set and the test set of the normal participants as a test data set of the member inference model, and calculating the member inference accuracy and the F1 score according to the inference result of the attack model on the test set. This example uses Python 3.7 and PyTorch 1.3.1 to implement a federal learned membership inference method based on a prediction confidence sequence. The configuration of the server running all experiments is as follows: intel Golden 6240CPU, 384GB memory and an NVIDIA V100 GPU with 32GB GDDR 5X.
The experimental scenario of this example employs CIFAR-10, CIFAR-100, Purchase100, and Texas100 datasets. For CIFAR-10 and CIFAR-100 data sets, the Alexnet model is adopted as a model for federal learning training in the embodiment; for the purchasse 100 and Texas100 data sets, the present embodiment is trained using a fully connected network containing four hidden layers, with 1024, 512, 256, and 128 cells, respectively. In addition, the embodiment constructs the member inference attack model by using a hidden layer with 64 units and a Sigmoid layer.
The present embodiment provides 5 participants to evaluate the local member inference effect and the global member inference effect of the above data set. In the local membership inference experiment, 1 participant was an attacker and the other 4 participants were normal users. This embodiment sets 50 federal learning iterations. As shown in FIG. 3, the local member inference accuracy on CIFAR-10, CIFAR-100, Purchase100 and Texas100 data sets reached 99.954%, 99.958%, 98.488% and 99.748%, respectively, and the F1 scores were 0.99954, 0.99958, 0.98495 and 0.99748, respectively. Experiments prove that the local member inference method based on the prediction confidence coefficient sequence can enable a malicious participant to effectively infer member records and non-member records in a target data set. The results of the global member inference experiments are shown in fig. 4, the accuracy rates of global member inference on the four data sets respectively reach 99.934%, 99.996%, 94.15% and 100%, the F1 scores are 0.99934, 0.9999, 0.93787 and 1 respectively, and the results prove that the malicious server can successfully infer whether the sample is a member of the participant training set, and the member inference is completed.
In order to further evaluate the member inference method provided by the present invention, this embodiment compares the local member inference effect of the purchasse 100 data set under different total numbers of participants. The experiment in turn tested the accuracy of local member inference and the F1 score under {10, 20.., 100} participants. As shown in fig. 5, the effect of local member inference decreases gradually as the total number of participants increases, and when 70 participants are involved in federal learning, the accuracy of local member inference can reach 59.907%, and an attacker can still try to infer member samples of the training set of normal participants.

Claims (2)

1. A prediction confidence sequence-based federal learning member inference method comprises the following steps:
(1) before the start of federal learning, an attacker divides the own data set into two parts: a member subset and a non-member subset;
(2) in each iteration of the federal learning, an attacker blends member subsets into the training of the federal learning, saves the shadow model and the target model of the iteration of the current iteration, participates in the training of the global model at the same time, and executes different operations according to the role of the attacker in the federal learning system, which is specifically as follows:
if the attacker is one of the participants of the federal study, the attacker firstly saves a copy of the received global model as a shadow model and a target model of the iteration of the current round, then uses the member subset to train the global model, calculates new local model parameters and uploads the new local model parameters to the server, and the server can form a new global model after aggregating the model parameters uploaded by the participants and enters the next iteration;
if the attacker is a server for federal learning, the attacker firstly trains a current global model by using a member subset, calculates local model parameters, uses the local model as a shadow model of the iteration of the current round, uses a model uploaded by a target participant as a target model, then aggregates the local model and all received participant models to form a new global model, sends the new global model to the participants and enters the next iteration;
the step (2) is a cyclic iteration process, and if the federate learning training process comprises r rounds of iteration, the shadow model sequence and the target model sequence finally saved by the attacker respectively comprise r models;
(3) after the federal learning is finished, an attacker selects a member to deduce the structure of an attack model, a plurality of stored shadow models are utilized to generate a prediction confidence sequence for each sample of a member subset and a non-member subset, a corresponding label 1 or 0 is marked, and the generated data is used for training the attack model;
(4) and the attacker generates a prediction confidence sequence for a given target sample by using the stored multiple target models, and delivers the prediction confidence sequence to the member inference attack model to infer whether the target sample is a member of the training set of the normal participant.
2. The method according to claim 1, wherein the step (3) comprises in particular the following:
(3.1) selecting a two-classification neural network model by an attacker, determining a specific structure and a hyper-parameter of the model, and using the model as a member inference attack model;
(3.2) for each sample (x, y) of the member subset, the attacker respectively calculates the prediction confidence coefficient of each shadow model pair (x, y) in the shadow model sequence to form a prediction confidence coefficient sequence of (x, y), and a label 1 is marked to be used as a training sample of the attack model; for each sample of the non-member subset, the attacker also calculates the prediction confidence sequence of the shadow model to the sample, and marks a label 0 as a training sample of the attack model;
and (3.3) training the member inference model determined in the step (3.1) by an attacker by using the attack model training sample generated in the step (3.2).
CN202111200539.8A 2021-10-15 2021-10-15 Prediction confidence sequence-based federal learning member inference method Pending CN113850399A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111200539.8A CN113850399A (en) 2021-10-15 2021-10-15 Prediction confidence sequence-based federal learning member inference method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111200539.8A CN113850399A (en) 2021-10-15 2021-10-15 Prediction confidence sequence-based federal learning member inference method

Publications (1)

Publication Number Publication Date
CN113850399A true CN113850399A (en) 2021-12-28

Family

ID=78978441

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111200539.8A Pending CN113850399A (en) 2021-10-15 2021-10-15 Prediction confidence sequence-based federal learning member inference method

Country Status (1)

Country Link
CN (1) CN113850399A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117150422A (en) * 2023-11-01 2023-12-01 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117150422A (en) * 2023-11-01 2023-12-01 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system
CN117150422B (en) * 2023-11-01 2024-02-02 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system

Similar Documents

Publication Publication Date Title
Zhang et al. Gan enhanced membership inference: A passive local attack in federated learning
Yu et al. Transfer learning with dynamic adversarial adaptation network
CN112668044B (en) Privacy protection method and device for federal learning
CN111311107A (en) Risk assessment method and device based on user relationship and computer equipment
CN114417427B (en) Deep learning-oriented data sensitivity attribute desensitization system and method
WO2022267960A1 (en) Federated attention dbn collaborative detection system based on client selections
WO2020143253A1 (en) Method employing sparse autoencoder to cluster power system operation modes
CN113780002A (en) Knowledge reasoning method and device based on graph representation learning and deep reinforcement learning
CN111985562A (en) End cloud collaborative training system for protecting end-side privacy
Xiao et al. Network security situation prediction method based on MEA-BP
CN115409155A (en) Information cascade prediction system and method based on Transformer enhanced Hooke process
Yang et al. Gradient leakage attacks in federated learning: Research frontiers, taxonomy and future directions
Cheng et al. GFL: Federated Learning on Non-IID data via Privacy-preserving Synthetic data
CN116227624A (en) Federal knowledge distillation method and system oriented to heterogeneous model
CN113850399A (en) Prediction confidence sequence-based federal learning member inference method
CN117574429A (en) Federal deep learning method for privacy enhancement in edge computing network
CN111259264A (en) Time sequence scoring prediction method based on generation countermeasure network
CN111192158A (en) Transformer substation daily load curve similarity matching method based on deep learning
CN117391816A (en) Heterogeneous graph neural network recommendation method, device and equipment
CN116187469A (en) Client member reasoning attack method based on federal distillation learning framework
CN116523001A (en) Method, device and computer equipment for constructing weak line identification model of power grid
CN116310642A (en) Variable dynamic discriminator differential privacy data generator based on PATE framework
CN113033410B (en) Domain generalization pedestrian re-recognition method, system and medium based on automatic data enhancement
CN115131605A (en) Structure perception graph comparison learning method based on self-adaptive sub-graph
CN113656833A (en) Privacy stealing defense method based on evolutionary computation under vertical federal architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination