CN116644433A - Data privacy and model safety test method for longitudinal federal learning - Google Patents

Data privacy and model safety test method for longitudinal federal learning Download PDF

Info

Publication number
CN116644433A
CN116644433A CN202310619376.XA CN202310619376A CN116644433A CN 116644433 A CN116644433 A CN 116644433A CN 202310619376 A CN202310619376 A CN 202310619376A CN 116644433 A CN116644433 A CN 116644433A
Authority
CN
China
Prior art keywords
attacker
data
model
server
federal learning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310619376.XA
Other languages
Chinese (zh)
Inventor
陈艳姣
徐文渊
张含蕾
白怡杰
马忠明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202310619376.XA priority Critical patent/CN116644433A/en
Publication of CN116644433A publication Critical patent/CN116644433A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
    • G06F21/577—Assessing vulnerabilities and evaluating computer system security
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G06N20/20—Ensemble learning
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer And Data Communications (AREA)

Abstract

The invention discloses a data privacy and model safety test method for longitudinal federal learning, which is applied to the technical field of safety of artificial intelligent distributed federal learning. The label reasoning attack step tests the privacy protection capability of a longitudinal federal learning algorithm and a corresponding defense method on label information by implementing a simple and efficient intermediate result replacement method designed by us. And in the back door attack step, a hidden and efficient back door attack method is realized by designing a trigger, enhancing the back door and adjusting the learning rate, so that the model security risk of a longitudinal federal learning algorithm and a corresponding defense method is tested.

Description

Data privacy and model safety test method for longitudinal federal learning
Technical Field
The invention relates to the technical field of safety of artificial intelligent distributed federal learning, in particular to a data privacy and model safety test method for longitudinal federal learning.
Background
Federal learning is a distributed learning paradigm in the field of artificial intelligence that allows multiple data owners to collaboratively train a global model without revealing their local private data. Federation learning can be classified into horizontal federation learning and vertical federation learning according to a data sharing mode between clients.
In lateral federal learning, participants use the same tagged dataset of feature space, but a different sample space. In the training process, each participant trains a local classifier through supervised learning, and sends weights to a centralized server for aggregation and updating. In longitudinal federal learning, participants possess datasets with different feature spaces but the same sample space. In the training process, each participant trains a local model through gradients from a server side, and transmits an intermediate result output by the local model to the server side for aggregation. In this process, only the server holds the data labels, and the server can train the final classifier by supervised learning based on the aggregated intermediate results. Since backdoor attacks have been extensively studied in horizontal federal learning, the present invention focuses on vertical federal learning.
Back door attacks aim to train a back door model that works properly on normal input samples, but that incorrectly classifies a particular input (input sample with an attacker maliciously designed trigger) as a target tag or other incorrect tag. Federal learning background is an ideal choice for an attacker to conduct a back-door attack as the server does not allow to check the local data and local model of the participants. Backdoor attacks have been widely studied in a horizontal federal learning scenario, in which an attacker uploads model updates of malicious clients to a server in order to insert backdoors in a global model.
Unlike traditional federal learning, longitudinal federal learning presents new challenges for back door attacks, the most imminent challenge being the lack of access to training data tags and server models. To address these challenges, it is important that the proper design approach capture the target data tag information and carefully construct the triggers to enable implantation of the back door into the black box model on the server. Therefore, the invention provides a label reasoning attack and backdoor attack method aiming at longitudinal federal learning, has effective label reasoning and backdoor attack strategies, and provides a new tool for testing privacy protection and model security capability of a longitudinal federal learning algorithm and a corresponding defense method.
Therefore, a data privacy and model security test method for longitudinal federal learning is provided to solve the difficulty existing in the prior art, which is a problem to be solved by those skilled in the art.
Disclosure of Invention
In view of the above, the invention provides a data privacy and model security test method for longitudinal federal learning, which tests the defending capability of a longitudinal federal learning algorithm and a corresponding defending method in terms of data privacy and model security by implementing label reasoning attack and back door attack aiming at longitudinal federal learning.
In order to achieve the above object, the present invention provides the following technical solutions:
a data privacy and model safety test method for longitudinal federal learning comprises the following steps:
(1) A label reasoning attack step comprising the following:
s101, an attacker takes part in a training process of longitudinal federal learning as a malicious client;
s102, an attacker steals the tag information of the data through an intermediate result replacement method;
s103, dynamically adjusting by an attacker to perform hidden label reasoning attack; in a certain round of training, a set of samples T is given, which is inferred by a label reasoning attack to have labels, and the sample with the smallest return gradient is selected, namelyFor use in the next replacement work;
(2) A back door attack step comprising the following:
s201, obtaining tag information of data through tag reasoning attack;
s202, designing a trigger;
s203, adding a trigger to the intermediate result of the batch with the target label, and enhancing the attack effect by adopting a random strategy;
s204, mixing the poisoning data with other normal data, and transmitting the mixed poisoning data to a server;
s205, adjusting the learning rate, and updating the model by using gradient information transmitted back by the server; and judging whether the number of training rounds required by the back door attack is reached, stopping the back door attack, and continuing S203-S205 is not reached.
In the above method, optionally, in step S101, the specific steps of the attacker, as a malicious client, participating in the training process of longitudinal federal learning are as follows:
each client transmits local data into a local model held by the client to obtain an intermediate result output by the local model, and transmits the intermediate result to a server; the server performs data aggregation on the received intermediate results, and transmits the aggregated results to a server-side model to obtain a final prediction result of the model; the server calculates a loss function, performs back propagation, and transmits the obtained gradient information to a corresponding client; each client updates the local model held by the client according to the received gradient information; the training step in S101 repeats the number of rounds until the model gradually converges.
In the above method, optionally, the specific steps of the attacker stealing the tag information of the data by the intermediate result replacing method in step S102 are as follows:
during normal training, an attacker transmits the correct intermediate results to the serverAnd gets the gradient information transmitted back by the server +.>Wherein f a Is a local model held by an attacker, +.>Local data, y, of unknown tags owned by an attacker i Is the label of the sample, but y for an attacker i Is unknown; the same data will be transferred again by the attacker next time +.>When the server is provided, an attacker can firstly conduct screening work to determine the unknown label y i Whether it is likely that the tag y of interest to the attacker is t 。
The method, optionally, judgingWhether the tag is the target tag y t The criteria of (2) are:
wherein I II 2 Represents L 2 Norms, θ and μ are two threshold parameters.
In the above method, optionally, step S102 further includes:
training a binary classifier H with data of all known tags at present, with target tag y t Data of (2)As a positive sampleWithout target tag y t Data of->As a negative sample; wherein (1)>Is known to an attacker to have a target tag y t Local data of->Is local data known to an attacker without the target tag.
In the above method, optionally, step S102 further includes:
the attacker inputs all samples in the same batch into the binary classifier to conduct label prediction, selects the first n samples with the highest prediction results to conduct the next intermediate result replacement work, and uses the final label inference result to use the samples in the training work of the binary classifier.
In the above method, optionally, step S102 further includes:
for samples selected by the screening work, the attacker will use intermediate results of other known tags Replacement->To the server and to obtain the gradient information returned by the server>Wherein y is t Tag information of interest to an attacker, and +.>Is that an attacker has alreadyKnown as having a label y t Is a local data of (a) a (b).
In the above method, optionally, the specific steps of designing the trigger in step S202 are as follows:
using the superposition trigger E and defining the data poisoning process as:
where +. ∈ represents Element-wise addition and flip-flop E can be expressed as:
wherein M is a trigger mask, the trigger area value is 1, the other area values are 0,is the Element-wise multiplication, beta is the parameter controlling the trigger amplitude, delta = [ +delta + delta, -delta, -delta, & delta +
δ,-δ,-δ]。
In the above method, optionally, in step S203, the following two random strategies are adopted to enhance the attack effect:
a random policy Dropout and a random policy shift.
Compared with the prior art, the invention discloses a data privacy and model safety test method for longitudinal federal learning, which has the following beneficial effects:
(1) The method comprises the steps of testing the defending capability of a longitudinal federal learning algorithm and a corresponding defending method in terms of data privacy and model safety by implementing a label reasoning attack and a back door attack aiming at longitudinal federal learning;
(2) The testing tool comprises two modules, namely a tag reasoning attack module and a back door attack module, for testing whether a malicious participant can successfully steal tag information under the current longitudinal federal learning algorithm and the defense method and implanting a back door for a model to realize malicious attack;
(3) The label reasoning attack module tests the privacy protection capability of a longitudinal federal learning algorithm and a corresponding defense method on label information by implementing a simple and efficient intermediate result replacement method designed by us. The back door attack module realizes a hidden and efficient back door attack method by designing a trigger, enhancing the back door and adjusting the learning rate so as to test the model security risk of a longitudinal federal learning algorithm and a corresponding defense method.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is an example of the invention testing the privacy protection and model security capabilities of a vertical federal learning algorithm and corresponding defense methods;
FIG. 2 is a schematic workflow diagram of a label reasoning attack module of the present invention;
fig. 3 is a schematic workflow diagram of a back door attack module according to the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, a data privacy and model security test method for longitudinal federal learning includes the steps of:
(1) A label reasoning attack step comprising the following:
s101, an attacker takes part in a training process of longitudinal federal learning as a malicious client;
s102, an attacker steals the tag information of the data through an intermediate result replacement method;
s103, dynamically adjusting by an attacker to perform hidden label reasoning attack; in a certain round of training, a set of samples T is given, which is inferred by a label reasoning attack to have labels, and the sample with the smallest return gradient is selected, namelyFor use in the next replacement work;
(2) A back door attack step comprising the following:
s201, obtaining tag information of data through tag reasoning attack;
s202, designing a trigger;
s203, adding a trigger to the intermediate result of the batch with the target label, and enhancing the attack effect by adopting a random strategy;
s204, mixing the poisoning data with other normal data, and transmitting the mixed poisoning data to a server;
s205, adjusting the learning rate, and updating the model by using gradient information transmitted back by the server; and judging whether the number of training rounds required by the back door attack is reached, stopping the back door attack, and continuing S203-S205 is not reached.
Further, in step S101, the specific steps of the attacker taking part in the training process of longitudinal federal learning as a malicious client are as follows:
each client transmits local data into a local model held by the client to obtain an intermediate result output by the local model, and transmits the intermediate result to a server; the server performs data aggregation on the received intermediate results, and transmits the aggregated results to a server-side model to obtain a final prediction result of the model; the server calculates a loss function, performs back propagation, and transmits the obtained gradient information to a corresponding client; each client updates the local model held by the client according to the received gradient information; the training step in S101 repeats the number of rounds until the model gradually converges.
Further, the specific steps of the attacker stealing the tag information of the data by the intermediate result replacing method in step S102 are as follows:
during normal training, an attacker transmits the correct intermediate results to the serverAnd gets the gradient information transmitted back by the server +.>Wherein f a Is a local model held by an attacker, +.>Local data, y, of unknown tags owned by an attacker i Is the label of the sample, but y for an attacker i Is unknown; the same data will be transferred again by the attacker next time +.>When the server is provided, an attacker can firstly conduct screening work to determine the unknown label y i Whether it is likely that the tag y of interest to the attacker is t 。
Further, judgeWhether the tag is the target tag y t The criteria of (2) are:
wherein the method comprises the steps of||·|| 2 Represents L 2 Norms, θ and μ are two threshold parameters.
Further, step S102 further includes:
training a binary classifier H with data of all known tags at present, with target tag y t Data of (2)As a positive sample, there is no target tag y t Data of->As a negative example. Wherein (1)>Is known to an attacker to have a target tag y t Local data of->Is local data known to an attacker without the target tag.
Still further, step S102 further includes:
the attacker inputs all samples in the same batch into the binary classifier to conduct label prediction, selects the first n samples with the highest prediction results to conduct the next intermediate result replacement work, and uses the final label inference result to use the samples in the training work of the binary classifier.
Further, step S102 further includes:
for samples selected by the screening work, the attacker will use intermediate results of other known tags Replacement->To the server and to obtain the gradient information returned by the server>Wherein y is t Tag information of interest to an attacker, and +.>Is known to an attacker to have tag y t Is a local data of (a) a (b).
Further, the specific steps for designing the trigger in step S202 are as follows:
using the superposition trigger E and defining the data poisoning process as:
where +. ∈ represents Element-wise addition and flip-flop E can be expressed as:
wherein M is a trigger mask, the trigger area value is 1, the other area values are 0,is the Element-wise multiplication, beta is the parameter controlling the trigger amplitude, delta = [ +delta + delta, -delta, -delta, & delta +
δ,-δ,-δ]。
Further, in step S203, the following two random strategies are adopted to enhance the attack effect:
a random policy Dropout and a random policy shift.
Referring to fig. 2, a tag inference attack module includes the steps of:
s101, an attacker takes part in a training process of longitudinal federal learning as a malicious client side:
each client transmits local data into a local model held by the client to obtain an intermediate result output by the local model, and transmits the intermediate result to a server; the server performs data aggregation on the received intermediate results, and transmits the aggregated results to a server-side model to obtain a final prediction result of the model; the server calculates a loss function, performs back propagation, and transmits the obtained gradient information to a corresponding client; each client updates the local model held by the client according to the received gradient information; the above steps are repeated for a plurality of rounds until the model gradually converges.
S102, an attacker carries out an intermediate result replacement method to steal the tag information of the data, and the method specifically comprises the following steps:
during normal training, an attacker transmits the correct intermediate results to the serverAnd gets the gradient information transmitted back by the server +.>Wherein f a Is a local model held by an attacker, +.>Local data, y, of unknown tags owned by an attacker i Is the label of the sample, but y for an attacker i Is unknown.
The attacker will again transmit the same data the next timeWhen the server is provided, an attacker can firstly conduct screening work to determine the unknown label y i Whether it is likely that the tag y of interest to the attacker is t The method specifically comprises the following steps:
(1) Training a binary classifier H with data of all known tags at present, with target tag y t Data of (2)As a positive sample, there is no target tag y t Data of->As a negative example. Wherein (1)>Is known to an attacker to have a target tag y t Local data of->Is local data known to an attacker without the target tag.
(2) The attacker inputs all samples in the same batch into the binary classifier H for label prediction, selects the first n samples with highest prediction results given by the H for next intermediate result replacement work, and uses the final label inference result to apply the samples to training work of the binary classifier H.
For samples selected by the screening work, the attacker will use intermediate results of other known tags Replacement->To the server and to obtain the gradient information returned by the server>Wherein y is t Is tag information of interest to an attacker, that is to say that the attacker wants to know which data has tag y t And->Is an attackKnown to have label y t Is a local data of (a) a (b).
Comparing the gradient changes before and after replacement, determining by the formula (1) (2)Whether the tag is the target tag y t ：
Wherein I II 2 Represents L 2 Norms, θ and μ are two threshold parameters.
S103, using static state in large quantity to avoid repetitionThe label switching is performed to cause the server to sound an alarm, and an attacker dynamically adjusts to perform hidden label reasoning attacks. In a certain training round, a set of samples T is given, and the samples T are inferred to have labels y through label reasoning attack t Selecting the sample with the smallest return gradient, i.eFor use in the next replacement work.
Referring to fig. 3, a back door attack module includes the steps of:
s201, constructing a trigger: using the superposition trigger E and defining the data poisoning process as:
where +. ∈ represents Element-wise addition and flip-flop E can be expressed as:
wherein M is a trigger mask, the trigger area value is 1, the other area values are 0,is the Element-wise multiplication, beta is the parameter controlling the trigger amplitude, delta = [ +delta + delta, -delta, -delta, & delta +
δ,-δ,-δ](every second positive value is followed by two negative values, every second negative value is followed by two positive values, until terminated). It should be noted that here, the trigger is added directly to the intermediate resultRather than the original data samples. Because the attacker only needs to upload the intermediate result to the server as the client-side participant of the longitudinal federal learning, the attacker superimposes the trigger on the intermediate result, and more effective back door attack can be realized.
Back gate injection in S202, vertical federal learning is more difficult than normal back gates because the attacker cannot control intermediate results from other benign participants. In order to enhance back door injection, the invention introduces randomness to the poisoning data during training to improve the performance of back door attacks during longitudinal federal learning model testing. Thus, the present invention employs two randomization strategies for back door enhancement.
(1) The first random strategy Dropout is inspired by Dropout method, which is commonly used to alleviate the over-fitting problem, and the attacker randomly zeroes out some elements in the trigger mask each time during the implantation of the back gate. Evaluation shows that the Dropout strategy makes the backdoor attack method more robust against backdoor defenses based on trigger elimination.
(2) Second random strategy shifting, randomly multiplying the trigger mask M by a uniform distribution Random number gamma in the range to slightly change the trigger amplitude. Wherein (1)>Refers to the lower limit value of the random number γ, and γ refers to the upper limit value of the random number γ.
S203, in the process of updating model parameters by using gradient information, an attacker can properly improve the learning rate of the local model owned by the attacker and change the convergence rate of the local model, so that the influence of the local model owned by the attacker on the final classification result is enhanced, and the influence of poisoning data on the server-side model is further enhanced.
Taking the scenario of participating in longitudinal federal learning for two client models and one server model as an example, the implementation process of the label reasoning attack and the back door attack is specifically described, and the test is performed in MNIST, CIFAR10, imageNette, CINIC-10 several image datasets and a Band Marking (BM), give-Me-Some-Credit (GM) table dataset (structured dataset), and the corresponding model selected by each dataset is as follows in Table 1:
TABLE 1
Data set Server-side model Client model Accuracy of normal training model
MNIST 3-layer full-connection network 4-layer full-connection network 94.51％
CIFAR-10 3-layer full-connection network VGG16 network 76.92％
ImageNette 3-layer full-connection network VGG16 network 69.91％
CINIC-10 3-layer full-connection network VGG16 network 62.10％
BM 3-layer full-connection network 4-layer full-connection network 91.98％
GM 3-layer full-connection network 4-layer full-connection network 77.91％
The specific implementation comprises the following steps:
a label reasoning attack module, the module comprising the steps of:
for MNIST, CIFAR-10, imageNette, CINIC-10, BM and GM, the number of samples per training lot was 128, 50, 64, 100, 1000, respectively, and the number of insert exchanges selected was n=14, 14, 10, 8, 6, 40, respectively. The threshold μ in the label inference attack is set to the average of the gradient L2 norms, since the gradient L2 norms of misclassified samples are typically larger than the average. In practice, μ=0.01 is found to be suitable for different data sets, and θ is taken to be 5 in practice, and is adjustable, if θ is smaller, the selection of the target label sample is more strict, but the target label sample may be ignored.
S1, an attacker carries out an intermediate result replacement method to steal the tag information of the data, and the method specifically comprises the following steps:
s101, in the normal training process, an attacker transmits the correct intermediate result to the server And gets the gradient information transmitted back by the server +.>Wherein f a Is a local model held by an attacker, +.>Local data, y, of unknown tags owned by an attacker i Is the label of the sample, but y for an attacker i Is unknown.
S102, transmitting the same data again by the attacker at the next timeWhen the server is provided, an attacker can firstly conduct screening work to determine the unknown label y i Whether it is likely that the tag y of interest to the attacker is t The method specifically comprises the following steps:
(1) Training a binary classifier H with data of all known tags at present, with target tag y t Data of (2)As a positive sample, there is no target tag y t Data of->As a negative example. Wherein (1)>Is known to an attacker to have a target tag y t Local data of->Is local data known to an attacker without the target tag.
(2) The attacker inputs all samples in the same batch into the binary classifier H for label prediction, selects the first n samples with highest prediction results given by the H for next intermediate result replacement work, and uses the final label inference result to apply the samples to training work of the binary classifier H.
S103, for the sample selected by the screening work, the attacker uses the intermediate results of other known labelsReplacement->To the server and to obtain the gradient information returned by the server>Wherein y is t Is tag information of interest to an attacker, that is to say that the attacker wants to know which data has tag y t And->Is known to an attacker to have tag y t Is a local data of (a) a (b).
S104, comparing gradient changes before and after replacement, and determining by the formulas (1) and (2)Whether the tag is the target tag y t ：
Wherein I II 2 Represents L 2 Norms, θ and μ are two threshold parameters.
S2, using static state in order to avoid repetitionThe label switching is performed to cause the server to sound an alarm, and an attacker dynamically adjusts to perform hidden label reasoning attacks. In a certain training round, a set of samples T is given, and the samples T are inferred to have labels y through label reasoning attack t Selecting the sample with the smallest return gradient, i.eFor use in the next replacement work.
According to the above procedure, the present invention performs tests on various data sets, label inferences are made on them, and the results are shown in Table 2 below:
TABLE 2
Data set MNIST CIFAR10 ImageNette CINIC-10 BM GM
Success rate of label reasoning 96.24％ 98.11％ 92.15％ 98.06％ 99.03％ 100.00％
It can be seen that the label reasoning algorithm proposed by the present invention achieves a high degree of accuracy on each dataset. The accuracy of the label reasoning attack on MNIST, CIFAR-10, CINIC-10, BM and GM is more than 96%. On the more complex dataset imagenet, the attack accuracy is slightly degraded, but still higher than 92%. In addition, the label reasoning attack algorithm provided by the invention is very time-saving, because only one round of training (a few minutes) is needed to infer labels, while the former label reasoning attack requires more time to train a semi-supervised model to carry out subsequent reasoning tasks.
A back door attack module, the module comprising the steps of:
for all data sets, the poisoning rate was 1%, the default value was 0.4, to maintain the flip-flop concealment, gamma was set to 0.6,set to 1.2. Select pairsAll tags were attacked to test the overall validity, and the dropout ratio used was 0.75.
S1, constructing a trigger: using the superposition trigger E and defining the data poisoning process as:
where +. ∈ represents Element-wise addition and flip-flop E can be expressed as:
wherein M is a trigger mask, the trigger area value is 1, the other area values are 0,is the Element-wise multiplication, beta is the parameter controlling the trigger amplitude, delta = [ +delta + delta, -delta, -delta, & delta +
δ,-δ,-δ](every second positive value is followed by two negative values, every second negative value is followed by two positive values, until terminated). It should be noted that here, the trigger is added directly to the intermediate resultRather than the original data samples. Because the attacker only needs to upload the intermediate result to the server as the client-side participant of the longitudinal federal learning, the attacker superimposes the trigger on the intermediate result, and more effective back door attack can be realized.
S2, in order to enhance back door injection, an attacker introduces randomness into the poisoning data during training so as to improve the performance of back door attacks during the test of the longitudinal federal learning model. Thus, an attacker employs two randomization strategies for back-gate enhancement.
S201, a first random strategy Dropout is inspired by a droopout method commonly used for relieving the fitting problem, and an attacker randomly zeroes part of elements in a trigger mask every time in the process of implanting a back gate. Evaluation shows that the Dropout strategy makes the present back door attack more robust against back door defenses based on elimination triggers.
S202, a second random strategy shifting, wherein the trigger mask M is randomly multiplied and uniformly distributed Random number gamma in the range to slightly change the trigger amplitude. Wherein (1)>Refers to the lower limit value of the random number γ, and γ refers to the upper limit value of the random number γ.
S3, in the process of updating model parameters by using gradient information, an attacker can properly improve the learning rate of the local model owned by the attacker, and change the convergence rate of the local model, so that the influence of the local model owned by the attacker on the final classification result is enhanced, and the influence of poisoning data on the server-side model is further enhanced.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (9)

1. The data privacy and model safety test method for longitudinal federal learning is characterized by comprising the following steps of:
(1) A label reasoning attack step comprising the following:
s101, an attacker takes part in a training process of longitudinal federal learning as a malicious client;
s102, an attacker steals the tag information of the data through an intermediate result replacement method;
s103, dynamically adjusting by an attacker to perform hidden label reasoning attack; in a certain round of training, a set of samples T is given, which is inferred by a label reasoning attack to have labels, and the sample with the smallest return gradient is selected, namelyFor use in the next replacement work;
(2) A back door attack step comprising the following:
s201, obtaining tag information of data through tag reasoning attack;
s202, designing a trigger;
s203, adding a trigger to the intermediate result of the batch with the target label, and enhancing the attack effect by adopting a random strategy;
s204, mixing the poisoning data with other normal data, and transmitting the mixed poisoning data to a server;
s205, adjusting the learning rate, and updating the model by using gradient information transmitted back by the server; and judging whether the number of training rounds required by the back door attack is reached, stopping the back door attack, and continuing S203-S205 is not reached.
2. The method for data privacy and model security testing for vertical federal learning of claim 1,
in step S101, the specific steps of the attacker taking part in the training process of longitudinal federal learning as a malicious client are as follows:
each client transmits local data into a local model held by the client to obtain an intermediate result output by the local model, and transmits the intermediate result to a server; the server performs data aggregation on the received intermediate results, and transmits the aggregated results to a server-side model to obtain a final prediction result of the model; the server calculates a loss function, performs back propagation, and transmits the obtained gradient information to a corresponding client; each client updates the local model held by the client according to the received gradient information; the training step in S101 repeats the number of rounds until the model gradually converges.
3. The method for data privacy and model security testing for vertical federal learning of claim 1,
the specific steps of the attacker stealing the tag information of the data by the intermediate result replacement method in step S102 are as follows:
during normal training, an attacker transmits the correct intermediate results to the serverAnd gets the gradient information transmitted back by the server +.>Wherein f a Is a local model held by an attacker, +.>Local data, y, of unknown tags owned by an attacker i Is the label of the sample, but y for an attacker i Is unknown; the same data will be transferred again by the attacker next time +.>When the server is provided, an attacker can firstly performScreening work to determine unknown tags y i Whether it is likely that the tag y of interest to the attacker is t 。
4. A data privacy and model security test method for vertical federal learning according to claim 3,
step S102 further includes:
training a binary classifier H with data of all known tags at present, with target tag y t Data of (2)As a positive sample, there is no target tag y t Data of->As a negative sample; wherein (1)>Is known to an attacker to have a target tag y t Local data of->Is local data known to an attacker without the target tag.
5. The method for data privacy and model security testing for vertical federal learning of claim 4,
step S102 further includes:
the attacker inputs all samples in the same batch into the binary classifier to conduct label prediction, selects the first n samples with the highest prediction results to conduct the next intermediate result replacement work, and uses the final label inference result to use the samples in the training work of the binary classifier.
6. The method for data privacy and model security testing for vertical federal learning of claim 5,
step S102 further includes:
for samples selected by the screening work, the attacker will use intermediate results of other known tags Replacement ofTo the server and to obtain the gradient information returned by the server>Wherein y is t Tag information of interest to an attacker, and +.>Is known to an attacker to have tag y t Is a local data of (a) a (b).
7. The method for data privacy and model security testing for vertical federal learning of claim 6,
judgingWhether the tag is the target tag y t The criteria of (2) are:
wherein I II 2 Represents L 2 Norms, θ and μ are two threshold parameters.
8. The method for data privacy and model security testing for vertical federal learning of claim 1,
the specific steps for designing the trigger in step S202 are as follows:
using the superposition trigger E and defining the data poisoning process as:
wherein the method comprises the steps ofRepresenting Element-wise addition, and the flip-flop E can be expressed as:
wherein M is a trigger mask, the trigger area value is 1, the other area values are 0,is the Element-wise multiplication, beta is the parameter controlling the trigger amplitude, delta = [ +delta + delta, -delta, the +delta+delta, -delta]。
9. The method for data privacy and model security testing for vertical federal learning of claim 1,
in step S203, the following two random strategies are adopted to enhance the attack effect:
a random policy Dropout and a random policy shift.
CN202310619376.XA 2023-05-29 2023-05-29 Data privacy and model safety test method for longitudinal federal learning Pending CN116644433A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310619376.XA CN116644433A (en) 2023-05-29 2023-05-29 Data privacy and model safety test method for longitudinal federal learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310619376.XA CN116644433A (en) 2023-05-29 2023-05-29 Data privacy and model safety test method for longitudinal federal learning

Publications (1)

Publication Number Publication Date
CN116644433A true CN116644433A (en) 2023-08-25

Family

ID=87643043

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310619376.XA Pending CN116644433A (en) 2023-05-29 2023-05-29 Data privacy and model safety test method for longitudinal federal learning

Country Status (1)

Country Link
CN (1) CN116644433A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117150422A (en) * 2023-11-01 2023-12-01 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system
CN118366010A (en) * 2024-06-18 2024-07-19 浙江大学 Model back door attack vulnerability analysis method and system for segmentation learning

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117150422A (en) * 2023-11-01 2023-12-01 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system
CN117150422B (en) * 2023-11-01 2024-02-02 数据空间研究院 Label inference attack method based on sample exchange in longitudinal federal learning system
CN118366010A (en) * 2024-06-18 2024-07-19 浙江大学 Model back door attack vulnerability analysis method and system for segmentation learning

Similar Documents

Publication Publication Date Title
Rodríguez-Barroso et al. Survey on federated learning threats: Concepts, taxonomy on attacks and defences, experimental study and challenges
CN116644433A (en) Data privacy and model safety test method for longitudinal federal learning
Wu et al. Adapting membership inference attacks to GNN for graph classification: Approaches and implications
Du et al. Active learning with human-like noisy oracle
US20220215104A1 (en) Methods of providing data privacy for neural network based inference
CN108717550A (en) A kind of image confrontation verification code generation method and system based on confrontation study
Hossain et al. Desmp: Differential privacy-exploited stealthy model poisoning attacks in federated learning
Lin et al. A hybrid neural network for fast automatic modulation classification
CN112365005B (en) Federal learning poisoning detection method based on neuron distribution characteristics
Nuding et al. Data poisoning in sequential and parallel federated learning
CN107612878A (en) Dynamic window system of selection and wireless network trust management system based on game theory
Lai et al. Two-phase defense against poisoning attacks on federated learning-based intrusion detection
CN117875455A (en) Federal learning data poisoning defense method based on data enhancement
Chen et al. Patch selection denoiser: An effective approach defending against one-pixel attacks
McClintick et al. Countering physical eavesdropper evasion with adversarial training
CN112883377A (en) Feature countermeasure based federated learning poisoning detection method and device
Li et al. An Adaptive Communication‐Efficient Federated Learning to Resist Gradient‐Based Reconstruction Attacks
Yuhang et al. FLVoogd: Robust and privacy preserving federated learning
Zhang et al. Improved colour‐to‐grey method using image segmentation and colour difference model for colour vision deficiency
Rácz et al. Finding a planted clique by adaptive probing
WO2019080844A1 (en) Data reasoning method and apparatus, and computer device
Hu et al. Training graph neural networks by graphon estimation
Balagura et al. Mathematical models of cognitive interaction identification in the social networks
Chen et al. A meta approach to defend noisy labels by the manifold regularizer PSDR
Shen et al. Learning to attack distributionally robust federated learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination