CN114202397B - Longitudinal federal learning backdoor defense method based on neuron activation value clustering - Google Patents
Longitudinal federal learning backdoor defense method based on neuron activation value clustering Download PDFInfo
- Publication number
- CN114202397B CN114202397B CN202210146719.0A CN202210146719A CN114202397B CN 114202397 B CN114202397 B CN 114202397B CN 202210146719 A CN202210146719 A CN 202210146719A CN 114202397 B CN114202397 B CN 114202397B
- Authority
- CN
- China
- Prior art keywords
- backdoor
- clustering
- commodity
- commodity guide
- embedded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Finance (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Accounting & Taxation (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Business, Economics & Management (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Probability & Statistics with Applications (AREA)
- Development Economics (AREA)
- Economics (AREA)
- Marketing (AREA)
- Strategic Management (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a longitudinal federal learning backdoor defense method based on neuron activation value clustering, which comprises the following steps: constructing a longitudinal federal recommendation system comprising a plurality of participants and collaborators; the method comprises the steps that federated learning is carried out, after a cooperative party obtains a built commodity guide link corresponding to aggregation embedded representation, the commodity guide link which is under the attack of a backdoor is effectively screened out through classification of the commodity guide link, and the commodity guide link under the attack of the backdoor is repaired by utilizing a clustering result to guide a commodity sample under the attack of the backdoor to learn towards a correct prediction direction, so that the commodity sample of a participating party does not need to be obtained, and a repaired commodity recommendation model can defend against the backdoor attack; and filtering or repairing the combined embedded representation with the same ID as the aggregated embedded representation of the backdoor attack to prevent or improve the parameter optimization of the aggregated embedded representation of the backdoor attack on the commodity recommendation model and improve the defense capability of the commodity recommendation model on the backdoor attack.
Description
Technical Field
The invention belongs to the technical field of privacy security of commodity recommendation, and particularly relates to a longitudinal federal learning backdoor defense method based on neuron activation value clustering.
Background
The deep learning achieves excellent performance in complex and variable recommendation tasks, so that the deep learning is applied and deployed in a commodity recommendation system in a real scene in a large scale. Compared with the traditional commodity recommendation system, the commodity recommendation system constructed through deep learning has further improved performance, and the commodity recommendation system is mainly benefited from abundant computing resources and sufficient computing data. However, in recent years, some countries and regions have come out of data privacy protection regulations to limit large-scale collection of business data, which brings great challenges to the deep learning constructed commodity recommendation system, and along with the data islanding phenomenon among enterprises.
A commercial recommendation system built using vertical federal learning techniques is referred to as a vertical federal recommendation system. The longitudinal federal recommendation system is regarded as a technical solution for breaking the data island problem in the commercial recommendation system. Enterprises participating in longitudinal federal learning can construct a complete commodity recommendation system by sharing embedded representations or gradients locally without sharing commodity attribute data. However, since it is difficult to guarantee that all enterprises are trusted in all processes of participating in the vertical federal recommendation system, it is highly likely that malicious parties exist. Participants attempt to inject backdoors into the merchandise recommendation system by tampering with data in the longitudinal federal recommendation system or manipulating the longitudinal federal learning training process. For example, a participant tries to be pushed to a user in a malicious manner after a particular identification is added to a certain commodity, which greatly destroys the security of the commodity recommendation system.
In the training process of the longitudinal federal recommendation system, two different paradigms of attack launching the back door exist, on one hand, when the longitudinal federal recommendation system is used as an active party (a party participating in the longitudinal federal recommendation system and provided with a label) to carry out the back door attack, the active party in the longitudinal federal recommendation system only adds a mode trigger on data characteristics in the training process and modifies label information for corresponding sample data to achieve the purpose of injecting the back door; on the other hand, when the rear door attack is carried out as a passive party (a party participating in the longitudinal federal recommendation system and only providing the characteristics), the passive party of the longitudinal federal recommendation system injects the rear door by tampering the gradient information in the training process. The purpose of the two injection backdoors is that in the testing stage of the longitudinal federal recommendation system, an attacker can enable the longitudinal federal recommendation system to recommend a specific commodity sample to a specific user maliciously and accurately.
In order to defend the backdoor attack threat of the longitudinal federal recommendation system, a defense method needs to be deployed on a cooperative party. There are 2 common defense methods in the prior art: differential privacy techniques and gradient sparseness techniques. However, these 2 techniques are not suitable for the commodity recommendation system to defend against the threat of backdoor attack, where the accuracy of the commodity recommendation task is drastically reduced by adding random noise in the differential privacy technique, and the performance of the backdoor defense and the performance of the recommendation task cannot be balanced. The sparse gradients cannot defend against backdoor attacks by the active side of the longitudinal federal recommendation system, mainly because the active side does not rely on the gradient information of the model to launch the backdoor attacks.
Disclosure of Invention
In view of the above, an object of the present invention is to provide a longitudinal federated learning backdoor defense method based on neuron activation value clustering, so that a top model in a longitudinal federated learning system can defend backdoor attacks, and robustness of the top model is improved.
In order to achieve the purpose, the invention provides the following technical scheme:
a longitudinal federal learning backdoor defense method based on neuron activation value clustering comprises the following steps:
the method comprises the steps that a longitudinal federal recommendation system comprising a plurality of participants and a cooperative party is built, and the longitudinal federal recommendation system is used for building a commodity recommendation model through longitudinal federal learning;
performing federal learning of a longitudinal federal recommendation system, comprising: each participant trains a local model by using a local commodity sample, and uploads an embedded representation corresponding to the commodity sample to a cooperative party; the cooperative party aggregates the embedded representations uploaded by all the participating parties to obtain aggregated embedded representations, obtains neuron activation values of the aggregated embedded representations in the top model to construct commodity guide links, screens the commodity guide links attacked by the backdoor by clustering the commodity guide links, repairs the commodity guide links attacked by the backdoor according to clustering results, updates parameters of the top model according to the repaired commodity guide links, and downloads the updated parameters to the participating parties for the next round of federal learning;
and extracting the top model of the end of the federal learning as a commodity recommendation model capable of defending against backdoor attacks.
In one embodiment, the collaborator aggregates the embedded representations uploaded by all the participants by using a splicing operation to obtain an aggregated embedded representation.
In one embodiment, the obtaining of the aggregate embedded neuron activation values represented in the top model to construct the commodity guide link comprises:
and performing forward conduction calculation on the input aggregation embedded expression once by using the top model to obtain the activation value of each layer of neurons of the top model in the forward conduction process, extracting the neuron with the maximum activation value of each layer as a commodity guide neuron, and connecting the commodity guide neurons of all layers according to the forward conduction direction to form a commodity guide link.
In one embodiment, the screening of the commodity guide links attacked by the backdoor by clustering the commodity guide links includes:
based on a prediction label obtained by carrying out primary forward conduction calculation on the input aggregation embedded representation by the top model, carrying out classified storage on the aggregation embedded representation and the corresponding commodity guide link;
clustering all aggregation embedded representations corresponding to each type of prediction labels, screening a cluster with the minimum aggregation embedded representation number as a first abnormal cluster in view of the fact that the number of attackers in longitudinal federal learning is less than that of participants participating in normal training, and taking the ratio of the aggregation embedded representation number contained in the first abnormal cluster to all aggregation embedded representation numbers participating in clustering as a first back door risk score;
clustering all commodity guide links corresponding to each type of prediction label, screening a cluster containing the smallest number of the commodity guide links as a second abnormal cluster, and taking the ratio of the number of the commodity guide links contained in the second abnormal cluster to the number of all the commodity guide links participating in clustering as a second back door risk score;
for each type of prediction label, comprehensively aggregating and embedding a first back door risk score corresponding to the indication and a second back door risk score corresponding to the commodity guide link to obtain a total back door risk score corresponding to each type of prediction label;
and when the total backdoor risk score is larger than a preset threshold value, considering that the aggregation embedding representation in the first abnormal clustering cluster is the joint embedding representation of backdoor attack, and simultaneously considering that the commodity guide link in the second abnormal clustering cluster is the commodity guide link of the backdoor attack.
In one embodiment, the classifying and storing the aggregate embedded representation and the corresponding commodity guide link includes:
and constructing a commodity index lookup table according to the prediction tags represented by aggregation embedding, wherein each type of prediction tag corresponds to each family of the commodity index lookup table, and the combined embedded representation belonging to the same type of prediction tags and the corresponding commodity guide links are arranged in the same family so as to realize classified storage according to the prediction tags.
In one embodiment, when all aggregation embedded representations corresponding to each type of prediction tag are clustered, two aggregation embedded representations are randomly selected as an initial clustering center, the clustering distance is the sum of the Euclidean distance from the aggregation embedded representation to be clustered to the clustering center and the cosine similarity distance from the aggregation embedded representation to be clustered to the clustering center, and after each round of clustering is finished, the clustering center is updated by adopting an average clustering algorithm according to the dimension average of all aggregation embedded representations in the clustering;
when all commodity guide links corresponding to each type of prediction label are clustered, two commodity guide links are randomly selected as an initial clustering center, the clustering distance is the sum of the Euclidean distance from the commodity guide link to be clustered to the clustering center and the cosine similarity distance from the commodity guide link to be clustered to the clustering center, and after each round of clustering is finished, the clustering center is updated by adopting an average clustering algorithm according to the dimension average of all the commodity guide links in the clustering.
In one embodiment, the comprehensive aggregation embedding represents that the corresponding first rear door risk score and the corresponding second rear door risk score of the commodity guide link obtain a total rear door risk score corresponding to each type of prediction tag, and the method comprises the following steps:
and weighting and summing the corresponding first rear-entrance risk score and the corresponding second rear-entrance risk score of the commodity guide link according to the weight to obtain a total rear-entrance risk score corresponding to each type of prediction label, wherein the value of the weight is 0-1, and the sum of the weight of the first rear-entrance risk score and the weight of the second rear-entrance risk score is 1.
In one embodiment, the repairing the commodity guide link of the backdoor attack according to the clustering result includes:
randomly selecting a normal commodity guide link from the clustering results to cover the commodity guide link attacked by the backdoor so as to realize repair;
or, calculating the average value of all normal commodity guide links in the clustering result, and covering the commodity guide link attacked by the backdoor with the average value to realize repair.
In one embodiment, when a commodity guide link of a backdoor attack is determined, a joint embedded representation of the backdoor attack corresponding to the commodity guide link is obtained and recorded;
the joint embedded representation of the recorded backdoor attacks is used to guide the next round of federal learning.
In one embodiment, the joint embedded representation of the recorded backdoor attacks is used to guide the next round of federal learning, including:
during the next round of federal learning, the cooperative party deletes the joint embedded representation with the same ID from the joint embedded representation uploaded by the participator according to the ID of the joint embedded representation of the latest backdoor attack, and the rest joint embedded representation is used for updating the top model of the participator;
or, in the next round of federal learning, the cooperative party screens the joint embedded representation with the same ID from the joint embedded representation uploaded by the participating party according to the ID of the joint embedded representation of the latest backdoor attack, and carries out attack repair on the joint embedded representation with the same ID, and the repaired joint embedded representation is used for updating the top model of the participating cooperative party;
wherein, attack repair includes: and replacing the normal maximum joint embedding representation in the adjacent fixed time range of the joint embedding representation with the same ID to realize attack repair.
Compared with the prior art, the invention has the beneficial effects that at least:
after the cooperative party obtains the constructed commodity guide link corresponding to the aggregation embedded representation, the cooperative party effectively screens out the commodity guide link which is under the attack of the backdoor by classifying the commodity guide link and utilizes the commodity guide link
The clustering result repairs the commodity guide link attacked by the backdoor to guide the commodity sample attacked by the backdoor to learn towards the correct prediction direction, so that the commodity sample of a participant does not need to be obtained, and the repaired top model can defend the backdoor attack;
and when the commodity guide link of the backdoor attack is determined, the aggregation embedded representation of the backdoor attack can be obtained, and the combined embedded representation with the same ID as the aggregation embedded representation of the backdoor attack is filtered or attack-repaired to prevent or improve the parameter optimization of the aggregation embedded representation of the backdoor attack on the top model and improve the defense capability of the top model on the backdoor attack.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a flowchart of a longitudinal federated learning back-door defense method based on neuron activation value clustering provided by an embodiment;
FIG. 2 is a schematic structural diagram of a longitudinal federal recommendation system provided by an embodiment;
fig. 3 is a schematic diagram of a commodity guide link of a cluster screening backdoor attack according to an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
In view of the risk of backdoor attacks after the longitudinal federal recommendation system is widely deployed in a commercial scene, an effective defense method needs to be researched for the problem of how to protect the safety of the model in the longitudinal federal learning. The invention aims to provide a longitudinal federal learning backdoor defense method based on neuron activation value clustering from the perspective of a cooperative party in a longitudinal federal recommendation system, and the specific technical concept is as follows: the back door attacks initiated by the active side and the passive side in the longitudinal federal recommendation system finally show the effect of back door injection in the top model with the prediction classification capability, and particularly, the technical concept is based on a realistic observation, and the neuron activation values of the top model with the back door injection and the clean top model show different rules in the classification task. Generally speaking, the same law is true for the clean model for the commodity guide links of the same class of sample categories, whereas two different laws are true for the back door model for the commodity guide links of the same class of sample categories. Based on this, a K-means clustering technique was introduced to analyze the neuron activation values of the top model, while analyzing the distribution features of the embedded representation, for which purpose an assessment was made as to whether the model was injected into the backdoor. Finally, the risk and challenge of repairing the backdoor by applying neuron activation values and embedding representative replacement operations to the backdoor potential of the longitudinal federal recommendation system.
Fig. 1 is a flowchart of a longitudinal federal learning backdoor defense method based on neuron activation value clustering according to an embodiment. As shown in fig. 1, the longitudinal federal learning backdoor defense method based on neuron activation value clustering provided by the embodiment includes the following steps:
step 1, constructing a longitudinal federal recommendation system comprising a plurality of participants and collaborators.
In an embodiment, the constructed longitudinal federal recommendation system is used for constructing a commodity recommendation model through longitudinal federal learning. According to the longitudinal federal recommendation system protocol, the participants have local commodity samples and local models, and the collaborators have top models capable of classification. The multiple participants are used for training the local model by using the local commodity sample, acquiring the embedded representation of the commodity sample in the forward propagation process of the local model of the participants, and uploading the embedded representation corresponding to the commodity sample to the cooperative party. The collaborators are used for aggregating the uploaded embedded representations and updating the top model parameters with the aggregated embedded representations, and are also used for deploying the model parameters of the local models of the participants in the next round by using the updated parameters of the top models.
In an embodiment, the participant has a sample of the item with item attribute information, wherein the item attribute includes a type of the item, a color of the item, a sales amount of the item, and the like. The recommendation of the commodity can be realized by using the commodity recommendation model based on the commodity attribute information.
In an embodiment, the deep learning model adopted by the local model of the participant may include a fully-connected neural network model, a convolutional neural network model, or a graph-convolutional neural network model, and the like, and the cooperator model is typically a fully-connected neural network model with classification capability. The top model and the local model both comprise an input layer, a hidden layer and an output layer, a link relation exists between different layers, each layer is a neural network composed of a plurality of neurons, for example, the circles of the local model and the top model in fig. 2 both represent the neurons, and the neurons of each layer of the neural network output activation values through activation functions to reflect the activation states of the neurons. The neuron with the largest neuron activation value in each layer is defined as a guide neuron, such as a black circle in the top model in fig. 2, which often plays an important role in the prediction task. The input layer of the local model corresponds to the original data dimension of the participant, and the output layer of the local model corresponds to the output dimension required in the training protocol of the longitudinal federal recommendation system.
Based on the data security and privacy protocol of the longitudinal federal recommendation system, the commodity samples of the participants are published to the outside and are not uploaded to the collaborators, so that the collaborators cannot obtain the local commodity samples. Meanwhile, the participant may be a malicious attacker, with the risk of backdoor attacks. An attacker poisons the longitudinal federal recommendation system by injecting elaborate backdoor commodity recommendation samples in the federal learning training process, and directly causes the top model of a cooperative party to be injected into the backdoor, so that the robustness of the final top model is influenced. Therefore, the mode of step 2 is adopted to carry out the back door attack defense learning of the top model so as to improve the defense capability of the top model against the back door attack.
And 2, carrying out federal learning of a longitudinal federal recommendation system.
As shown in fig. 2, in the federal learning process, each participant trains a local model by using a local commodity sample, and uploads an embedded representation corresponding to the commodity sample to a collaborator. Since each participant has different sample characteristics of the item, the embedded representation extracted by each participant is also different.
As shown in fig. 2, in the federal learning process, the cooperative parties aggregate the embedded representations uploaded by all the participating parties to obtain an aggregate embedded representation, acquire neuron activation values of the aggregate embedded representation on the top model to construct commodity guide links, screen the commodity guide links attacked by the backdoor by clustering the commodity guide links, repair the commodity guide links attacked by the backdoor according to the clustering result, update the parameters of the top model according to the repaired commodity guide links, and download the updated parameters to the participating parties for the next round of federal learning.
In an embodiment, a collaborator receives an embedded representation uploaded by all participants
And aggregating the embedded representations uploaded by all participants
Obtaining an aggregate embedded representation
Wherein, in the step (A),sfor the index of the embedded representation, also corresponding to the index of the participant,xindexes of embedded representations are aggregated. Specifically, the following methods can be used
And a splicing operation of a function and the like aggregates the embedded representations uploaded by all the participants to obtain an aggregated embedded representation, wherein the splicing operation is to splice a plurality of embedded representations along the same dimension. The dimension of the aggregate embedded representation, i.e., the input dimension of the top model, is
Wherein, in the step (A),
the dimensions of the embedded representation output for the participant,
representing the number of participants, it is assumed here without loss of generality that the output embedding representation dimension of the local model of each participant is the same by default.
In an embodiment, the obtained aggregate embedding represents neuron activation values of the top model to construct the commodity guide link, including:
inputting the aggregation embedded representation into a top model, performing forward conduction calculation on the aggregation embedded representation by using the top model to obtain the activation value of each layer of neurons of the top model in the forward conduction process, extracting the neuron with the maximum activation value of each layer as a commodity guide neuron, and connecting the commodity guide neurons of all layers according to the forward conduction direction to form a commodity guide link
。
In an embodiment, the screening of the commodity guide links attacked by the backdoor by clustering the commodity guide links includes:
(a) and performing classified storage on the aggregation embedded representation and the corresponding commodity guide link based on a prediction label obtained by performing forward conduction calculation on the input aggregation embedded representation once on the basis of the top model.
In an embodiment, aggregate embedded representations
Inputting the data into the top model, and obtaining the commodity guide link through one forward conduction calculation
And a commodity prediction tag
The goods are recommended to a particular predictive tag. For example, the anti-skid shoes are accurately recommended to the user group of the aged according to the prediction result. The cooperative party establishes a commodity index lookup table for the commodity sample, and the commodity index lookup table can be stored in a memory in a dictionary or list mode. The memory may be a volatile memory at the near end, such as RAM, a non-volatile memory, such as ROM, FLASH, floppy disk, mechanical hard disk, etc., or a remote storage cloud.
In the embodiment, a commodity index lookup table is constructed according to the prediction tags represented by aggregation embedding, each type of prediction tag corresponds to each family of the commodity index lookup table, and the joint embedding representation belonging to the same type of prediction tags and the corresponding commodity guide links are arranged in the same family, so that classified storage according to the prediction tags is realized.
As shown in FIG. 3, the collaborator performs a back-door risk score for each family using K-means clustering. The back door risk scoring mechanism consists of 2 parts: a back-door risk score for the merchandise guide link and a back-door risk score for the consolidated embedded representation. Commodity-oriented links
And embedded representation
Are considered to be vector form for clustering. And (4) judging whether the embedded potential backdoor risk exists in each family or not by retrieving the abnormal classification condition of each family. The collaborator sorts the families in sequence and carries out the back door risk scoring of each family one by one.
(b) And clustering all aggregation embedded representations corresponding to each type of prediction labels, screening the cluster with the minimum number of the aggregation embedded representations as a first abnormal cluster, and taking the ratio of the number of the aggregation embedded representations contained in the first abnormal cluster to the number of all the aggregation embedded representations participating in clustering as a first backdoor risk score.
In an embodiment, when all the aggregation embedded representations corresponding to each type of prediction tag are clustered, two aggregation embedded representations are randomly selected as initial clustering centers
Respectively representing the clustering center of the normal clustering cluster and the clustering center of the abnormal clustering cluster; and then, after the clustering distances from the rest joint embedded representations to two clustering centers are calculated, clustering the joint embedded representations according to the clustering distances, namely clustering the joint embedded representations to the clustering cluster corresponding to the minimum clustering distance.
In an embodiment, the cluster distance
Using two metrics, including Euclidean distance
And cosine similarity distance
And representing the Euclidean distance to the cluster center by the aggregate embedding to be clustered
Embedding cosine similarity distance from representation to cluster center with aggregation to be clustered
The sum as the clustering distance, i.e.
。
After each round of clustering is finished, updating the clustering center by adopting an average clustering algorithm according to the average number of the dimensionalities embedded and expressed in all the clusters, namely the updated clustering center is expressed as follows:
wherein the content of the first and second substances,
is shown asiThe cluster center of each update is then updated,
the number of embedded representations for each family of aggregates.
After clustering is finished, a cluster containing the minimum number of aggregation embedding representations is selected as a first abnormal cluster, namely, a cluster with the small number of aggregation embedding representations is selected from two clusters as a first abnormal cluster, the rest cluster is a first normal cluster, and the aggregation embedding representation contained in the first normal cluster is a normal aggregation embedding representation; and then taking the ratio of the number of the aggregation embedded representations contained in the first abnormal cluster to the number of all the aggregation embedded representations participating in the clustering as a first backdoor risk score, and simultaneously recording the ID of the aggregation embedded representations in the first abnormal cluster.
(c) And clustering all the commodity guide links corresponding to each type of prediction label, screening the cluster containing the smallest commodity guide link number as a second abnormal cluster, and taking the ratio of the commodity guide link number contained in the second abnormal cluster to the commodity guide link numbers participating in clustering as a second back door risk score.
In an embodiment, the commodity guide links are clustered in the same clustering manner as the aggregation embedding representation to obtain the backdoor risk score of the commodity guide links. When all commodity guide links corresponding to each type of prediction label are clustered, two commodity guide links are randomly selected as an initial clustering center
Respectively representing the clustering center of the normal clustering cluster and the clustering center of the abnormal clustering cluster; and then, after the clustering distances from the rest commodity guide links to two clustering centers are calculated, clustering the commodity guide links according to the clustering distances, namely clustering the commodity guide links to the clustering cluster corresponding to the minimum clustering distance.
In the embodiment, the clustering distance is adopted for clustering all commodity guide links corresponding to each type of prediction labels
Also using two metrics, including Euclidean distance
And cosine similarity distance
And representing the Euclidean distance to the cluster center by the aggregate embedding to be clustered
Embedding cosine similarity distance from aggregation to be clustered to representation to clustering center
The sum as the clustering distance, i.e.
。
After each round of clustering is finished, updating the clustering center by adopting an average clustering algorithm according to the dimension average of all commodity guide links in the clustering, namely the updated clustering center is expressed as:
wherein the content of the first and second substances,
is shown asiThe cluster center of each update is then updated,
the number of links is directed to each group of items.
After clustering is finished, the cluster containing the minimum number of the commodity guide links is selected as a second abnormal cluster, namely, the cluster containing the minimum number of the commodity guide links is selected from the two clusters as the second abnormal cluster, the rest cluster is a second normal cluster, and the commodity guide links contained in the second normal cluster are normal commodity guide links; and then taking the ratio of the number of the commodity guide links contained in the second abnormal clustering cluster to the number of all the commodity guide links participating in clustering as a second backdoor risk score, and simultaneously recording the ID of the commodity guide links in the second abnormal clustering cluster.
(d) And for each type of prediction label, comprehensively aggregating and embedding a first back door risk score corresponding to the indication and a second back door risk score corresponding to the commodity guide link to obtain a total back door risk score corresponding to each type of prediction label.
After the scoring in the step (b) and the step (c) is completed, the collaborator sums the first rear door risk score corresponding to the aggregation embedded representation and the second rear door risk score corresponding to the commodity guide link according to the weight weighting to obtain a total rear door risk score corresponding to each type of prediction label, wherein the value of the weight is 0-1, and the sum of the weight of the first rear door risk score and the weight of the second rear door risk score is 1.
Preferably, the weight of the first rear door risk score and the weight of the second rear door risk score are both 0.5, and then the arithmetic mean value of the first rear door risk score and the second rear door risk score is used as the total rear door risk score corresponding to each type of prediction labelscore。
(e) And when the total backdoor risk score is larger than a preset threshold value, considering that the aggregation embedding expression in the first abnormal cluster is the joint embedding expression of backdoor attack, and simultaneously considering that the commodity guide link in the second abnormal cluster is the commodity guide link of the backdoor attack.
In the embodiment, the threshold M is obtained through experimental investigation as a criterion for measuring whether the backdoor attack exists, and is preferably set to 0.2-0.3, and the threshold can better judge whether the backdoor attack exists. Further preferably, the threshold value M is set to 0.2.
When total backdoor risk scorescoreAnd when the value is larger than the preset threshold value M, the top model is considered to have backdoor attack, namely the aggregation embedded representation in the first abnormal clustering cluster is considered to be the joint embedded representation of the backdoor attack, and meanwhile, the commodity guide link in the second abnormal clustering cluster is considered to be the commodity guide link of the backdoor attack.
When total backdoor risk scorescoreWhen the number of the top model is smaller than or equal to a preset threshold value M, the top model is considered to have no backdoor attack, and when the top model is considered to have no backdoor attack, the storage result is released from the memory, and the occupation information of the family in the memory is erased; otherwise, the detection result is stored in the memory, and the information of the original family is not erased.
Notably, the recommended commodity back-door risk scoring link may occur at any stage of longitudinal federal recommendation system training, and there are typically 2 scoring strategies: single point scores and interval scores. The single-point scoring refers to scoring of a single turn which is relied on by a cooperative party only for a top model in the training process of the longitudinal federal recommendation system, and often occurs when the longitudinal federal recommendation system approaches a convergence stage or a training middle stage. And the interval scoring refers to randomly extracting a top model for scoring in a plurality of turns in the training process of the longitudinal federal recommendation system by a cooperative policy, and selecting a maximum scoring value as a final back-door risk scoring.
In the embodiment, the commodity guide link of the backdoor attack is repaired according to the clustering result, the parameters of the top model are updated according to the repaired commodity guide link, and the updated parameters are downloaded to the participants for the next round of federal learning.
And when the cooperative party judges that the top model has potential backdoor risks, reading the commodity guide link attacked by the backdoor from the memory to carry out backdoor repairing operation. Specifically, repairing the commodity guide link attacked by the backdoor according to the clustering result, namely, performing purification operation on the neuron activation value in the commodity guide link, wherein the method comprises the following two modes:
and directly replacing coverage, namely, randomly selecting a normal commodity guide link from the clustering result to cover the commodity guide link attacked by the backdoor so as to realize repair. Specifically, the activation value of a normal commodity guide link (considered as an un-attacked normal commodity guide link) is randomly selected from the second normal cluster to directly replace the activation value of the commodity guide link of the backdoor attack. The method is suitable for the scene with low computing resources of the collaborators.
The second method comprises the following steps: and (4) average covering, namely calculating the average value of all normal commodity guide links in the clustering result, and covering the commodity guide link attacked by the backdoor with the average value to realize repairing. Specifically, the average activation value of all the normal commodity guide links in the second normal clustering cluster is calculated, and the average activation value is used for replacing the activation value of the commodity guide link of the backdoor attack.
And the repaired commodity guide link is used for updating parameters of the top model and downloading the updated parameters to the participants for the next round of federal study. The updated model parameters can guide the commodity sample of the backdoor attack to learn towards the correct prediction direction, so that the commodity sample of the participant does not need to be obtained, and the repaired top model can defend the backdoor attack.
In the embodiment, when the commodity guide link of the backdoor attack is determined, the joint embedded representation of the backdoor attack corresponding to the commodity guide link can be obtained and recorded; the joint embedding representation of the recorded backdoor attacks is used for guiding the next round of federal learning and comprises the following two ways:
the first method is as follows: and when the next round of federal learning is carried out, deleting the joint embedded representation with the same ID from the joint embedded representation uploaded by the participator by the cooperator according to the ID of the joint embedded representation of the latest backdoor attack, and utilizing the rest joint embedded representation to update the top model of the participator.
In the embodiment, the joint embedded representation of the backdoor attack recorded close to the latest federal learning round of the next round is used as the joint embedded representation of the latest backdoor attack, the joint embedded representation which is considered to have the same ID as the joint embedded representation of the latest backdoor attack in the joint embedded representation uploaded by the participants is deleted to prevent the aggregation embedded representation of the backdoor attack risk from optimizing the parameters of the top model, and the defense capability of the top model against the backdoor attack is improved.
The second method comprises the following steps: and during the next round of federal learning, screening the joint embedded representation with the same ID from the joint embedded representation uploaded by the participator by the cooperative party according to the ID of the joint embedded representation of the latest backdoor attack, carrying out attack repair on the joint embedded representation with the same ID, and updating the top model of the participator by the repaired joint embedded representation.
Wherein, attack repair includes: and replacing the maximum normal joint embedded representation in the adjacent fixed time range of the joint embedded representation with the same ID to realize attack repair. Specifically, according to the index value of the joint embedded representation, all normal joint embedded representations are selected from the adjacent fixed time range of the joint embedded representation with the same ID, the maximum normal joint embedded representation is selected as a replacement target, and the joint embedded representation with the same ID is replaced by the replacement target, so that attack repair is realized.
It should be noted that, in each round of training, the order of the merchandise samples adopted by the participants is not changed, so that the joint embedded representation of the current round which may have the backdoor attack can be locked according to the ID of the joint embedded representation of the latest backdoor attack recorded before. Furthermore, the order of the commodity samples adopted by the participants is unchanged, the time order of the joint embedded representation forming and inputting the top model is unchanged, so that the normal joint embedded representation in the adjacent fixed time range before and after the joint embedded representation with the same ID can be conveniently locked, the normal joint embedded representation is used for replacement, the repaired joint embedded representation is replaced to participate in the updating of the top model of the collaborator, the parameter optimization of the top model by the aggregation embedded representation of the backdoor attack can be improved, and the defense capacity of the top model to the backdoor attack is improved.
And 3, extracting the top model after the federal learning is finished as a commodity recommendation model capable of defending backdoor attacks.
After the federal learning is finished, a top model of a cooperative party and optimized model parameters are extracted to serve as a final commodity recommendation model, and the commodity recommendation model adopts the repairing of a commodity guide link of the backdoor attack and the filtering and repairing of the joint embedded expression of the backdoor attack in the training process, so that the commodity recommendation model has the capability of defending against the attack, and even aiming at a commodity sample of the backdoor attack, the accurate recommendation of commodities can be realized.
The above-mentioned embodiments are intended to illustrate the technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only the most preferred embodiments of the present invention, and are not intended to limit the present invention, and any modifications, additions, equivalents, etc. made within the scope of the principles of the present invention should be included in the scope of the present invention.
Claims (9)
1. A longitudinal federal learning backdoor defense method based on neuron activation value clustering is characterized by comprising the following steps:
the method comprises the steps that a longitudinal federal recommendation system comprising a plurality of participants and a cooperative party is built, and the longitudinal federal recommendation system is used for building a commodity recommendation model through longitudinal federal learning;
performing federal learning of a longitudinal federal recommendation system, comprising: each participant trains a local model by using a local commodity sample, and uploads an embedded representation corresponding to the commodity sample to a cooperative party; the cooperative party aggregates the embedded representations uploaded by all the participating parties to obtain aggregated embedded representations, obtains neuron activation values of the aggregated embedded representations in the top model to construct commodity guide links, screens the commodity guide links attacked by the backdoor by clustering the commodity guide links, repairs the commodity guide links attacked by the backdoor according to clustering results, updates parameters of the top model according to the repaired commodity guide links, and downloads the updated parameters to the participating parties for the next round of federal learning;
extracting a top model after the federal learning is finished as a commodity recommendation model capable of defending backdoor attacks;
the screening of the commodity guide links attacked by the backdoor by clustering the commodity guide links comprises the following steps:
based on a prediction label obtained by carrying out primary forward conduction calculation on the input aggregation embedded representation by the top model, carrying out classified storage on the aggregation embedded representation and the corresponding commodity guide link;
clustering all aggregation embedded representations corresponding to each type of prediction labels, screening the cluster with the minimum number of the aggregation embedded representations as a first abnormal cluster, and taking the ratio of the number of the aggregation embedded representations contained in the first abnormal cluster to the number of all the aggregation embedded representations participating in clustering as a first back door risk score;
clustering all commodity guide links corresponding to each type of prediction label, screening a cluster containing the smallest number of the commodity guide links as a second abnormal cluster, and taking the ratio of the number of the commodity guide links contained in the second abnormal cluster to the number of all the commodity guide links participating in clustering as a second back door risk score;
for each type of prediction label, comprehensively aggregating and embedding a first back door risk score corresponding to the expression and a second back door risk score corresponding to the commodity guide link to obtain a total back door risk score corresponding to each type of prediction label;
and when the total backdoor risk score is larger than a preset threshold value, considering that the aggregation embedding representation in the first abnormal clustering cluster is the joint embedding representation of backdoor attack, and simultaneously considering that the commodity guide link in the second abnormal clustering cluster is the commodity guide link of the backdoor attack.
2. The longitudinal federated learning backdoor defense method based on neuron activation value clustering according to claim 1, wherein a collaborator aggregates embedded representations uploaded by all participants by adopting a splicing operation to obtain an aggregated embedded representation.
3. The longitudinal federal learning backdoor defense method based on neuron activation value clustering according to claim 1, wherein the obtaining aggregate embedded neuron activation values represented in a top model to construct a commodity guide link comprises:
and performing forward conduction calculation on the input aggregation embedded expression once by using the top model to obtain the activation value of each layer of neurons of the top model in the forward conduction process, extracting the neuron with the maximum activation value of each layer as a commodity guide neuron, and connecting the commodity guide neurons of all layers according to the forward conduction direction to form a commodity guide link.
4. The longitudinal federal learning backdoor defense method based on neuron activation value clustering according to claim 1, wherein the classified storage of the aggregate embedded representation and the corresponding commodity guide link comprises:
and constructing a commodity index lookup table according to the prediction tags represented by aggregation embedding, wherein each type of prediction tag corresponds to each family of the commodity index lookup table, and the combined embedded representation belonging to the same type of prediction tags and the corresponding commodity guide links are arranged in the same family so as to realize classified storage according to the prediction tags.
5. The longitudinal federal learning backdoor defense method based on neuron activation value clustering as claimed in claim 1, characterized in that when all aggregation embedded representations corresponding to each type of prediction tags are clustered, two aggregation embedded representations are randomly selected as an initial clustering center, the clustering distance is the sum of the Euclidean distance from the aggregation embedded representation to be clustered to the clustering center and the cosine similarity distance from the aggregation embedded representation to be clustered to the clustering center, and after each round of clustering is finished, the clustering center is updated by adopting an average clustering algorithm according to the dimension average of all aggregation embedded representations in the clustering;
when all commodity guide links corresponding to each type of prediction label are clustered, two commodity guide links are randomly selected as an initial clustering center, the clustering distance is the sum of the Euclidean distance from the commodity guide link to be clustered to the clustering center and the cosine similarity distance from the commodity guide link to be clustered to the clustering center, and after each round of clustering is finished, the clustering center is updated by adopting an average clustering algorithm according to the dimension average of all the commodity guide links in the clustering.
6. The longitudinal federal learning backdoor defense method based on neuron activation value clustering according to claim 1, wherein the comprehensive aggregation embedding represents that the corresponding first backdoor risk score and the corresponding second backdoor risk score of a commodity guide link obtain a total backdoor risk score corresponding to each type of prediction label, and the method comprises the following steps:
and weighting and summing the corresponding first rear-entrance risk score and the corresponding second rear-entrance risk score of the commodity guide link according to the weight to obtain a total rear-entrance risk score corresponding to each type of prediction label, wherein the value of the weight is 0-1, and the sum of the weight of the first rear-entrance risk score and the weight of the second rear-entrance risk score is 1.
7. The longitudinal federal learning backdoor defense method based on neuron activation value clustering as claimed in claim 1, wherein the repairing of the commodity guide link of backdoor attack according to the clustering result comprises:
randomly selecting a normal commodity guide link from the clustering results to cover the commodity guide link attacked by the backdoor so as to realize repair;
or, calculating the average value of all normal commodity guide links in the clustering result, and covering the commodity guide link attacked by the backdoor with the average value to realize repair.
8. The longitudinal federal learning backdoor defense method based on neuron activation value clustering according to claim 1, characterized in that when a commodity guide link of a backdoor attack is determined, a joint embedded representation of the backdoor attack corresponding to the commodity guide link is obtained and recorded;
the joint embedded representation of the recorded backdoor attacks is used to guide the next round of federal learning.
9. The longitudinal federal learning backdoor defense method based on neuron activation value clustering according to claim 8, wherein the joint embedded representation of the recorded backdoor attacks is used for guiding the next round of federal learning, and comprises:
during the next round of federal learning, the cooperative party deletes the joint embedded representation with the same ID from the joint embedded representation uploaded by the participator according to the ID of the joint embedded representation of the latest backdoor attack, and the rest joint embedded representation is used for updating the top model of the participator;
or, in the next round of federal learning, the cooperative party screens the joint embedded representation with the same ID from the joint embedded representation uploaded by the participating party according to the ID of the joint embedded representation of the latest backdoor attack, and carries out attack repair on the joint embedded representation with the same ID, and the repaired joint embedded representation is used for updating the top model of the participating cooperative party;
wherein, attack repair includes: and replacing the normal maximum joint embedding representation in the adjacent fixed time range of the joint embedding representation with the same ID to realize attack repair.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210146719.0A CN114202397B (en) | 2022-02-17 | 2022-02-17 | Longitudinal federal learning backdoor defense method based on neuron activation value clustering |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210146719.0A CN114202397B (en) | 2022-02-17 | 2022-02-17 | Longitudinal federal learning backdoor defense method based on neuron activation value clustering |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114202397A CN114202397A (en) | 2022-03-18 |
| CN114202397B true CN114202397B (en) | 2022-05-10 |
Family
ID=80645624
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210146719.0A Active CN114202397B (en) | 2022-02-17 | 2022-02-17 | Longitudinal federal learning backdoor defense method based on neuron activation value clustering |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114202397B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116502709A (en) * | 2023-06-26 | 2023-07-28 | 浙江大学滨江研究院 | Heterogeneous federal learning method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287244A (en) * | 2020-10-29 | 2021-01-29 | 平安科技（深圳）有限公司 | Product recommendation method and device based on federal learning, computer equipment and medium |
| CN112836130A (en) * | 2021-02-20 | 2021-05-25 | 四川省人工智能研究院(宜宾) | Context-aware recommendation system and method based on federated learning |
| CN113298267A (en) * | 2021-06-10 | 2021-08-24 | 浙江工业大学 | Vertical federal model defense method based on node embedding difference detection |
| CN113411329A (en) * | 2021-06-17 | 2021-09-17 | 浙江工业大学 | DAGMM-based federated learning backdoor attack defense method |
| CN113919513A (en) * | 2021-10-22 | 2022-01-11 | 全球能源互联网研究院有限公司南京分公司 | Method and device for aggregating security of federated learning and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020185973A1 (en) * | 2019-03-11 | 2020-09-17 | doc.ai incorporated | System and method with federated learning model for medical research applications |
| CN112540926A (en) * | 2020-12-17 | 2021-03-23 | 杭州趣链科技有限公司 | Resource allocation fairness federal learning method based on block chain |
| CN113095512A (en) * | 2021-04-23 | 2021-07-09 | 深圳前海微众银行股份有限公司 | Federal learning modeling optimization method, apparatus, medium, and computer program product |
-
2022
- 2022-02-17 CN CN202210146719.0A patent/CN114202397B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112287244A (en) * | 2020-10-29 | 2021-01-29 | 平安科技（深圳）有限公司 | Product recommendation method and device based on federal learning, computer equipment and medium |
| CN112836130A (en) * | 2021-02-20 | 2021-05-25 | 四川省人工智能研究院(宜宾) | Context-aware recommendation system and method based on federated learning |
| CN113298267A (en) * | 2021-06-10 | 2021-08-24 | 浙江工业大学 | Vertical federal model defense method based on node embedding difference detection |
| CN113411329A (en) * | 2021-06-17 | 2021-09-17 | 浙江工业大学 | DAGMM-based federated learning backdoor attack defense method |
| CN113919513A (en) * | 2021-10-22 | 2022-01-11 | 全球能源互联网研究院有限公司南京分公司 | Method and device for aggregating security of federated learning and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| Federated learning with hierarchical clustering of local updates to improve training on non-IID data;Christopher Briggs et al.;《 2020 International Joint Conference on Neural Networks (IJCNN)》;20200928;第1-9页 * |
| 基于纵向联邦学习的推荐系统技术研究;李鸣;《中国优秀硕士学位论文数据库 信息科技辑（月刊）》;20220115(第01期);第178-185页 * |
| 联邦推荐系统的协同过滤冷启动解决方法;王健宗等;《智能系统学报》;20210131;第16卷(第1期);第I138-3112页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114202397A (en) | 2022-03-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yu et al. | CloudLeak: Large-Scale Deep Learning Models Stealing Through Adversarial Examples. | |
| CN110070183B (en) | Neural network model training method and device for weakly labeled data | |
| Dai et al. | Adversarial attack on graph structured data | |
| CN111460443B (en) | Security defense method for data manipulation attack in federated learning | |
| Ren et al. | Building an effective intrusion detection system by using hybrid data optimization based on machine learning algorithms | |
| Zhang et al. | Trustworthy graph neural networks: Aspects, methods and trends | |
| US11907955B2 (en) | System and method for blockchain automatic tracing of money flow using artificial intelligence | |
| Papernot et al. | The limitations of deep learning in adversarial settings | |
| CN111783442A (en) | Intrusion detection method, device, server and storage medium | |
| CN113204745B (en) | Deep learning back door defense method based on model pruning and reverse engineering | |
| Sharma | Z-CRIME: A data mining tool for the detection of suspicious criminal activities based on decision tree | |
| Zhang et al. | Feature augmentation for imbalanced classification with conditional mixture WGANs | |
| Nguyen et al. | Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions | |
| CN114202397B (en) | Longitudinal federal learning backdoor defense method based on neuron activation value clustering | |
| CN113283590A (en) | Defense method for backdoor attack | |
| Xiao et al. | A multitarget backdooring attack on deep neural networks with random location trigger | |
| Li et al. | Selective and collaborative influence function for efficient recommendation unlearning | |
| CN113609394A (en) | Information flow-oriented safety recommendation system | |
| Zhao et al. | Natural backdoor attacks on deep neural networks via raindrops | |
| Wang et al. | Attention‐guided black‐box adversarial attacks with large‐scale multiobjective evolutionary optimization | |
| Sheng et al. | Backdoor attack of graph neural networks based on subgraph trigger | |
| CN110347669A (en) | Risk prevention method based on streaming big data analysis | |
| Bilgin et al. | Explaining Inaccurate Predictions of Models through k-Nearest Neighbors. | |
| Lu et al. | Counting crowd by weighing counts: A sequential decision-making perspective | |
| CN115496227A (en) | Method for training member reasoning attack model based on federal learning and application |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |