CN112884161B - Cooperative learning method, device, equipment and medium for resisting label turning attack - Google Patents

Cooperative learning method, device, equipment and medium for resisting label turning attack Download PDF

Info

Publication number
CN112884161B
CN112884161B CN202110142654.8A CN202110142654A CN112884161B CN 112884161 B CN112884161 B CN 112884161B CN 202110142654 A CN202110142654 A CN 202110142654A CN 112884161 B CN112884161 B CN 112884161B
Authority
CN
China
Prior art keywords
machine learning
formula
label
learning model
attack
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110142654.8A
Other languages
Chinese (zh)
Other versions
CN112884161A (en
Inventor
王鑫
杨明
吴晓明
杨美红
穆超
陈振娅
王彪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Computer Science Center National Super Computing Center in Jinan
Original Assignee
Shandong Computer Science Center National Super Computing Center in Jinan
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Computer Science Center National Super Computing Center in Jinan filed Critical Shandong Computer Science Center National Super Computing Center in Jinan
Priority to CN202110142654.8A priority Critical patent/CN112884161B/en
Publication of CN112884161A publication Critical patent/CN112884161A/en
Application granted granted Critical
Publication of CN112884161B publication Critical patent/CN112884161B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16Y—INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]
    • G16Y40/00—IoT characterised by the purpose of the information processing
    • G16Y40/50—Safety; Security of things, users, data or systems

Abstract

The application discloses a collaborative learning method, a device, equipment and a medium for resisting label turning attack, which comprise the following steps: constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method; selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of an optimization problem; iteratively updating the machine learning model using an alternating direction multiplier method; analyzing the anti-label-flipping attack performance of the collaborative learning mechanism according to the misclassification rate of the l-risk calculation model; and calculating the generalization error of the model according to the label flipping probability, the data property and the elastic loss function property. Therefore, by skillfully fusing the elastic loss function and the alternative direction multiplier method, the classification precision of the model based on the turnover label training is ensured to be the same as that of the model obtained by correct label training, the internal relation between the turnover label and the generalization error of the learning model is further disclosed, and a foundation is laid for collaborative machine learning research under large-range error data injection attack.

Description

Cooperative learning method, device, equipment and medium for resisting label turning attack
Technical Field
The invention relates to the field of information security, in particular to a collaborative learning method, a device, equipment and a medium for resisting label flip attack.
Background
Sensing and computing nodes in the Internet of things generate abundant high-value data, and the cloud server can collect the data and deeply mine and learn the data, so that services such as personalized recommendation, health monitoring and intelligent home are provided for users, and the convenience degree of daily life of people is greatly improved. With the continuous expansion of the node scale, the machine learning method in the single machine mode cannot effectively deal with the processing task of mass data. The cooperative machine learning technology realizes cooperative calculation among the servers by reasonably scheduling a plurality of server resources, is an effective way for solving the problems, and has important research value and practical application significance.
However, with the continuous opening of the internet of things, the quality of data generated by nodes may be damaged by endless security threats, and the performance of collaborative machine learning is seriously affected. Limited by volume and power consumption, the nodes of the internet of things only have limited computing, storage and communication resources, so that complex security protection means are difficult to prepare. Therefore, in the environment of the internet of things, deploying elastic defense measures at the server end to deal with data corruption caused by malicious attacks becomes one of the problems that need to be solved in cooperation with machine learning.
In the data generated by the nodes, the labels directly determine the correction direction of the machine learning model. If the label is turned over maliciously by an attacker, a model obtained according to the training of the label generates large deviation, so that the model cannot be applied to the service of the Internet of things. In addition, in one data record, the label only occupies a small field, and particularly in the two-class machine learning problem, the label can be represented by only one binary bit, so that the cost of an attacker for implementing the label flipping attack is low. By means of the characteristics of obvious attack effect and low cost, the label turning attack obtains wide attention in the field of security machine learning, and a series of defense strategies aiming at the attack are developed. However, in the context of cooperative machine learning, there is still no elastic defense method that is effective against tag flipping attacks.
Disclosure of Invention
In view of this, the present invention provides a collaborative learning method, apparatus, device and medium for resisting tag flipping attack, which can ensure that training of a high-performance machine learning model is achieved when a training data tag is randomly flipped by a malicious attacker. The specific scheme is as follows:
a cooperative learning method for resisting label flipping attack comprises the following steps:
constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of the optimization problem;
iteratively updating the machine learning model using an alternating direction multiplier method;
calculating the misclassification rate of the updated machine learning model according to the l-risk, and analyzing the anti-label turning attack performance of the cooperative machine learning mechanism;
and calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
Preferably, in the above collaborative learning method for resisting a tag flipping attack provided by the embodiment of the present invention, the constructed optimization problem includes a first formula and a second formula; the first formula and the second formula are sequentially:
Figure BDA0002929776640000021
Figure BDA0002929776640000022
wherein, wiAnd wkRespectively representing machine learning models obtained by training the ith and the kth servers, N (w)i) Is a regularization function, a is a regularization coefficient, n is the total number of servers, miThe number of corresponding nodes when collecting node data for the ith server is (x)i,j,yi,j)，xi,jWith a representation dimension dCharacteristic field, yi,jIndicating a tag field.
Preferably, in the above collaborative learning method for resisting a label flipping attack provided by the embodiment of the present invention, the bounded hyper-sphere is represented by a third formula; the third formula is:
W＝{w|||w||2≤C}
wherein W represents the bounded hypersphere, W represents any machine learning model, | | · | | luminance2The expression is the two-norm of the vector, and C is a preset normal number.
Preferably, in the above collaborative learning method for resisting a tag flipping attack provided in the embodiment of the present invention, applying an elastic loss function to an objective function of the optimization problem specifically includes:
taking the loss function in the first formula as the elastic form of a fourth formula; the fourth formula is:
l(yi,j,<wi,xi,j>)＝1-yi,j·<wi,xi,j>
let all servers use the fourth formula as a loss function in the objective function of the optimization problem.
Preferably, in the collaborative learning method for resisting a tag flipping attack provided in the embodiment of the present invention, the iterative updating of the machine learning model by using an alternating direction multiplier method specifically includes:
solving an optimization problem given by the first formula, the second formula and the fourth formula by using an alternating direction multiplier method, and expressing a model of the ith server in the t-th iteration solution as wi(t) and using w (t) ═ w1(t)；…；wn(t)) aggregating n servers to perform iterative updating solution of the machine learning model by adopting a fifth formula, a sixth formula and a seventh formula in the t round; the fifth formula, the sixth formula and the seventh formula are sequentially:
Figure BDA0002929776640000031
Figure BDA0002929776640000032
γ(t+1)-γ(t)-βV-w(t+1)＝0
wherein J is an objective function of the optimization problem,
Figure BDA0002929776640000033
expressing the gradient of the objective function J to the machine learning model, gamma (t) expressing Lagrange multiplier, beta being penalty coefficient, V+And V-An extended unsigned laplacian matrix and an extended signed laplacian matrix representing the server communication topology, respectively.
Preferably, in the above method for collaborative learning to resist a tag flipping attack provided in the embodiment of the present invention, analyzing the performance of the cooperative machine learning mechanism to resist the tag flipping attack specifically includes:
judging whether the updated machine learning model is the machine learning model with the minimized l-risk or not according to the calculated misclassification rate;
and if so, judging that the cooperative machine learning mechanism has the elastic performance of resisting the label overturning attack.
Preferably, in the collaborative learning method for resisting the label flipping attack provided by the embodiment of the present invention, an eighth formula is used to calculate a generalization error of the updated machine learning model; the eighth formula is:
Figure BDA0002929776640000034
wherein the content of the first and second substances,
Figure BDA0002929776640000041
P0the probability is represented by the number of bits in the bit stream,
Figure BDA0002929776640000042
indicating the label after flipping, p-1And p+1To be controlled by an attackerThe label turnover probability of (1) satisfies that p is more than or equal to 0-y,
Figure BDA0002929776640000043
Delta epsilon (0,1) is the convergence probability,
Figure BDA0002929776640000044
representing an updated machine learning model, w*Optimum machine learning model found for known data distribution P, Pmax＝{p+1,p-1Denotes the maximum value of the label flip probability, X is the data feature vector Xi,j,
Figure BDA0002929776640000045
Upper bound of j, L1Represents the elastic loss function l (y)i,j,<wi,xi,j>) The upper bound of (c).
The embodiment of the invention also provides a cooperative learning device for resisting the label flip attack, which comprises:
the optimization problem construction module is used for constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
the elastic mechanism establishing module is used for selecting a bounded hyper-sphere as a candidate set of a machine learning model and applying an elastic loss function to a target function of the optimization problem;
the model iteration updating module is used for iteratively updating the machine learning model by using an alternate direction multiplier method;
the misclassification rate calculation module is used for calculating the misclassification rate of the updated machine learning model according to the l-risk and analyzing the anti-label turning attack performance of the collaborative machine learning mechanism;
and the generalization error calculation module is used for calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
The embodiment of the invention also provides a cooperative learning device for resisting the label flipping attack, which comprises a processor and a memory, wherein the processor realizes the cooperative learning method for resisting the label flipping attack provided by the embodiment of the invention when executing the computer program stored in the memory.
The embodiment of the present invention further provides a computer-readable storage medium, configured to store a computer program, where the computer program, when executed by a processor, implements the above collaborative learning method for resisting a tag flipping attack.
According to the technical scheme, the cooperative learning method for resisting the label flipping attack, provided by the invention, comprises the following steps: constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method; selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of an optimization problem; iteratively updating the machine learning model using an alternating direction multiplier method; calculating the misclassification rate of the updated machine learning model according to the l-risk, and analyzing the anti-label turning attack performance of the cooperative machine learning mechanism; and calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
In order to effectively resist the influence of label turning attack on a learning model, an elastic defense mechanism is designed for a server side, the elastic defense on the label turning attack is realized by skillfully integrating an elastic loss function and an alternative direction multiplier method, the classification precision of a model based on turning label training is ensured to be the same as that of a model obtained by correct label training, the internal relation between a turning label and a machine learning model generalization error is further revealed, and a foundation is laid for cooperative machine learning research under large-range error data injection attack. In addition, the invention also provides a corresponding device, equipment and a computer readable storage medium aiming at the label turning attack resisting collaborative learning method, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in related arts, the drawings used in the description of the embodiments or related arts will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a flowchart of a collaborative learning method for resisting a tag flipping attack according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a label flipping attack for collaborative machine learning according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a cooperative learning apparatus for resisting a tag flipping attack according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a collaborative learning method for resisting label flipping attack, which comprises the following steps as shown in figure 1:
s101, constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
s102, selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of an optimization problem;
s103, iteratively updating the machine learning model by using an alternative direction multiplier method;
s104, calculating the misclassification rate of the updated machine learning model according to the l-risk, and analyzing the anti-label turning attack performance of the cooperative machine learning mechanism;
and S105, calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
In the collaborative learning method for resisting the label flipping attack provided by the embodiment of the invention, firstly, an optimization problem of collaborative machine learning is established by using an empirical risk minimization method, then a bounded hyper-sphere is selected as a candidate machine learning model, a server applies an elastic loss function to a target function of the optimization problem, then the machine learning model is iteratively updated by using an alternative direction multiplier method, the misclassification rate of the machine learning model is calculated according to l-risk, the label flipping attack resistance of the collaborative machine learning mechanism is analyzed, the generalization error of the model is further obtained according to the label flipping probability, the data property and the elastic loss function property, thus an elastic defense mechanism is designed for a server end, the elastic defense of the label flipping attack can be realized by skillfully fusing the elastic loss function and the alternative direction multiplier method, the model based on the flipping label training is ensured to realize the same classification precision as the model obtained by correct label training, and further reveals the internal relation between the turnover label and the machine learning model generalization error, and lays a foundation for collaborative machine learning research under the large-range error data injection attack.
It should be noted that before step S101 is executed, a tag flipping attack model including two types of tag flipping probabilities may be defined. As shown in fig. 2, when there is a malicious attacker that can control all nodes to send data or tamper with data transmitted in a communication channel, a specific attack manner is to obtain a true tag of a node, and then randomly flip the tag according to the following pattern:
Figure BDA0002929776640000061
wherein, P0Representing the probability, yi,jA field of a presentation tag is provided,
Figure BDA0002929776640000062
indicating the label after flipping, p-1And p+1For the label turnover probability controlled by an attacker, p is more than or equal to 0-y,
Figure BDA0002929776640000063
Further, in specific implementation, in the above collaborative learning method for resisting a tag flipping attack provided in the embodiment of the present invention, when there are n servers participating in collaborative machine learning, the optimization problem constructed in step S101 includes a first formula and a second formula; the first formula and the second formula may be, in order:
Figure BDA0002929776640000064
Figure BDA0002929776640000065
wherein, wiAnd wkRespectively representing machine learning models obtained by training the ith and the kth servers, N (w)i) Is a regularization function, a is a regularization coefficient, n is the total number of servers, miThe number of corresponding nodes when collecting node data for the ith server is (x)i,j,yi,j)，xi,jA feature field of dimension d, yi,jIndicating a tag field.
In specific implementation, in the above collaborative learning method for resisting a tag flipping attack provided by the embodiment of the present invention, in step S102, the candidate set of machine learning models is set as a bounded hyper-sphere, and the bounded hyper-sphere may be represented by a third formula; the third formula is:
W＝{w|||w||2≤C} (3)
wherein W represents a bounded hypersphere, W represents any machine learning model, | | · | | luminancexThe expression is the two-norm of the vector, and C is a preset normal number.
In specific implementation, in the above collaborative learning method for resisting a tag flipping attack provided in the embodiment of the present invention, the step S102 applies an elastic loss function to an objective function of an optimization problem, which may specifically include applying the elastic loss function to the objective function of the optimization problem: the loss function l (y) in the first formulai,j,<wi,xi,j>) May take the elastic form of a fourth formula; the fourth formula is:
l(yi,j,<wi,xi,j>)＝1-yi,j·<wi,xi,j> (4)
let all servers use the fourth formula as a loss function in the objective function of the optimization problem.
In specific implementation, in the above collaborative learning method for resisting a tag flipping attack provided in the embodiment of the present invention, step S103 iteratively updates the machine learning model by using an alternating direction multiplier method, which may specifically include: and solving an optimization problem given by the first formula, the second formula and the fourth formula by using an alternative direction multiplier method, wherein each server only needs to interact with part of servers to train results to complete the cooperative calculation of the machine learning model. Solving the optimization problem in an iterative mode by an alternative direction multiplier method, and expressing a model of an ith server in the iterative solution of the t-th round as wi(t) and using w (t) ═ w1(t)；…；wn(t)) aggregating n servers, in the t-th round, performing iterative update solution on the machine learning model by adopting a fifth formula, a sixth formula and a seventh formula; the fifth formula, the sixth formula and the seventh formula are sequentially as follows:
Figure BDA0002929776640000071
Figure BDA0002929776640000072
γ(t+1)-γ(t)-βV-w(t+1)＝0 (7)
wherein J is an objective function of the optimization problem, V represents the gradient of solving the objective function J to the machine learning model, gamma (t) represents a Lagrange multiplier, beta is a penalty coefficient, and V is a penalty coefficient+And V-Extended unsigned Laplace matrices andthe signed laplacian matrix is extended.
In specific implementation, in the above collaborative learning method for resisting tag flipping attack provided in the embodiment of the present invention, step S104 calculates the misclassification rate of the updated machine learning model according to the l-risk, and analyzes the performance of the collaborative machine learning mechanism for resisting tag flipping attack, which may specifically include:
defining the l-risk of the machine learning model w as Rl,P＝Ε(x,y)∝Pl (y, < w, x >), where P denotes the distribution of node data, E(x,y)∝PRepresenting the mathematical expectation of the loss function l (-) under the data distribution P, and then calculating the misclassification rate of the updated machine learning model according to the defined l-risk;
by using
Figure BDA0002929776640000081
A data distribution representing a tag flip attack,
Figure BDA0002929776640000082
representing the updated machine learning model, for any model W ∈ W, there are:
Figure BDA0002929776640000083
judging whether the updated machine learning model is the machine learning model with the minimized l-risk or not according to the calculated misclassification rate; since 0 is not more than p-y,
Figure BDA0002929776640000084
Then there is
Figure BDA0002929776640000085
Description of the invention
Figure BDA0002929776640000086
Is also a machine learning model that minimizes l-risk, and thus a machine learning model h trained with correct labels*The same misclassification rate exists; at this time, the cooperative device is judgedThe learning mechanism has the elastic performance of resisting label overturning attack.
In specific implementation, in the above collaborative learning method for resisting tag flipping attack provided in the embodiment of the present invention, on the basis of the defined tag flipping attack model, all servers use the sixth formula and the seventh formula to solve the optimization problem with the fourth formula as a loss function, and then in step S105, the eighth formula may be used to calculate and train to obtain a generalization error of the updated machine learning model; the eighth formula is:
Figure BDA0002929776640000087
where δ ∈ (0,1) is the convergence probability, w*Optimum machine learning model found for known data distribution P, Pmax＝{p+1,p-1Denotes the maximum value of the label flip probability, X is the data feature vector Xi,j,
Figure BDA0002929776640000088
Upper bound of j, L1Represents the elastic loss function l (y)i,j,<wi,xi,j>) The upper bound of (c).
Based on the same inventive concept, the embodiment of the invention also provides a cooperative learning device for resisting the label flip attack, and as the problem solving principle of the device is similar to that of the cooperative learning method for resisting the label flip attack, the implementation of the device can refer to the implementation of the cooperative learning method for resisting the label flip attack, and repeated parts are not repeated.
In specific implementation, the cooperative learning apparatus for resisting a tag flipping attack provided by the embodiment of the present invention, as shown in fig. 3, specifically includes:
the optimization problem construction module 11 is used for constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
an elastic mechanism establishing module 12, configured to select a bounded hyper-sphere as a candidate set of a machine learning model, and apply an elastic loss function to a target function of an optimization problem;
a model iteration updating module 13, configured to update the machine learning model iteratively by using an alternating direction multiplier method;
the misclassification rate calculation module 14 is used for calculating the misclassification rate of the updated machine learning model according to the l-risk and analyzing the label turning attack resistance of the collaborative machine learning mechanism;
and the generalization error calculation module 15 is configured to calculate a generalization error of the updated machine learning model according to the label flipping probability, the data property, and the elastic loss function property.
In the cooperative learning device for resisting the label flipping attack, provided by the embodiment of the invention, elastic defense for the label flipping attack can be realized through interaction of the five modules, the model based on the flipping label training is ensured to realize the classification precision same as that of the model obtained by correct label training, the internal relation between the flipping label and the generalization error of the machine learning model is further disclosed, and a foundation is laid for cooperative machine learning research under large-scale error data injection attack.
For more specific working processes of the modules, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Correspondingly, the embodiment of the invention also discloses a cooperative learning device for resisting the label turning attack, which comprises a processor and a memory; when the processor executes the computer program stored in the memory, the cooperative learning method for resisting the tag flipping attack disclosed in the foregoing embodiment is implemented.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
Further, the present invention also discloses a computer readable storage medium for storing a computer program; the computer program is executed by a processor to realize the cooperative learning method for resisting the label turning attack disclosed in the foregoing.
For more specific processes of the above method, reference may be made to corresponding contents disclosed in the foregoing embodiments, and details are not repeated here.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device, the equipment and the storage medium disclosed by the embodiment correspond to the method disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the method part for description.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The embodiment of the invention provides a label flip attack resistant collaborative learning method, which comprises the following steps: constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method; selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of an optimization problem; iteratively updating the machine learning model using an alternating direction multiplier method; calculating the misclassification rate of the updated machine learning model according to the l-risk, and analyzing the anti-label turning attack performance of the cooperative machine learning mechanism; and calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property. Because the collaborative machine learning can realize the excavation and learning of mass data, the label turning attack can seriously affect the performance of the learning model, in order to effectively resist the influence of the label turning attack on the learning model, an elastic defense mechanism is designed for a server side, the elastic defense of the label turning attack can be realized by skillfully integrating an elastic loss function and an alternative direction multiplier method, the classification precision of the model based on the turning label training is ensured to be the same as that of the model obtained by correct label training, the internal relation between the turning label and the generalization error of the machine learning model is further revealed, and the foundation is laid for the collaborative machine learning research under the large-range error data injection attack. In addition, the invention also provides a corresponding device, equipment and a computer readable storage medium aiming at the label turning attack resisting collaborative learning method, so that the method has higher practicability, and the device, the equipment and the computer readable storage medium have corresponding advantages.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The method, the device, the equipment and the medium for the cooperative learning against the label flipping attack provided by the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the above embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (9)

1. A collaborative learning method for resisting label flipping attack is characterized by comprising the following steps:
constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
selecting a bounded hyper-sphere as a candidate set of a machine learning model, and applying an elastic loss function to a target function of the optimization problem; the bounded hyper-sphere is represented by a third formula; the third formula is:
W＝{w|||w||2≤C}
wherein W represents the bounded hypersphere, W represents any machine learning model, | | · | | luminance2Representing the two norms of the vector, wherein C is a preset normal number;
iteratively updating the machine learning model using an alternating direction multiplier method;
calculating the misclassification rate of the updated machine learning model according to the l-risk, and analyzing the anti-label turning attack performance of the cooperative machine learning mechanism; the l-risk of the machine learning model w is defined as Rl,P＝E(x,y)∝Pl(y,<w,x>) Where P denotes the distribution of node data (x, y), E(x,y)∝PRepresents the mathematical expectation of the loss function l (-) under the data distribution P;
and calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
2. The cooperative learning method for resisting label flipping attacks according to claim 1, wherein the constructed optimization problem comprises a first formula and a second formula; the first formula and the second formula are sequentially:
Figure FDA0003252623250000011
Figure FDA0003252623250000012
wherein, wiAnd wkRespectively representing machine learning models obtained by training the ith and the kth servers, N (w)i) Is a regularization function, a is a regularization coefficient, n is the total number of servers, miThe number of corresponding nodes when collecting node data for the ith server is (x)i,j,yi,j)，xi,jA feature field of dimension d, yi,jIndicating a tag field.
3. The cooperative learning method for resisting label flipping attack according to claim 2, wherein applying an elastic loss function to the objective function of the optimization problem specifically comprises:
taking the loss function in the first formula as the elastic form of a fourth formula; the fourth formula is:
l(yi,j,<wi,xi,j>)＝1-yi,j·<wi,xi,j>
let all servers use the fourth formula as a loss function in the objective function of the optimization problem.
4. The cooperative learning method for resisting tag flipping attacks according to claim 3, wherein the iterative updating of the machine learning model by using an alternating direction multiplier method specifically comprises:
solving an optimization problem given by the first formula, the second formula and the fourth formula by using an alternating direction multiplier method, and expressing a model of the ith server in the t-th iteration solution as wi(t) and using w (t) ═ w1(t)；…；wn(t)) polymerizationThe n servers adopt a fifth formula, a sixth formula and a seventh formula to carry out iterative updating solution on the machine learning model in the t round; the fifth formula, the sixth formula and the seventh formula are sequentially:
Figure FDA0003252623250000021
Figure FDA0003252623250000022
γ(t+1)-γ(t)-βV-w(t+1)＝0
wherein J is an objective function of the optimization problem,
Figure FDA0003252623250000023
expressing the gradient of the objective function J to the machine learning model, gamma (t) expressing Lagrange multiplier, beta being penalty coefficient, V+And V-An extended unsigned laplacian matrix and an extended signed laplacian matrix representing the server communication topology, respectively.
5. The cooperative learning method for resisting tag flipping attack according to claim 4, wherein analyzing the tag flipping attack resistance of the cooperative machine learning mechanism specifically comprises:
judging whether the updated machine learning model is the machine learning model with the minimized l-risk or not according to the calculated misclassification rate;
and if so, judging that the cooperative machine learning mechanism has the elastic performance of resisting the label overturning attack.
6. The cooperative learning method for resisting label flipping attack according to claim 5, wherein an eighth formula is used to calculate a generalization error of the updated machine learning model; the eighth formula is:
Figure FDA0003252623250000024
wherein the content of the first and second substances,
Figure FDA0003252623250000025
P0the probability is represented by the number of bits in the bit stream,
Figure FDA0003252623250000026
indicating the label after flipping, p-1And p+1For the label turning probability controlled by an attacker, the method meets the requirement
Figure FDA0003252623250000027
Delta epsilon (0,1) is the convergence probability,
Figure FDA0003252623250000028
representing an updated machine learning model, w*Optimum machine learning model found for known data distribution P, Pmax＝{p+1,p-1Denotes the maximum value of the label flip probability, X is the data feature vector Xi,j,
Figure FDA0003252623250000031
Upper bound of, L1Represents the elastic loss function l (y)i,j,<wi,xi,j>) The upper bound of (c).
7. A cooperative learning device for resisting label flipping attack is characterized by comprising:
the optimization problem construction module is used for constructing an optimization problem of collaborative machine learning by using an empirical risk minimization method;
the elastic mechanism establishing module is used for selecting a bounded hyper-sphere as a candidate set of a machine learning model and applying an elastic loss function to a target function of the optimization problem; the bounded hyper-sphere is represented by a third formula; the third formula is:
W＝{w|||w||2≤C}
wherein W represents the bounded hypersphere, W represents any machine learning model, | | · | | luminance2Representing the two norms of the vector, wherein C is a preset normal number;
the model iteration updating module is used for iteratively updating the machine learning model by using an alternate direction multiplier method;
the misclassification rate calculation module is used for calculating the misclassification rate of the updated machine learning model according to the l-risk and analyzing the anti-label turning attack performance of the collaborative machine learning mechanism; the l-risk of the machine learning model w is defined as Rl,P＝E(x,y)∝Pl(y,<w,x>) Where P denotes the distribution of node data (x, y), E(x,y)∝PRepresents the mathematical expectation of the loss function l (-) under the data distribution P;
and the generalization error calculation module is used for calculating the generalization error of the updated machine learning model according to the label overturning probability, the data property and the elastic loss function property.
8. A cooperative learning apparatus for resisting a tag flipping attack, comprising a processor and a memory, wherein the processor implements the cooperative learning method for resisting a tag flipping attack according to any one of claims 1 to 6 when executing a computer program stored in the memory.
9. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the collaborative learning method for tag flipping attack resistance according to any of claims 1 to 6.
CN202110142654.8A 2021-02-02 2021-02-02 Cooperative learning method, device, equipment and medium for resisting label turning attack Active CN112884161B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110142654.8A CN112884161B (en) 2021-02-02 2021-02-02 Cooperative learning method, device, equipment and medium for resisting label turning attack

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110142654.8A CN112884161B (en) 2021-02-02 2021-02-02 Cooperative learning method, device, equipment and medium for resisting label turning attack

Publications (2)

Publication Number Publication Date
CN112884161A CN112884161A (en) 2021-06-01
CN112884161B true CN112884161B (en) 2021-11-02

Family

ID=76055749

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110142654.8A Active CN112884161B (en) 2021-02-02 2021-02-02 Cooperative learning method, device, equipment and medium for resisting label turning attack

Country Status (1)

Country Link
CN (1) CN112884161B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222848A (en) * 2019-05-05 2019-09-10 阿里巴巴集团控股有限公司 The determination method and device for the integrated model that computer executes
CN110827351A (en) * 2020-01-09 2020-02-21 西南交通大学 Automatic generation method of voice tag of new target for robot audio-visual collaborative learning
CN111260614A (en) * 2020-01-13 2020-06-09 华南理工大学 Convolutional neural network cloth flaw detection method based on extreme learning machine
CN111325324A (en) * 2020-02-20 2020-06-23 浙江科技学院 Deep learning confrontation sample generation method based on second-order method
CN111581466A (en) * 2020-05-15 2020-08-25 北京交通大学 Multi-label learning method for characteristic information with noise
CN112200380A (en) * 2020-10-23 2021-01-08 支付宝(杭州)信息技术有限公司 Method and device for optimizing risk detection model
CN112232524A (en) * 2020-12-14 2021-01-15 北京沃东天骏信息技术有限公司 Multi-label information identification method and device, electronic equipment and readable storage medium

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10229195B2 (en) * 2017-06-22 2019-03-12 International Business Machines Corporation Relation extraction using co-training with distant supervision
CN107577945B (en) * 2017-09-28 2021-03-23 创新先进技术有限公司 URL attack detection method and device and electronic equipment
GB2591374B (en) * 2018-08-06 2023-05-10 Walmart Apollo Llc Artificial intelligence system and method for auto-naming customer tree nodes in a data structure

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110222848A (en) * 2019-05-05 2019-09-10 阿里巴巴集团控股有限公司 The determination method and device for the integrated model that computer executes
CN110827351A (en) * 2020-01-09 2020-02-21 西南交通大学 Automatic generation method of voice tag of new target for robot audio-visual collaborative learning
CN111260614A (en) * 2020-01-13 2020-06-09 华南理工大学 Convolutional neural network cloth flaw detection method based on extreme learning machine
CN111325324A (en) * 2020-02-20 2020-06-23 浙江科技学院 Deep learning confrontation sample generation method based on second-order method
CN111581466A (en) * 2020-05-15 2020-08-25 北京交通大学 Multi-label learning method for characteristic information with noise
CN112200380A (en) * 2020-10-23 2021-01-08 支付宝(杭州)信息技术有限公司 Method and device for optimizing risk detection model
CN112232524A (en) * 2020-12-14 2021-01-15 北京沃东天骏信息技术有限公司 Multi-label information identification method and device, electronic equipment and readable storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Diverse reduct subspaces based co-training for partially labeled data;Duoqian Miao 等;《International Journal of Approximate Reasoning》;20111130;第52卷(第8期);第1103-1117页 *
半监督的文本分类—两阶段协同学习;郝秀兰 等;《小型微型计算机系统》;20091031;第30卷(第10期);第1921-1926页 *
基于门限代理签名的协同学习研究;李凤英 等;《电化教育研究》;20131231(第10期);第66-70页 *

Also Published As

Publication number Publication date
CN112884161A (en) 2021-06-01

Similar Documents

Publication Publication Date Title
CN112329073B (en) Distributed data processing method, device, computer equipment and storage medium
CN111541570B (en) Cloud service QoS prediction method based on multi-source feature learning
Ray et al. A surrogate assisted parallel multiobjective evolutionary algorithm for robust engineering design
Jia et al. A semi-supervised online sequential extreme learning machine method
Robinson et al. Physics guided neural networks for modelling of non-linear dynamics
Hireche et al. Deep data plane programming and AI for zero-trust self-driven networking in beyond 5G
Abdelbari et al. A computational intelligence‐based method to ‘learn’causal loop diagram‐like structures from observed data
CN108009635A (en) A kind of depth convolutional calculation model for supporting incremental update
CN112748941A (en) Feedback information-based target application program updating method and device
Yao et al. Predicting long-term impact of CQA posts: a comprehensive viewpoint
Shi et al. Twin bridge transfer learning for sparse collaborative filtering
CN114707641A (en) Training method, device, equipment and medium for neural network model of double-view diagram
Belenguer et al. GöwFed: A novel federated network intrusion detection system
CN112884161B (en) Cooperative learning method, device, equipment and medium for resisting label turning attack
Zhang et al. Quality of web service prediction by collective matrix factorization
CN116992151A (en) Online course recommendation method based on double-tower graph convolution neural network
Xu et al. Distributed information-theoretic semisupervised learning for multilabel classification
CN115965078A (en) Classification prediction model training method, classification prediction method, device and storage medium
CN115631008B (en) Commodity recommendation method, device, equipment and medium
Huang et al. Generalized regression neural network optimized by genetic algorithm for solving out-of-sample extension problem in supervised manifold learning
CN114896977A (en) Dynamic evaluation method for entity service trust value of Internet of things
Ge et al. A cooperative framework of learning automata and its application in tutorial-like system
Smahi et al. An encoder-decoder architecture for the prediction of web service qos
CN112231571A (en) Information data processing method, device, equipment and storage medium
Ma et al. Federated Learning Framework Based on Data Value Evaluation in Industrial IoT

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant