CN116961939B - Trusted federation learning method based on alliance chain - Google Patents
Trusted federation learning method based on alliance chain Download PDFInfo
- Publication number
- CN116961939B CN116961939B CN202210394304.5A CN202210394304A CN116961939B CN 116961939 B CN116961939 B CN 116961939B CN 202210394304 A CN202210394304 A CN 202210394304A CN 116961939 B CN116961939 B CN 116961939B
- Authority
- CN
- China
- Prior art keywords
- client
- gradient
- node
- model
- federation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000012549 training Methods 0.000 claims abstract description 41
- 238000007906 compression Methods 0.000 claims abstract description 22
- 230000006835 compression Effects 0.000 claims abstract description 18
- 230000007246 mechanism Effects 0.000 claims description 16
- 238000001816 cooling Methods 0.000 claims description 6
- 230000003542 behavioural effect Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000009977 dual effect Effects 0.000 claims 1
- 238000007620 mathematical function Methods 0.000 claims 1
- 230000002829 reductive effect Effects 0.000 abstract description 14
- 231100000572 poisoning Toxicity 0.000 abstract description 13
- 230000000607 poisoning effect Effects 0.000 abstract description 13
- 238000004891 communication Methods 0.000 abstract description 6
- 238000013461 design Methods 0.000 abstract description 4
- 230000006399 behavior Effects 0.000 description 19
- 230000002776 aggregation Effects 0.000 description 9
- 238000004220 aggregation Methods 0.000 description 9
- 238000002474 experimental method Methods 0.000 description 8
- 238000009826 distribution Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000012360 testing method Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000004806 packaging method and process Methods 0.000 description 3
- 238000012856 packing Methods 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/08—Network architectures or network communication protocols for network security for authentication of entities
- H04L63/083—Network architectures or network communication protocols for network security for authentication of entities using passwords
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
- G06F21/57—Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
- H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
- H04L63/1416—Event detection, e.g. attack signature detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3297—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving time stamps, e.g. generation of time stamps
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention aims to design a trusted federation learning method based on a federation chain. Firstly, the problem of single point of failure in traditional federal learning is solved by using a blockchain; secondly, communication pressure is reduced by using gradient compression, and member reasoning attack can be prevented by uploading the compressed gradient, so that the risk of gradient leakage is reduced; then, when the global model is aggregated, parameter selection is carried out according to the model quality, so that the influence of poisoning attack and low-quality data on the global model is reduced; finally, a PBFT consensus algorithm based on double factor contribution is designed, and the high-quality nodes can be stimulated to participate in system training on the premise of preventing system centralization. The invention can improve the poisoning attack resistance of federal learning by 30%, reduce the risk of gradient leakage and improve the communication efficiency.
Description
Technical Field
The invention relates to the technical field of blockchains, in particular to a trusted federation learning method based on a federation chain.
Background
In the era of interconnection of things, big data application has two major dilemmas: 1. barriers exist among different industries and different departments, so that a data island is formed; 2. data security and data privacy cannot be guaranteed. In order to solve the problems, google corporation proposed a federal learning architecture in 2016, clients participating in federal learning do not need to upload local data, only need to upload local parameters of each round of training, aggregate into a global model through a central server, and all clients can jointly complete machine learning tasks under the condition of not sharing original data. However, conventional federal learning requires a trusted third party server to aggregate the global model, thereby introducing a single point of failure threat to the federal learning system. Moreover, the third party server can acquire the updated gradient uploaded by the client, and even if the third party server only has the gradient information of the model, an attacker can also reversely push out part of the original data, so that gradient leakage is also a threat. Finally, traditional federal learning lacks an incentive mechanism to attract clients to participate in training. The blockchain technology has the characteristics of decentralization, traceability, tamper resistance and the like, so that the blockchain technology can effectively solve a plurality of pain points of federal learning.
Based on the problem, the inventor designs a federation learning framework based on a federation chain in a trusted federation learning method based on the federation chain, solves the problems of single-point fault and gradient leakage of the traditional federation learning, and improves the capability of federation learning for resisting poisoning attack.
Disclosure of Invention
In order to solve the single point failure risk of federal learning, the invention introduces a blockchain technology, and utilizes the distributed processing capacity of the blockchain to replace a central server. And secondly, performing model training by gradient verification and selection of node gradients meeting the precision requirement, so that the threat of poisoning attack to the model is reduced. Then, gradient compression technology is utilized, so that the threat of gradient leakage in the federal learning training process is reduced, and the communication pressure is reduced. Finally, a block chain consensus algorithm based on the two-factor contribution is provided, and the block chain accounting node is selected according to the contribution size. And provides incentives for clients participating in model training using the incentives mechanism of the blockchain.
A trusted federation learning method based on a federation chain comprises the following steps:
(1) The model requester performs task initialization;
(2) The client trains the model locally by utilizing local data;
(3) Uploading the sparse model gradient by the client;
(4) Block link point validating the transaction of the client;
(5) Selecting a leader node and a consensus committee in a block chain consensus stage;
(6) In the block release stage, the leader node calculates the training results of the round and packages and releases the blocks;
(7) The client downloads the global model and retrains the phase;
Drawings
FIG. 1 is a flow chart of a trusted federation learning method based on federation chains of the present invention.
Fig. 2 is a graph of the results of the present invention for different compression rates during training of 50 client nodes in an independent co-distribution scenario.
Fig. 3 is a graph of the results of different compression ratios during training of 50 client nodes under non-independent co-distribution conditions according to the present invention.
FIG. 4 is a graph showing the results of model loss functions of different poisoning rates and selection rates under the condition of independent identical distribution.
FIG. 5 is a graph of the model loss function results for different poisoning rates and selection rates under non-independent and uniform distribution conditions.
FIG. 6 is a graph of the behavior contribution versus blockchain length for 10 blockchain nodes used in the present invention.
FIG. 7 is a graph of completion of behavior versus number of blocks packed when 10 blockchain nodes are used in accordance with the present invention.
Fig. 8 is a graph of the common-knowledge delay when different numbers of clients participate in training according to the present invention.
Detailed Description
For the purpose of promoting an understanding of the principles and advantages of the invention, reference will now be made in detail to the drawings and specific examples.
The invention relates to the technical field of information management, in particular to a federation learning method based on a federation chain. As shown in fig. 1, a federation learning method based on federation chains includes the steps of:
(1) Task initialization: the model requester issues a task request, and designs and initializes a federal learning model;
(2) Client local training: the client downloads the latest global model from the consensus node bound thereto, and then uses the global model to replace the local model parameters and uses the local privacy dataset for training. In order to improve training efficiency, the client adopts small Batch gradient descent (Mini-Batch GRADIENT DESCENT, MBGD) for training, and the calculation formula is as follows:
Wherein D b represents a small-lot training dataset, Model parameters representing the kth round of client i,Representing the gradient of the model on D b, η is the learning rate, D n is the training sample,Representing the parameters asThe model of (c) is lost at d n. Gradient compression can effectively prevent gradient leakage, communication pressure can be reduced, and overall communication efficiency of the system is improved. The scheme adopts gradient sparsification to realize gradient compression. The gradient compression process is as follows. After the client node i finishes the kth round of training, the gradient of the round is obtainedWill beAnd residual gradientAdding, i.e.Then each layer of gradient of the accumulated residual is sorted according to absolute value, and each layer of gradient is selected according to compression rate CCompression threshold of (2)For layer I networks, the absolute value will be less thanThe gradient values of (2) are accumulated into a local residual gradient and then set to 0, and the gradient sparsification process is formulated as follows:
(3) The client uploads local parameters: after training, the client node uploads metadata in the form of transactions, compressed local model gradients (1.Ltoreq.k.ltoreq.T) to a consensus node. Wherein,Is the local model gradient of client i for this round of federal learning submission.
(4) Node verification stage: after receiving the transaction of the client node, the consensus node verifies the digital signature of the client node by using the public key of the client, and forwards the transaction to other blockchain nodes after confirming the identity, and performs parameter quality scoring.
(5) After all client transactions are received or a maximum waiting period T await is reached, a consensus committee node election phase is entered. Election criteria are the blockchain node's contribution to the system, which is divided into model contribution and behavioral contribution.Representing the score of the kth round of client i,Representing model contributions of the kth round of blockchain nodes j. The several blockchain consensus nodes with the highest model contribution form the consensus committee. In order to improve the stability of the system, a node behavior contribution assessment mechanism is designed, and a node with the highest behavior contribution in the consensus committee becomes a leader node of the round. In order to reduce the centralisation of the system, a node cooling mechanism is added when electing the leader node, i.e. the node that has been elected as the leader is no longer elected as the leader node in the next rounds.
(6) The leader node generates a new block: the leader node packages all legal transactions of the round into a new Block k, the hash value hash k-1 of the previous Block, the hash value hash k of the current Block, the timestamp k, the public key of the leader, the digital signature of the leader and the global gradient g k of the round are stored in the Block header, and the Block body stores all legal transactions in a data structure of a Merkle tree. In order to reduce the influence of poisoning attack on the model, the client-side score is used as a selection standard when the global gradient is calculated, and the client-side gradient is selected according to the selection rate S to aggregate the global gradient. After passing the authentication of the two-thirds common committee node, the new block is broadcasted to other block chain nodes for uplink preservation.
(7) The client downloads the block and continues training: and the block chain link receiving the new block forwards the new block to all client nodes bound with the new block, and the client updates the local model according to the global gradient therein after downloading and starts the next training round.
Repeating the steps until the training times reach the federal learning target training round number.
In order to improve the stability and efficiency of a federation learning system based on a federation chain, the invention designs a PBFT consensus mechanism based on double factor contribution, and the mechanism firstly contributes to election of a consensus committee according to a federation learning model, and then elects a leader node through node behavior contribution, so that a client with a high-quality data set can be effectively stimulated to participate in model training, and meanwhile, the stability of a blockchain system is improved.
Firstly, a model contribution evaluation mechanism is adopted, wherein block chain nodes in a system are divided into a leader node, a committee node and a common node, the committee node is selected from the common node, and the leader node is the committee node with the highest contribution. Each node binds several federal learning clients. In each round of training, clients are scored according to the quality of parameters submitted by the clients. The contribution of each blockchain node is determined by the overall score of the client to which it is bound. But only calculating the current round of contribution is unfair to nodes with higher history contributions but lower round of contribution, so the node history contribution needs to be considered. To prevent nodes with higher historical contributions from decreasing the enthusiasm to participate in the system, higher weights need to be given to the newer contribution values. Therefore, in order to comprehensively consider the historical contribution of the node, the invention uses a time decay function based on Newton's law of cooling to calculate the historical contribution value of the node, and the calculation formula is as follows: Wherein k is the current training wheel number, The contribution score of j in the round r is obtained by summing scores of client nodes bound with the contribution score, e -α(k-r) is weight of the score in the round k, the more new contribution score weight is larger, alpha is an attenuation factor of Newton's law of cooling, and the larger alpha is, the faster the weight is attenuated.
Blockchain systems require nodes to participate in maintenance together, and the stability of the system depends on the behavior of the nodes. Therefore, the invention introduces the behavior contribution mechanism of the node to reflect the performance and the performance of the node, and increases the probability of the trusted node to participate in the transaction of the blockchain system through the behavior contribution mechanism of the node. Nodes that successfully complete the system task will get the behavior contribution score and nodes that fail to complete the job on time will be deducted from the partial behavior contribution score. After electing the consensus committee node, the node's behavioral contribution is used to elect the leader node of the packaging block from. The behavior of the node includes: packing blocks, participating committee consensus, validating client parameters. The contribution corresponding to the three behaviors is denoted by < b 1,b2,b3 >, and for blockchain node j, each behavior contribution is calculated as follows:
wherein count 1 (j) represents the number of times that a chunk was successfully packaged on time after the leader node was selected, Representing the number of times a new block has not been generated within a specified time, alpha 1 is a penalty factor for a block packing failure, count 2 (j) represents the number of times a committee has successfully participated in consensus,Representing the number of times that committee consensus is engaged but the consensus task is not completed, α 2 is a penalty factor for committee node consensus failure, and Vr (j) represents the number of clients that node j verifies in round r. The comprehensive behavior contribution BS (j) of the node j is calculated by three behavior contribution weights: BS (j) =δ 1×b1(j)+δ2×b2(j)+δ3×b3 (j). Wherein δ 1,δ2,δ3 represents weights corresponding to three behaviors, satisfying δ 1+δ2+δ3 =1. Packaging blocks and achieving consensus is critical to the stability of the blockchain system, and therefore, the assignment of three weights should follow the principle of delta 1＞δ2≥δ3.
The experimental data of the invention adopts a classical MNIST data set, the data set consists of 60000 training samples and 10000 test samples, each data sample is a 28 multiplied by 28 gray scale image, and the corresponding label is 0-9. In order to test the system performance under the Non-IID condition, 60000 training samples are firstly ordered according to labels, and then uniformly divided into 200 parts, 300 samples each, and most of each sample has one label. The number of experimental clients was 50 and 100, respectively. With 50 clients co-training, each client has 4 copies of data. In the IID case, the training samples were randomly disturbed and evenly distributed in this experiment. The verification set for scoring the client is from 100 samples uniformly selected according to the labels in the test set, and each round of global model test set is all 10000 test samples.
The experimental model of the invention adopts a convolutional neural network (Conventional neural network, CNN) as a training model. The method comprises 3 layers of 3×3 convolution layers, wherein the number of convolution kernels of each layer is 32, 64 and 64, and the largest pooling layer of one layer is added to the two previous convolution layers respectively, and finally two full connection layers are added. The neural network selects Relu as the activation function and the output layer uses a softmax function. The learning rate is 0.01 and the loss function is a cross entropy loss function. The programming language for the experiment was python3.7, and the neural network was designed using the tensorflow2.5 and Keras framework.
Experiments are carried out by changing the gradient compression rate, so that the convergence rate of the model is reduced along with the increase of the gradient compression rate. When iid is distributed, compared with a federal learning algorithm without compression, the model precision is almost not lost by adopting 90% and 95% gradient compression rate, and the precision is slightly reduced by 1.52% when 99% compression rate is adopted. Under Non-iid distribution, the model accuracy is hardly reduced by adopting gradient compression rate below 95%, and the accuracy is reduced by 1.05% when adopting 99% compression rate.
The invention tests the model performance by setting different poisoning rates and selection rates. Under the condition of two different distribution, the invention can achieve better convergence effect. Under the condition that the client data set accords with iid, 30% of the poisoning attack can lead the model adopting the traditional federal learning aggregation algorithm to be unable to converge, the model convergence performance adopting the aggregation algorithm with 70% of the selection rate can almost reach the level of no poisoning attack, and the condition of convergence speed reduction does not occur. The aggregation algorithm provided by the invention can achieve good convergence effect under the condition that the client data accords with Non-iid, and under the condition of 30% poisoning attack, the convergence speed and the final model loss of the aggregation algorithm are superior to those of the traditional federal learning aggregation algorithm, and the model loss is close to the Non-poisoning attack level. Experiments show that the model performance of the aggregation algorithm using the method is superior to that of the traditional algorithm under the condition of two different distributions. The aggregation algorithm is selected according to the parameter quality of the client, the gradient of the client with low quality is discarded in a small amount under the condition of no poisoning attack, the whole model is not greatly influenced, and the convergence speed of the aggregation algorithm is faster than that of the aggregation algorithm which only uses the weight distribution based on the parameter quality under the condition of facing the poisoning attack.
The invention analyzes a block chain consensus mechanism based on double factor contribution through experiments. As the blockchain length increases, the behavior contribution score of a node generally increases, and the behavior contribution score has a positive correlation with the behavior completion ac. Sometimes the low ac node contribution score is greater than the high ac node because only nodes with higher model contributions qualify to enter the consensus committee's ability to participate in consensus and contend for the packaging block. The behavioral contribution score decreases because the node fails to generate new blocks on time after selecting as the leader, or becomes a committee node and fails to complete the consensus task. The ac high node has more opportunities to become the leader node, and can improve the stability of the blockchain system. However, the situation of packing rights of monopoly blocks of a single node does not occur, which indicates that the node election cooling mechanism can effectively reduce the system centralization degree. Experiments show that the block chain consensus mechanism based on the double-factor contribution can effectively excite nodes to participate in model training and maintain a block chain system, and the node cooling mechanism can effectively reduce the system centralization degree.
The invention finally analyzes the system time delay through experiments, the consensus time is greatly reduced after 95% gradient compression is used, and the average consensus time is reduced to 18.75% of the original algorithm. Experiments prove that the communication pressure can be effectively reduced while the model quality is ensured, and the designed block chain consensus mechanism based on the double factor contribution can attract high-quality nodes to actively participate in training, so that the consensus efficiency and stability of the block chain system are improved.
The foregoing examples are illustrative of the present invention and are not intended to be limiting, and other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principles of the invention are intended to be equivalent in scope.
Claims (7)
1. A trusted federation learning method based on a federation chain is characterized by comprising the following steps:
step S1, initializing a federation learning task by a model requester;
s2, the client side participating in training utilizes local data to train a model locally and compress gradients;
s3, uploading the sparse model gradient to a blockchain node by the client;
S4, the block chain link point verifies the transaction of the client and forwards the transaction to other block chain nodes;
S5, selecting a leader node and a consensus committee in a block chain consensus stage;
step S6, in the block release stage, the leader node calculates the training result of the round and packages and releases the block;
s7, the client downloads the global model from the blockchain;
Step S2 is that the client downloads the latest global model from the common node bound with the client, then replaces local model parameters with the global model, trains by using a local privacy data set, and performs gradient compression, and the specific steps are as follows:
(1) The client uses small Batch gradient descent (Mini-Batch GRADIENT DESCENT, MBGD) to train, and the calculation formula is as follows
Wherein D b represents a small batch training dataset;
Model parameters representing the kth round of client i;
Representing the gradient of the model on D b, wherein eta is the learning rate;
dn represents a training sample;
Representing the parameters as The model of (c) is lost at d n;
i represents a federal learning client;
k represents the global federal learning round number;
w represents a parameter of the mathematical function;
(2) Gradient compression is realized by adopting gradient sparsification, and the gradient compression process is as follows: after the client node i finishes the kth round of training, the gradient of the round is obtained Will beAnd residual gradientAdding, i.e.
Then each layer of gradient of the accumulated residual is sorted according to absolute value, and each layer of gradient is selected according to compression rate CCompression threshold of (2)For layer I networks, the absolute value will be less thanIs accumulated into the local residual gradient and then set to 0.
2. For the federation chain-based trusted federation learning method of claim 1, step S1 is for a task requester to issue federation learning tasks, initialize federation learning models, and parameters for client training.
3. The federation chain-based trusted federation learning method of claim 1, wherein step S3 is a step of, after client training is completed, uploading metadata in the form of transactions, compressed local model gradients by the client nodesTo a consensus node, whereIs the local model gradient of client i for this round of federal learning submission.
4. The federation chain-based trusted federation learning method of claim 1, wherein step S4 is to verify the digital signature of the public key of the client after the consensus node receives the transaction from the client, forward the transaction to other blockchain nodes after the identity is confirmed, and perform the parameter quality scoring.
5. The federation chain-based trusted federation learning method according to claim 1, wherein step S5 is to select consensus committee nodes according to a consensus mechanism based on dual factor contribution, and a number of the consensus nodes with highest model contribution form the consensus committee, wherein the node with highest behavioral contribution becomes the leader node of the current round, and a node cooling mechanism is added when the leader node is selected, i.e. the node that has been selected as the leader is not selected any more in the subsequent rounds.
6. For the federation chain-based trusted federation learning method of claim 1, step S6 the leader node generates a new chunk, the leader node packages all legal transactions of the round into a new chunk, and the hash value of the previous chunk, the hash value of the current chunk, the timestamp, the leader 'S public key, the leader' S digital signature, and the global gradient of the round are stored in the chunk header.
7. The federation chain-based trusted federation learning method of claim 1, wherein step S7 downloads the block for the client, continues training, receives the block link point of the new block, forwards it to all client nodes bound thereto, updates the local model according to the global gradient therein after the client downloads, and starts the next training round.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210394304.5A CN116961939B (en) | 2022-04-14 | 2022-04-14 | Trusted federation learning method based on alliance chain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210394304.5A CN116961939B (en) | 2022-04-14 | 2022-04-14 | Trusted federation learning method based on alliance chain |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116961939A CN116961939A (en) | 2023-10-27 |
| CN116961939B true CN116961939B (en) | 2024-07-19 |
Family
ID=88447833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210394304.5A Active CN116961939B (en) | 2022-04-14 | 2022-04-14 | Trusted federation learning method based on alliance chain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116961939B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN118520052A (en) * | 2024-07-23 | 2024-08-20 | 中国移动紫金(江苏)创新研究院有限公司 | Trusted federation learning method based on blockchain |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111125779A (en) * | 2019-12-17 | 2020-05-08 | 山东浪潮人工智能研究院有限公司 | Block chain-based federal learning method and device |
| CN112541592A (en) * | 2020-12-06 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Federal learning method and device based on differential privacy and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102430986B1 (en) * | 2020-08-24 | 2022-08-08 | 공주대학교 산학협력단 | Method for Performing Federated Learning Based on Block Chain in Hybrid System |
| CN112765559B (en) * | 2020-12-29 | 2024-08-20 | 平安科技（深圳）有限公司 | Processing method and device for model parameters in federal learning process and related equipment |
| CN113794675B (en) * | 2021-07-14 | 2023-04-07 | 中国人民解放军战略支援部队信息工程大学 | Distributed Internet of things intrusion detection method and system based on block chain and federal learning |
-
2022
- 2022-04-14 CN CN202210394304.5A patent/CN116961939B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111125779A (en) * | 2019-12-17 | 2020-05-08 | 山东浪潮人工智能研究院有限公司 | Block chain-based federal learning method and device |
| CN112541592A (en) * | 2020-12-06 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Federal learning method and device based on differential privacy and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116961939A (en) | 2023-10-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114186237B (en) | Robust federal learning model aggregation method based on true value discovery | |
| Nikolaenko et al. | Powers-of-tau to the people: Decentralizing setup ceremonies | |
| CN116961939B (en) | Trusted federation learning method based on alliance chain | |
| Skidanov et al. | Nightshade: Near protocol sharding design | |
| CN111414420B (en) | Improved PBFT block chain consensus method | |
| WO2019009897A1 (en) | Systems and methods for compression and distribution of machine learning models | |
| CN117527834B (en) | Improved PBFT consensus method based on reputation scoring mechanism | |
| CN116015672B (en) | PBFT consensus mechanism based on reputation model | |
| CN114398634A (en) | Federal learning participant weight calculation method based on information entropy | |
| CN112822051B (en) | Service acceleration method based on service perception | |
| CN115865378A (en) | Streaming media real-time evidence storing and checking method based on block chain | |
| CN113052329A (en) | Method and device for jointly updating service model | |
| CN116363449A (en) | Image recognition method based on hierarchical federal learning | |
| CN112671746A (en) | Block chain-based federated learning model poisoning detection method | |
| CN116566712A (en) | Internet of things block chain consensus method based on trust score | |
| CN117574429A (en) | Federal deep learning method for privacy enhancement in edge computing network | |
| CN114912022A (en) | Prediction model training method, system, computer device and storage medium | |
| CN116541831B (en) | Dual defense method based on blockchain and federal learning | |
| CN117852068A (en) | Federal learning privacy protection system and method based on blockchain and privacy computing technology | |
| CN117391858A (en) | Inductive blockchain account distribution method and device based on graphic neural network | |
| CN117094411A (en) | Server partition federation learning method based on blockchain slicing | |
| CN116233132A (en) | Energy block chain link point consensus method based on improved Raft consensus mechanism | |
| CN116911383A (en) | Federal recommendation model training method, terminal equipment, server and medium | |
| Muhlbaier et al. | Incremental learning from unbalanced data | |
| CN113297310B (en) | Method for selecting block chain fragmentation verifier in Internet of things |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |