CN112100659A

CN112100659A - Block chain federal learning system and Byzantine attack detection method

Info

Publication number: CN112100659A
Application number: CN202010963388.0A
Authority: CN
Inventors: 李宗航; 虞红芳; 李晴; 周天遥; 罗龙; 范末婵; 孙罡
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2020-09-14
Filing date: 2020-09-14
Publication date: 2020-12-18
Anticipated expiration: 2040-09-14
Also published as: CN112100659B

Abstract

The invention discloses a block chain federal learning system and a Byzantine attack detection method, wherein the block chain federal learning system comprises a data holder, a verifier, a miner and a task publisher; the system comprises a plurality of verifiers, a plurality of data holders and a plurality of data processing devices, wherein the plurality of verifiers form a verifier group, each verifier group is connected with a miner to form an edge cloud, and each data holder is randomly connected with any verifier in the verifier group of the adjacent edge cloud; and the miners of all the edge clouds construct a block chain network and are connected with the task publisher through the block chain network. The block chain federal learning system can be deployed in various edge computing scenes including mobile edge computing, micro data centers and micro clouds, and verification loads are balanced by using sufficient computing, communication and storage resources of edge infrastructure, so that verification time delay is reduced, and system efficiency is improved. The method can tolerate the local model with negative precision gain and can obtain high detection rate on the premise of ensuring the lossless model precision.

Description

Block chain federal learning system and Byzantine attack detection method

Technical Field

The invention relates to the field of federal learning, in particular to a block chain federal learning system and a byzantine attack detection method.

Background

Federal learning is an emerging distributed artificial intelligence technology for data privacy protection, and promotes the integration of edge intelligence. In order to share knowledge in a network edge privacy-protected manner, federated learning retains training data within training devices, and machine learning models are interacted between the training devices under coordination of a cloud parameter server. Such a centrally-based parameter server is subject to high pressure, is prone to collapse, is not trusted, and is prone to attack targets, and thus needs to have decentralization, non-tamper-ability, and traceability. The above properties are naturally present in the blockchain, which makes blockchain technology a viable solution to the above problems. Furthermore, the training devices of the edge environment are not trusted, unstable, and not active, and these problems can also be solved by means of block chains for intrinsic security of untrusted networks, support for dynamic joining and exiting, and sophisticated incentive mechanisms and virtual currency systems. Perfect matching of requirements and characteristics encourages federal learning to be combined with blockchain techniques and to develop into the next generation decentralized safe federal learning system: block-chain federal Learning (BFL).

With the wide deployment of edge servers, the network edge can provide sufficient computing, communication, and storage capabilities, attracting the deployment of many BFL systems at the network edge layer. In addition to the benefits of rich edge resources, malicious attackers that are hidden in the system also present new challenges to the security of BFL systems. Since the terminals and edge devices are held by users that are security-conscious and untrusted, these devices may submit inferior models to contaminate global models (poisoning attacks) or cheat training rewards (pick-up attacks), which are collectively referred to herein as byzantine attacks. The Byzantine attack seriously threatens the safety and fairness of the BFL system, so a fine-grained Byzantine attack detection method must be explored to identify and shield an attack source as soon as possible and prevent an attacker from further damaging the system.

However, fine-grained attack detection means that the system needs to validate the local model submitted by the device one by one, which would result in extremely high validation pressure and time cost in marginal scenarios where the number of devices reaches the billions. Edge computing technologies, typified by mobile edge computing, micro datacenters, and micro clouds, provide a solution to this problem. Under the support of the edge infrastructure, the BFL system can sink repeated and complicated verification processes to the edge of the network, and realize load balancing of calculation and communication at the edge of the network. Therefore, the BFL system architecture must be designed reasonably, and the verification load is balanced by using widely distributed edge infrastructure, reducing the verification delay.

Disclosure of Invention

Aiming at the defects in the prior art, the block chain federal learning system and the Byzantine attack detection method provided by the invention realize efficient and fine-grained Byzantine attack detection, and ensure that the model precision is lossless and the detection rate is high.

In order to achieve the purpose of the invention, the invention adopts the technical scheme that:

a blockchain federal learning system is provided, which comprises a data holder, a verifier, a miner and a task publisher; the plurality of verifiers form a verifier group, each verifier group is connected with a miner, and each data holder is randomly connected with any verifier in the verifier group; all miners construct a block chain network and are connected with a task publisher through the block chain network;

the data holder is provided with local training data of federal learning training and used for training a local model according to the local training data;

the verifier is used for verifying whether the local model submitted by the data holder is a Byzantine attack model or not, blocking the Byzantine attack model, and submitting the honest local model to a miner to form a transaction; wherein the Byzantine attack model comprises a poisoning attack model and a free vehicle attack model;

the miners are used for mining the average model meeting the conditions according to the local model submitted by the verifier, generating a new block, broadcasting the new block in the block chain network to achieve consensus and chaining the new block;

the task publisher is used for generating a creation block, publishing a training task and initial task configuration, and distributing training remuneration for a data holder, a verifier and a miner who participate in training;

the block chain structure of the block chain federal learning system comprises a created block and a sequence of common blocks, wherein each common block indexes the previous block according to the hash value of the previous block; the created block comprises a created block head and a created block body, wherein the created block head comprises a timestamp field, a target round number field, a first current round number field, a first verification precision field, a global model hash field, a minimum transaction number field, a tolerable precision oscillation threshold field and a tolerable precision deviation threshold field; the created region blocks contain hyper-parameters for machine learning;

the common block comprises a common block head and a common block body, wherein the common block head comprises all fields of a creature block head, a homomorphic hash key field, a previous block hash value field and a Merkle root field; the common block body comprises an actual transaction number and a Merkle tree constructed by a plurality of transactions;

the transaction comprises a second current round number field, a data holder ID field, a verifier ID field, a local model hash field, a homomorphic hash value field and a second verification precision field;

the block chain federal learning system adopts an interplanetary file system to store global model parameters and local model parameters, and adopts a block chain to record storage certificates of the global model parameters and the local model parameters in the interplanetary file system, wherein the storage certificates are used for positioning and downloading corresponding model parameters of block chain nodes in the interplanetary file system; the storage certificate comprises global model hash and local model hash and respectively corresponds to the storage certificate of the global model parameters and the local model parameters in the interplanetary file system.

A method for detecting Byzantine attack is provided, which comprises the following steps:

s1, generating a created block through a task publisher, and broadcasting the created block in the block chain network to publish a training task;

s2, constructing a model structure and a training algorithm according to configuration information in the founding block by miners, downloading the latest global model parameters from an interplanetary file system, and applying for a local verification set from a task publisher;

s3, downloading a model structure, a training algorithm, the latest global model parameters and a local verification set from the miners of the edge cloud through the verifier;

s4, downloading a model structure, a training algorithm and the latest global model parameters from the accessed verifier through a data holder, carrying out local training, and submitting the trained local model parameters to the verifier;

s5, verifying the local model: calculating, by the verifier, a second verification accuracy of the local model parameters on the local verification set; judging whether the second verification precision is greater than the threshold value, if so, entering the step S6, otherwise, abandoning the local model parameter, and returning to the step S4;

s6, packaging transaction: storing the local model parameters into an interplanetary file system through a verifier and obtaining a corresponding storage certificate, namely the local model hash; obtaining a homomorphic hash value of a local model parameter according to a homomorphic hash key of the edge cloud, packaging a second verification precision, the homomorphic hash value, the local model hash, a second current round number, an identity of a data holder and an identity of a verifier corresponding to the local model parameter into a transaction and putting the transaction into a transaction pool of miners;

s7, obtaining legal average model parameters: downloading local model parameters corresponding to each transaction from an interplanetary file system through a miner according to a storage certificate in a transaction pool, calculating average model parameters of the local model parameters which are not less than the minimum transaction number, and taking the average model parameters of which the first verification precision on a local verification set is greater than a threshold value as legal average model parameters;

s8, forming a new block and broadcasting: storing legal average model parameters into an interplanetary file system through miners and obtaining a corresponding storage certificate, namely global model hash; packaging all transactions corresponding to legal average model parameters, first verification precision, homomorphic hash keys, global model hash, target round number, first current round number, minimum transaction number, actual transaction number, tolerable precision oscillation threshold, tolerable verification precision deviation threshold, hash value of a previous block, Merkle roots of Merkle trees formed by all actual transactions and current time stamps into a new block, and broadcasting the new block in a block chain;

s9, preliminary verification: verifying the target round number, the first current round number, the first verification precision, the hash value of the previous block, Merkle roots of Merkle trees formed by all actual transactions, the minimum transaction number, the actual transaction number, a tolerable precision oscillation threshold value and a tolerable verification precision deviation threshold value by a non-broadcast miner, and entering step S10 if the verification is passed, or entering step S12 if the verification is passed;

s10, first verification accuracy comparison: downloading average model parameters from an interplanetary file system through a non-broadcast miner according to the global model hash in the new block, calculating first verification accuracy of the average model parameters on a local verification set of the average model parameters, and judging whether the absolute value of the difference between the first verification accuracy and the first verification accuracy recorded in the new block is smaller than the value of a tolerable verification accuracy deviation threshold, if so, entering step S11, otherwise, entering step S12;

s11, homomorphic hash value comparison: judging whether the homomorphic hash value of the average model parameter multiplied by the actual transaction times is equal to the sum of homomorphic hash values of the actual transaction in the new block or not by the non-broadcast miners, if so, passing the verification of the non-broadcast miners, and otherwise, not passing the verification of the non-broadcast miners;

s12, determining whether the new block is valid by voting: voting the non-broadcast miners participating in the step S9 to the step S11, judging whether the new block passes the verification of more than half of the non-broadcast miners, if so, determining that the new block is valid, and entering the step S13; otherwise, discarding the new block and punishing the miners broadcasting the new block, and returning to the step S2;

s13, finishing the current round training: the miners broadcasting the new blocks are elected as current-wheel leaders, the new blocks are chained, and the average model parameters corresponding to the new blocks are used as the global model parameters of the current wheel; enabling miners of other edge clouds to give up current round training, enabling the verifier to stop receiving local model parameters of a current round, and issuing the selected global model parameters of the current round to all data holders;

s14, miners who have mined the new block are rewarded through the block chain, validation rewards are distributed to the validators by the miners who have been rewarded, and training rewards are distributed to the data holders.

The invention has the beneficial effects that: the block chain federal learning system can be deployed in various edge computing scenes including mobile edge computing, micro data centers and micro clouds, and verification loads are balanced by using sufficient computing, communication and storage resources of edge infrastructure, so that verification time delay is reduced, and system efficiency is improved. Aiming at the problem of model precision loss caused by the existing Byzantine attack detection method, the invention provides the Byzantine attack detection method with negative precision gain tolerance, the method can tolerate the local model with negative precision gain, and high detection rate can be obtained on the premise of ensuring that the model precision is not damaged.

Drawings

FIG. 1 is a system block diagram of the present system;

FIG. 2 is a schematic flow diagram of the present method;

FIG. 3 is a block chain structure diagram of BytoChain of the interplanetary file system IPFS;

FIG. 4 is a schematic diagram of the structure of a creating block, a common block and a transaction body;

FIG. 5 is a schematic diagram of three attackers;

FIG. 6 is a schematic flow chart of the main steps of the method;

FIG. 7 is a schematic diagram of three edge implementation deployment scenarios of the present invention;

FIG. 8 is a graph showing verification accuracy curves of the present system (BytoChain) and Federal Learning (FL) under random and reverse poisoning attacks;

FIG. 9 is a schematic diagram of the missing detection rate and the false detection rate of the present invention under different tolerable precision oscillation thresholds γ;

FIG. 10 is a schematic diagram of verification accuracy after two kinds of poisoning attacks are defended by the present invention under different tolerable accuracy shock thresholds γ;

FIG. 11 is a graph of the median of the verification accuracy deviations for honest models and over-fit poisoning attacks at different sampling rates.

Detailed Description

The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined in the appended claims, and all matters produced by the invention using the inventive concept are protected.

As shown in fig. 1, the blockchain federal learning system (bytecain) includes four types of roles of data holder, verifier, miner, and task publisher.

The data holder has training data required by federal learning training, is used for training a local model according to the local training data, and can be deployed on intelligent terminal equipment such as mobile phones, vehicle-mounted terminals and industrial sensors.

The verifier is used for accessing the data holder and verifying whether the local model submitted by the data holder is a Byzantine model or not, and can be deployed on an edge server at the edge of the network. Several verifiers form a verifier group, and the local model submitted by the data holder is randomly distributed to any verifier in the verifier group.

Miners are used for mining the average model meeting the conditions according to the verified local models submitted by the verifier group, generating new blocks, broadcasting the new blocks in the block chain network to achieve consensus, and deploying the new blocks on the edge server at the edge of the network.

The task publisher is used for generating a creation block, publishing a training task and initial task configuration, distributing training remuneration for a data holder, a verifier and a miner who participate in training, and can be deployed in any block chain node.

In the four types of node roles, a data holder and a verifier can freely join or exit.

The verifier group and miners form an edge cloud, and edge servers in the edge cloud are interconnected through a local area network. The data holder accesses the nearby edge cloud through a wired or wireless edge network.

The block chain structure of the bytechain system is shown in fig. 3. The BytoChain system stores global model parameters and local model parameters by relying on an interplanetary file system (IPFS), and only records storage certificates of the model parameters in the IPFS on a block chain: and (6) carrying out model hash. The stored credentials may be used for blockchain nodes to locate and download corresponding model parameters in the IPFS.

The model hash comprises global model hash and local model hash, and the model hash respectively corresponds to storage certificates of global model parameters and local model parameters in the IPFS. The global model hash is recorded in the header of the created block and the normal block, and the local model hash is recorded in the transaction of the normal block.

The bychain block chain structure is composed of a created block and a sequence of normal blocks, and each normal block indexes to a previous block according to the hash value of the previous block.

The structure of the creation block, the normal block and the transaction body is shown in fig. 4. The creation block is generated by the task publisher, the common block is generated by the miners, and the transaction is generated by the verifier.

The creation block comprises a block head and a block body. The block header contains the following fields: the method comprises the steps of time stamping, target round number, first current round number, first verification precision, global model hash, minimum transaction number, tolerable precision oscillation threshold and tolerable precision deviation threshold. The block body comprises: model structure, learning rate, neuron drop rate, batch data size, local iteration number, optimizer, loss function, activation function, and other hyper-parameters needed to train a machine learning model.

The normal block comprises a block head and a block body. The chunk header contains all the fields of the created chunk header, as well as the homomorphic hash key, the previous chunk hash value, and the Merkle root. The block of blocks contains the actual number of transactions, as well as the Merkle tree built from several transactions.

The transaction body comprises a second current round number, a data holder ID, a verifier ID, a local model hash, a homomorphic hash value and a second verification precision.

In the foundational block and the normal block:

the time stamp records the creation time of the created blocks and the normal blocks.

The target round number specifies the maximum number of training rounds desired by the task publisher, and also specifies the upper limit on the length of the blockchain.

The first current round records the current training round when creating the founder block and the normal block. And adding 1 to the first current round of numerical value every time a block is added to the block chain, and terminating the training task until the target round of numerical value is reached.

The first validation accuracy records the validation accuracy of the global model parameters on the local validation set of the miners who generated the block.

The global model hash records the stored credentials of the global model parameters returned by the IPFS.

The minimum number of transactions specifies a lower limit on the number of transactions that should be packed in the block of regular tiles.

The homomorphic hash key records homomorphic hash keys adopted by a homomorphic hash algorithm when homomorphic hash values of the model parameters are calculated; wherein the same homomorphic hash key is used by the verifier and the miners in the same edge cloud;

the tolerable accuracy oscillation threshold specifies a reasonable interval of accuracy oscillation acceptable to the verifier and miners. When the first verification accuracy and the second verification accuracy fall within the interval, both the local model and the average model will be accepted.

The tolerable accuracy deviation threshold value specifies a reasonable interval of verification accuracy deviation acceptable to miners. When the first verification precision deviation of the average model on the local verification set of each miner is within the interval, the first verification precision of the common block header record is considered to be valid.

The previous block hash value records the hash value obtained by hashing the previous block as a whole, and the hash value is used to find the previous block, thereby forming a chain.

The Merkle root records the hash value recorded by the root of a Merkle tree consisting of a plurality of transactions in the block body of the common block. This value is used to check whether the packaged transaction has been tampered with.

The actual transaction number records the transaction number of the actual package in the block of the general block.

The Merkle tree is composed of a plurality of transactions, leaf nodes represent packaged transactions, intermediate nodes represent local hash values, and a tree root represents a hash value of the whole tree.

In the transaction body:

the second current round number records the current training round number at the time the transaction was generated.

The data holder ID records the identity of the data holder contributing the local model parameters in the transaction.

The validator ID records the identity of the validator in the transaction that validates the local model parameters.

The local model hash records the stored credentials of the local model parameters returned by the IPFS.

The homomorphic hash value records homomorphic hash values of local model parameters calculated by a homomorphic hash algorithm according to homomorphic hash keys specified by the current edge cloud.

The second validation accuracy records the validation accuracy of the local model parameters on the local validation set of the validator that generated the transaction.

In one embodiment of the invention, assume an edge cloud E, numbered s_sWith 1 miner node M therein_s，K_sAn authenticator node V_s，k(k＝1，...，K_s) And the edge cloud has access to N_sIndividual data holder H_s，n(n＝1，...，N_s)。

Data holder H_s，nPossess I_s，nLocal training sample

And training a machine learning model with l as a loss function based on the samples, the model being

Parameterization (i.e., local model parameters), where r is the current round number. The learning rate of the local optimizer is eta, the batch data size is b, and the number of local traversal rounds is E.

Verifier node V_s，k(k＝1，...，K_s) And miner node M_sSharing the same local authentication subset F_sThe subset is obtained by uniformly and randomly sampling the complete verification set F of the task publisher according to the sampling rate.

In the verifier V_s，kThe local model parameters it is responsible for

Locally verifying subset F_sHas the verification precision of

In miner M_sThereon, it polymerizes T_sAverage model parameter of local model parameters

Locally verifying subset F_sHas the verification precision of

Wherein T is_sIs the actual transaction number.

The r-th normal block B on the block chain^rThe corresponding global model parameters are denoted as ω^rVerification accuracy of block header record is ACC^r。

Let the target round number be R, the minimum transaction number be L, the homomorphic hash function be HomoHash, the homomorphic hash key be HomoKey, the tolerable precision oscillation threshold be γ, and the tolerable precision deviation threshold be τ.

The present invention focuses on addressing malicious data holders

Verifying device

And miners

A byzantine attack is initiated as shown in fig. 5.

Holder of malicious data

By submitting Byzantine model

To its edge cloud E_sTo implement the attack.

Malicious verifier

By contact with malicious data holders

Chuantong-help byzantine model

And bypassing local model verification, thereby realizing attack.

Malicious miner

By broadcasting Byzantine models directly in blockchain networks

To implement the attack.

The Byzantine attacks are divided into two categories: model poisoning attacks and pick-up car attacks. Model poisoning attacks compromise model accuracy and training speed by submitting a pollution model, and casualty vehicle attacks cheat training rewards by unfair competition.

The model poisoning attack specifically includes three types: random poisoning attack, reverse poisoning attack, overfitting poisoning attack.

Random poisoning attack means that the attacker

Using randomly generated model gradients

And (3) updating local model parameters:

at this time, the process of the present invention,

referred to as a random poisoning model.

The reverse poisoning attack refers to an attacker

Can intercept other T in the edge cloud_s1 honest data holders H_s，t(t ≠ n) submitted local model parameters

Estimating the correct model update direction:

and updating the local model parameters in the reverse direction:

where α is the reverse update step size. At this time, the process of the present invention,

referred to as the inverse detoxification model.

Overfitting poisoning attack means that the attacker

Verifier passing through edge cloud with which it is located

Collusion, or illegal theft verifier

Local authentication set F_sTo train local model parameters

Until overfitting, obtaining a local verification set F_sOverfitting poisoning model with upper accuracy of 100%

The casualty vehicle attack means an attacker

Untrained local model parameters

Directly to its edge cloud. At this time, the process of the present invention,

is called a relief vehicle model.

The random poisoning model, the inverse poisoning model, the overfitting poisoning model and the free-carrying model are collectively called as a Byzantine model.

The random poisoning attack, the reverse poisoning attack, the overfitting poisoning attack and the vehicle carrying attack can also be attacked by malicious miners

And (4) initiating.

In summary, one problem to be solved by the present invention is to detect and mask the four byzantine models in the edge clouds and in the blockchain networks, in the case where the data holders, verifiers and miners are all potentially malicious.

In a specific implementation process, as shown in fig. 6, the method for detecting a byzantine attack proposed by the present application is composed of 6 main steps: the method comprises the steps of creating a block and initializing a task, training a local model, verifying the model and generating a transaction, mining the model and the block, verifying the model and the block and distributing rewards.

The chunking block generation and task initialization steps are performed by the task publisher. The local model training step is performed by the data holder. The model validation and transaction generation steps are performed by a validator within the edge cloud. The modeling and block mining steps are performed by miners within the edge cloud. The model and block verification steps are performed by miners within other edge clouds. The prize allocation step is performed by an actuation mechanism of the blockchain system.

The method comprises the following steps: the creation block generation and task initialization specifically comprises the following sub-steps:

step 1.1: task publishers generate initial global model parameters ω⁰Initial global model parameters ω⁰The IPFS is logged to obtain a storage credential (global model hash).

Step 1.2: task publisher calculates initial global model parameter omega⁰Verification accuracy ACC on complete verification set F⁰。

Step 1.3: the task publisher packages the storage certificate (global model hash) in the step 1.1, the verification precision in the step 1.2, task configuration parameters such as a current timestamp, a target round number R, a current round number R, a minimum transaction number L, a tolerable precision oscillation threshold gamma and a tolerable precision deviation threshold tau, and hyper-parameters required by machine learning algorithms such as a model structure, a learning rate, a neuron discarding rate, a batch data size, a local iteration number, an optimizer, a loss function and an activation function into a creating block B⁰And broadcasting the foundational blocks in the blockchain network to distribute the training tasks.

Step 1.4: the task publisher pays remuneration to the blockchain incentive mechanism to reward data holders, validators, and miners that participate in the training.

Step 1.5: the method includes the steps that the edge cloud registers miners in a block chain network to join a training task, the miners who register successfully issue a public key, and the miners of other edge clouds verify the validity of a message source according to the public key.

Step 1.6: miners download creating block B from block chain⁰And constructing a model structure and a training algorithm according to the configuration information in the founding block. If only the founder blocks exist on the block chain, downloading the initial global model parameters omega from the IPFS according to the storage certificate recorded in the header of the founder blocks⁰(ii) a Otherwise, the miner downloads the latest common block B on the block chain^rDownloading the latest global model parameters ω from IPFS according to the stored credentials recorded in its block header^r。

Step 1.7: the miners apply for the local verification set from the task publisher, and the task publisher uniformly and randomly samples the verification subset with the proportion of equal from the complete verification set F and sends the verification subset to the miners who initiate the application. The suggested sampling ratio is less than 0.1%.

Step 1.8: the verifier is free to join and leave the set of verifiers of its nearby edge cloud. The newly added verifier downloads model structures, training algorithms, initial global model parameters omega from miners in their edge clouds⁰(or the latest global model parameter ω^r) And a local authentication subset.

Step 1.9: the data holder is free to join and leave the training task. The newly joined data holder accesses the verifier group of the edge cloud nearby through the edge network and randomly establishes a communication connection with one of the verifiers. The data holder downloads the model structure and training algorithms from the verifier it accesses to initiate the local training environment.

Step two: local model training to access edge cloud E_sData holder H of_s，nFor example, assume verifier V_s，kTaking charge of the data holder's access, and the current round is r, specifically comprising the following substeps:

step 2.1: data holder H_s，nDownloading global model parameters omega from its accessed verifier^r-1And copying a local modelParameter copy

Step 2.2: data holder H_s，nLocal training sample

Cut into numbers (0, a., j, a., I) according to the batch data size b_s，nSmall batch data of/b-1), and traversing E rounds of the small batch data to train local model parameters

Where g is the temporarily generated small batch average gradient. The batch number counter j ═ 0, and each time equation (1) and equation (2) are calculated, j ← (j +1) mod I_s，nB is the ratio of the total weight of the components to the total weight of the components. Here, the

Training samples for current local model parameters

Equation (1) and equation (2) will iteratively calculate EI based on the model gradient calculated by the gradient back-propagation algorithm_s，nB times, obtaining the trained local model parameters

Step 2.3: data holder H_s，nUploading trained local model parameters

Authenticator V to access_s，k。

Step three: model validation and transaction generation with edge cloud E_sInner validator V_s，kFor example, the method specifically comprises the following substeps:

step 3.1: verifier V_s，kCalculating local model parameters

In local authentication set F_sAccuracy of verification of

If not satisfied

The local model parameters

Will be considered to be a security risk and will be verified by the verifier V_s，kDiscarding; otherwise, step 3.2 is performed. Wherein ACC^r-1For the newest block B on the current block chain^r-1The first verification precision recorded in the block header, γ, is a tolerable precision oscillation threshold, and the suggested value γ is 0.002.

Step 3.2: verifier V_s，kAccording to edge cloud E_sUniformly specified homomorphic Hash key HomoKey calculation local model parameter

Homomorphic hash value of (3) ((iii))

HomoKey)。

Step 3.3: verifier V_s，kValidating the local model parameters

The IPFS is logged to obtain a storage credential (local model hash).

Step 3.4: verifier V_s，kThe verification precision of the step 3.1, the homomorphic hash value of the step 3.2, the storage certificate of the step 3.3, the current round number r and the data holder H_s，nAnd a verifier V_s，kAre packaged together as a transaction and put into miner M_sIn the transaction pool of (2).

Step four: model and block mining with edge clouds E_sInner miner M_sFor example, the method specifically comprises the following substeps:

step 4.1: miner M_sAnd downloading local model parameters corresponding to each transaction from the IPFS according to the storage certificate (local model hash) recorded by the transaction pool.

Step 4.2: miner M_sCalculating the average model parameter of the local model parameters until T is found_s(≧ L) local model parameters

So as to average the model parameters

In local authentication set F_sAccuracy of verification of

Satisfy the requirement of

Wherein T is_sL is the minimum transaction number. If a set of local model parameters is found to satisfy the above conditions, the miner M is considered to be_sSuccessful mining to legal average model parameters

Otherwise, the miner M_sContinue to wait for a new transaction and repeat steps 4.1-4.2.

Step 4.3: miner M_sAverage model parameters

The IPFS is logged to obtain a storage credential (global model hash).

Step 4.4: miner M_sConverting T of step 4.1_sIndividual transaction, verification accuracy of step 4.2, storage voucher of step 4.3, target round number R, current round number R, minimum transaction number L, actual transaction number T, edge cloud E_sA specified homomorphic Hash key HomoKey, a tolerable precision oscillation threshold value gamma, a tolerable precision deviation threshold value tau and a Hash value Hash (B) of a previous block^r ^-1)、T_sThe Merkle root and the current timestamp of the Merkle tree formed by the transaction are packaged into a new block B^rAnd broadcasting a new block B in a blockchain network^rTo achieve consensus. Wherein the new block B^rThe R, L, γ, τ settings are required to match the latest block B on the block chain^r-1The corresponding fields of (a) remain consistent.

Step five: model and block validation with edge cloud E_lInner miner M_lFor example, the method specifically comprises the following substeps:

step 5.1: miner M_lChecking the received new block B^rThe method comprises the following steps: the target round number R, the minimum transaction number L, the tolerable precision oscillation threshold gamma, the tolerable precision deviation threshold tau should be related to the latest block B on the block chain^r-1The corresponding fields of the data are kept consistent; the current round number r should be equal to the latest block B on the block chain^r-1Adding 1 to the current number of wheels; verification accuracy

Should satisfy

The previous block hash should be the newest block B in the block chain^r-1Hash value of (B)^r-1) (ii) a The Merkle root should be equal to the tree root hash value of the Merkle tree formed by all exchanges in the zone block; actual transaction number T_sThe minimum transaction number L should be not less than. If the verification is passed, executing step 5.2; otherwise, the verification is not passed and step 5.5 is performed.

Step 5.2: miner M_lAccording to the new block B^rIn (2) the recorded storage credential (global model hash) downloads the average model parameters from the IPFS

Step 5.3: miner M_lCalculating average model parameters

In local authentication set F_lAccuracy of verification of

If it satisfies

Step 5.4 is executed; otherwise, the verification is not passed and step 5.5 is performed. Where τ is a tolerable accuracy deviation threshold, the recommended value is τ 0.05.

Step 5.4: miner M_lValidating average model parameters

And new block B^rWhether the recorded transaction is satisfied

Wherein

Indicates a new block B^rMiddle T_sThe sum of the homomorphic hash values of the individual transaction records. If yes, the verification is passed; otherwise, the verification is not passed. Step 5.5 is performed.

Step 5.5: all miners participating in steps 5.1-5.4 initiate a vote if a new block B is present^rThe new block B is considered to be a new block B after more than half of miners verify^rIf so, perform step 5.6; otherwise, the new block B^rWill be discarded as an invalid block, generating the malicious miners of the block

Will also be punished for its dishonest behavior and other miners will return to step four.

Step 5.6: verified miner M_sIs elected as the leader of the current wheel and a new block B is formed^rUpLink, average model parameters generated thereof

Will also be global model parameters for the r-th round

Step 5.7: and giving up miners of other edge clouds to execute the fourth step, stopping receiving the local model parameters of the r-th round by all the verifiers, and issuing the global model parameters omega of the r-th round^rTo the data holder.

Step six: reward allocation, with marginal cloud E_sInner successfully digging new block B^rM of miner_sFor example, the method specifically comprises the following substeps:

step 6.1: the block chain is M for miners_sDispensing a virtual currency award in the amount

max{ACC^r-ACC ^r-10, where A is a reward reference value, β is a reward compensation factor, ACC^r-1And ACC^rRespectively last new block B for the block chain^r-1And the latest block B^rThe recorded verification precision, e is a natural constant;

step 6.2: miner M_sA validation reward is assigned to the validator based on the number of transactions submitted by the validator, and a training reward is assigned to the data holder based on a precision gain of a local model parameter submitted by the data holder.

Step 6.3: let R ← R +1, if the current round number R reaches the target round number R, the task publisher downloads the latest common block B from the block chain^R-1Downloading the trained global model parameters from the IPFS according to a storage certificate (global model hash) recorded in the block header, and stopping the training task; otherwise, returning to the step two.

In the present embodiment, only the generation process of one new tile is described, and in the actual use process, after a prize is allocated to the generation of one new tile, if a new tile needs to be generated continuously, the data holder may download the latest global model parameters from the edge cloud, and return to step two (step S4) to continue the execution.

When the deployment environment is implemented, as shown in fig. 7, the invention can be deployed in three edge computing scenarios, namely, mobile edge computing, micro data center and micro cloud, and can also be deployed in a scenario in which the three edge architectures are mixed.

Deployed in a moving edge computing scenario: the mobile edge computing platform includes a plurality of communication base stations and a mobile edge server group. The communication base stations are interconnected through a wired optical fiber network. The communication base station and the mobile edge server group are interconnected through a wired local area network. The mobile edge server group comprises a plurality of edge servers which are interconnected through a wired local area network. A plurality of geographically distributed communication base stations form a wireless cellular access network, which communication base stations cover nearby wireless terminal devices by radio.

Deployed in a micro datacenter scenario: the micro data center is light in volume, can be placed in remote suburbs and narrow rooms, and is provided with a whole set of equipment owned by a conventional data center, such as a built-in cabinet, power supply, monitoring, network and cooling. A plurality of high-performance edge servers are arranged in the micro data center, and the edge servers are interconnected through a built-in wired local area network. The micro data center is accessed to the wired or wireless terminal equipment nearby through the public network gateway and is interconnected with other micro data centers.

Deployed in a micro-cloud scenario: the micro cloud is composed of several commercial servers that are geographically adjacent to each other and can be provided by small businesses or users. The servers are interconnected through a wired local area network. A cloudlet contains multiple wireless access points, the most common of which is Wi-Fi, one wireless access point typically paired with one or several servers. The micro cloud covers wireless terminal equipment in a cell area through a plurality of wireless access points. The micro clouds are interconnected through a wired wide area network.

The method is deployed in a scene mixed by three edge architectures including a mobile edge computing architecture, a micro data center architecture and a micro cloud architecture: in this scenario, a portion of the edge cloud is deployed on the mobile edge computing architecture, a portion of the edge cloud is deployed on the micro data center architecture, and a portion of the edge cloud is deployed on the micro cloud architecture. The three types of edge clouds are interconnected by a wired wide area network.

Implementing a deployment environment for a mobile edge computing scenario. The data holder node is deployed at the wireless terminal device. In the mobile edge server group, one mobile edge server is used for deploying miner nodes, other all or part of mobile edge servers are used for deploying a plurality of verifier nodes, the verifier nodes form a verifier group, and the miner nodes and the verifier group form an edge cloud. Miners' nodes of all edge clouds participate in building the licensed blockchain network. The task publisher may be any node in the license blockchain network. In this embodiment, the data holder is provided by the user and may participate or quit voluntarily. The verifier node and the miner node are provided by a large-scale operator, and the running state of the nodes is stable.

The deployment environment is implemented as a micro data center scenario. The data holder node is deployed in a wired or wireless terminal device. In a group of high-performance edge servers in the micro data center, one edge server is used for deploying miner nodes, other all or part of edge servers are used for deploying a plurality of verifier nodes, the verifier nodes form a verifier group, and the miner nodes and the verifier group form an edge cloud. Miners' nodes of all edge clouds participate in building the licensed blockchain network. The task publisher may be any node in the license blockchain network. In this embodiment, the data holder is provided by the user and may participate or quit voluntarily. The verifier nodes are provided by the micro data center owner and can participate or quit voluntarily. The miner nodes can not be withdrawn at will, and each micro data center is ensured to have one and only one miner node.

The deployment environment is implemented as a micro-cloud scenario. The data holder node is deployed at the wireless terminal device. In a plurality of commercial servers in the micro cloud, one server is used for deploying miner nodes, all or part of other servers are used for deploying a plurality of verifier nodes, the verifier nodes form a verifier group, and the miner nodes and the verifier group form an edge cloud. Miners' nodes of all edge clouds participate in building the licensed blockchain network. The task publisher may be any node in the license blockchain network. In this embodiment, the data holder is provided by the user and may participate or quit voluntarily. Verifier nodes are provided by different small merchants or users and may participate or quit voluntarily. The miner nodes can not be withdrawn at will, and each micro cloud is ensured to have only one miner node.

The implementation and deployment environment is a mixed scene of mobile edge computing, a micro data center and a micro cloud architecture. If an edge cloud is deployed on the mobile edge computing architecture, the edge cloud is deployed for the embodiment of the mobile edge computing scenario according to the implementation deployment environment. If an edge cloud is deployed on the micro data center architecture, deployment is performed for the micro data center scenario embodiment according to the implementation deployment environment. If an edge cloud is deployed on the micro-cloud architecture, the edge cloud is deployed according to the embodiment in which the deployment environment is a micro-cloud. And between the isomorphic and heterogeneous edge clouds, a unique miner node inside the edge clouds participates in the construction of the permission block chain network. The task publisher may be any node in the license blockchain network.

In a specific implementation, the goal is to train a typical convolutional neural network model using the classical handwritten digit data set MNIST. The structure of the model is Conv16-MaxPool-Conv32-MaxPool-FC32-FC 10. The data set comprises 60000 training samples and 10000 verification samples, wherein the training samples are uniformly, independently and uniformly distributed and partitioned to all data holders, and the verification samples are held by task publishers. In each training round, the data holder runs a small batch stochastic gradient descent algorithm to traverse its local training set E1 time, wherein the optimization algorithm uses a learning rate η 0.001 and a batch data size b 32. This implementation simulates multiple virtual edge clouds, each edge cloud having 1 miner node and K_s5 verifier nodes and N is accessed_s10 data holders. The attacker can be any node of a data holder, a verifier and a miner, and the attacker can launch random poisoning attack, reverse poisoning attack and overfittingPoisoning attacks and casualty car attacks.

In the above experimental environment, the defense effects of random poisoning attacks and reverse poisoning attacks initiated by malicious data holders, malicious verifiers and malicious miners are explained by adopting the byzantine attack detection method of the present invention: setting γ to 0.002, fig. 8 shows the verification accuracy curves of the present invention (bytocain) and Federal Learning (FL) under random poisoning attack and reverse poisoning attack. As can be seen from fig. 8, the federal learning has a sharp oscillation and fails to converge in the accuracy curve under the random poisoning attack, and the training speed is significantly slowed down under the reverse poisoning attack. The verification precision curve of the Byzantine attack detection method under the random poisoning attack and the reverse poisoning attack almost completely coincides with the Federal learning verification precision curve under the attack-free scene, so that the precision is maintained to be lossless, the training speed is ensured, and the random poisoning attack and the reverse poisoning attack can be effectively resisted.

In this example, tolerable precision oscillation thresholds γ are set to [0, 0.001, 0.002, 0.003, 0.004, 0.005, 0.01, 0.05, 0.1], and fig. 9 shows the false detection rate and the false detection rate of the byzantine attack detection method provided by the present invention under different tolerable precision oscillation thresholds γ, where the false detection rate refers to the proportion of honest local models which are falsely detected as poisoning models to all honest local models, and the false detection rate refers to the proportion of undetected poisoning models to all poisoning models. As can be seen from fig. 9, the larger the tolerable accuracy oscillation threshold γ is, the lower the false detection rate is, and the higher the false detection rate is. In order to ensure low omission factor, gamma is not more than 0.003.

Based on the data shown in fig. 9, fig. 10 shows the highest verification accuracy of the byzantine attack detection method provided by the invention after defending against random poisoning attack and reverse poisoning attack under different tolerable accuracy concussion thresholds γ. As can be seen from fig. 10, with the increase of the tolerable precision oscillation threshold γ, the defense effect of the present invention exhibits a trend of first enhancing and then weakening, and when γ is greater than or equal to 0.002 and less than or equal to 0.003, the defense effect of the present invention is optimal, and compared with federal learning, the highest verification precision at this time is lossless. Also, the present example notes that the tolerable precision oscillation threshold γ is not as small as possible. When gamma is 0, the method for detecting the Byzantine attack is similar to the prior detection technology only accepting positive precision gain: only local models of positive precision gain are accepted, while local models of zero precision gain and negative precision gain are simply discarded. At the moment, the global model is locally excellent, the highest verification precision is reduced from 0.981 to 0.918, the model precision is seriously damaged, and the defects of the technology are proved.

For random poisoning attacks and reverse poisoning attacks, the attacker uses a uniform randomly generated model gradient

According to

Updating local model parameters

To generate a random poisoning model

And according to

Generating a reverse poisoning model

Where the reverse update step alpha is 9. For simplicity of analysis, symbols

And

both random and inverse poisoning models can be represented.

When the attacker is the data holder: random poisoning model and reverse poisoning model

Verification accuracy of

Compare global model ω^r-1Is verified to be accurate ACC^r-1Generally, 0.02 to 0.8 lower, with the recommended γ ═ 0.002 setting, the poisoning model

Violate the condition of step 3.1

And will therefore be directly discarded by the verifier.

When the verifier colludes with the data holder: poisoning model submitted by data holder

The detection of the verifier in the third step can be bypassed and directly submitted to the miner M_s. Model due to poisoning

Will reduce the mean model

Verification accuracy of

Make the average model

It is more difficult to satisfy the condition of step 4.2

Also enables the miner M_sIt is difficult to mine legal blocks to participate in block chain competition, and therefore, the poisoning model

Will not be on the global model omega^rAnd the step four is executed by other edge clouds without being influenced. Even if miner M_sThe lucky digs the legal average model and block, poisoning model

The effect of the generated noise has been greatly reduced, and the generated noise is applied to the average model

The precision damage caused by the method is less than gamma (less than 0.002).

When the attacker is the miner M_sThe method comprises the following steps: miner M_sThere may be malicious behavior that 1) does not use enough local models to generate the average model such that the verification accuracy of the average model

The condition of step 4.2 is not satisfied; 2) in the generated malicious block

High accuracy of verification of false head

To force the verification accuracy to meet the condition of step 4.2; 3) direct counterfeiting of average models without waiting for data holders to submit local models

Miner M_sMalicious blocks of a broadcast

Validation of miners who will accept other edge clouds, as edge cloud E_lInner miner M_l(l ≠ s) for example, the detection methods of the above three behaviors were analyzed. 1) Miner M_lBlocks will be detected in step 5.1

Verification accuracy of head records

Not meet the requirements of

2) Miner M_lBlocks will be detected in step 5.3

Verification accuracy of head records

Not meet the requirements of

3) Miner M_lA fake mean model will be detected in step 5.4

And block

Does not satisfy the sum of homomorphic hash values of the transaction sets recorded in (1)

Any malicious activity will trigger the detection conditions of steps 5.1-5.4 and will block the malicious block in step 5.5

And its average model

The vote is discarded.

Therefore, the method can detect the random poisoning model and the reverse poisoning model in a fine granularity manner, can identify the attack source, and can effectively resist the random poisoning attack and the reverse poisoning attack. Compared with the traditional federal learning, the method can obtain the model precision and the training convergence which are the same as those of the federal learning in a non-attack scene, so that the model precision can be ensured to be lossless and the training convergence can be ensured to be lossless.

In the above experimental environment, the defense effect of the overfitting poisoning attack initiated by the malicious data holder and the malicious miners is explained by adopting the byzantine attack detection method of the present invention:

this example sets the verification subset F_sThe ratio of samples from the complete validation set F was [0.001, 0.1, 0.2, 0.3, 0.4, 0.5%]I.e., the verification subset F that an attacker can use to train the over-fit poisoning model_sRespectively comprise [10, 1000, 2000, 3000, 4000, 5000]The strip validates the sample. FIG. 11 shows the mean values of the validation accuracy bias for honest models and overfit poisoning attacks at different sampling rates

The change curve of (2). As can be seen from fig. 11, the verification accuracy deviation median curves (green and yellow) of the honest local model and the overfit poisoning model have obvious boundaries, the smaller the sampling rate is, the higher the boundary discrimination is, but one curve (blue) can be always found to completely discriminate the honest local model and the overfit poisoning model. Therefore, the sampling rate is set to 0.001, the tolerable accuracy deviation threshold τ is set to 0.05, and the overfitting poisoning model can be correctly detected in step 5.3.

For overfitting poisoning attacks, 1) malicious data holders

Using illegally obtained authentication subsets F_sAs local training set and obtain overfitting poisoning model according to step 2.2

2) Malicious miner

Skipping steps two to four, instead of generating the average model from the local model, the verification subset F held by it is used_sAs local training set and obtain overfitting poisoning model according to step 2.2

Here, the symbols

And

represents an overfitting poisoning model.

When the attacker is the data holder: holder of malicious data

Submitted overfitting poisoning model

In the verification subset F_sHas high verification precision

Thus, can be easily passed through the verifier detection of step 3.1 and delivered directly to the mineworker M_s. Adulterated with overfitting poisoning model

Average model of

Will also be contaminated as an overfitting poisoning model

It is in the verification subset F_sHas high verification precision

And simplifies the M of miners_sIn step 4.2, it is found that the method is legalDifficulty of models and blocks, so that models are over-fitted to poisoning

It is easier to enter into the blockchain network.

When the attacker is a miner: malicious miner

Generated overfitting poisoning model

Will go directly into the blockchain network.

In the method for detecting the Byzantine attack, the two types of attackers are injected into an overfitting poisoning model of a block chain network

Will be detected by miners of other edge clouds. With edge cloud E_lMiner M in (l ≠ s)_lFor example, overfitting the poisoning model

High verification accuracy of

Will trigger miner M_lDetection of step 5.3 above, and due to violation

Is determined to be invalid and discarded. Thus, a malicious miner

Will be penalized by the blockchain network for its malicious behavior. If the attacker originates from the data holder, miner M_sIt can review its transaction pool where verification of a local model of unusually high precision is possible

Will be marked as overfitting poisoningModel, malicious data holder submitting the over-fitted poisoning model

Will be masked by the verifier due to its malicious behavior. Finally, overfitting poisoning attacks are not used for updating the global model, so that the safety of the system can be guaranteed. On the other hand, as the attack model is detected in time, the attack source can be traced, and an attacker can not uplink the false blocks, the training reward can not be obtained, thereby ensuring the fairness of participating in training.

Therefore, the over-fit poisoning model can be detected in a fine-grained manner, the attack source can be identified, and the over-fit poisoning attack can be effectively resisted. Compared with the traditional federal learning, the method can prevent attackers from cheating more training rewards by submitting the over-fit poisoning model or polluting the generalization capability of the global model, and can effectively guarantee the safety of the system and the fairness of the training participation of data holders, verifiers and miners.

For the attack of taking the free vehicle: 1) holder of malicious data

Direct submission of untrained local models

Or submitting a local model incorporating noise sigma

Where the noise sigma is negligibly small. Portable vehicle model

Verification accuracy of

With the global model omega^r-1Is verified byPrecision ACC^r-1Almost identical and can therefore easily be passed through the verifier detection of step 3.1 and directed to the mineworker M_s. Doped with a model for taking a convenient vehicle

Average model of

Will not be polluted and change M of miners_sThe difficulty of mining legal models and blocks in step 4.2, so that the detection of other miners in step five can be smoothly passed. 2) Malicious miner

Method for generating free-carrying vehicle model by using method similar to malicious data holder

The model of the vehicle for carrying the toilet is the global model omega^r-1The detection of other miners in the step five can be smoothly passed through due to high similarity. At this time, the symbol

And

all represent the pick-up model. However, due to the model of the vehicle

And

the accuracy gain of (3) is almost zero, and the prize that an attacker can obtain according to the prize distribution rules of steps 6.1-6.2 is also almost zero. The attacker loses the incentive of the attack because the attacker can not utilize the free-carrying attack to cheat the reward, thereby relieving the free-carrying attack.

Reward distribution fairness issues for participants at different time periods: the higher the verification precision will be as the current round number r increasesDifficulty, distributing rewards based only on accuracy gains, resulting in the reward amount available to late participants being much less than to early participants, would severely overwhelm the enthusiasm of late participants and also result in unfairness in the distribution of rewards at different times. The reward distribution rule of step 6.1 proposed by the invention is implemented by introducing a compensation term

The late participants are compensated, so that the fairness of the prize distribution of the participants in different periods is guaranteed.

Therefore, the reward mechanism in the step 6.1 can relieve the casual vehicle taking attack, prevent an attacker from cheating the training reward by submitting an casual vehicle taking model, balance the rewards of different difficulty training stages and effectively guarantee the fairness of reward distribution of participants in different periods.

For DDoS attacks: malicious miner

Directly generating the model of the vehicle for carrying the toilet by skipping the steps from two to four

Without waiting for the data holder and verifier to submit the local model, thereby enabling high frequency broadcast of dummy blocks to the blockchain network

And model for carrying vehicle

Multiple malicious miners perform the above operations and generate DDoS attacks aimed at slowing down the training efficiency of the system or completely paralyzing the system. However, the model of the vehicle for carrying

For the purpose of empty generation, the detection conditions of other miners in the step 5.4 are violated without support of homomorphic hash values of local model parameters in the transaction pool

Dummy block

Will be judged invalid and discarded in the voting stage of step 5.5, the source of the attack

Will also be penalized by the blockchain network, preventing the generation of DDoS attacks and continued damage to the system.

Therefore, the invention can prevent DDoS attack derived from the casual vehicle carrying attack, can prevent an attacker from submitting an casual vehicle carrying model to drag a slow or paralyzed block chain network through high frequency, and can effectively ensure the safety of the system.

In addition, the block chain federal learning system can realize efficient fine-grained Byzantine attack detection, and by introducing the role of the verifier, the data holder access, local model verification and model parameter storage work of miners are sunk to the verifier for execution, so that the pressure of the miners in communication, calculation and storage is relieved, and the node load balance in the edge cloud is realized. In addition, by using a plurality of verifier nodes, the edge cloud can access a larger number of data holders in parallel, and can execute local model verification in parallel to improve model verification efficiency and reduce model verification time delay, thereby improving system training efficiency. Meanwhile, miners can concentrate on model and block mining work, and generation of new blocks and updating of the global model can be accelerated.

The block chain federal learning system can be deployed in three edge computing scenes which are widely concerned, including a mobile edge computing scene, a micro data center scene, a micro cloud scene and a scene in which the three edge computing technologies are mixed. The block chain federal learning system does not need extra hardware cost, and only needs to be released to the existing edge computing infrastructure in a software mode for deployment.

The block chain federal learning system realizes decentralized federal learning, and can effectively relieve bottlenecks of communication, calculation, safety and the like of a server architecture with central parameters in traditional federal learning. By utilizing the characteristics of decentralization, the block chain federal learning system can effectively solve the problems of single-point faults and single-point attacks. For the single point of failure problem, failure and loss of any node in the system does not block the normal execution of the training task. For the single-point attack problem, a central node does not exist in the system, and an attacker loses the target of centralized attack, so that the safety protection pressure of each node can be relieved and shared.

The block chain federal learning system can support dynamic joining and exiting of most nodes. All data holders and verifiers can freely select nearby edge clouds and join the training task, and can also disconnect communication and quit the training task at any time. By utilizing the support of the blockchain network on the dynamic joining and withdrawing of the miner nodes, the edge cloud in the blockchain federal learning system can join and withdraw the training task at any time. The support for dynamic joining and quitting strengthens the robustness and flexibility of the system, and has important significance for forming a mutual benefit-benefit federal learning ecology between a task publisher and a task participant.

The block chain federal learning system can ensure the uncorruptable property of the block and model parameters. For the non-tamper property of the block, the technology of indexing by using the hash value of the previous block and forming a chain is utilized, and any tamper to the block changes the hash value of the block, so that the subsequent block cannot index to the block and the abnormal behavior of block chain disconnection is caused. For non-tamperability of the model parameters, tampering with the model parameters will result in the hash value of the actual model parameters not matching the storage credential returned by the IPFS, whereby tampering behaviour can be detected. Finally, the tampered blocks and model parameters can be restored by means of redundant backup technology of block chains and IPFS, thereby realizing non-tampering of the blocks and model parameters.

The block chain federal learning system realizes the traceability of block and model parameters. For the traceability of the block, the latest block can be indexed to the previous block according to the hash value of the previous block, and through repeated iteration, the index path can traverse the whole block chain, so that any historical block on the block chain can be traced back. For the traceability of the model parameters, since each block records the storage certificate of the global model parameters and the local model parameters corresponding to the training turns, the global model parameters and the local model parameters stored in the IPFS can be indexed according to the storage certificate. With the traceability of the blockchain and model parameters, the system can restore the blockchain and global model parameters to historical states.

The block chain federal learning system realizes auditability of block and model parameters. For auditability of blocks, a task publisher may utilize the block's traceability to review all blocks on the blockchain. For the auditability of the model parameters, the task publisher can acquire the global model parameters and all the local model parameters of each training turn by using the auditability of the model parameters and check whether the model parameters are legal or not online.

Claims

1. A block chain federal learning system is characterized by comprising a data holder, a verifier, a miner and a task publisher; the plurality of verifiers form a verifier group, each verifier group is connected with a miner, and each data holder is randomly connected with any verifier in the verifier group; all miners construct a block chain network and are connected with a task publisher through the block chain network;

the data holder has local training data of federal learning training and is used for training a local model according to the local training data;

the miners are used for mining the average model meeting the conditions according to the local model submitted by the verifier, generating a new block, broadcasting the new block in the block chain network to achieve consensus, and chaining the new block;

2. The blockchain federal learning system of claim 1, wherein the interconnected set of validators and miners form an edge cloud, and the edge servers in each edge cloud are interconnected by a local area network; the data holder is deployed on the intelligent terminal device and is accessed to the edge cloud through a wired and/or wireless edge network.

3. The blockchain federal learning system of claim 1, wherein the timestamp field is used to record creation time of created blocks and ordinary blocks;

the target round number field is used for setting the maximum number of training rounds expected by a task publisher and setting the length upper limit of the block chain;

the first current round number field is used for recording the current training round number when the creation block and the common block are generated, namely the first current round number; adding a block every time when the block chain is added, and accumulating the numerical value of the first current round number by 1 until the target round number is reached, and terminating the training task;

the first verification precision field is used for recording the verification precision of the global model parameters on the local verification set of the miners generating the block, namely the first verification precision;

the global model hash field is used for recording a storage certificate of global model parameters returned by the interplanetary file system;

the minimum transaction number field is used for setting the lower limit of the transaction number packaged in the common block body;

the tolerable precision oscillation threshold field is used for setting a tolerable precision oscillation threshold of the verifier and the miner, and when the first verification precision and the second verification precision fall ranges are within the precision oscillation threshold, both the local model and the average model are accepted;

the tolerable precision deviation threshold field is used for setting a tolerable verification precision deviation threshold of miners, and when the first verification precision deviation of the broadcast model on the local verification set of each miner is within the verification precision deviation threshold, the first verification precision recorded by the common block head is considered to be valid;

the homomorphic hash key field is used for recording homomorphic hash keys adopted by a homomorphic hash algorithm when homomorphic hash values of the model parameters are calculated; wherein the same homomorphic hash key is used by the verifier and the miners in the same edge cloud;

the previous block hash value field is used for recording a hash value obtained by carrying out hash operation on the whole previous block, and the hash value is used for searching the previous block so as to form a chain;

the Merkle root field is used for recording hash values recorded by the root of a Merkle tree consisting of a plurality of transactions in the common block, and the hash values recorded by the root of the Merkle tree consisting of a plurality of transactions are used for checking whether the packaged transactions are tampered;

the actual transaction number is used for recording the actual packaged transaction number in the common area block;

leaf nodes of the Merkle tree represent packaged transactions, intermediate nodes represent local hash values, and a tree root represents a hash value of the whole tree;

the second current round number field is used for recording the current training round number when the transaction is generated, namely the second current round number;

the data holder ID field is used for recording the identification of the data holder contributing the local model parameters in the transaction;

the verifier ID field is used for recording the identifier of a verifier for verifying the local model parameters in the transaction;

the local model hash field is used for recording storage certificates of local model parameters returned by the interplanetary file system;

the homomorphic hash value field is used for recording homomorphic hash values of local model parameters calculated by a homomorphic hash algorithm according to homomorphic hash keys specified by the current edge cloud;

the second verification accuracy field is used for recording the verification accuracy of the local model parameters on a local verification set of the verifier generating the transaction, namely the second verification accuracy.

4. A Byzantine attack detection method is characterized by comprising the following steps:

5. The Byzantine attack detection method according to claim 4, characterised in that said specific method of step S1 comprises the following sub-steps:

s1-1, generating initial global model parameter omega by task publisher⁰Initial global model parameters ω⁰Storing the data into an interplanetary file system and obtaining a storage certificate, namely a global model hash;

s1-2, calculating initial global model parameter omega by task publisher⁰First verification accuracy ACC on complete verification set F⁰Hashing the global model and obtaining the first verification precision ACC⁰The task configuration parameters and the hyper-parameters for machine learning are packaged into an innovation block B⁰And broadcasting the creation block B in the blockchain network⁰Issuing a training task; the task configuration parameters comprise a current timestamp, a target round number R, a first current round number R, a minimum transaction number L, a tolerable precision oscillation threshold value gamma and a tolerable verification precision deviation threshold value tau; the hyper-parameters include model structure, learning rate, neuron drop rate, batch data size, local iteration number, optimizer, loss function, and activation function.

6. The Byzantine attack detection method according to claim 5, characterised in that said specific method of step S4 comprises the following sub-steps:

s4-1, passing through data holder H_s,nDownloading global model parameters omega of previous round from verifier accessed by the same^r-1And using the local model parameters as the local model parameters;

s4-2, passing through data holder H_s,nLocal training sample

Cut into numbers (0,. eta., j.. eta., I) according to the batch data size b_s,nSmall batch data of/b-1);

an ith local training sample representing an nth data holder in the edge cloud s; h_s,nRepresenting the nth data holder in the edge cloud s; i is_s,nRepresenting the total number of local training samples of the nth data holder in the edge cloud s;

s4-3, according to the formula:

by using I_s,nB, circularly training local model parameter EI by small batch data_s,nB, obtaining local model parameters after training of traversing E rounds of the local training set; wherein

For the local model parameters after the training,

representing the local model parameters before the training, eta is the learning rate,

for local model parameters

In training sample

The model gradient calculated according to the gradient back propagation algorithm is calculated, j is a small batch data number counter, the initial value is 0, and the value is changed into (j +1) mod (I) when the local model parameter is trained once by using small batch data_s,n/b), mod (·) represents a complementation function;

s4-3, training the local model parameters by the data holder

And submitting to a verifier.

7. The Byzantine attack detection method of claim 4, wherein the thresholds in step S5 and step S7 are both ACC^r-1- γ; wherein ACC^r-1A first verification precision recorded in a block header of a latest block on a current block chain; gamma is a tolerable precision oscillation threshold.

8. The Byzantine attack detection method of claim 4, wherein the specific method of step S9 is:

whether a target round number, a minimum transaction number, a tolerable precision oscillation threshold value and a tolerable verification precision deviation threshold value in the received new block are consistent with corresponding fields of a latest block on a block chain or not is checked through a non-broadcast miner, whether a first current round number is equal to the first current round number of the latest block on the block chain plus 1 or not, whether first verification precision is larger than a threshold value or not, whether a hash value of a previous block is equal to the hash value of the latest block on the block chain or not, whether Merkle roots of Merkle trees formed by all actual transactions are equal to the tree root hash values of Merkle trees formed by all transactions in the block body or not, and whether the actual transaction numbers are larger than or equal to the minimum transaction number or not, if yes, the step S10 is carried out, and if not, the step S12 is carried out.

9. The Byzantine attack detection method according to claim 6, characterised in that said specific method of step S14 comprises the following sub-steps:

s14-1, distributing virtual currency to miners who excavate to new blocks as rewards by block chains, wherein the amount is

Wherein A is reward reference value, beta is reward compensation coefficient, ACC^rFirst verification precision, ACC, recorded for a new block^r-1A first verification precision, max { }, recorded for a block preceding the new blockTaking a maximum function, wherein e is a natural constant;

s14-2, miners mining to the new block allocate validation rewards to the validators according to the transaction numbers submitted by the validators, and allocate training rewards to the data holders according to the precision gains of the local model parameters submitted by the data holders.

10. The Byzantine attack detection method of claim 4, wherein the tolerable accuracy oscillation threshold is 0.002 and the tolerable verification accuracy deviation threshold is 0.05.