CN114764707A - Federal learning model training method and system - Google Patents

Abstract

The invention provides a method and a system for training a federated learning model, wherein the method comprises the following steps: creating a blockchain network for use in the training of the federal learning model; receiving local training model parameters uploaded by an organization user in each round of the training process of the federal learning model, wherein the organization user is a block chain user for locally training the federal learning model by using local user data; sending the local training model parameters uploaded by all organization users to worker users, and performing joint training on the federal learning model by the worker users by using all the local training model parameters; updating the federal learning model by adopting the parameters of the joint training model uploaded by the worker user who successfully creates the block; and sending the joint training model parameters to the organization user to update the local training model parameters of the organization user. In the invention, users of other participants except the data provider participate in the training of the federal learning model, so that the user range is expanded.

Description

Federated Learning Model Training Method and System

technical field

Embodiments of the present invention relate to the technical field of business support, and in particular, to a federated learning model training method and system.

Background technique

Federated Learning is to carry out efficient machine learning among multiple participants or multiple computing nodes under the premise of ensuring information security during big data exchange, protecting the privacy of terminal data and personal data, and ensuring legal compliance. An emerging artificial intelligence basic technology. Currently, some federated learning models are trained through federated learning.

However, the current method of training federated learning models through federated learning has the following problems: federated learning participants are limited to data providers, resulting in a small range of users and low training efficiency.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a federated learning model training method and system, which are used to solve the problems of small user scope and low training efficiency in the current method for training federated learning models through federated learning.

In order to solve the above-mentioned technical problems, the present invention is achieved in this way:

In a first aspect, an embodiment of the present invention provides a federated learning model training method, including:

Create a blockchain network for training federated learning models;

During each round of federated learning model training, receive local training model parameters uploaded by organizational users, who are blockchain users who locally use local user data to train the federated learning model;

Sending the local training model parameters uploaded by all the organizational users to the worker users, and the worker users use the local training model parameters uploaded by all the organizational users to jointly train the federated learning model;

Update the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created blocks;

Sending the joint training model parameters to the organization user to update the organization user's local training model parameters.

Optionally, the local training model parameters uploaded by the organizational user are encrypted using a homomorphic encryption algorithm.

Optionally, the organization user uses a self-defined key to perform homomorphic encryption on the parameters of the local training model.

Optionally, in each round of training of the federated learning model, the blockchain node of the blockchain network receives the local training model parameters uploaded by the organization user including:

When receiving the local training model parameters uploaded by the organizational user, determine whether the organizational user has uploaded an encrypted certificate at the same time, and the encrypted certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof;

If the organization user does not send the encryption certificate, it is determined that the organization user is a malicious participant.

Optionally, the method further includes:

Before the training of the federated learning model is performed, an additional sample uploaded by an organization user is received to form an additional sample library, where the additional sample includes a plurality of pairs of input samples and output samples, and the input samples are generated by the organization user using a random algorithm, The output sample is an output obtained by the organization user adding the input sample to its own local user data to train the federated learning model;

Receive local training model parameters uploaded by organization users;

Any N endorsement nodes of the blockchain network randomly select K samples from the additional sample library, and perform accuracy discrimination on the local training model parameters according to the K samples;

If the average accuracy of the N endorsement nodes is higher than the preset threshold, an encryption certificate is issued for the user of the organization.

Optionally, the method further includes:

After the blockchain network is created, initial assets are given to each user;

After the converged federated learning model is obtained, the assets are settled for each user according to the transactions that occur in each round.

Optionally, according to the transactions that occur in each round, before the asset settlement for each user, it also includes:

After the user joins the blockchain network, a smart contract set is deployed on the user's blockchain node. If the user is an organizational user, the smart contract set includes at least one of the following: a status query contract, a model update contract and model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract and a model download contract;

Wherein, when the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

When the model update contract is called by the organizational user, upload the local training model parameters of the organizational user to the blockchain network to complete a model parameter update transaction;

When the model download contract is called by the worker user, it downloads all the local training model parameters uploaded by the organization user from the blockchain network. The blockchain network downloads the joint training model parameters uploaded by the worker user to complete a model parameter download transaction;

When the model processing contract is invoked by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

In a second aspect, an embodiment of the present invention provides a federated learning model training system, including:

Create modules for creating blockchain networks for training federated learning models;

The first receiving module is used to receive the local training model parameters uploaded by the organizational user during each round of training of the federated learning model, and the organizational user is a blockchain that uses local user data to train the federated learning model locally user;

a first sending module, configured to send the local training model parameters uploaded by all the organizational users to the worker users, and the worker users use the local training model parameters uploaded by all the organizational users to jointly train the federated learning model;

an update module for updating the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created the block;

The second sending module is configured to send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

Optionally, the local training model parameters uploaded by the organizational user are encrypted using a homomorphic encryption algorithm.

Optionally, the organization user uses a self-defined key to perform homomorphic encryption on the parameters of the local training model.

Optionally, the system further includes:

The encryption module is used to determine whether the organization user has uploaded an encryption certificate at the same time when receiving the local training model parameters uploaded by the organization user. The encryption certificate is based on zero-knowledge by any N endorsement nodes of the blockchain network. Certificate issuance; if the organization user does not send an encrypted certificate, it is determined that the organization user is a malicious participant.

Optionally, the system further includes:

The first receiving module is configured to receive additional samples sent by organizational users before training the federated learning model to form an additional sample library, where the additional samples include a plurality of pairs of input samples and output samples, and the input samples are composed of the The organization user is generated by using a random algorithm, and the output sample is the output obtained by the organization user adding the input sample to its own local user data to train the federated learning model;

The second receiving module is used to receive the local training model parameters uploaded by the organization user;

A discrimination module, configured to randomly extract K samples from the additional sample library, and perform accuracy discrimination on the parameters of the local training model according to the K samples;

The certification module is used to issue an encryption certificate for the user of the organization if the average accuracy of the N endorsement nodes is higher than the preset threshold.

Optionally, the system further includes:

The initial asset assignment module is used to assign initial assets to each user after the blockchain network is created;

The settlement module is used to perform asset settlement for each user according to the transactions that occur in each round after the converged federated learning model is obtained.

Optionally, the system further includes:

The contract deployment module is used to deploy a smart contract set on the user's blockchain node after the user joins the blockchain network. If the user is an organizational user, the smart contract set includes at least one of the following: status query contract, model update contract and model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract and a model download contract;

Wherein, when the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

When the model update contract is called by the organizational user, upload the local training model parameters of the organizational user to the blockchain network to complete a model parameter update transaction;

When the model download contract is called by the worker user, it downloads all the local training model parameters uploaded by the organization user from the blockchain network. The blockchain network downloads the joint training model parameters uploaded by the worker user to complete a model parameter download transaction;

When the model processing contract is invoked by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

In a third aspect, an embodiment of the present invention provides a computing device, including: a processor, a memory, and a program stored on the memory and executable on the processor, when the program is executed by the processor The steps of implementing the federated learning model training method of the first aspect above.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the federated learning model training of the above-mentioned first aspect is implemented steps of the method.

In the embodiment of the present invention, combining blockchain and federated learning, by introducing users other than the data provider, users who do not have sufficient user data or do not want to share user data also have the right to participate in federated learning. In this way, the scope of users can be expanded, the efficiency of model training can be effectively improved, and rewards can be obtained by generating blocks through model training. These rewards can be used to deduct model usage fees, etc., and are excellent in terms of user scope and incentive mechanism. in existing technical solutions.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

1 is a schematic flowchart of a federated learning model training method according to an embodiment of the present invention;

2 is a schematic structural diagram of a federated learning model training system according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a computing device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Blockchain technology is a joint innovative application of four technologies: smart contracts, peer-to-peer networks, encryption algorithms and consensus mechanisms. The blockchain is essentially a timestamp-based linked list storage structure, also known as a distributed ledger or a decentralized database, because each node of the blockchain forms a peer-to-peer network, showing the characteristics of decentralization. The blockchain determines the writer based on the consensus mechanism, and at the same time ensures that the data of the entire network is consistent; based on the encryption algorithm, the data is safe and reliable, and the smart contracts deployed on the nodes are used for automated transaction processing. To sum up, the blockchain has the characteristics of decentralization, collective maintenance, cross-validation, openness and transparency, not easy to tamper with, and data traceability. Various application scenarios for trusting third parties, typical examples are digital finance, food and drug traceability, and judicial evidence storage.

As a decentralized, encrypted and non-tamperable distributed shared database, blockchain can provide data confidentiality for federated learning data exchange to protect user privacy and ensure data security between participants. The value-driven incentive mechanism of the data consistency blockchain that ensures that multiple participants provide data for model training can also increase the enthusiasm of each participant to provide data and update network model parameters.

In the present invention, the blockchain and federated learning are combined to train the federated learning model. Among them, the federated learning model can be a business recommendation model, etc.

Referring to FIG. 1, an embodiment of the present invention provides a federated learning model training method, including:

Step 11: Create a blockchain network for training federated learning models;

In the embodiment of the present invention, the participants of the blockchain network are collectively referred to as blockchain users, and blockchain users can be divided into organizational users and worker users according to their contribution methods.

(a) Organizational users refer to business vendors who own user data and are willing to use user data to participate in federated learning. Organizations contribute by sharing samples and participating in the training of federated learning models. Every time an organization submits updated model parameters, it can get corresponding rewards.

(b) Worker users refer to users who do not have enough user data, or who have user data but are unwilling to use their own user data to participate in federated learning, but still want to use the federated learning model, such as small-scale business manufacturers. The contribution method of worker users includes the work of federated learning model training and updating model parameters, and producing new blocks. Like Bitcoin miners, multiple worker users compete to complete the work of a block, and the first worker user to complete the block and confirm it through consensus can be rewarded.

In the embodiment of the present invention, multiple service providers in the same field jointly negotiate a deep learning model, which is used as a federated learning model. Considering that the services provided by multiple service providers belong to the same field, the characteristic fields of their data tend to have a high degree of coincidence, while the degree of coincidence of user groups is low. Therefore, the sample-based federated learning method is adopted, and the partial data whose feature fields overlap and user samples do not overlap are taken out as shared samples.

When the service provider provides services, according to the actual needs of the business, a local data collection module is built into the software (such as e-commerce, news and other APPs) or hardware (such as home theater video-on-demand, cloud game services and other equipment) to collect data. The user data is uploaded to the manufacturer's server to form the manufacturer's local user database.

Step 12: During each round of training of the federated learning model, receive the local training model parameters uploaded by the organizational user, and the organizational user is a blockchain user who locally uses the local user data to train the federated learning model;

Organize users to use local user data to train the federated learning model locally without uploading local user data, thus protecting the privacy of user data and avoiding user data leakage.

Step 13: Send the local training model parameters uploaded by all the organization users to the worker users, and the worker users use the local training model parameters uploaded by all the organization users to jointly train the federated learning model;

Step 14: Update the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created the block;

Step 15: Send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

In the embodiment of the present invention, combining blockchain and federated learning, by introducing users other than the data provider, users who do not have sufficient user data or do not want to share user data also have the right to participate in federated learning. In this way, the scope of users can be expanded, the efficiency of model training can be effectively improved, and rewards can be obtained by generating blocks through model training. These rewards can be used to deduct model usage fees, etc., and are excellent in terms of user scope and incentive mechanism. in existing technical solutions.

The reward mechanism of the embodiment of the present invention will be described below.

In this embodiment of the present invention, optionally, the method further includes:

After the blockchain network is created, each user is given initial assets; among them, the participants include organizational users and worker users, and the initial assets are used to reflect the cost (money cost or data cost) paid by the user to participate in federated learning.

After the converged federated learning model is obtained, the assets are settled for each user according to the transactions that occur in each round.

In this embodiment of the present invention, optionally, according to transactions that occur in each round, before performing asset settlement for each user, the method further includes:

After the user joins the blockchain network, a smart contract set S is deployed on the user's blockchain node. If the user is an organizational user, the smart contract set S includes at least one of the following: _a state query contract S1 , a model update contract S2 and a model download contract S3 _; if the user is a worker user, the smart contract set includes at least one _of the following: a state query contract S1, _a model processing contract S4 and a model download contract _{S3 ;} It should be noted that, in addition to this contract, the smart contract set S should be scalable, and other smart contracts can be added according to actual business needs.

Wherein, when the state query contract S1 is called by the user, it submits _a query transaction to query the world state on all distributed ledgers owned by the participant, that is, query the current remaining available assets of the user; _A party can submit a query transaction using the state query contract S1.

When the model update contract S2 is called by the organization user, upload the local training model parameters of the organization user to the _blockchain network to complete a model parameter update transaction; that is, the organization user can update the contract by calling the model S2 _uploads encrypted locally trained model parameters to the blockchain. In each iteration of federated learning, all organizational users (organization users) need to complete the model parameter upload for worker users to use all model parameters for joint training.

When the model download contract S3 is called by the worker user, it downloads all the local training model parameters uploaded by the organization user from the _blockchain network, and when the model download contract _S3 is called by the organization user, The joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction; that is, a participant can submit a model parameter download transaction by invoking the model download contract S ₃ .

When the model processing contract S4 is _invoked by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction. After the joint training is completed, the worker users who have successfully created the block can call the model processing contract S ₄ and use the joint training model parameters to update the parameters of the federated learning model.

A complete federated learning iteration consists of the following steps:

(a) Organize users to use local user data to train the federated learning model, and obtain the local training model parameters. _The model update contract S2 is called to generate a transaction, and the local training model parameters are encrypted and uploaded to the blockchain, that is, all users exchange the encrypted local training model parameters. For a transaction initiated by multiple users, each user needs to pay transaction fees based on the amount of data they own. Generally speaking, the larger the amount of data a party has, the less it will pay.

(b) _The worker user collects the model update contract S2 transaction proposed by all organizational users, calls the model download contract S3 to download all encrypted model parameters, uses all the model parameters to complete the joint training _of the federated learning model, and successfully creates the worker user of the block _The model processing contract S4 is called to complete the parameter update of the model.

( _c ) Organize the user to call the model download contract S3 to download the joint training model parameters, and update the local training model parameters after decryption.

At this point, a complete federated learning iteration is completed. At this point, a block in the blockchain was successfully created, the worker user who created the block was rewarded for mining, and the organization user who participated in the training completed an update of the local model. At this time, each organization user calculates whether the loss function meets the requirements. If not, it continues to initiate the next round of iteration, and repeats the above three steps (a), (b), (c) until the loss function converges.

When all users complete multiple rounds of federated learning iterations and obtain a converged federated learning model, actual asset settlement is performed according to the changes in user assets caused by transactions that occur in each iteration.

Although the organization user trains the federated learning model locally, the model parameters obtained from the training contain some features of the local user data. If a malicious party obtains plaintext model parameters and a predetermined federated learning model, the features of local user data can be inversely deduced from them, resulting in data leakage. Therefore, in this embodiment of the present invention, in order to avoid leakage of data feature information caused by leakage of model parameters, optionally, the local training model parameters uploaded by the organizational user are encrypted using a homomorphic encryption algorithm. Further optionally, the organization user uses a self-defined key to perform homomorphic encryption on the parameters of the local training model.

The characteristic of homomorphic encryption is that processing the homomorphically encrypted data to obtain an output, and decrypting this output, the result is the same as the output result obtained by processing the unencrypted original data by the same method. Therefore, as a worker user who actually performs joint training, the training task of the model can be completed without decrypting the gradient. That is, worker users cannot access any unencrypted data in the whole process, which avoids the leakage of local model data feature information.

In the embodiment of the present disclosure, by using the homomorphic encryption algorithm, the model parameters uploaded by the organization user are encrypted in one layer, which prevents information leakage caused by the plaintext model parameters. The data that worker users contact when training the model is encrypted data, and worker users cannot decrypt it at all, so it also solves the problem that organizational users and worker users do not trust each other.

In the federated learning process, there are multiple participants, and there may be malicious participants among the participants. The malicious participants maliciously upload wrong or forged model parameters to obtain the joint training model without paying or paying less cost. parameters, while reducing the accuracy of the jointly trained model.

In order to avoid malicious participant attacks, in the embodiment of the present invention, in each round of training of the federated learning model, the blockchain node of the blockchain network receives the local training model parameters uploaded by the organization user including:

When receiving the local training model parameters uploaded by the organizational user, determine whether the organizational user has uploaded an encrypted certificate at the same time, and the encrypted certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof;

If the organization user does not send the encryption certificate, it is determined that the organization user is a malicious participant.

In this embodiment of the present invention, optionally, the method further includes:

Before the training of the federated learning model is performed, an additional sample uploaded by an organization user is received to form an additional sample library, where the additional sample includes a plurality of pairs of input samples and output samples, and the input samples are generated by the organization user using a random algorithm, The output sample is an output obtained by the organization user adding the input sample to its own local user data to train the federated learning model;

Receive local training model parameters uploaded by organization users;

Any N endorsement nodes of the blockchain network randomly select K samples from the additional sample library, and perform accuracy discrimination on the local training model parameters according to the K samples;

If the average accuracy of the N endorsement nodes is higher than the preset threshold, an encryption certificate is issued for the user of the organization.

Among them, N is a positive integer greater than 1, and K is a positive integer greater than 1.

That is, before federated learning starts, organizational users generate additional samples using a random algorithm, and all organizational users jointly generate an additional sample pool. The data in this additional repository is automatically generated and has nothing to do with any organization's own user data. Each user needs to add the additional sample library to its own user database as a supplementary training set for model training. In each round of federated learning iteration, in the step of organizing users to upload encrypted model parameters, each organizational user needs to submit an encrypted proof at the same time. The cryptographic proof is issued by any N endorsing nodes based on zero-knowledge proof. The specific method of issuing is as follows: the endorsement node randomly selects K samples from the additional sample library, and discriminates the encrypted model parameters uploaded by users of the organization. If the average accuracy rate of N endorsement nodes is higher than the threshold λ, the encrypted model parameters uploaded by users of the organization are considered to be correct. That is, input the input samples in the samples into the federated learning model composed of the encrypted model parameters, obtain the output samples, and judge the average accuracy rate of the output samples of the N endorsement nodes.

In the embodiment of the present invention, based on the idea of zero-knowledge proof, an encryption proof mechanism is proposed. This mechanism can verify the correctness of the encrypted model parameters without obtaining the model parameters uploaded by the organization user, preventing malicious participants from uploading wrong parameters or forging parameters, and solving the problem that federated learning partners may face in the prior art. The malicious attack is difficult to trace and punish.

Referring to FIG. 2, an embodiment of the present invention further provides a federated learning model training system 20, including:

Create a module 21 for creating a blockchain network for training a federated learning model;

The first receiving module 22 is used to receive the local training model parameters uploaded by the organizational user during each round of training of the federated learning model, where the organizational user is the block that uses the local user data to train the federated learning model locally chain user;

The first sending module 23 is configured to send the local training model parameters uploaded by all the organizational users to the worker users, and the worker users use the local training model parameters uploaded by all the organizational users to jointly train the federated learning model;

The updating module 24 is used for updating the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created the block;

The second sending module 25 is configured to send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

Optionally, the local training model parameters uploaded by the organizational user are encrypted using a homomorphic encryption algorithm.

Optionally, the organization user uses a self-defined key to perform homomorphic encryption on the parameters of the local training model.

Optionally, also include:

The encryption module is used to determine whether the organization user has uploaded an encryption certificate at the same time when receiving the local training model parameters uploaded by the organization user. The encryption certificate is based on zero-knowledge by any N endorsement nodes of the blockchain network. Certificate issuance; if the organization user does not send an encrypted certificate, it is determined that the organization user is a malicious participant.

Optionally, the system further includes:

The first receiving module is configured to receive additional samples sent by organizational users before training the federated learning model to form an additional sample library, where the additional samples include a plurality of pairs of input samples and output samples, and the input samples are composed of the The organization user is generated by using a random algorithm, and the output sample is the output obtained by the organization user adding the input sample to its own local user data to train the federated learning model;

The second receiving module is used to receive the local training model parameters uploaded by the organization user;

A discrimination module, configured to randomly extract K samples from the additional sample library, and perform accuracy discrimination on the parameters of the local training model according to the K samples;

The certification module is used to issue an encryption certificate for the user of the organization if the average accuracy of the N endorsement nodes is higher than the preset threshold.

Optionally, the system further includes:

The initial asset assignment module is used to assign initial assets to each user after the blockchain network is created;

The settlement module is used to perform asset settlement for each user according to the transactions that occur in each round after the converged federated learning model is obtained.

Optionally, the system further includes:

The contract deployment module is used to deploy a smart contract set on the user's blockchain node after the user joins the blockchain network. If the user is an organization user, the smart contract set includes at least one of the following: status query contract, model update contract and model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract and a model download contract;

Wherein, when the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

When the model update contract is called by the organizational user, upload the local training model parameters of the organizational user to the blockchain network to complete a model parameter update transaction;

When the model download contract is called by the worker user, it downloads all the local training model parameters uploaded by the organization user from the blockchain network. The blockchain network downloads the joint training model parameters uploaded by the worker user to complete a model parameter download transaction;

When the model processing contract is invoked by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

To sum up, the scheme of the present invention mainly consists of the following parts:

A federated learning network. Multiple businesses in the same business domain (such as cloud services, video, or communications, etc.) provide user data collected by themselves, and perform deep learning model training locally. The model parameters of the intermediate model obtained by training are encrypted, uploaded to a blockchain network composed of all manufacturers for joint training, and the model parameters after joint training are downloaded for local model update. A federated learning model in this business domain, such as a business recommendation model, can be used by each manufacturer to make more accurate business recommendations to users. During the entire training process, all user privacy data are stored locally in the business vendor, which avoids user privacy data leakage due to shared samples.

A blockchain network used to ensure that the on-chain data of all parties involved in federated learning cannot be tampered with and can be traced. When malicious parties try to use wrong parameter updates to destroy the overall model effect, the cross-validation mechanism of the blockchain network can be used to quickly locate malicious parties and avoid erroneous updates. In addition, the inherent incentive mechanism of the blockchain network can allow participants with more contributions to obtain additional income compensation.

A set of cryptographic proof mechanisms based on homomorphic encryption and zero-knowledge proofs are used to ensure that the local data characteristics of each participant are not leaked and avoid malicious participant attacks.

A federated learning model training system for local data collection, local modeling, model parameters plus multi-party joint training of the final model, data upload to the blockchain, blockchain contribution incentives, and malicious attacks that may be faced by federated learning partners. All functions involved in this technical solution, such as traceability.

In one embodiment, the federated learning model training system involved in this solution includes: a user information management module, a local model training module, a blockchain operation module, a settlement module, and an encryption module.

The user information management module is used for basic information management such as user registration, information recording and information modification. In addition, this module also has the ability to classify users according to the user's asset certificate (money assets or data assets), and limit the user's functional rights according to the user's category.

The local model training module is used to complete the training, update and maintenance of the user's local model. Organization users call this module to use the user data in the local database to perform preliminary model training, and use the federated learning model parameters after joint training by worker users to update the local model; worker users call this module to collect all encrypted data. The models are jointly trained.

The blockchain operation module is used to provide users with functions such as node joining or exiting the blockchain, initiating asset transactions, uploading or downloading model parameters, submitting encrypted certificates, and data traceability, and submitting the user's operation results to the blockchain. .

The encryption module is used to provide users with the function of homomorphic encryption based on a user-defined key, and the encryption correctness audit function based on zero-knowledge proof. In the step of uploading encrypted model parameters, the function of presenting encryption proof is completed by this module.

Referring to FIG. 3, an embodiment of the present invention further provides a computing device 30, including a processor 31, a memory 32, and a computer program stored in the memory 32 and running on the processor 31, the computer program being executed by the processor When 31 is executed, each process of the above-mentioned embodiment of the federated learning model training method is implemented, and the same technical effect can be achieved. In order to avoid repetition, details are not described here.

Embodiments of the present invention also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, each process of the foregoing federated learning model training method embodiment is implemented, and can To achieve the same technical effect, in order to avoid repetition, details are not repeated here. The computer-readable storage medium is, for example, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a computing device, etc.) execute the methods described in the various embodiments of the present invention.

The embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the inspiration of the present invention, without departing from the spirit of the present invention and the scope protected by the claims, many forms can be made, which all belong to the protection of the present invention.

Claims (10)

1. a federated learning model training method, is characterized in that, comprises: Create a blockchain network for training federated learning models; During each round of federated learning model training, receive local training model parameters uploaded by organizational users, who are blockchain users who locally use local user data to train the federated learning model; Sending the local training model parameters uploaded by all the organizational users to the worker users, and the worker users use the local training model parameters uploaded by all the organizational users to jointly train the federated learning model; Update the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created blocks; Sending the joint training model parameters to the organization user to update the organization user's local training model parameters. 2 . The method according to claim 1 , wherein the local training model parameters uploaded by the organization user are encrypted by using a homomorphic encryption algorithm. 3 . 3 . The method according to claim 2 , wherein the organization user performs homomorphic encryption on the parameters of the local training model by using a self-defined key. 4 . 4. The method according to claim 1, wherein, in each round of training of the federated learning model, the blockchain node of the blockchain network receives the local training model parameters uploaded by the organization user comprising: When receiving the local training model parameters uploaded by the organizational user, determine whether the organizational user has uploaded an encrypted certificate at the same time, and the encrypted certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof; If the organization user does not send the encryption certificate, it is determined that the organization user is a malicious participant. 5. The method of claim 4, further comprising: Before the training of the federated learning model is performed, an additional sample uploaded by an organization user is received to form an additional sample library, where the additional sample includes a plurality of pairs of input samples and output samples, and the input samples are generated by the organization user using a random algorithm, The output sample is an output obtained by the organization user adding the input sample to its own local user data to train the federated learning model; Receive local training model parameters uploaded by organization users; Any N endorsement nodes of the blockchain network randomly select K samples from the additional sample library, and perform accuracy discrimination on the local training model parameters according to the K samples; If the average accuracy of the N endorsement nodes is higher than the preset threshold, an encryption certificate is issued for the user of the organization. 6. The method of claim 1, further comprising: After the blockchain network is created, initial assets are given to each user; After the converged federated learning model is obtained, the assets are settled for each user according to the transactions that occur in each round. 7. The method according to claim 6, wherein, according to the transactions that occur in each round, before performing asset settlement for each user, the method further comprises: After the user joins the blockchain network, a smart contract set is deployed on the user's blockchain node. If the user is an organizational user, the smart contract set includes at least one of the following: a status query contract, a model update contract and model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract and a model download contract; Wherein, when the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user; When the model update contract is called by the organizational user, upload the local training model parameters of the organizational user to the blockchain network to complete a model parameter update transaction; When the model download contract is called by the worker user, all the local training model parameters uploaded by the organization user are downloaded from the blockchain network to complete a model parameter download transaction; the model download contract is called by the organization When the user calls, download the joint training model parameters uploaded by the worker user from the blockchain network to complete a model parameter download transaction; When the model processing contract is invoked by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction. 8. A federated learning model training system, comprising: Create modules for creating blockchain networks for training federated learning models; The first receiving module is used to receive the local training model parameters uploaded by the organizational user during each round of training of the federated learning model, and the organizational user is a blockchain that uses local user data to train the federated learning model locally user; a first sending module, configured to send the local training model parameters uploaded by all the organizational users to the worker users, and the worker users use the local training model parameters uploaded by all the organizational users to jointly train the federated learning model; an update module for updating the federated learning model using the joint training model parameters uploaded by the worker users who have successfully created the block; The second sending module is configured to send the joint training model parameters to the organization user to update the local training model parameters of the organization user. 9. A computing device, comprising: a processor, a memory, and a program stored on the memory and executable on the processor, the program being executed by the processor to achieve as claimed in the claims Steps of the federated learning model training method described in any one of 1 to 7. 10. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program according to any one of claims 1 to 7 is implemented. The steps of the federated learning model training method.

Images