WO2021114929A1

WO2021114929A1 - Blockchain-based model combination training method and device

Info

Publication number: WO2021114929A1
Application number: PCT/CN2020/124489
Authority: WO
Inventors: 倪翔; 汲小溪; 王维强
Original assignee: 支付宝(杭州)信息技术有限公司
Priority date: 2019-12-09
Filing date: 2020-10-28
Publication date: 2021-06-17
Also published as: CN111062044B; CN113268776A; TWI732557B; CN113268776B; CN111062044A; TW202123047A

Abstract

A blockchain-based model combination training method and device, wherein in the training method, a current layer node of a blockchain receives a first training result sent by a corresponding upper layer node. The first training result is determined at least on the basis of a first type of result, the first type of result is obtained by the upper layer node on the basis of its private data maintained off the chain, after training a target model in multiple models maintained on the chain. The current layer node trains the target model in the multiple models maintained on the chain on the basis of the private data maintained off the chain to obtain a second training result. The first training result and the second training result are fused to obtain a fusion result. It is determined whether the current layer node is in the last layer. If the current layer node is in the last layer, the fusion result is broadcast on the blockchain, so that multiple nodes update the target model in the multiple models maintained on the chain on the basis of the fusion result.

Description

Model joint training method and device based on blockchain

Technical field

One or more embodiments of this specification relate to the field of computer technology, and in particular to a method and device for joint training of models based on blockchain.

Background technique

Federated Learning, also known as joint learning, is an emerging basic technology of artificial intelligence. Its design goal is to carry out high-efficiency machine learning among multiple parties (or computing nodes or subjects) under the premise of ensuring information security during big data exchange, protecting terminal data and personal data privacy, and ensuring legal compliance.

In the traditional model joint learning or training process, in order to ensure the security of the data of all parties, the data interacting between the parties is usually encrypted, but there is still a risk of data leakage.

Therefore, it is necessary to provide a safer model joint training method.

Summary of the invention

One or more embodiments of this specification describe a method and device for joint training of models based on blockchain, which can efficiently perform joint training on models while ensuring the security of data of all parties.

In a first aspect, a method for joint training of models based on blockchain is provided, including: receiving a first training result sent by a corresponding upper node; the first training result is determined based on at least a first type of result; the first The type result is obtained by the upper-level node based on its private data maintained off-chain, after training the target model in the multiple models maintained on the chain; based on the private data maintained off-chain, it is obtained on-chain The target model among the multiple models maintained is trained to obtain the second training result; the first training result and the second training result are fused to obtain the fusion result; it is judged whether the current layer node is the last One level; if the current layer node is at the last level, the fusion result is broadcast on the blockchain, so that the multiple nodes can compare the various models maintained on the chain based on the fusion result The target model in is updated.

In a second aspect, a method for joint training of models based on blockchain is provided, which includes: receiving a first training result sent by a corresponding upper node; the first training result includes at least a first type of result; the first type The result is obtained by the upper node after training the target model in the multiple models maintained on the chain based on its private data maintained off-chain; based on the private data maintained off-chain, it is maintained on-chain Training the target model of the multiple models to obtain the second training result; judging whether the current layer node is in the last level; if the current layer node is in the last level, then the first training result Fusion with the second training result to obtain the fusion result; broadcast the fusion result on the blockchain, so that the multiple nodes can compare each of the multiple models maintained on the chain based on the fusion result The target model is updated.

In a third aspect, there is provided a block chain-based model joint training device, including: a receiving unit for receiving a first training result sent by a corresponding upper node; the first training result is determined based on at least a first type of result The first type of result is obtained by the upper node based on its private data maintained under the chain after training the target model in the multiple models maintained on the chain; the training unit is used to base its off-chain For the maintained private data, train the target model among the multiple models maintained on the chain to obtain the second training result; the fusion unit is used to perform the training on the first training result received by the receiving unit and all the training results. The second training result obtained by the training unit is fused to obtain the fusion result; the judging unit is used to judge whether the current layer node is at the last level; the broadcasting unit is used to if the judging unit judges the current layer If the layer node is at the last level, the fusion result is broadcast on the blockchain, so that the multiple nodes perform the target model of the multiple models maintained on the chain based on the fusion result. Update.

In a fourth aspect, a block chain-based model joint training device is provided, including: a receiving unit configured to receive a first training result sent by a corresponding upper node; the first training result includes at least a first type result; The first type of result is obtained by the upper node after training the target model in the multiple models maintained on the chain based on its private data maintained off-chain; the training unit is used for maintenance based on its off-chain To train the target model in the multiple models maintained on the chain to obtain the second training result; the judgment unit is used to judge whether the current layer node is at the last level; the fusion unit is used If the judging unit judges that the current layer node is at the last level, then the first training result and the second training result are fused to obtain the fusion result; the broadcasting unit is used for logging in the blockchain The fusion result obtained by the fusion unit is broadcast on the above, so that the multiple nodes update the target model in the multiple models maintained on the chain based on the fusion result.

In a fifth aspect, a computer storage medium is provided with a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect or the method of the second aspect.

In a sixth aspect, a computing device is provided, including a memory and a processor, the memory stores executable code, and when the processor executes the executable code, the method of the first aspect or the method of the second aspect is implemented method.

One or more embodiments of this specification provide a blockchain-based model joint training method and device. Each layer node of the blockchain can receive the training result sent by the corresponding upper node, and can compare the received training result with its own The training results are fused and sent to the corresponding lower-level nodes, or the two training results are directly forwarded to the corresponding lower-level nodes, until the last-level node is reached. The last layer of nodes integrates all the training results to obtain the final training result, and broadcasts the final training result on the blockchain, so that each node can update the corresponding model, thus achieving the blockchain-based model Joint training. That is to say, the solution provided in this specification can realize the joint training of the model based on the blockchain, thereby ensuring the safety and reliability of the joint training process of the model.

Description of the drawings

In order to explain the technical solutions of the embodiments of this specification more clearly, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the specification. A person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

Figure 1 is a schematic diagram of the implementation scenario of the blockchain-based model joint training method provided in this manual;

FIG. 2 is a flowchart of a model joint training method based on blockchain provided by an embodiment of this specification;

FIG. 3 is a flowchart of a model joint training method based on blockchain provided by another embodiment of this specification;

Figure 4 is a schematic diagram of a block chain-based model joint training device provided by an embodiment of this specification;

Fig. 5 is a schematic diagram of a block chain-based model joint training device provided by another embodiment of this specification.

Detailed ways

The following describes the solutions provided in this specification with reference to the accompanying drawings.

Before describing the solution provided in this specification, the following description of the inventive concept of this solution is provided.

As described in the background art, the traditional model joint training method has the risk of data leakage. In order to ensure the security of the data of all parties, this solution will implement the joint training of the model based on the blockchain. Blockchain is a series of text records that are connected and protected by cryptography. Each block contains the encrypted hash of the previous block, the corresponding time stamp and transaction data. This design makes the block content difficult to tamper with. The distributed ledger connected by blockchain technology allows two parties to effectively record the transaction, and the transaction can be permanently checked.

The specific realization idea can be as follows: first, each node in the blockchain can be divided into multiple levels, and the corresponding relationship between the nodes of two adjacent levels can be established. After that, each layer node of the blockchain can receive the training result sent by the corresponding upper node, and can merge the received training result with its own training result and send it to the corresponding lower node, or directly forward the two training results To the corresponding lower-level node, until reaching the last-level node. The last layer of nodes integrates all the training results to obtain the final training result, and broadcasts the final training result on the blockchain, so that each node can update the corresponding model, thus achieving the blockchain-based model Joint training.

The above is the inventive concept provided in this specification, and the solution can be obtained based on the inventive concept. The solution will be described in detail below.

Figure 1 is a schematic diagram of the implementation scenario of the blockchain-based model joint training method provided in this specification. In Figure 1, nodes 1 to 15 are nodes in the blockchain, and each node is organized according to a hierarchical relationship. Specifically, node 1 to node 8 is in the first level, node 9 to node 12 is in the second level, node 13 to node 14 is in the third level, and node 15 is in the fourth level (that is, the last level) . In addition, node 1-node 2 of the first level corresponds to node 9 of the second level, node 3-node 4 of the first level corresponds to node 10 of the second level, and node 5 of the first level. The node 6 corresponds to the node 11 of the second level, and the node 7-node 8 of the first level corresponds to the node 12 of the second level. The node 9-node 10 of the second level corresponds to the node 13 of the third level, and the node 11-node 12 of the second level corresponds to the node 14 of the third level. The node 13-node 14 of the third level corresponds to the node 15 of the last level.

Each node in Figure 1 can represent a participant. Each of the above nodes can maintain multiple models on the chain. The multiple models here may include at least one of a Logistic Regression (LR) model, a Gradient Boosting Decision Tree (GBDT) model, and a (Deep Neural Networks, DNN) model. Any one of the above-mentioned multiple models may be submitted to the blockchain in advance by a certain node. It is understandable that for the model submitted to the blockchain, each node can perform corresponding calculations (eg, training). In addition, each node in the blockchain can also maintain its own private data under the chain. In an example, each node can implement the maintenance of the off-chain data by including a pointer to the off-chain data in the corresponding block record.

In Figure 1, the task rebuilder can publish model training tasks on the blockchain. The task processor here can be a separate server, or it can be set on any node in the blockchain. In addition, the aforementioned model training task at least indicates the target model to be trained. It should be noted that after the above-mentioned model training task is released, only the nodes of the first level will perform the training task, and the nodes of other levels will wait to receive the training result of the corresponding node before starting the training task.

Take node 1 to node 2 of the first level as an example. After publishing the above model training task, node 1 to node 2 can select the target model from a variety of models maintained on the chain, and based on its The private data maintained under the chain trains the selected target model. After that, node 1 and node 2 may send their respective training results (eg, model gradient and/or model parameters) to node 9. The node 9 can obtain the corresponding training result, and merge the received training result with the training result obtained by itself, or directly forward all the training results to the node 13 and so on, and so on, until the node 15 is reached. The node 15 can fuse all the training results and publish the fusion result to the blockchain, so that each node updates the target model maintained by each node based on the fusion result.

It is explained here that due to space limitations, Figure 1 only lists four levels of nodes, and each node of the next level only corresponds to the two nodes of the previous level. In fact, the blockchain can also include more than There are four levels of nodes, and each node of the next level can correspond to more than two nodes of the previous level, which is not limited in this specification.

It should be noted that the chain hash pointer on the blockchain can ensure that it is almost impossible to change or delete records on the blockchain. In addition, records on the blockchain can also contain pointers to private data off-chain. The pointer and hash pointer together can be used to verify the integrity of the data off the chain. In this way, the blockchain can also protect the integrity of off-chain data. Protecting the integrity of on-chain and off-chain records essentially prevents fraudulent deletion or alteration of records for personal gain.

In addition, the blockchain provides near real-time transparency of records attached to the blockchain, that is, these new records are visible to each node of the blockchain. This improves the ability of such nodes to detect fraudulent records. The existence of this transparency and the mere possibility of being discovered will greatly deter fraud.

In summary, this solution can ensure the safety of the model joint training process.

Fig. 2 is a flowchart of a model joint training method based on blockchain provided by an embodiment of this specification. The execution subject of the method can be any node among the nodes of the 2-4th level of the blockchain in FIG. 1. As shown in FIG. 2, the method may specifically include step 202 to step 210.

Step 202: Receive the first training result sent by the corresponding upper node.

Regarding the above-mentioned first training result, here is explained in two cases. In the first case, the above-mentioned upper-level node does not have a corresponding upper-level node, that is, the upper-level node is a node of the first level, and the first training result is based on the first type The result is determined, for example, the first type of result is used as the first training result. The first type of result here is obtained by the upper node based on the private data maintained off-chain by the target model of the multiple models maintained on the chain. In the second type, the above-mentioned upper-level node also has a corresponding upper-level node, that is, the upper-level node is a non-first-level node, then the first training result is determined based on the first-type result and the second-type result, where the second-type result It is determined by the upper node based on the result received from its corresponding upper node. In an example, the first training result can be obtained by averaging the results of the first type and the second type, or by calculating the weighted average.

It should be noted that the above-mentioned target model may be specified by the task processor when issuing the model training task. For example, it can be any of LR model, GBDT model or DNN model. In addition, the above-mentioned first type result or second type result may refer to the intermediate model gradient and/or the intermediate model parameter of the target model.

It should also be noted that the number of the aforementioned first training results may be determined based on the number of corresponding upper-level nodes. That is, when the number of corresponding upper-layer nodes is multiple, the number of the above-mentioned first training results may also be multiple.

The above-mentioned first training result will be described below in conjunction with FIG. 1.

Taking node 9 at the second level in Figure 1 as an example, node 9 can receive the first training results sent by node 1 and node 2, that is, the number of first training results received by node 9 is two . Among them, the first training result sent by node 1 (or node 2) is the first type result, that is, when node 1 (or node 2) receives the model training task issued by the task processor on the blockchain, It is obtained after training the target model based on the private data maintained under the chain.

For the node 13 of the third level, it can receive the first training results sent by the node 9 and the node 10 respectively, that is, the number of the first training results received by the node 13 is two. Wherein, the first training result sent by the node 9 (or the node 10) is determined based on the first type result and the second type result. For node 9, the corresponding first type result is obtained by node 9 after receiving the first training result sent by node 1 and node 2, based on the private data maintained under the chain, after training the target model. The corresponding second type result is obtained by node 9 after fusing the first training results sent by node 1 and node 2.

Step 204, based on the private data maintained off-chain, train the target model among the multiple models maintained on the chain to obtain a second training result.

For example, the node 9 of the second level can train the target model to obtain the second training result. For another example, the node 13 of the third level can train the target model to obtain the second training result.

In step 206, the first training result and the second training result are fused to obtain a fusion result.

For example, the first training result can be used as the new second type result, and the second training result can be used as the new first type result for averaging or weighted average to obtain the fusion result.

For example, the node 9 of the second level can use the two first training results received from node 1 and node 2 as two new second-type results, in addition, it can also use the second training results obtained by itself. The result is the new first type result. After that, the two new second-type results and the new first-type results are averaged or weighted to obtain the fusion result. It is understandable that the fusion result is the first training result sent by node 9 to node 13.

For another example, the node 13 at the third level can use the two first training results received from the node 9 and the node 10 as two new second-type results, and in addition, it can also use the second-type results obtained by itself. The training result is regarded as the new first type result. After that, the two new second-type results and the new first-type results are averaged or weighted to obtain the fusion result. It can be understood that the fusion result is the first training result sent by node 13 to node 15.

Step 208: Determine whether the current layer node is in the last layer.

Step 210: If the current layer node is at the last level, broadcast the fusion result on the blockchain, so that multiple nodes update the target model in the multiple models maintained on the chain based on the fusion result.

Taking the node 15 of the fourth level in FIG. 1 as an example, since the node 15 is at the last level, it can broadcast the fusion result on its blockchain. It is understandable that the fusion result here may refer to the total model gradient or the total model parameters of the target model.

If the fusion result is the total model gradient, each node can determine the total model parameters based on the total model gradient after receiving the total model gradient. After that, based on the total model parameters, the corresponding target models are updated.

If the fusion result is the overall model parameter, after each node receives the overall model parameter, it can directly update the corresponding target model based on the overall model parameter.

The above is the description of the case where the current layer node is at the last level, and the following describes the case where the current layer node is not at the last level.

If the current layer node is not in the last layer, the fusion result is used as the first training result sent to the corresponding lower layer node.

Taking the node 9 of the second level as an example, it can send the corresponding fusion result to the node 13 as the first training result.

Taking the node 13 of the third level as an example, it can send the corresponding fusion result to the node 15 as the first training result.

It should be noted that after step 210 is executed, only one round of iterative training of the target model is completed. If after one round of iteration, the loss function corresponding to the target model reaches the predetermined convergence condition, the joint model training ends; otherwise, the next round of iteration is entered until the loss function corresponding to the target model reaches the predetermined convergence condition. It is understandable that the next iteration of the process is executed based on the updated target model.

When the loss function corresponding to the target model meets the predetermined convergence condition, the joint training of the model ends. The task processor can call the smart contract, and the smart contract is used to calculate the respective rewards of multiple nodes, and distribute the respective rewards to the multiple nodes, so as to increase the respective virtual resources of the multiple nodes.

In an example, the implementation principle of a smart contract is as follows: Assuming that the t-th submitted data (or algorithm) gains A_t to the existing model, for each submitted data (or algorithm) h, define a loss function L(h,t ). The reward corresponding to the last node is A_t–L(h,t).

The above model gain can be obtained by calculating the Shapley value. Shapley value is to fairly distribute the benefits of cooperation by considering the contributions made by each node.

It should also be noted that step 202 to step 210 are operations performed by nodes other than the first level in the model joint training process. It is understandable that, for each node of the first level, when it receives the model training task issued by the task processor on the blockchain, it can maintain it on the chain based on the private data maintained under the chain. The target model among multiple models is trained, and the second training result is obtained. After that, the second training result can be used as the first training result sent to the corresponding lower-level node (that is, the second-level node). Therefore, the corresponding lower-level node can execute the above-mentioned step 202-step 210 until the node of the last level is reached.

In summary, the blockchain-based model joint training method provided by the embodiments of this specification has made the following innovations: First, innovatively propose to introduce blockchain into federated learning. Second, use the hash pointer technology of the blockchain to solve the integrity of the data on and off the chain. Third, use the digital signature technology of the blockchain to solve the authenticity of the data source of the machine learning model. Fourth, the use of blockchain's smart contract technology solves the problem of incentive and punishment mechanisms that promote data sharing and algorithm co-construction in federated learning. Fifth, the process of federated learning is realized on the blockchain.

Based on the above innovations, the authenticity, completeness, and traceability of the private data and model training results of each participant can be guaranteed. In addition, the use of smart contracts can promote collaboration capabilities among various participants.

Fig. 3 is a flowchart of a model joint training method based on blockchain provided by another embodiment of this specification. The execution subject of the method can be any node among the nodes of the 2-4th level of the blockchain in FIG. 1. As shown in FIG. 3, the method may specifically include step 302 to step 310.

Step 302: Receive the first training result sent by the corresponding upper node.

Regarding the above-mentioned first training result, here are two cases for explanation. In the first case, the above-mentioned upper-level node does not have a corresponding upper-level node, that is, the upper-level node is a node of the first level, and the first training result only includes the first-level node. Type results. The first type results here are obtained by the upper node after training the target model among the multiple models maintained on the chain based on the private data maintained off-chain by the upper node. In the second type, the above-mentioned upper-level node also has a corresponding upper-level node, that is, the upper-level node is a non-first-level node, and the first training result includes a first-type result and a second-type result. Wherein, the second type result is determined by the upper node based on the result received from its corresponding upper node. For example, the second type result is composed of the training results of each upper node corresponding to the upper node.

Taking node 9 at the second level in Figure 1 as an example, node 9 can receive the first training results sent by node 1 and node 2, that is, the number of first training results received by node 9 is two . Among them, the first training result sent by node 1 (or node 2) only includes the first type of result, that is, when node 1 (or node 2) receives the model training task issued by the task processor on the blockchain, It is obtained after training the target model based on the private data maintained under the chain.

For the node 13 of the third level, it can receive the first training results sent by the node 9 and the node 10 respectively, that is, the number of the first training results received by the node 13 is two. Wherein, the first training result sent by the node 9 (or the node 10) includes the first type result and the second type result. For node 9, the corresponding first type result is obtained by node 9 after receiving the first training result sent by node 1 and node 2, based on the private data maintained under the chain, after training the target model. The corresponding second type result is composed of the first training result sent by node 1 and node 2 corresponding to node 9.

Step 304: Based on the private data maintained off-chain, train the target model among the multiple models maintained on-chain to obtain a second training result.

Step 306: Determine whether the current layer node is in the last layer.

Step 308: If the current layer node is in the last layer, the first training result and the second training result are fused to obtain the fusion result.

For example, the first training result and the second training result can be averaged or weighted averaged to obtain the fusion result.

For example, the node 15 of the fourth level can average or calculate the weighted average of the two first training results received from the node 13 and the node 14 and the second training result obtained by itself to obtain the fusion result.

It is understandable that this is equivalent to a summary calculation of the training results of all nodes.

If the current layer node is not in the last layer, the first training result is used as the second type result of the first training result sent to the corresponding lower layer node, and the second training result is used as the first training result sent to the corresponding lower layer node. The first type of training result.

For example, the node 9 of the second level can use the two first training results received from node 1 and node 2 as two new second-type results, in addition, it can also use the second training results obtained by itself. The result is the new first type result. After that, the two new results of the second type and the new results of the first type are used as two parts of the first training result sent to the node 13.

Step 310: Broadcast the fusion result on the blockchain, so that multiple nodes update the target model in the multiple models maintained on the chain based on the fusion result.

It is understandable that the fusion result here may refer to the total model gradient or the total model parameters of the target model.

Similarly, after performing step 310, only one round of iterative training of the target model is completed. If after one round of iteration, the loss function corresponding to the target model reaches the predetermined convergence condition, the joint model training ends; otherwise, the next round of iteration is entered until the loss function corresponding to the target model reaches the predetermined convergence condition. It is understandable that the next iteration of the process is executed based on the updated target model.

It should also be noted that steps 302 to 310 are operations performed by nodes other than the first level in the model joint training process. It is understandable that, for each node of the first level, when it receives the model training task issued by the task processor on the blockchain, it can maintain it on the chain based on the private data maintained under the chain. The target model among multiple models is trained, and the second training result is obtained. After that, the second training result can be used as the first training result sent to the corresponding lower-level node (that is, the second-level node). Therefore, the corresponding lower-level node can execute the above-mentioned steps 302-310 until the node of the last level is reached.

The block chain-based model joint training method provided in the embodiment of this specification can realize the joint training of the model based on the block chain, thereby ensuring the safety and reliability of the model joint training process.

Corresponding to the aforementioned blockchain-based model joint training method, an embodiment of this specification also provides a blockchain-based model joint training device. The blockchain includes multiple nodes with hierarchical relationships and adjacent There is a correspondence between the nodes of the two levels. Each of the multiple nodes maintains private data off-chain, and there are multiple models for on-chain maintenance. The device is set at the current layer node of the blockchain. As shown in Figure 4, the device may include the following units.

The receiving unit 402 is configured to receive the first training result sent by the corresponding upper node. The first training result is determined based on at least the first type of result, and the first type of result is obtained by the upper node after training the target model among the multiple models maintained on the chain based on the private data maintained off-chain by the upper node.

If the upper node still has a corresponding upper node, the first training result is determined based on the first type result and the second type result, and the second type result is determined by the upper node based on the result received from its corresponding upper node.

The training unit 404 is used to train the target model among the multiple models maintained on the chain based on the private data maintained off-chain to obtain the second training result.

The fusion unit 406 is configured to fuse the first training result received by the receiving unit 402 and the second training result obtained by the training unit 404 to obtain a fusion result.

The fusion unit 406 is specifically configured to: use the first training result as the new second-type result, and use the second training result as the new first-type result to average or calculate a weighted average to obtain the fusion result.

The fusion result here may include the total model gradient and/or the total model parameters of the target model.

The judging unit 408 is used to judge whether the current layer node is in the last layer.

The broadcasting unit 410 is configured to, if the judging unit 408 judges that the current layer node is at the last level, broadcast the fusion result on the blockchain, so that multiple nodes can respond to the targets in the multiple models maintained on the chain based on the fusion result The model is updated.

Optionally, the device may further include: a sending unit (not shown in the figure), configured to use the fusion result as the first training result sent to the corresponding lower-layer node if the current layer node is not in the last layer.

Optionally, the device may further include: a calling unit (not shown in the figure) for calling a smart contract when the loss function corresponding to the target model meets a predetermined convergence condition, and the smart contract is used to calculate multiple nodes Respectively corresponding rewards, and issue respective rewards to multiple nodes, so as to increase the virtual resources corresponding to multiple nodes.

The function of each functional module of the device in the above-mentioned embodiment of this specification can be realized by the steps of the above-mentioned method embodiment. Therefore, the specific working process of the device provided in an embodiment of this specification will not be repeated here.

The block chain-based model joint training device provided in an embodiment of this specification can ensure the safety and reliability of the model joint training process.

Corresponding to the aforementioned blockchain-based model joint training method, an embodiment of this specification also provides a blockchain-based model joint training device. The blockchain includes multiple nodes with hierarchical relationships and adjacent There is a correspondence between the nodes of the two levels. Each of the multiple nodes maintains private data off-chain, and there are multiple models for on-chain maintenance. The device is set at the current layer node of the blockchain. As shown in Figure 5, the device may include the following units.

The receiving unit 502 is configured to receive the first training result sent by the corresponding upper node. The first training result includes at least a first type result, and the first type result is obtained by the upper node after training the target model among the multiple models maintained on the chain based on the private data maintained off-chain by the upper node.

If the upper node still has a corresponding upper node, the first training result also includes a second type result, and the second type result is determined by the upper node based on the result received from its corresponding upper node.

The training unit 504 is used to train the target model of the multiple models maintained on the chain based on the private data maintained off-chain to obtain the second training result.

The judging unit 506 is used to judge whether the current layer node is in the last layer.

The fusion unit 508 is used for fusing the first training result and the second training result to obtain the fusion result if the judging unit 506 judges that the current layer node is at the last level.

The fusion unit 508 may be specifically configured to: average or obtain a weighted average of the first training result and the second training result to obtain the fusion result.

The broadcasting unit 510 is configured to broadcast the fusion result obtained by the fusion unit 508 on the blockchain, so that multiple nodes update the target model in the multiple models maintained on the chain based on the fusion result.

Optionally, the device may further include: a sending unit (not shown in the figure), configured to, if the judging unit 506 judges that the current layer node is not in the last layer, use the first training result as the first training result sent to the corresponding lower layer node. A result of the second type in the training result, using the second training result as the result of the first type in the first training result sent to the corresponding lower-level node.

On the other hand, the embodiments of this specification provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method shown in FIG. 2 or FIG. 3.

On the other hand, the embodiment of this specification provides a computing device, including a memory and a processor, the memory is stored with executable code, and when the processor executes the executable code, the implementation shown in FIG. Indicates the method.

The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment.

The steps of the method or algorithm described in conjunction with the disclosure of this specification can be implemented in a hardware manner, or can be implemented in a manner in which a processor executes software instructions. Software instructions can be composed of corresponding software modules, which can be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, mobile hard disk, CD-ROM or any other form of storage known in the art Medium. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and the storage medium may be located in the ASIC. In addition, the ASIC may be located in the server. Of course, the processor and the storage medium may also exist as discrete components in the server.

Those skilled in the art should be aware that, in one or more of the above examples, the functions described in the present invention can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on the computer-readable medium. The computer-readable medium includes a computer storage medium and a communication medium, where the communication medium includes any medium that facilitates the transfer of a computer program from one place to another. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than in the embodiments and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The specific implementations described above further describe the purpose, technical solutions and beneficial effects of this specification. It should be understood that the above are only specific implementations of this specification and are not intended to limit the description of this specification. The scope of protection, any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of this specification shall be included in the scope of protection of this specification.

Claims

A joint training method for a model based on a block chain. The block chain includes a plurality of nodes having a hierarchical relationship, and there is a corresponding relationship between the nodes of two adjacent hierarchies; each node of the plurality of nodes Private data is maintained off-chain, and there are multiple models for on-chain maintenance; the method is executed by the current layer nodes of the blockchain, including:

Receive the first training result sent by the corresponding upper-layer node; the first training result is determined based on at least the first-type result; the first-type result is determined by the upper-layer node based on the private data maintained off-chain by the upper-layer node The target model in the multiple models maintained on the chain is obtained after training;

Based on the private data maintained off-chain, train the target model among the multiple models maintained on-chain to obtain the second training result;

Fusing the first training result and the second training result to obtain a fusion result;

Judging whether the current layer node is in the last layer;

If the current layer node is at the last level, the fusion result is broadcast on the blockchain, so that the multiple nodes can compare all of the multiple models maintained on the chain based on the fusion result. The target model is updated.
According to the method of claim 1, if the upper node still has a corresponding upper node, the first training result is determined based on the first type result and the second type result; the second type result is determined by The upper node is determined based on the result received from its corresponding upper node.
The method according to claim 2, said fusing the first training result and the second training result to obtain a fusion result, comprising:

The first training result is used as a new second type result, and the second training result is used as a new first type result for averaging or weighted average to obtain a fusion result.
The method according to claim 1, further comprising:

If the current layer node is not in the last layer, the fusion result is used as the first training result sent to the corresponding lower layer node.
The method according to claim 1, wherein the fusion result includes a total model gradient and/or a total model parameter of the target model.
The method according to claim 1, further comprising:

When the loss function corresponding to the target model satisfies a predetermined convergence condition, a smart contract is called, and the smart contract is used to calculate the respective rewards of the multiple nodes, and issue respective rewards to the multiple nodes , So that the virtual resources corresponding to each of the multiple nodes are increased.
A joint training method for a model based on a block chain. The block chain includes a plurality of nodes having a hierarchical relationship, and there is a corresponding relationship between the nodes of two adjacent hierarchies; each node of the plurality of nodes Private data is maintained off-chain, and there are multiple models for on-chain maintenance; the method is executed by the current layer nodes of the blockchain, including:

Receive the first training result sent by the corresponding upper node; the first training result includes at least the first type of result; the first type of result is registered by the upper node on the chain based on the private data maintained by the upper node The target model in the multiple models maintained on the above is obtained after training;

Based on the private data maintained off-chain, train the target model among the multiple models maintained on-chain to obtain the second training result;

Judging whether the current layer node is in the last layer;

If the current layer node is in the last layer, fusing the first training result and the second training result to obtain a fusion result;

Broadcasting the fusion result on the blockchain, so that the multiple nodes update the target model in the multiple models maintained on the chain based on the fusion result.
According to the method of claim 7, if the upper node still has a corresponding upper node, the first training result further includes a second type result, and the second type result is determined by the upper node based on its corresponding The result received by the upper node is determined.
The method according to claim 8, further comprising:

If the judgment unit judges that the current layer node is not in the last layer, the first training result is taken as the second type result in the first training result sent to the corresponding lower layer node, and the second training result is As a result of the first type in the first training result sent to the corresponding lower-level node.
The method according to claim 7, said fusing the first training result and the second training result to obtain a fusion result, comprising:

The first training result and the second training result are averaged or weighted averaged to obtain a fusion result.
A model joint training device based on a block chain, the block chain includes a plurality of nodes having a hierarchical relationship, and there is a corresponding relationship between the nodes of two adjacent hierarchies; each node of the plurality of nodes Private data is maintained under the chain, and there are multiple models for maintenance on the chain; the device is set at the current layer node of the blockchain, including:

The receiving unit is configured to receive the first training result sent by the corresponding upper node; the first training result is determined based on at least the first type of result; the first type of result is determined by the upper node based on the privately maintained off-chain Data, obtained after training the target model in the multiple models maintained on the chain;

The training unit is used to train the target model of the multiple models maintained on the chain based on the private data maintained off-chain to obtain the second training result;

A fusion unit, configured to fuse the first training result received by the receiving unit and the second training result obtained by the training unit to obtain a fusion result;

A judging unit for judging whether the current layer node is in the last layer;

The broadcasting unit is configured to, if the judging unit judges that the current layer node is at the last level, broadcast the fusion result on the blockchain, so that the plurality of nodes are in each position based on the fusion result. The target model among the multiple models maintained on the chain is updated.
The device according to claim 11, if the upper node still has a corresponding upper node, the first training result is determined based on the first type result and the second type result; the second type result is determined by The upper node is determined based on the result received from its corresponding upper node.
The device according to claim 12, wherein the fusion unit is specifically configured to:

The first training result is used as a new second type result, and the second training result is used as a new first type result for averaging or weighted average to obtain a fusion result.
The device according to claim 11, further comprising:

The sending unit is configured to, if the current layer node is not in the last layer, use the fusion result as the first training result sent to the corresponding lower layer node.
The device according to claim 11, wherein the fusion result includes a total model gradient and/or a total model parameter of the target model.
The device according to claim 11, further comprising:

The calling unit is used to call a smart contract when the loss function corresponding to the target model meets a predetermined convergence condition, and the smart contract is used to calculate the rewards corresponding to each of the multiple nodes and send them to the multiple nodes. Respectively corresponding rewards are issued so as to increase the virtual resources corresponding to each of the multiple nodes.
A model joint training device based on a block chain, the block chain includes a plurality of nodes having a hierarchical relationship, and there is a corresponding relationship between the nodes of two adjacent hierarchies; each node of the plurality of nodes Private data is maintained under the chain, and there are multiple models for maintenance on the chain; the device is set at the current layer node of the blockchain, including:

The receiving unit is configured to receive a first training result sent by a corresponding upper-layer node; the first training result includes at least a first-type result; the first-type result is determined by the upper-layer node based on its private data maintained off-chain , Which is obtained after training the target model among the multiple models maintained on the chain;

The training unit is used to train the target model among the multiple models maintained on the chain based on the private data maintained off-chain to obtain the second training result;

A judging unit for judging whether the current layer node is in the last layer;

A fusion unit, configured to merge the first training result and the second training result to obtain a fusion result if the judgment unit judges that the current layer node is at the last level;

The broadcasting unit is configured to broadcast the fusion result obtained by the fusion unit on the blockchain, so that the multiple nodes can, based on the fusion result, report on the target model among the multiple models maintained on the chain. Update.
According to the device of claim 17, if the upper node still has a corresponding upper node, the first training result further includes a second type result, and the second type result is determined by the upper node based on its corresponding The result received by the upper node is determined.
The device according to claim 18, further comprising:

The sending unit is configured to, if the judging unit judges that the current layer node is not in the last layer, use the first training result as the second type result in the first training result sent to the corresponding lower layer node, and then The second training result is taken as the first type result in the first training result sent to the corresponding lower-level node.
The device according to claim 17, wherein the fusion unit is specifically configured to:

The first training result and the second training result are averaged or weighted averaged to obtain a fusion result.
A computer-readable storage medium with a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the method described in any one of claims 1-6 or any one of claims 7-10. The method described in one item.
A computing device, comprising a memory and a processor, the memory is stored with executable code, and when the processor executes the executable code, the method or claim in any one of claims 1 to 6 is implemented The method of any one of 7-10.