WO2022077720A1 - Method and apparatus for sharing medical data - Google Patents

Abstract

The present invention relates to the technical field of data processing. Disclosed are a method and apparatus for sharing medical data. The method comprises: using a consortium chain to connect a plurality of data islands; publishing a federated machine learning task in the form of a smart contract on the consortium chain; adding an image-finding assistant to federated machine learning by means of a smart contract API interface; and summarizing and updating medical data of each node in the image-finding assistant. Decentralization is achieved by means of blockchain technology and federated machine learning training, which ameliorates the shortcoming of traditional federated learning relying on a single server.

Description

Method and device for sharing medical data technical field

The present invention relates to the technical field of data processing, and in particular, to a medical data sharing method and device.

Background technique

Federated machine learning (Federated machine learning/Federated Learning): also known as federated learning, federated learning, and federated learning. Federated Machine Learning is a machine learning framework that can effectively help multiple agencies conduct data usage and machine learning modeling while meeting user privacy protection, data security, and government regulations. As a distributed machine learning paradigm, federated learning can effectively solve the problem of data silos, allowing participants to jointly model without sharing data, which can technically break the data silos and realize AI collaboration.

Federated learning defines a machine learning framework under which the problem of different data owners collaborating without exchanging data is solved by designing virtual models. The virtual model is the optimal model for all parties to aggregate data together, and each region serves local goals according to the model. Federated learning requires that this modeling result should be infinitely close to the traditional model, that is, the result of aggregating the data of multiple data owners into one place for modeling. Under the federal mechanism, the identity and status of each participant are the same, and a shared data policy can be established. Since the data is not transferred, it does not reveal user privacy or affect data compliance. To protect data privacy and meet legal compliance requirements.

Although traditional federated learning can solve the problem of data silos well, the problem that cannot be solved is the problem of centralization. In traditional federated learning, the final summary update of parameters needs to be performed on a centralized server, which consumes computing power and bandwidth.

technical problem

The main purpose of the present invention is to propose a medical data sharing method and device, which realizes decentralization through blockchain technology and federated machine learning training, and solves the shortcomings of traditional federated learning relying on a single server.

technical solutions

The main purpose of the present invention is to propose a medical data sharing method and device, which realizes decentralization through blockchain technology and federated machine learning training, and solves the shortcomings of traditional federated learning relying on a single server.

To achieve the above purpose, a medical data sharing method provided by the present invention includes:

Use consortium chains to connect multiple data silos;

Publish federated machine learning tasks in the form of smart contracts on the consortium chain;

Add the shadow-seeking assistant to federated machine learning through the smart contract API interface;

Summarize and update the medical data of each node in the shadow-seeking assistant.

Optionally, the use of consortium chains to connect multiple data islands includes:

setting a one-to-one corresponding multiple distributed asynchronous discriminators in the multiple data islands;

forming an adversarial network of the plurality of distributed asynchronous discriminators and a temporary center generator;

Synthetic training is performed by the temporary center generator to generate synthetic data samples that are similar to the data in the plurality of data islands.

Optionally, the summary and update of the medical data of each node in the shadow-seeking assistant is specifically:

Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.

Optionally, an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has OCT image quality control, AI lesion screening, AI lesion automatic detection, OCT image quality enhancement, AI-assisted labeling and multi-node. The function of collaborative training.

Optionally, the smart ophthalmology OCT module is mounted on a Zhilian platform.

As another aspect of the present invention, a medical data sharing device is provided, comprising:

The connection module is used to connect multiple data islands using the alliance chain;

The publishing module is used to publish federated machine learning tasks in the form of smart contracts on the alliance chain;

Add a module for adding the shadow-seeking assistant to federated machine learning through the smart contract API interface;

The update module is used to summarize and update the medical data of each node in the shadow-seeking assistant.

Optionally, the connection module includes:

a setting unit for setting a one-to-one corresponding multiple distributed asynchronous discriminators in the multiple data islands;

an adversarial unit, configured to form an adversarial network of the plurality of distributed asynchronous discriminators and a temporary center generator;

A training unit, configured to perform synthetic training through the temporary center generator to generate synthetic data samples that are similar to the data in the plurality of data islands.

Optionally, the summary and update of the medical data of each node in the shadow-seeking assistant is specifically:

Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.

Optionally, an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has OCT image quality control, AI lesion screening, AI lesion automatic detection, OCT image quality enhancement, AI-assisted labeling and multi-node. The function of collaborative training.

Optionally, the smart ophthalmology OCT module is mounted on a Zhilian platform.

beneficial effect

A medical data sharing method and device provided by the present invention include: using a consortium chain to connect multiple data islands; publishing a federated machine learning task in the form of a smart contract on the consortium chain; The shadow assistant joins federated machine learning; it aggregates and updates the medical data of each node in the shadow assistant; through blockchain technology and federated machine learning training, it achieves decentralization and solves the shortcomings of traditional federated learning relying on a single server.

Description of drawings

FIG. 1 is a flowchart of a medical data sharing method according to Embodiment 1 of the present invention;

Fig. 2 is the method flow chart of step S10 in Fig. 1;

3 is an exemplary structural block diagram of a medical data sharing apparatus according to Embodiment 2 of the present invention;

FIG. 4 is an exemplary structural block diagram of a connection module according to Embodiment 2 of the present invention.

Embodiments of the present invention

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Example 1

As shown in FIG. 1, in this embodiment, a medical data sharing method includes:

S10. Use the alliance chain to connect multiple data islands;

S20. Publish federated machine learning tasks in the form of smart contracts on the alliance chain;

S30. Add the shadow-seeking assistant to the federated machine learning through the smart contract API interface;

S40, summarize and update the medical data of each node in the shadow-seeking assistant.

In this embodiment, the decentralization of medical data is realized through blockchain technology and federated machine learning training, which solves the disadvantage that traditional federated learning relies on a single server.

In this embodiment, the smart ophthalmology OCT module in the shadow-seeking assistant is configured with a smart contract API interface, and nodes on the chain can join federated machine learning training through this interface, and the blockchain can continuously store each node in such an iterative process. The update record of , achieves decentralization, and solves the shortcomings of traditional federated learning relying on a single server. The system selects temporary nodes at regular intervals to summarize and update the parameters, and feed the results back to the nodes and auxiliary systems.

As shown in FIG. 2, in this embodiment, the step S10 includes:

S11, setting a one-to-one corresponding multiple distributed asynchronous discriminators in the multiple data islands;

S12, forming an adversarial network with the multiple distributed asynchronous discriminators and a temporary center generator;

S13. Perform synthetic training through the temporary center generator to generate synthetic data samples that are similar to the data in the multiple data islands.

In this embodiment, in the mode of machine learning, the Zhilian platform introduces federated machine learning to realize multi-party collaborative training, so that enterprises and institutions can collaborate on the shared model without sharing patient data to achieve decentralized Neural network training.

After acquiring the training task, the Zhilian platform forms an adversarial network by combining distributed asynchronous discriminators located in multiple data islands and a temporary central generator, so that the temporary central generator can also conduct training without touching the original private data. Synthetic training, so that synthetic data samples that are similar to the original data in each data island can be generated for downstream tasks. On this basis, two loss functions are also used, so that the temporary center generator has a certain lifelong learning ability, and can continue to learn in the dynamically changing data island environment. There are cases where institutions drop out and learn incrementally from different data silos to approximate the distribution of homogeneous data or even different data. By avoiding direct access to the original data, the Zhilian platform adheres to the core advantages of federated learning and solves the problem of privacy protection of medical data well. Not only that, compared with the traditional federated learning, the Zhilian platform also effectively reduces the amount of communication data between the temporary center generator and the data island due to the "different approach" in the implementation method, only the transmission of synthetic image data and feedback errors are not required. Not all parameter data of the entire model, and there is no need to exchange any data or parameters between data islands, which can significantly reduce the cost of research through federated learning between medical institutions, and speed up research efficiency and AI model production.

In this embodiment, the summary and update of the medical data of each node in the shadow-seeking assistant is specifically:

Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.

In this embodiment, each node adopts an algorithm suitable for nodes to update model parameters in a distributed manner, and completes the distributed update and convergence of machine learning model parameters with the support of a decentralized parameter server. At the beginning of each training round, a node will be randomly selected by the smart contract as a temporary server, which is responsible for processing the gradient information of all nodes. The smart contract will send the public key of the ephemeral server to other nodes participating in the training. Each node will train the model locally, obtain gradient information, store the gradient information locally, and then encrypt the storage path with the public key, and write the encrypted storage path into the smart contract. Subsequently, the ephemeral server reads all the collected encrypted storage paths from the smart contract. After the temporary server reads the encrypted paths, it decrypts them, and then downloads the gradient information of this round from the corresponding nodes. After the temporary server obtains all the gradient information, it will integrate the gradient information. The simplest way to integrate is to average all the gradients. After integration, the temporary server stores the integrated parameters locally, encrypts the storage paths with the public keys of all nodes, and writes them into the smart contract. Each node obtains the parameter path encrypted by its own public key by reading the smart contract. After decrypting the parameter path, connect to the temporary server to download the parameter information updated in this round. Because the temporary server for each round of training is randomly selected by the smart contract, the temporary server for each round is almost different, so no node can obtain all the intermediate gradient information. However, each node only grasps the intermediate gradients of very few rounds, thus eliminating the risk of privacy leakage caused by intermediate gradients.

In this embodiment, an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has the functions of OCT image quality control, AI lesion screening, AI lesion automatic detection, OCT image quality enhancement, AI auxiliary annotation and The function of multi-node collaborative training.

In this embodiment, using the system can help the ophthalmologist to complete the preliminary fundus disease screening, and greatly save the patient's examination time. Intelligent ophthalmology OCT combines OCT fundus examination and AI lesion screening. The AI-assisted labeling system will also continue to learn during this process, and perform image processing and labeling, so that each new labelled image can become training data. Further Improve the accuracy of the model.

In this embodiment, the smart ophthalmic OCT module is mounted on the Zhilian platform and Clara. The Clara platform is a software toolkit launched by Nvidia that provides medical application developers with a series of libraries for GPUs, such as computing, advanced visualization, and AI technology-related libraries. The smart ophthalmology oct system is also equipped with a smart contract API interface. Nodes on the chain can join federated machine learning training through this interface. The blockchain can continuously store the update records of each node in such an iterative process, realizing decentralization. It solves the shortcomings of traditional federated learning relying on a single server. The system selects temporary nodes at regular intervals to summarize and update the parameters, and feed the results back to the nodes and auxiliary systems.

Embodiment 2

As shown in Figure 3, a medical data sharing device includes:

The connection module 10 is used to connect multiple data islands by using the alliance chain;

The publishing module 20 is used to publish federated machine learning tasks in the form of smart contracts on the alliance chain;

The joining module 30 is used to add the shadow-seeking assistant to the federated machine learning through the smart contract API interface;

The updating module 40 is used for summarizing and updating the medical data of each node in the shadow-seeking assistant.

In this embodiment, the decentralization of medical data is realized through blockchain technology and federated machine learning training, which solves the disadvantage that traditional federated learning relies on a single server.

In this embodiment, the smart ophthalmology OCT module in the shadow-seeking assistant is configured with a smart contract API interface, and nodes on the chain can join federated machine learning training through this interface, and the blockchain can continuously store each node in such an iterative process. The update record of , achieves decentralization, and solves the shortcomings of traditional federated learning relying on a single server. The system selects temporary nodes at regular intervals to summarize and update the parameters, and feed the results back to the nodes and auxiliary systems.

As shown in Figure 4, in this embodiment, the connection module includes:

A setting unit 11 is used to set a plurality of distributed asynchronous discriminators in one-to-one correspondence in the plurality of data islands;

an adversarial unit 12, configured to form an adversarial network with the multiple distributed asynchronous discriminators and a temporary center generator;

The training unit 13 is configured to perform synthetic training through the temporary center generator to generate synthetic data samples that are similar to the data in the multiple data islands.

In this embodiment, in the mode of machine learning, the Zhilian platform introduces federated machine learning to realize multi-party collaborative training, so that enterprises and institutions can collaborate on the shared model without sharing patient data to achieve decentralized Neural network training.

After acquiring the training task, the Zhilian platform forms an adversarial network by combining distributed asynchronous discriminators located in multiple data islands and a temporary central generator, so that the temporary central generator can also conduct training without touching the original private data. Synthetic training, so that synthetic data samples that are similar to the original data in each data island can be generated for downstream tasks. On this basis, two loss functions are also used, so that the temporary center generator has a certain lifelong learning ability, and can continue to learn in the dynamically changing data island environment. There are cases where institutions drop out and learn incrementally from different data silos to approximate the distribution of homogeneous data or even different data. By avoiding direct access to the original data, the Zhilian platform adheres to the core advantages of federated learning and solves the problem of privacy protection of medical data well. Not only that, compared with the traditional federated learning, the Zhilian platform also effectively reduces the amount of communication data between the temporary center generator and the data island due to the "different approach" in the implementation method, only the transmission of synthetic image data and feedback errors are not required. Not all parameter data of the entire model, and there is no need to exchange any data or parameters between data islands, which can significantly reduce the cost of research through federated learning between medical institutions, and speed up research efficiency and AI model production.

In this embodiment, the summary and update of the medical data of each node in the shadow-seeking assistant is specifically:

Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.

In this embodiment, an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has the functions of OCT image quality control, AI lesion screening, AI lesion automatic detection, OCT image quality enhancement, AI auxiliary annotation and The function of multi-node collaborative training.

In this embodiment, each node adopts an algorithm suitable for nodes to update model parameters in a distributed manner, and completes the distributed update and convergence of machine learning model parameters with the support of a decentralized parameter server. At the beginning of each training round, a node will be randomly selected by the smart contract as a temporary server, which is responsible for processing the gradient information of all nodes. The smart contract will send the public key of the ephemeral server to other nodes participating in the training. Each node will train the model locally, obtain gradient information, store the gradient information locally, and then encrypt the storage path with the public key, and write the encrypted storage path into the smart contract. Subsequently, the ephemeral server reads all the collected encrypted storage paths from the smart contract. After the temporary server reads the encrypted paths, it decrypts them, and then downloads the gradient information of this round from the corresponding nodes. After the temporary server obtains all the gradient information, it will integrate the gradient information. The simplest way to integrate is to average all the gradients. After integration, the temporary server stores the integrated parameters locally, encrypts the storage paths with the public keys of all nodes, and writes them into the smart contract. Each node obtains the parameter path encrypted by its own public key by reading the smart contract. After decrypting the parameter path, connect to the temporary server to download the parameter information updated in this round. Because the temporary server for each round of training is randomly selected by the smart contract, the temporary server for each round is almost different, so no node can obtain all the intermediate gradient information. However, each node only grasps the intermediate gradients of very few rounds, thus eliminating the risk of privacy leakage caused by intermediate gradients.

In this embodiment, an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has the functions of OCT image quality control, AI lesion screening, AI lesion automatic detection, OCT image quality enhancement, AI auxiliary annotation and The function of multi-node collaborative training.

In this embodiment, using the system can help the ophthalmologist to complete the preliminary fundus disease screening, and greatly save the patient's examination time. Intelligent ophthalmology OCT combines OCT fundus examination and AI lesion screening. The AI-assisted labeling system will also continue to learn during this process, and perform image processing and labeling, so that each new labelled image can become training data. Further Improve the accuracy of the model.

In this embodiment, the smart ophthalmic OCT module is mounted on the Zhilian platform and Clara. The Clara platform is a software toolkit launched by Nvidia that provides medical application developers with a series of libraries for GPUs, such as computing, advanced visualization, and AI technology-related libraries. The smart ophthalmology oct system is also equipped with a smart contract API interface. Nodes on the chain can join federated machine learning training through this interface. The blockchain can continuously store the update records of each node in such an iterative process, realizing decentralization. It solves the shortcomings of traditional federated learning relying on a single server. The system selects temporary nodes at regular intervals to summarize and update the parameters, and feed the results back to the nodes and auxiliary systems.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A method for sharing medical data, comprising:

   Use consortium chains to connect multiple data silos;

   Publish federated machine learning tasks in the form of smart contracts on the consortium chain;

   Add the shadow-seeking assistant to federated machine learning through the smart contract API interface;

   Summarize and update the medical data of each node in the shadow-seeking assistant.
2. The method for sharing medical data according to claim 1, wherein the connecting a plurality of data islands by using a consortium chain comprises:

   setting a one-to-one corresponding multiple distributed asynchronous discriminators in the multiple data islands;

   forming an adversarial network of the plurality of distributed asynchronous discriminators and a temporary center generator;

   Synthetic training is performed by the temporary center generator to generate synthetic data samples that are similar to the data in the plurality of data islands.
3. The method for sharing medical data according to claim 1, wherein the summarizing and updating the medical data of each node in the shadow-seeking assistant is specifically:

   Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.
4. The medical data sharing method according to claim 1, wherein an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has the functions of OCT image quality control, AI lesion screening, AI lesion Functions of automatic detection, OCT image quality enhancement, AI-assisted annotation and multi-node collaborative training.
5. The medical data sharing method according to claim 4, wherein the intelligent ophthalmology OCT module is mounted on a Zhilian platform.
6. A medical data sharing device, comprising:

   The connection module is used to connect multiple data islands with the alliance chain;

   The publishing module is used to publish federated machine learning tasks in the form of smart contracts on the alliance chain;

   Add a module for adding the shadow-seeking assistant to federated machine learning through the smart contract API interface;

   The update module is used to summarize and update the medical data of each node in the shadow-seeking assistant.
7. A medical data sharing device according to claim 6, wherein the connection module comprises:

   a setting unit for setting a one-to-one corresponding multiple distributed asynchronous discriminators in the multiple data islands;

   an adversarial unit, configured to form an adversarial network of the plurality of distributed asynchronous discriminators and a temporary center generator;

   A training unit, configured to perform synthetic training through the temporary center generator to generate synthetic data samples that are similar to the data in the plurality of data islands.
8. The medical data sharing device according to claim 6, wherein the summarizing and updating the medical data of each node in the shadow-seeking assistant is specifically:

   Through the distributed update model parameter algorithm, the medical data of each node in the shadow-seeking assistant is summarized and updated.
9. The medical data sharing method according to claim 6, wherein an intelligent ophthalmology OCT module is set in the shadow-seeking assistant, and the intelligent ophthalmology OCT module has the functions of OCT image quality control, AI lesion screening, AI lesion Functions of automatic detection, OCT image quality enhancement, AI-assisted annotation and multi-node collaborative training.
10. The medical data sharing method according to claim 9, wherein the intelligent ophthalmology OCT module is mounted on a Zhilian platform.