Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/23—Updating
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/25—Integrating or interfacing systems involving database management systems
  • G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
  • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
  • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F9/00—Arrangements for program control, e.g. control units
  • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
  • G06F9/46—Multiprogramming arrangements
  • G06F9/48—Program initiating; Program switching, e.g. by interrupt
  • G06F9/4806—Task transfer initiation or dispatching
  • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system

Definitions

• the invention belongs to the field of data collaborative processing, and in particular relates to a multi-center biomedical data collaborative processing system and method without patient data sharing.
• the asynchronous collaborative processing mode needs to wait for the calculation progress of each medical center to update the cloud server model, which causes a waste of collaboration time.
• the parameters of the multi-center collaborative model need to be frequently accessed by all nodes. When the machine performance varies greatly, there will be different waiting times. At the same time, the fault tolerance in the cloud environment also needs to be considered, which will improve the efficiency of medical data collaborative analysis. There is a certain impact.
• the purpose of the present invention is to solve the problem of user data privacy leakage and too long waiting time for synchronization update model in the existing multi-center medical data collaboration process, and propose a new multi-center medical data collaborative processing system and method without patient data sharing.
• the present invention uses a distributed collaborative processing method to solve the problem of medical big data analysis under the premise of fully avoiding the risk of external circulation and leakage of all patient data, and completes the collaborative optimization and optimization of the model through the method of asynchronous sharing of the model. Increased computing efficiency.
• a multi-center biomedical data collaborative processing system without patient data sharing the system includes a cloud server used for coordinating model parameters and asynchronous calculations of various medical centers and used for data development Local high-performance computing medical center client;
• the medical center client has two roles: initiator and participant; as the initiator, it sends a request for collaborative processing of medical data to the cloud server, and at the same time transmits the user's predefined analysis model and model parameters, and the medical center that intends to invite collaborative processing List, waiting for the initial collaborative analysis model and model parameters sent back by the task scheduler; as a participant, after receiving the collaborative processing content and invitation issued by the task scheduler, determine whether to participate in the collaborative processing, and if it is determined to participate in the collaboration, the task will be scheduled
• the device sends a confirmation participation coordination instruction, and sends the initial local analysis model and model parameters of the participant to the parameter manager, and waits for the initial collaborative analysis model and model parameters returned by the task scheduler, otherwise the collaborative processing flow of the participant is ended ;
• the medical center client After the medical center client receives the initial collaborative analysis model and model parameters, it prepares local medical data, stores the local medical data and the initial collaborative analysis model and model parameters together in the cache, and performs model iteration calculation locally.
• the calculated model parameters and calculation progress are uploaded to the task scheduler; the task scheduler is sent back
• the updated collaborative analysis model and model parameters are released, the local cache is released, and the updated collaborative analysis model and model parameters are stored; iterate continuously until the calculation accuracy meets the set threshold, and the calculation is terminated;
• the cloud server includes a parameter manager and a task scheduler; after the task scheduler receives a medical data collaborative processing request initiated by a medical center client, it sends to all medical center clients in the list of medical centers parsed from the request Collaborative processing content and invitations; the task scheduler judges the invitation response status of all medical center clients in the list of medical centers, and the response status includes receipt of a confirmation to participate in collaboration instruction, receipt of a refusal to participate in collaboration instruction, and overtime unprocessed judgment; After the end, open the parameter manager. According to the pre-defined analysis model and model parameters of the initiator, as well as the initial local analysis model and model parameters of the participants, the parameter manager constructs the initial collaborative analysis model, and confirms the participants and The initiator sends the initial collaborative analysis model and model parameters;
• the task scheduler After the task scheduler receives the model parameters and calculation progress uploaded by the medical center client, it divides the resources of the model, and at the same time stores the model parameters in the parameter manager, and the calculation status in the task scheduler; task scheduler When the calculation status uploaded by the corresponding medical center client is obtained, the current model parameters are retrieved from the parameter manager and the gradient is calculated, the model and model parameters are updated, and the updated model and model parameters are overwritten by the previous medical center customer The terminal stores the model and model parameters in the parameter manager, and transmits the updated model and model parameters back to the medical center client.
• the medical center client and the cloud server both have their own network interface layer.
• the network interface layer is developed based on the Flask lightweight web application framework under the python language, including but not limited to the use of web application frameworks such as Spring Boot. To realize the communication of medical data collaborative processing request and model parameters.
• the medical center client stores the collaborative processing request, model, and data in a cache.
• the cache uses the non-relational memory database Redis in the Nosql database to relieve the client processing pressure caused by too frequent collaborative requests.
• the medical center client uses GPU to accelerate data calculation and processing.
• the medical center client analysis algorithm is based on the Python language, including but not limited to Scala, C++, R, Julia, GO, etc.
• the scientific computing libraries used include but are not limited to Numpy, Pandas, Scipy, Breeze, Blitz++, POOMA, BLAS Wait.
• the task scheduler assigns a service node and a working node to each medical center client; the service node is only responsible for maintaining the management and update of the model parameters and calculation progress of the medical center client to which it is assigned;
• the new service node uses the distributed hash table in the system to dynamically insert it into the service group at any time; the worker node is only responsible for the processing tasks of the medical center client assigned to it.
• the parameters are expressed as a collection of (key, value), and the synchronization and update of the gradient between each medical center and cloud server are realized through push and pull operations; the task scheduler is responsible for maintaining the consistency of metadata, such as the consistency of each node Status, allocation of parameters, etc.
• the parameter manager updates the collaborative model parameters of the client of a single medical center on the cloud server through the following rules: suppress the small gradient value of the client of a single medical center, and wait for each medical center When the client gradient value accumulates to a custom threshold, the medical center client gradient is updated to balance the algorithm convergence speed and system performance.
• a multi-center biomedical data collaborative processing method without patient data sharing includes the following steps:
• the medical center client initiator sends a medical data collaborative processing request to the cloud server, and transmits the user's predefined analysis model and model parameters, as well as the list of medical centers to be invited for collaborative processing, and waits for the initial collaborative analysis returned by the task scheduler Model and model parameters;
• the cloud server task scheduler After receiving the medical data collaborative processing request initiated by the medical center client, the cloud server task scheduler sends the collaborative processing content and invitation to all medical center clients in the medical center list parsed from the request;
• the medical center client participant After the medical center client participant receives the collaborative processing content and invitation issued by the task scheduler, it determines whether to participate in the collaborative processing. Initialize the local analysis model and model parameters and send them to the parameter manager, wait for the initial collaborative analysis model and model parameters returned by the task scheduler, otherwise terminate the collaborative processing process of the participant;
• the cloud server task scheduler judges the invitation response status of all medical center clients in the list of medical centers.
• the response status includes the receipt of a confirmation to participate in the coordination instruction, the receipt of a refusal to participate in the coordination instruction, and the timeout not processed;
• the parameter manager After the judgment of the cloud server task scheduler is over, the parameter manager is opened.
• the parameter manager constructs an initial collaborative analysis model based on the initiator’s predefined analysis model and model parameters, as well as the participant’s initial local analysis model and model parameters. And send the initial collaborative analysis model and model parameters to the participants and initiators who are confirmed to participate in the collaboration;
• the medical center client After receiving the initial collaborative analysis model and model parameters, the medical center client prepares local medical data, stores the local medical data and the initial collaborative analysis model and model parameters together in the cache, and performs model iteration calculations locally. When the predefined number of iterations is reached or when the updated collaborative analysis model and model parameters are received from the task scheduler, upload the calculated model parameters and calculation progress to the task scheduler;
• the cloud server task scheduler After the cloud server task scheduler receives the model parameters and calculation progress uploaded by the medical center client, it divides the resources of the model, saves the model parameters in the parameter manager, and saves the calculation status in the task scheduler; When the task scheduler gets the calculation status uploaded by the corresponding medical center client, it retrieves the current model parameters from the parameter manager and calculates the gradient, updates the model and model parameters, and overwrites the updated model and model parameters. The medical center client stores the model and model parameters in the parameter manager, and transmits the updated model and model parameters back to the medical center client;
• the medical center client When the medical center client receives the updated collaborative analysis model and model parameters returned by the task scheduler, it releases the local cache and stores the updated collaborative analysis model and model parameters; iterates continuously until the calculation accuracy meets the requirements. Set the threshold and terminate the calculation.
• the present invention connects and manages the parameter sharing mechanism and asynchronous communication mechanism through the parameter manager and task scheduler of the cloud server, and jointly applies to the multi-center medical data collaborative computing; it can meet the requirements of the medical center data and the cloud server.
• Safe isolation fully protect the privacy of patient data in the medical center, and ensure the high privacy of data, while realizing the calculation of medical problems in the collaborative processing of multi-center medical data; at the same time, compared to directly synchronizing and exchanging data calculation results, it effectively reduces the calculation waiting time and greatly improves Analysis efficiency and data processing capabilities of multi-center collaborative processing.
• Figure 1 is a flowchart of the implementation of a multi-center biomedical data collaborative processing system without patient data sharing according to the present invention
• Figure 2 is a working principle diagram of the task scheduler
• FIG. 1 Schematic diagram of parameter manager gradient update
• the present invention provides a multi-center biomedical data collaborative processing system without patient data sharing.
• the system includes a cloud server for coordinating model parameters and asynchronous calculations of various medical centers and a cloud server for performing local data processing.
• Medical center client for high-performance computing.
• the network interface layer is developed based on the Flask lightweight web application framework under the python language, including but not limited to the use of web application frameworks such as Spring Boot to achieve medical data collaboration Handle the communication of requests and model parameters.
• the medical center client has two roles: initiator and participant; as the initiator, it sends a request for collaborative processing of medical data to the cloud server, and transmits the user's predefined analysis model and model parameters, as well as the list of medical centers to be invited for collaborative processing. Wait for the initial collaborative analysis model and model parameters returned by the task scheduler; as a participant, after receiving the collaborative processing content and invitation issued by the task scheduler, determine whether to participate in the collaborative processing, and if it is determined to participate in the collaboration, send it to the task scheduler Confirm the participation coordination instruction, and send the initial local analysis model and model parameters of the participant to the parameter manager, and wait for the initial coordination analysis model and model parameters returned by the task scheduler, otherwise the cooperative processing flow of the participant is ended.
• the medical center client After receiving the initial collaborative analysis model and model parameters, the medical center client prepares local medical data, stores the local medical data and the initial collaborative analysis model and model parameters together in the cache, and performs the iterative calculation of the model locally.
• the calculated model parameters and calculation progress are uploaded to the task scheduler; the update returned by the task scheduler is received
• the subsequent collaborative analysis model and model parameters are released, the local cache is released, and the updated collaborative analysis model and model parameters are stored; iterate continuously until the calculation accuracy meets the set threshold, and the calculation is terminated.
• the medical center client stores collaborative processing requests, models, and data in the cache.
• the cache can use the non-relational memory database Redis in the Nosql database to relieve the client processing pressure caused by too frequent collaborative requests.
• the medical center client uses GPU (Graphic Processing Unit) to accelerate data calculation and processing.
• the medical center client analysis algorithm is based on the Python language, including but not limited to Scala, C++, R, Julia, GO, etc.
• the scientific computing libraries used include but are not limited to Numpy, Pandas, Scipy, Breeze, Blitz++, POOMA, BLAS, etc.
• the cloud server includes a parameter manager and a task scheduler; after the task scheduler receives the medical data collaborative processing request initiated by the medical center client, it sends the collaborative processing to all medical center clients in the medical center list parsed from the request Content and invitation; the task scheduler judges the invitation response status of all medical center clients in the list of medical centers, and the response status includes receipt of a confirmation to participate in collaboration instruction, receipt of a refusal to participate in collaboration instruction, and overtime unprocessed; after the judgment is over Open the parameter manager, in the parameter manager, according to the initiator’s predefined analysis model and model parameters, as well as the participant’s initial local analysis model and model parameters, construct the initial collaborative analysis model, and confirm the participants and initiators participating in the collaboration Send the initial collaborative analysis model and model parameters.
• the task scheduler After the task scheduler receives the model parameters and calculation progress uploaded by the medical center client, it divides the resources of the model, saves the model parameters in the parameter manager, and stores the calculation status in the task scheduler; the task scheduler is getting When the calculation status uploaded by the corresponding medical center client is reached, the current model parameters are retrieved from the parameter manager and the gradient is calculated, the model and model parameters are updated, and the updated model and model parameters are overwritten by the medical center client before The model and model parameters stored in the parameter manager, and the updated model and model parameters are transmitted back to the medical center client.
• the main working principle of the task scheduler is shown in Figure 2.
• the task scheduler assigns a service node and a working node to each medical center client; the service node is only responsible for maintaining the model parameters and calculation progress of the medical center client assigned to it Management and update; when a new medical center client joins the collaborative processing, the new service node uses the distributed hash table in the system to dynamically insert it into the service group at any time; the worker node is only responsible for the medical care assigned to it
• the processing task of the central client in which the parameters are expressed as a collection of (key, value), the synchronization and update of the gradient between each medical center and the cloud server are realized through push and pull operations; the task scheduler is responsible for maintaining metadata Consistency, such as the status of each node, the allocation of parameters, etc.
• the specific steps of collaborative processing include:
• the medical center client initiator sends a medical data collaborative processing request to the cloud server, and transmits the user's predefined analysis model and model parameters, as well as the list of medical centers to be invited for collaborative processing, and waits for the initial collaborative analysis returned by the task scheduler Model and model parameters;
• the cloud server task scheduler After receiving the medical data collaborative processing request initiated by the medical center client, the cloud server task scheduler sends the collaborative processing content and invitation to all medical center clients in the medical center list parsed from the request;
• the medical center client participant After the medical center client participant receives the collaborative processing content and invitation issued by the task scheduler, it determines whether to participate in the collaborative processing. Initialize the local analysis model and model parameters and send them to the parameter manager, wait for the initial collaborative analysis model and model parameters returned by the task scheduler, otherwise terminate the collaborative processing process of the participant;
• the cloud server task scheduler judges the invitation response status of all medical center clients in the list of medical centers.
• the response status includes the receipt of a confirmation to participate in the coordination instruction, the receipt of a refusal to participate in the coordination instruction, and the timeout not processed;
• the parameter manager After the judgment of the cloud server task scheduler is over, the parameter manager is opened.
• the parameter manager constructs an initial collaborative analysis model based on the initiator’s predefined analysis model and model parameters, as well as the participant’s initial local analysis model and model parameters. And send the initial collaborative analysis model and model parameters to the participants and initiators who are confirmed to participate in the collaboration;
• the medical center client After receiving the initial collaborative analysis model and model parameters, the medical center client prepares local medical data, stores the local medical data and the initial collaborative analysis model and model parameters together in the cache, and performs model iteration calculations locally. When the predefined number of iterations is reached or when the updated collaborative analysis model and model parameters are received from the task scheduler, upload the calculated model parameters and calculation progress to the task scheduler;
• the cloud server task scheduler After the cloud server task scheduler receives the model parameters and calculation progress uploaded by the medical center client, it divides the resources of the model, saves the model parameters in the parameter manager, and saves the calculation status in the task scheduler; When the task scheduler gets the calculation status uploaded by the corresponding medical center client, it retrieves the current model parameters from the parameter manager and calculates the gradient, updates the model and model parameters, and overwrites the updated model and model parameters. The medical center client stores the model and model parameters in the parameter manager, and transmits the updated model and model parameters back to the medical center client;
• the medical center client When the medical center client receives the updated collaborative analysis model and model parameters returned by the task scheduler, it releases the local cache and stores the updated collaborative analysis model and model parameters; iterates continuously until the calculation accuracy meets the requirements. Set the threshold and terminate the calculation.
• users use a visual interface to publish collaboration requirements based on the problems to be studied, listing their own problems to be studied, data requirements, results attribution, responsibility rules and other information in detail, and wait for interested collaboration centers to join the research together . If a similar queue already exists, the user only needs to select the corresponding queue in the queue pool. If a new queue is needed, the user should share the entry conditions with all collaborative centers participating in the research, and each center creates a new queue and filters out the queue Data, complete the queue data selection.
• the user can choose one or more analysis methods, customize the parameters of the adaptation and the required calculation results.
• the page will synchronously display the result generation progress and the current result reference value. If you are unsatisfied with the result or find a wrong choice of variables, etc., you can selectively terminate the processing. If you do not choose to terminate, it will continue to wait until the end of the collaborative processing, and the final result will be generated on the page. The user can save the summary report of the result or use it for reference only.
• the present invention performs collaborative processing on the biomedical data of multiple medical centers under the condition of no sharing of multi-center patient data, and analyzes the corresponding results for the established collaborative medical problem.
• the data is stored in the medical center client, and the medical center client submits the parameters and calculation processes generated in the local model calculation process to the cloud server.
• the cloud server uses the parameter manager to uniformly manage the computing process and update the model.
• the updated model and model parameters are sent to the medical center client to achieve multi-center biomedical data collaboration without patient data sharing deal with.
• the invention can effectively protect the privacy of patient data in each medical center and simultaneously utilize the collaborative processing model to perform big data analysis.
• patient data is not circulated in various medical centers, and there is no risk of patient data privacy leakage, and the asynchronous parameter sharing mechanism can be used to effectively reduce the calculation waiting time, improve the efficiency of collaborative processing, and ensure the privacy of medical data Get the most effective use.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Databases & Information Systems (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Data Mining & Analysis (AREA)
• Software Systems (AREA)
• Computing Systems (AREA)
• Medical Treatment And Welfare Office Work (AREA)
• Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Priority Applications (1)

Applications Claiming Priority (2)

Publications (1)

Family

ID=68218957

Family Applications (1)

Country Status (3)

Cited By (3)

Families Citing this family (3)

Citations (5)

Family Cites Families (6)

• 2019
  • 2019-07-12 CN CN201910629257.6A patent/CN110362586B/zh active Active
• 2020
  • 2020-04-07 JP JP2021533305A patent/JP7074306B2/ja active Active
  • 2020-04-07 WO PCT/CN2020/083587 patent/WO2020233257A1/zh active Application Filing

Patent Citations (5)

Also Published As

Similar Documents

Legal Events