CN114417428B - Privacy calculation method and system based on distributed cooperation - Google Patents
Privacy calculation method and system based on distributed cooperation Download PDFInfo
- Publication number
- CN114417428B CN114417428B CN202210321981.4A CN202210321981A CN114417428B CN 114417428 B CN114417428 B CN 114417428B CN 202210321981 A CN202210321981 A CN 202210321981A CN 114417428 B CN114417428 B CN 114417428B
- Authority
- CN
- China
- Prior art keywords
- privacy
- controller
- computing
- sdk
- calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Computer Hardware Design (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Storage Device Security (AREA)
Abstract
The application discloses a privacy calculation method and system based on distributed cooperation, and belongs to the technical field of data processing. The method comprises the following steps: each privacy all-in-one computer registers a data set stored by the privacy all-in-one computer with a controller; the SDK applies for using a data set to the n privacy computing all-in-one machines through the controller; after the n privacy all-in-one computers agree to the SDK to use the data sets, the SDK sends the developed privacy calculation application to the controller, and the privacy calculation application contains references to the data sets in the n privacy all-in-one computers; the controller decomposes the privacy calculation application and sends the obtained calculation tasks to the n privacy calculation all-in-one machines respectively; and each privacy computing all-in-one machine computes the computing task according to the data set stored by the privacy computing all-in-one machine, and sends the obtained computing result to the SDK through the controller. The application development terminal and the application development method can ensure data safety and enable application development on the application development terminal to be lighter.
Description
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a privacy computing method and system based on distributed collaboration.
Background
Data are integrated into the aspects of social life, and the release of intrinsic value of the data continuously pushes the development of society and economy. However, data privacy is not sufficiently protected in the data sharing and application processes, so that data security problems are frequent, and privacy computing technologies are gradually emerging in order to effectively deal with privacy leakage problems.
MPSaaS is an end-to-end automation system oriented to large-scale multi-party full-safety computation, and a platform environment is constructed for multiple parties with cooperation requirements through compatibility of heterogeneous computing equipment (virtual machines, mobile phones, browsers and the like) and automatic deployment of multi-party safety computing protocols.
However, in the platform environment created by the MPSaaS, all private computing is integrated on the public cloud, which causes data security problem, and the application development environment and tool chain (such as multi-party secure computing application and compiler) are tightly coupled, which makes the application development difficult.
Disclosure of Invention
The application provides a privacy computing method and system based on distributed cooperation, which are used for solving the problems that when all privacy computing is integrated on a public cloud, the data security problem is caused, and the application development is difficult. The technical scheme is as follows:
in one aspect, a privacy computing method based on distributed collaboration is provided, and is used in a privacy computing system based on distributed collaboration, where the privacy computing system includes a software development kit SDK, a controller and multiple privacy computing all-in-one machines, the controller is deployed in an edge cloud, and the multiple privacy computing all-in-one machines are deployed in a private cloud, and the method includes:
each privacy all-in-one computer registers a data set stored by the privacy all-in-one computer with the controller;
the SDK applies for using a data set to the n privacy computing all-in-one machines through the controller;
after the n privacy all-in-one computers agree to the SDK to use the data set, the SDK sends the developed privacy computing application to the controller, the privacy computing application contains references to the data set in the n privacy all-in-one computers, and n is larger than or equal to 1;
the controller decomposes the privacy calculation application and sends the obtained calculation tasks to the n privacy calculation all-in-one machines respectively;
and each privacy calculation all-in-one machine calculates the calculation task according to the data set stored by the privacy calculation all-in-one machine, and sends the obtained calculation result to the SDK through the controller.
In one possible implementation, the method further includes: the controller generates a data mart from the data set;
the SDK applies for a usage data set from the n privacy computing all-in-one machines through the controller, and the usage data set comprises: the SDK performs data screening and usage statement in the data mart, generates a data usage application containing screening information, usage statement information and the identification of the SDK, and sends the data usage application to the controller; and the controller forwards the data use application to the n privacy computing all-in-one machines.
In one possible implementation, the method further includes:
after receiving the data use application, each privacy all-in-one machine sends an application result to the controller, wherein the application result is used for indicating whether to approve or reject the SDK use data set;
the controller forwards the application result to the SDK;
and the SDK determines whether the privacy all-in-one computer agrees to use the data set by the SDK according to the application result.
In a possible implementation manner, the forwarding, by the controller, the data usage application to the n privacy-computing all-in-one machines includes:
the controller detects whether the data use application meets a preset forwarding condition;
and if the data use application meets the forwarding condition, the controller forwards the data use application to the n privacy calculation all-in-one machines.
In one possible implementation, the method further includes:
the privacy all-in-one computer sends authorization information to the controller, wherein the authorization information is used for indicating that the SDK of the data set can be used;
the controller generates the forwarding condition according to the authorization information.
In a possible implementation manner, each private computing all-in-one machine calculates the computing task according to a data set stored in the private computing all-in-one machine, and sends an obtained computing result to the SDK through the controller, including:
in the process of calculating the calculation task according to the data set stored in the privacy all-in-one machine, each privacy all-in-one machine exchanges parameters with the SDK or other privacy all-in-one machines through the controller;
after a calculation result is obtained, the privacy calculation all-in-one machine sends the calculation result to the controller;
the controller forwards the calculation result to the SDK.
In one possible implementation, the method further includes: after the privacy all-in-one computer agrees that the SDK uses the data set, the controller generates an index of the data set and binds the index and the identification of the SDK;
the controller decomposes the privacy computing application, and sends the obtained computing tasks to the n privacy computing all-in-one machines respectively, including: and the controller decomposes the privacy calculation application to obtain a plurality of calculation tasks, determines a corresponding privacy calculation all-in-one machine according to the index and the reference of the data set contained in each calculation task, and sends each calculation task to the corresponding privacy calculation all-in-one machine.
In a possible implementation manner, the controller includes a parser, an orchestrator, and an optimizer connected in series, and the controller decomposes the privacy computing application to obtain a plurality of computing tasks, including:
the analyzer analyzes the privacy calculation application, and sends analyzed participants and corresponding privacy calculation requirements to the orchestrator, wherein the participants comprise the privacy calculation all-in-one machine;
the orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the plurality of privacy computation back-end types and the corresponding participant deployment schemes to the optimizer;
and the optimizer selects a final privacy computation back-end type and a participant deployment scheme from the privacy computation back-end types and the corresponding participant deployment schemes based on network resources and computing resources to obtain a plurality of computation tasks.
In one aspect, a privacy computing system based on distributed collaboration is provided, and comprises a Software Development Kit (SDK), a controller and a plurality of privacy computing all-in-one machines, wherein the controller is deployed in an edge cloud, and the plurality of privacy computing all-in-one machines are deployed in a private cloud;
each privacy all-in-one computer is used for registering a data set stored by the privacy all-in-one computer with the controller;
the SDK is used for applying for using a data set to the n privacy all-in-one computers through the controller;
the SDK is further used for sending the developed privacy calculation application to the controller after the n privacy calculation all-in-one machines agree to use the data sets by the SDK, the privacy calculation application comprises references to the data sets in the n privacy calculation all-in-one machines, and n is larger than or equal to 1;
the controller is used for decomposing the privacy calculation application and respectively sending the obtained calculation tasks to the n privacy calculation all-in-one machines;
each privacy calculation all-in-one machine is also used for calculating the calculation tasks according to the data sets stored in the privacy calculation all-in-one machine, and the obtained calculation results are sent to the SDK through the controller.
In a possible implementation manner, the controller is further configured to generate an index of the data set after the private computing all-in-one machine agrees that the SDK uses the data set, and bind the index and an identifier of the SDK;
the controller is further configured to decompose the privacy computing application, and send the obtained computing tasks to the n privacy computing all-in-one machines, respectively, and includes: and the controller decomposes the privacy calculation application to obtain a plurality of calculation tasks, determines a corresponding privacy calculation all-in-one machine according to the index and the reference of the data set contained in each calculation task, and sends each calculation task to the corresponding privacy calculation all-in-one machine.
The technical scheme provided by the application has the beneficial effects that:
each privacy computing all-in-one machine registers a data set stored by the privacy computing all-in-one machine to the controller, the SDK applies for using the data set to the n privacy computing all-in-one machines through the controller, after the n privacy computing all-in-one machines agree to the SDK to use the data set, the SDK sends a developed privacy computing application to the controller, the privacy computing application contains citations to the data sets in the n privacy computing all-in-one machines, the controller decomposes the privacy computing application, the obtained computing tasks are respectively sent to the n privacy computing all-in-one machines, each privacy computing all-in-one machine calculates the computing tasks according to the data set stored by the privacy computing all-in-one machine, the obtained computing results are sent to the SDK through the controller, therefore, the data sets can be stored on a private cloud, the data safety can be guaranteed, the management of the privacy data can be brought into the whole work flow, and the data sets can be cited and created in the SDK, so that developers can conveniently organize and manage. In addition, the SDK, the controller and the privacy computing all-in-one machine are decoupled, so that the application development on the application development end is lighter.
The controller comprises a parser, a composer and an optimizer which are connected in series, actual arrangement and optimization of privacy calculation application can be sunk onto the controller, and generation of a mixed scheme integrating multiple privacy back-end types is facilitated.
The orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the privacy computation back-end types and the corresponding participant deployment schemes to the optimizer, and the optimizer selects a final privacy computation back-end type and a participant deployment scheme from the privacy computation back-end types and the corresponding participant deployment schemes based on network resources and computing resources to obtain a plurality of computation tasks.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic block diagram illustrating a privacy computing system in accordance with some exemplary embodiments;
FIG. 2 is a flowchart of a method for distributed collaboration based privacy computation according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for distributed collaboration based privacy computation according to another embodiment of the present application;
FIG. 4 is a schematic structural diagram of a privacy all-in-one computer provided in another embodiment of the present application;
FIG. 5 is a schematic structural diagram of a controller according to another embodiment of the present application;
fig. 6 is a schematic structural diagram of an SDK according to another embodiment of the present application.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail with reference to the accompanying drawings.
Please refer to the mechanism diagram of the privacy computing system based on distributed collaboration shown in fig. 1, which includes an SDK (Software Development Kit) 110, at least one controller 120, and at least one privacy computing kiosk 130. The controller 120 is typically deployed in a cluster in an edge cloud, and the plurality of privacy all-in-one computers 130 are deployed in a private cloud.
The SDKs 110 are used by the mobile-side or host-side developers to provide the ability to develop private computing applications, and each SDK has a unique identifier that characterizes the creator of the private computing application.
The controller 120 has the following main functions: 1) is responsible for the management registration of the data set; 2) the system is responsible for receiving the privacy calculation application submitted by the application development terminal, and performing scheduling optimization and conversion to a specific calculation task; 3) and performing calculation task coordination.
The integrated privacy computing machine 130 is a specific software and hardware device for performing privacy computation, and is responsible for computing a computing task issued by the controller 120, and the integrated privacy computing machine 130 is managed by an operation of a data element holder.
Referring to fig. 2, a flowchart of a distributed collaboration-based privacy computation method according to an embodiment of the present application is shown. The privacy calculation method based on distributed collaboration can comprise the following steps:
The data element holder can register the data set with the controller through the privacy all-in-one computer, and the registered contents include but are not limited to: characteristics of the data, data source, data quantity, data usage right quotation, privacy rules of the data and the like.
The SDK can send a data use application to the controller according to a data set required to be used in privacy calculation, the controller forwards the data use application to the n privacy calculation all-in-one machines, and each privacy calculation all-in-one machine determines whether the SDK agrees to use the data set stored by the SDK.
And 203, after the n privacy all-in-one computers agree to the SDK to use the data set, the SDK sends the developed privacy calculation application to the controller, the privacy calculation application contains references to the data set in the n privacy all-in-one computers, and n is larger than or equal to 1.
If the n privacy computing all-in-one machines agree to the SDK to use the data set, a developer can develop privacy computing application in the SDK, and creates reference of the data set in the development process, so that the reference can be used as a local variable in the development process, a privacy computing application is finally obtained, and the privacy computing application is sent to the controller.
And 204, decomposing the privacy calculation application by the controller, and respectively sending the obtained calculation tasks to the n privacy calculation all-in-one machines.
The controller can decompose the privacy calculation application to obtain a plurality of calculation tasks, and then sends each calculation task to the corresponding privacy calculation all-in-one machine.
And step 205, each privacy computing all-in-one machine computes a computing task according to the data set stored by the privacy computing all-in-one machine, and sends the obtained computing result to the SDK through the controller.
To sum up, in the privacy computing method based on distributed collaboration provided by the embodiments of the present application, each privacy computing all-in-one machine registers its own stored data set with the controller, the SDK applies for using the data set to n privacy computing all-in-one machines through the controller, after the n privacy computing all-in-one machines agree to the SDK to use the data set, the SDK sends the developed privacy computing application to the controller, the privacy computing application includes references to the data sets in the n privacy computing all-in-one machines, the controller decomposes the privacy computing application, sends the obtained computing tasks to the n privacy computing all-in-one machines respectively, each privacy computing all-in-one machine computes the computing tasks according to its own stored data set, and sends the obtained computing results to the SDK through the controller, so that the data set can be stored in a private cloud, which can ensure data security, the management of the private data can be incorporated into the whole workflow, and the data set can be referenced and created in the SDK, so that developers can conveniently organize and manage the data set. In addition, the SDK, the controller and the privacy computing all-in-one machine are decoupled, so that the application development on the application development end is lighter.
Referring to fig. 3, a flowchart of a distributed collaboration-based privacy computation method according to an embodiment of the present application is shown. The privacy calculation method based on distributed cooperation can comprise the following steps:
The data element holder can register the data set with the controller through the privacy all-in-one computer, and the registered content includes but is not limited to: characteristics of the data, data source, data quantity, data usage right quotation, privacy rules of the data and the like.
The privacy computing all-in-one machine is mainly designed in a layered mode, a fine-grained audit framework is matched for the flowing of data elements, and as shown in fig. 4, the structure of the privacy computing all-in-one machine is introduced from top to bottom:
(A) an application layer: the common basic algorithm is mainly packaged;
(B) a computation layer: the method comprises the steps of aiming at operators of machine learning application or SQL (Structured Query Language) application to realize heterogeneous privacy calculation paradigms, wherein the paradigms mainly comprise a federal learning paradigms, a difference paradigms, a credible execution paradigms and a multiparty safety paradigms;
(C) hardware layer: a collection of various hardware is supported. Besides a general GPU (graphic Processing Unit) and a CPU (Central Processing Unit), a trusted abstraction module (used for providing trusted requirements in the implementation of a privacy computing-related protocol) is constructed based on trusted execution environment technology; constructing a circuit generating module (used for converting a multi-party safety circuit into hardware realization) based on an FPGA (Field Programmable Gate Array); a key management Module (used for key generation and management in privacy calculation) is constructed based on HSM (Hardware Security Module);
(D) and (3) an audit layer: audits are performed for algorithm usage, operator calls, hardware calls, and network communications.
In step 302, the controller generates a data mart from the data set.
The controller is mainly divided into a data management part and a task management part, wherein the data management part mainly manages data registration and authorization of a data element provider.
As shown in fig. 5, the data management part includes a data mart for screening the data set by the SDK, a data registration module for managing the registered and registered data set of the privacy all-in-one computer, and a data authorization management module for managing the authorization of the data set.
And step 303, the SDK performs data screening and usage declaration in the data mart, generates a data usage application including screening information, usage declaration information, and an identifier of the SDK, and sends the data usage application to the controller.
The screening information can include, but is not limited to, data screening conditions and data screening quantity, and can be set according to actual use requirements of the meal provider.
Usage declaration information may include, but is not limited to, number of data uses, application type, model type, for purposes of usage declaration.
The SDK in the embodiment mainly comprises a control console, an algorithm template library and a data management library, wherein the control console is mainly used for submitting privacy calculation application and obtaining callback related to data statistics or results; the algorithm template library mainly provides basic algorithm templates for developers to directly fill and develop calculation programs; the data management library mainly provides a relevant interface for data mart and manages the reference of the applied data, as shown in fig. 6.
Specifically, the SDK may perform data screening and usage declaration in a data mart in the controller through the data management library, obtain an identifier of the SDK, generate a data usage application including the screening information, the usage declaration information, and the identifier of the SDK, and send the data usage application to the controller.
And step 304, the controller forwards the data use application to the n privacy computing all-in-one machines.
The controller can firstly carry out preliminary examination on the data use application according to the forwarding conditions, and then forwards the data use application to the n privacy calculation all-in-one machines after the examination is correct. Specifically, the forwarding, by the controller, the data use application to the n privacy computing all-in-one machines may include: the controller detects whether the data use application meets a preset forwarding condition; and if the data use application meets the forwarding condition, the controller forwards the data use application to the n privacy computing all-in-one machines.
The content of the preliminary audit can be various, for example, whether the format or syntax of the application for using the audit data is wrong or not, or whether the SDK has the right to use the data set is audited, i.e., the data authorization management module in fig. 5 is used for authorization management. If the SDK is checked to have the permission to use the data set, the privacy all-in-one computer sends authorization information to the controller, wherein the authorization information is used for indicating the SDK which can use the data set; the controller generates a forwarding condition according to the authorization information.
When the authorization information is a blacklist, the forwarding condition may be that the identifier of the SDK is not in the blacklist; when the authorization information is a white list, the forwarding condition may be that the identification of the SDK is in the white list. The controller can audit the data use application according to the set forwarding condition.
In step 306, the controller forwards the application result to the SDK.
The controller can read the application result, and if the application result is used for indicating that the SDK refuses to use the data set, the application result is forwarded to the SDK; if the application result is used to indicate that the SDK agrees to use the data set, the controller not only needs to forward the application result to the SDK, but also needs to generate an index of the data set, and bind the index and the identifier of the SDK, that is, execute step 309.
And 307, the SDK determines whether the integrated privacy computing machine agrees to use the data set by the SDK according to the application result.
And 308, after the n privacy all-in-one computers agree to the SDK to use the data sets, the SDK sends the developed privacy calculation application to the controller, the privacy calculation application contains references to the data sets in the n privacy all-in-one computers, and n is larger than or equal to 1.
If the n privacy calculation all-in-one machines agree to the SDK to use the data set, a developer can develop privacy calculation application in the SDK, reference of the data set is created in the development process, the data set can be used as a local variable in the development process, finally, privacy calculation application is obtained, and then the privacy calculation application is sent to the controller.
In step 309, the controller generates an index of the data set and binds the index to the SDK identifier.
And 310, decomposing the privacy calculation application by the controller to obtain a plurality of calculation tasks, determining a corresponding privacy calculation all-in-one machine according to the indexes and the references of the data sets contained in each calculation task, and sending each calculation task to the corresponding privacy calculation all-in-one machine.
As shown in fig. 5, the controller further includes a task management part, where the task management part may include a parser, an orchestrator, and an optimizer connected in series, and the decomposing, by the controller, of the privacy computing application to obtain a plurality of computing tasks may include: the analyzer analyzes the privacy calculation application, and sends the analyzed participants and the corresponding privacy calculation requirements to the orchestrator, wherein the participants comprise a privacy calculation all-in-one machine; the orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the privacy computation back-end types and the corresponding participant deployment schemes to the optimizer; the optimizer selects a final privacy computation back-end type and a participant deployment scheme from a plurality of privacy computation back-end types and corresponding participant deployment schemes based on network resources and computing power resources to obtain a plurality of computation tasks.
Because the privacy requirements of each participant are possibly different, the participant deployment scheme meeting the conditions needs to be generated based on the privacy requirement limits of all the participants, so that various privacy computing back-end capabilities can be integrally delivered through the privacy computing all-in-one machine, a client operating the privacy computing all-in-one machine can control the expression of specific computing tasks through the scheduling of network resources and computing resources, and a foundation is provided for the production-level privacy computing service management.
And 311, each privacy calculation all-in-one machine calculates a calculation task according to the data set stored by the privacy calculation all-in-one machine, and sends the obtained calculation result to the SDK through the controller.
Specifically, each privacy computing all-in-one machine computes a computing task according to a data set stored in the privacy computing all-in-one machine, and sends an obtained computing result to the SDK through the controller, and the method may include: in the process of calculating a calculation task according to a data set stored in the privacy all-in-one machine, each privacy all-in-one machine exchanges parameters with the SDK or other privacy all-in-one machines through the controller; after the calculation result is obtained, the privacy calculation all-in-one machine sends the calculation result to the controller; the controller forwards the calculation result to the SDK.
After the privacy computing all-in-one machine receives the computing task, the privacy computing all-in-one machine can carry out concrete calling and computing layer by layer, then carries out parameter exchange with other privacy computing all-in-one machines and the SDK through the coordination of the controller, finally obtains a computing result, sends the computing result to the controller, and the controller forwards the computing result to the SDK.
In the embodiment, the data set in the private cloud can be flexibly accessed into the privacy computing platform, the computation, management and development of the privacy computing are deeply decoupled, and the dynamic capacity expansion capability of the computing infrastructure is strong. The privacy computing all-in-one machine integrates delivery of privacy computing capacity, can be compatible with different privacy computing rear-end types, provides acceleration capacity through exclusive hardware and improves overall efficiency.
To sum up, in the privacy computing method based on distributed collaboration provided in the embodiments of the present application, each privacy computing all-in-one machine registers its own stored data set with the controller, the SDK applies for using the data set to n privacy computing all-in-one machines through the controller, after the n privacy computing all-in-one machines agree to the SDK to use the data set, the SDK sends the developed privacy computing application to the controller, the privacy computing application includes references to the data sets in the n privacy computing all-in-one machines, the controller decomposes the privacy computing application, sends the obtained computing tasks to the n privacy computing all-in-one machines, each privacy computing all-in-one machine computes the computing tasks according to its own stored data set, and sends the obtained computing results to the SDK through the controller, so that the data set can be stored in the privacy cloud, which can ensure data security, the management of the private data can be incorporated into the whole workflow, and the data set can be referenced and created in the SDK, so that developers can conveniently organize and manage the data set. In addition, the SDK, the controller and the privacy computing all-in-one machine are decoupled, so that the application development on the application development end is lighter.
The controller comprises a parser, a composer and an optimizer which are connected in series, actual arrangement and optimization of privacy calculation application can be sunk onto the controller, and generation of a mixed scheme integrating multiple privacy back-end types is facilitated.
The orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the privacy computation back-end types and the corresponding participant deployment schemes to the optimizer, and the optimizer selects a final privacy computation back-end type and a participant deployment scheme from the privacy computation back-end types and the corresponding participant deployment schemes based on network resources and computing resources to obtain a plurality of computation tasks.
Referring to fig. 1, a block diagram of a distributed collaboration based privacy computing system according to an embodiment of the present application is shown. The privacy computing system based on distributed collaboration can comprise a Software Development Kit (SDK) 110, a controller 120 and a plurality of privacy all-in-one computers 130, wherein the controller 120 is deployed in an edge cloud, and the plurality of privacy all-in-one computers 130 are deployed in a private cloud;
each privacy all-in-one computer 130 is used for registering a data set stored by the privacy all-in-one computer 130 with the controller 120;
the SDK110 is used for applying for using a data set to the n privacy all-in-one computers 130 through the controller 120;
the SDK110 is further configured to send the developed privacy computing application to the controller 120 after the n privacy computing all-in-one machines 130 agree to the SDK110 to use the data set, where the privacy computing application includes references to the data sets in the n privacy computing all-in-one machines 130, and n is greater than or equal to 1;
the controller 120 is configured to decompose the privacy computing application, and send the obtained computing tasks to the n privacy computing all-in-one machines 130 respectively;
each privacy computing all-in-one machine 130 is further configured to compute a computing task according to the data set stored in the privacy computing all-in-one machine, and send an obtained computing result to the SDK110 through the controller 120.
In an alternative embodiment, the controller 120 is further configured to generate a data mart from the data set;
the SDK110 is further configured to perform data screening and usage declaration in the data mart, generate a data usage application including screening information, usage declaration information, and an identifier of the SDK110, and send the data usage application to the controller 120;
the controller 120 is further configured to forward the data use application to the n privacy all-in-one computers 130.
In an alternative embodiment, after receiving the data usage application, each of the private all-in-one computers 130 is further configured to send an application result to the controller 120, where the application result is used to indicate agreement or rejection to use the data set by the SDK 110;
the controller 120 is further configured to forward the application result to the SDK 110;
the SDK110 is further configured to determine whether the privacy all-in-one computer 130 agrees to use the data set by the SDK110 according to the application result.
In an optional embodiment, the controller 120 is further configured to detect whether the data usage application satisfies a preset forwarding condition;
if the data use application meets the forwarding condition, the controller 120 is further configured to forward the data use application to the n integrated privacy computing machines 130.
In an alternative embodiment, the privacy all-in-one computer 130 is further configured to send authorization information to the controller 120, the authorization information indicating that the SDK110 of the data set can be used;
the controller 120 is further configured to generate a forwarding condition according to the authorization information.
In an alternative embodiment, in the process of computing a computing task according to the data set stored in the privacy all-in-one computer, each privacy all-in-one computer 130 is further configured to perform parameter exchange with the SDK110 or other privacy all-in-one computers 130 through the controller 120;
after the calculation result is obtained, the privacy and calculation all-in-one machine 130 is further configured to send the calculation result to the controller 120;
the controller 120 is further configured to forward the calculation result to the SDK 110.
In an optional embodiment, the controller 120 is further configured to generate an index of the data set after the privacy all-in-one computer 130 agrees that the SDK110 uses the data set, and bind the index and the identifier of the SDK 110;
the controller 120 is further configured to decompose the privacy computing application, and send the obtained computing tasks to the n privacy computing all-in-one machines 130, including: the controller 120 decomposes the privacy calculation application to obtain a plurality of calculation tasks, determines a corresponding privacy calculation all-in-one machine 130 according to the index and the reference of the data set contained in each calculation task, and sends each calculation task to the corresponding privacy calculation all-in-one machine 130.
In an optional embodiment, the controller 120 includes a parser 121, a composer 122, and an optimizer 123 connected in series, where the parser 121 is configured to parse the privacy computation application, and send the parsed participants and the corresponding privacy computation requirements to the composer 122, where the participants include the integrated privacy computation machine 130;
the orchestrator 122 is configured to generate a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and send the plurality of privacy computation back-end types and the corresponding participant deployment schemes to the optimizer 123;
and the optimizer 123 is configured to select a final privacy computation back-end type and a participant deployment scheme from the multiple privacy computation back-end types and the corresponding participant deployment schemes based on the network resources and the computation resources, so as to obtain multiple computation tasks.
To sum up, in the privacy computing system based on distributed collaboration provided by the embodiment of the present application, each privacy computing all-in-one machine registers its own stored data set with the controller, the SDK applies for using the data set to n privacy computing all-in-one machines through the controller, after the n privacy computing all-in-one machines agree to the SDK to use the data set, the SDK sends the developed privacy computing application to the controller, the privacy computing application includes references to the data sets in the n privacy computing all-in-one machines, the controller decomposes the privacy computing application, sends the obtained computing tasks to the n privacy computing all-in-one machines respectively, each privacy computing all-in-one machine computes the computing tasks according to its own stored data set, and sends the obtained computing results to the SDK through the controller, so that the data set can be stored in a privacy cloud, which can ensure data security, the management of the private data can be incorporated into the whole workflow, and the data set can be referenced and created in the SDK, so that developers can conveniently organize and manage the data set. In addition, the SDK, the controller and the privacy and calculation all-in-one machine are decoupled, so that application development on an application development end is lighter.
The controller comprises a parser, a composer and an optimizer which are connected in series, actual arrangement and optimization of privacy calculation application can be sunk onto the controller, and generation of a mixed scheme integrating multiple privacy back-end types is facilitated.
The orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the privacy computation back-end types and the corresponding participant deployment schemes to the optimizer, and the optimizer selects a final privacy computation back-end type and a participant deployment scheme from the privacy computation back-end types and the corresponding participant deployment schemes based on network resources and computing resources to obtain a plurality of computation tasks.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
The above description should not be taken as limiting the embodiments of the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the embodiments of the present application.
Claims (8)
1. A privacy computing method based on distributed cooperation is used in a privacy computing system based on distributed cooperation, the privacy computing system comprises a Software Development Kit (SDK), a controller and a plurality of privacy computing all-in-one machines, the controller is deployed in an edge cloud, the plurality of privacy computing all-in-one machines are deployed in a private cloud, and the method comprises the following steps:
each privacy all-in-one computer registers a data set stored by the privacy all-in-one computer with the controller;
the SDK applies for using a data set to the n privacy all-in-one computers through the controller;
after the n privacy computing integrators agree to use the data sets by the SDK, the SDK sends the developed privacy computing applications to the controller, the privacy computing applications comprise references to the data sets in the n privacy computing integrators, and the references to the data sets in the n privacy computing integrators are created in the process of developing the privacy computing applications in the SDK after the n privacy computing integrators agree to use the data sets by the SDK, so that the privacy computing applications can be used as local variables, and n is larger than or equal to 1;
the controller decomposes the privacy calculation application and sends the obtained calculation tasks to the n privacy calculation all-in-one machines respectively;
each privacy calculation all-in-one machine calculates the calculation task according to the data set stored by the privacy calculation all-in-one machine, and sends the obtained calculation result to the SDK through the controller;
the method further comprises the following steps: after the private all-in-one computer agrees that the SDK uses the data set, the controller generates an index of the data set and binds the index and the identification of the SDK;
the controller decomposes the privacy computing application, and sends the obtained computing tasks to the n privacy computing all-in-one machines respectively, including: and the controller decomposes the privacy calculation application to obtain a plurality of calculation tasks, determines a corresponding privacy calculation all-in-one machine according to the index and the reference of the data set contained in each calculation task, and sends each calculation task to the corresponding privacy calculation all-in-one machine.
2. The method of claim 1,
the method further comprises the following steps: the controller generates a data mart from the data set;
the SDK applies for a usage data set from the n privacy all-in-one computers through the controller, and the method comprises the following steps: the SDK performs data screening and usage statement in the data mart, generates a data usage application containing screening information, usage statement information and the identification of the SDK, and sends the data usage application to the controller; and the controller forwards the data use application to the n privacy computing all-in-one machines.
3. The method of claim 2, further comprising:
after receiving the data use application, each privacy all-in-one computer sends an application result to the controller, wherein the application result is used for indicating agreement or refusal of the SDK use data set;
the controller forwards the application result to the SDK;
and the SDK determines whether the privacy all-in-one computer agrees to use the data set by the SDK according to the application result.
4. The method of claim 2, wherein the controller forwards the data usage application to the n private computing kiosks, comprising:
the controller detects whether the data use application meets a preset forwarding condition;
and if the data use application meets the forwarding condition, the controller forwards the data use application to the n privacy computing all-in-one machines.
5. The method of claim 4, further comprising:
the privacy all-in-one computer sends authorization information to the controller, wherein the authorization information is used for indicating that the SDK of the data set can be used;
the controller generates the forwarding condition according to the authorization information.
6. The method of claim 1, wherein each of the private computing all-in-one machines computes the computing task according to a data set stored in the private computing all-in-one machine, and sends an obtained computation result to the SDK through the controller, and the method includes:
in the process of calculating the calculation task according to the data set stored in the privacy all-in-one machine, each privacy all-in-one machine exchanges parameters with the SDK or other privacy all-in-one machines through the controller;
after a calculation result is obtained, the privacy calculation all-in-one machine sends the calculation result to the controller;
the controller forwards the calculation result to the SDK.
7. The method of claim 1, wherein the controller comprises a parser, an orchestrator, and an optimizer connected in series, and the controller decomposes the privacy computing application to obtain a plurality of computing tasks, including:
the analyzer analyzes the privacy calculation application, and sends analyzed participants and corresponding privacy calculation requirements to the orchestrator, wherein the participants comprise the privacy calculation all-in-one machine;
the orchestrator generates a plurality of privacy computation back-end types and corresponding participant deployment schemes according to the participants and corresponding privacy computation requirements, and sends the plurality of privacy computation back-end types and the corresponding participant deployment schemes to the optimizer;
and the optimizer selects a final privacy computation back-end type and a participant deployment scheme from the privacy computation back-end types and the corresponding participant deployment schemes based on network resources and computing resources to obtain a plurality of computation tasks.
8. The privacy computing system based on distributed collaboration is characterized by comprising a Software Development Kit (SDK), a controller and a plurality of privacy computing all-in-one machines, wherein the controller is deployed in an edge cloud, and the privacy computing all-in-one machines are deployed in a private cloud;
each privacy all-in-one computer is used for registering a data set stored by the privacy all-in-one computer with the controller;
the SDK is used for applying for using a data set to the n privacy all-in-one computers through the controller;
the SDK is further used for sending the developed privacy computing application to the controller after the n privacy computing all-in-one machines agree to the SDK to use the data set, the privacy computing application comprises references to the data set in the n privacy computing all-in-one machines, the references to the data set in the n privacy computing all-in-one machines are created in the process of sending the privacy computing application in the SDK after the n privacy computing all-in-one machines agree to the SDK to use the data set, so that the privacy computing application can be used as a local variable, and n is larger than or equal to 1;
the controller is used for decomposing the privacy calculation application and respectively sending the obtained calculation tasks to the n privacy calculation all-in-one machines;
each privacy computing all-in-one machine is also used for computing the computing task according to the data set stored by the privacy computing all-in-one machine and sending the obtained computing result to the SDK through the controller;
the controller is further configured to generate an index of the data set after the privacy all-in-one computer agrees that the SDK uses the data set, and bind the index and the identifier of the SDK;
the controller is further configured to decompose the privacy computing application, and send the obtained computing tasks to the n privacy computing all-in-one machines, respectively, and includes: and the controller decomposes the privacy calculation application to obtain a plurality of calculation tasks, determines a corresponding privacy calculation all-in-one machine according to the index and the reference of the data set contained in each calculation task, and sends each calculation task to the corresponding privacy calculation all-in-one machine.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210321981.4A CN114417428B (en) | 2022-03-30 | 2022-03-30 | Privacy calculation method and system based on distributed cooperation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210321981.4A CN114417428B (en) | 2022-03-30 | 2022-03-30 | Privacy calculation method and system based on distributed cooperation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114417428A CN114417428A (en) | 2022-04-29 |
| CN114417428B true CN114417428B (en) | 2022-08-26 |
Family
ID=81263107
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210321981.4A Active CN114417428B (en) | 2022-03-30 | 2022-03-30 | Privacy calculation method and system based on distributed cooperation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114417428B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114764509B (en) * | 2022-06-14 | 2022-08-26 | 深圳致星科技有限公司 | Interconnection and intercommunication method and device for privacy calculation, privacy data and federal learning |
| CN116090019B (en) * | 2023-04-12 | 2023-06-16 | 北京数力聚科技有限公司 | Privacy computing method and system based on distributed collaboration |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109889575B (en) * | 2019-01-15 | 2020-08-25 | 北京航空航天大学 | Collaborative computing platform system and method under edge environment |
| CN110955463B (en) * | 2019-12-03 | 2022-07-29 | 缀初网络技术(上海)有限公司 | Internet of things multi-user computing unloading method supporting edge computing |
| CN112199734A (en) * | 2020-09-24 | 2021-01-08 | 北京冲量在线科技有限公司 | Multi-party data circulation system |
| CN112910785B (en) * | 2021-01-15 | 2021-11-09 | 北京理工大学 | NDN-based edge calculation routing table establishing and using method |
-
2022
- 2022-03-30 CN CN202210321981.4A patent/CN114417428B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN114417428A (en) | 2022-04-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114417428B (en) | Privacy calculation method and system based on distributed cooperation | |
| WO2022160707A1 (en) | Human-machine interaction method and apparatus combined with rpa and ai, and storage medium and electronic device | |
| CN110781082B (en) | Method, device, medium and equipment for generating test case of interface | |
| CN109710236B (en) | Service development and implementation method, device, platform and medium based on shared service | |
| CN112835560A (en) | WEB multi-terminal low-code intelligent software development platform | |
| CN111382174A (en) | Multi-party data combined query method, device, server and storage medium | |
| US20040176968A1 (en) | Systems and methods for dynamically configuring business processes | |
| CN102024204B (en) | Constructing method of reliability design analysis service system of service oriented architecture | |
| US20230028947A1 (en) | Cost-aware integration process modeling in multi-cloud computing environment | |
| JP7519646B2 (en) | Method, device, electronic device, and storage medium for creating a basic database of quantum vehicle parts | |
| Südholt | A model of components with non-regular protocols | |
| CN116361137A (en) | Service call chain tracking method, device, electronic equipment and readable storage medium | |
| Zacharewicz et al. | Short-lived ontology approach for agent/HLA federated enterprise interoperability | |
| CN113992736B (en) | Interconnection method of structured data based on cloud computing service platform and server | |
| CN103729451B (en) | A kind of information input method of database, apparatus and system | |
| CN115757589A (en) | Data exchange and sharing method and device of database and readable storage medium | |
| Le et al. | Modeling and verifying ws-cdl using event-b | |
| CN115729535A (en) | Complex process application development framework based on automatic code generation | |
| CN108304219B (en) | Secondary development platform and method | |
| KR20090039176A (en) | Workflow system based on a rule-script engine | |
| CN112001715B (en) | Application method and system of distributed front-end tool | |
| CN109857390B (en) | Annotation transmission method of Git warehouse file annotation system | |
| CN113495723A (en) | Method and device for calling functional component and storage medium | |
| Fitsilis | Object-oriented development for telecommunication services | |
| Pham et al. | Building parallel time-constrained HLA federates: A case study with the PARSEC parallel simulation language |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |