CN111062492B - Method and system for model training based on optional private data - Google Patents
Method and system for model training based on optional private data Download PDFInfo
- Publication number
- CN111062492B CN111062492B CN201911329551.1A CN201911329551A CN111062492B CN 111062492 B CN111062492 B CN 111062492B CN 201911329551 A CN201911329551 A CN 201911329551A CN 111062492 B CN111062492 B CN 111062492B
- Authority
- CN
- China
- Prior art keywords
- data
- terminal
- type
- party
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
One or more embodiments of the present specification relate to a method and system for model training based on optional private data, the method including: receiving data of a first type from at least a first terminal and a second terminal; calculating first-class accumulated data based on the received first-class data and the corresponding model parameters; acquiring an accumulated loss value in a multi-party safety calculation mode; the accumulated loss value is determined at least by the first terminal and the second terminal based on the first type of accumulated data, the second type of data and the sample label; the method comprises the steps that a cumulative loss value participates in calculation of a first type of cumulative gradient and a second type of cumulative gradient, and the first type of cumulative gradient and the second type of cumulative gradient are used for updating parameters of a joint training model at least based on models of a first terminal and a second terminal; the first terminal and the second terminal respectively hold first training data and second training data, and the first training data and the second training data comprise first class data and second class data corresponding to different privacy levels.
Description
Technical Field
One or more embodiments of the present specification relate to multi-party data collaboration, and more particularly, to a method and system for model training based on optional private data.
Background
In the fields of data analysis, data mining, economic prediction and the like, the machine learning model can be used for analyzing and finding potential data values. Since data held by a single data owner may be incomplete, and thus it is difficult to accurately characterize the target, joint training of models by cooperation of data from multiple data owners is widely used for better model prediction results. But in the process of multi-party data cooperation, problems such as data security and model security are involved.
Therefore, there is a need to provide a secure solution for joint modeling based on multi-party data.
Disclosure of Invention
One aspect of embodiments of the present specification provides a method of model training based on selectable privacy data; the method comprises the following steps: a third party receives data of a first type at least from a first terminal and a second terminal; calculating first-class accumulated data based on the received first-class data and the corresponding model parameters; a third party obtains an accumulated loss value in a multi-party secure calculation mode; the accumulated loss value is at least determined by the first terminal and the second terminal based on the first type of accumulated data, the second type of data and the sample label; the third party participates the accumulated loss value in the calculation of a first type of accumulated gradient and a second type of accumulated gradient, and the first type of accumulated gradient and the second type of accumulated gradient are used for updating parameters of a joint training model at least based on the models of the first terminal and the second terminal; the method comprises the steps that a first terminal and a second terminal respectively hold first training data and second training data, and the first training data and the second training data correspond to the same characteristics of different samples; at least one of the first training data and the second training data comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same training samples.
Another aspect of an embodiment of the present specification provides a system for model training based on optional private data; the system comprises: the first-class data receiving module is used for receiving first-class data at least from a first terminal and a second terminal; the first-class accumulated data determining module is used for calculating first-class accumulated data based on the received first-class data and the corresponding model parameters thereof; the accumulated loss value determining module is used for acquiring an accumulated loss value in a multi-party safety calculation mode; the accumulated loss value is determined at least by the first terminal and the second terminal based on the first type of accumulated data, the second type of data and the sample label; the model parameter updating module is used for enabling the accumulated loss value to participate in calculation of a first type of accumulated gradient and a second type of accumulated gradient, and the first type of accumulated gradient and the second type of accumulated gradient are used for updating parameters of a joint training model at least based on models of the first terminal and the second terminal; the method comprises the steps that a first terminal and a second terminal respectively hold first training data and second training data, and the first training data and the second training data correspond to the same characteristics of different samples; at least one of the first training data and the second training data comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same training samples.
Another aspect of embodiments of the present specification provides an apparatus for model training based on selectable privacy data, the apparatus comprising a processor and a memory; the memory is used for storing instructions, and the processor is used for executing the instructions to realize operations corresponding to the model training method based on the optional privacy data.
Another aspect of an embodiment of the present specification provides a method of model training based on selectable privacy data; the method comprises the following steps: the first terminal transmits the first type of data to a third party; the method comprises the steps that a first terminal receives first-class accumulated data and second-class model parameters from a third party; the first terminal calculates a first loss value based on self training data and a sample label; the first terminal participates in the joint training of the first loss value at least based on the models of the first terminal and the second terminal in a multi-party safety calculation mode to obtain a model with updated parameters; the training data of the first terminal comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same characteristics of different samples.
Another aspect of an embodiment of the present specification provides a system for model training based on optional private data; the system comprises: the first-class data transmission module is used for transmitting the first-class data to a third party; the first data receiving module is used for receiving the first type accumulated data and the second type model parameters from a third party; the first loss value calculating module is used for calculating a first loss value based on self training data and the sample label; and the model parameter updating module is used for participating the first loss value in joint training at least based on the models of the first terminal and the second terminal in a multi-party safety calculation mode to obtain a parameter updated model.
Another aspect of embodiments of the present specification provides an apparatus for model training based on selectable privacy data, the apparatus comprising a processor and a memory; the memory is used for storing instructions, and the processor is used for executing the instructions to realize operations corresponding to the model training method based on the optional privacy data.
Drawings
The present description will be further described by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not intended to be limiting, and in these embodiments like numerals are used to indicate like structures, wherein:
FIG. 1 is a diagram of an exemplary application scenario for an alternative privacy data based model training system in accordance with some embodiments of the present description;
FIG. 2 is an exemplary flow diagram of a method for model training based on optional privacy data, according to some embodiments of the present description; and
FIG. 3 is an exemplary diagram illustrating a method for model training based on optional privacy data according to some further embodiments of the present description.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only examples or embodiments of the application, from which the application can also be applied to other similar scenarios without inventive effort for a person skilled in the art. Unless otherwise apparent from the context, or otherwise indicated, like reference numbers in the figures refer to the same structure or operation.
It should be understood that "system", "device", "unit" and/or "module" as used in this specification is a method for distinguishing different components, elements, parts or assemblies at different levels. However, other words may be substituted by other expressions if they accomplish the same purpose.
As used in this specification and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural, unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.
Flow charts are used in this description to illustrate operations performed by a system according to embodiments of the present description. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, the various steps may be processed in reverse order or simultaneously. Meanwhile, other operations may be added to the processes, or a certain step or several steps of operations may be removed from the processes.
A large amount of information data, which is abundant in various industries such as economy, culture, education, medical care, public management, etc., is widely used in more and more scenes for performing data processing analysis such as data analysis, data mining, and trend prediction. The data cooperation mode can enable a plurality of data owners to obtain better data processing results. For example, more accurate model parameters may be obtained through joint training of multi-party data.
In some embodiments, the joint training system for model based on optional privacy data can be applied to a scenario in which parties train a machine learning model in cooperation for use by multiple parties while ensuring the data security of the parties. In this scenario, multiple data parties have their own data, and they want to use each other's data together for unified modeling (e.g., linear regression model, logistic regression model, etc.), but do not want the respective data (especially private data) to be revealed. For example, an internet deposit institution a has one batch of user data, a government bank B has another batch of user data, and a training sample set determined based on the user data of a and B can be trained to obtain a better machine learning model. Both a and B would like to participate in model training together with each other's user data, but for some reasons a and B would not like their own user data information to be revealed, or at least would not like to let the other party know their own user data information.
The model training system based on the selectable privacy data can enable the machine learning model used together to be obtained through the joint training of the multi-party data under the condition that the multi-party privacy data are not leaked, and a win-win cooperation state is achieved.
In some embodiments, in the joint training based on multi-party data, in order to prevent the leakage of private data, a garbled circuit (garbled circuit) or secret sharing manner may be adopted. When the feature dimension is large, the operation efficiency of a garbled circuit (garbled circuit) or a secret sharing scheme is not high. In some embodiments, the privacy data of each party can also be homomorphically encrypted, and then the privacy data of each party participates in the model training operation in the encrypted state. However, homomorphic encryption only supports product operation and/or sum operation, and during the use process, corresponding operation formulas need to be correspondingly converted according to needs. In some scenarios where the number of data owners is particularly large, the homomorphic encryption scheme is not computationally efficient.
In some scenarios, there are some training samples whose feature data require privacy protection, e.g., user's income, gender, consumption level, etc. in the sample data; there are some characteristic data that are publicly available, e.g., the home of the user, the city location, etc., in the sample data. For the above scenario, in some embodiments, all data in the training sample is treated as privacy protection data, and then the privacy data is participated in the joint training by means of secret sharing or asymmetric encryption. In some embodiments, the private data features of each data party can be processed safely according to the principle that non-private data can be disclosed and private data needs to be protected, and then the data parties participate in the joint training. For example, the operation product of the privacy features of each data party and the model parameters can participate in model training, so as to achieve the purpose of not revealing the privacy features.
In some embodiments, operational efficiency may also be improved through the intervention of third parties. For example, the plurality of data owners respectively transmit the non-private data to the third party, and then the non-private data is uniformly collected and processed by the third party and then uniformly distributed to the data owners. By the participation of the third-party server, the multi-party data owner can deliver the non-private data to a third party for operation, then the calculated result is returned to the multi-party data owner, the multi-party data owner performs learning calculation based on the private data of the multi-party data owner, and the third party obtains the learning result of the multi-party data owner in a multi-party safety calculation mode. The third party does not need to carry out any encryption and decryption work, and the operation efficiency can be improved.
FIG. 1 is a diagram of an exemplary application scenario for an alternative privacy data based model training system in accordance with some embodiments of the present description.
In some embodiments, model training based on optional privacy data system 100 includes a first terminal 110, a second terminal 120, a third party 130, and a network 140. The first terminal 110 may be understood as a first party data owner, including the processing device 110-1, the storage device 110-2; the second terminal 120 may be understood as a second party data owner including a processing device 120-1, a storage device 120-2; the third party 130 is not the data owner, but holds the training data for the model, including the processing device 130-1, the storage device 130-2. Specifically, a multi-party data owner trains non-private data to participate in a model through a third party to obtain a calculation result, the third party transmits the calculation result to the multi-party data owner, the multi-party data owner obtains a second calculation result according to the calculation result and private data of the multi-party data owner, and the second calculation result is collected based on multi-party safety calculation, so that the numerical value of the model parameter can be updated.
It should be noted that the number of data owners in fig. 1 is two, and in other embodiments, a third party data owner and a fourth party data owner may be included.
The first terminal 110 and the second terminal 120 may each be a device with data acquisition, storage, and/or transmission capabilities. In some embodiments, the first terminal 110, the second terminal 120 may include, but are not limited to, a mobile device, a tablet, a laptop, a desktop, etc., or any combination thereof. In some embodiments, the first terminal 110 and the second terminal 120 may receive relevant data from the third party 130. For example, the first terminal 110 may receive the second type of model parameters from a third party. For example, the first terminal 110 and the second terminal 120 may receive the first type of accumulated data from the third party 130.
The processing devices 110-1 and 120-1 of the first and second terminals may perform data and/or instruction processing. Processing devices 110-1 and 120-1 may perform computations on the data and may also execute related algorithms and/or instructions. For example, the processing device 110-1 of the first terminal 110 may receive the second type model parameters from the third party 130, and calculate the first loss value using the stored second type data, or calculate the gradient of the first terminal using the first loss value.
The memory devices 110-2 and 120-2 of the first and second terminals may store data and/or instructions for execution by the corresponding processing devices 110-1 and 120-1, which the processing devices 110-1 and 120-1 may execute or use to implement the exemplary methods of this specification. Storage devices 110-2 and 120-2 may be used to store first training data and corresponding sample labels and second training data and corresponding sample labels, respectively; associated instructions may also be stored that instruct the first terminal and the second terminal to perform operations. Storage devices 110-2 and 120-2 may also store data processed by processing devices 110-1 and 120-1, respectively. For example, the memory devices 110-2 and 120-2 may also store loss values and associated gradient values for each terminal, respectively. In some embodiments, the storage device 110-2 and the storage device 120-2 may also be a storage device, wherein the first terminal and the second terminal can only obtain the data stored by themselves from the storage device. In some embodiments, the storage device may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), and the like, or any combination thereof.
The third party 130 may be a device with data acquisition, storage, and/or transmission functions, such as a cloud server, a terminal processing device, and the like. In some embodiments, the third party 130 may receive the relevant data from the first terminal 110 and the second terminal 120. For example, the third party 130 may receive the first type of data from the first terminal 110. In some embodiments, the third party 130 may send the data to the first terminal 110 and the second terminal 120. For example, the third party 130 may send the second type of model parameters to the first terminal 110.
The third party's storage device 130-2 may store data and/or instructions for execution by the processing device 130-1, which the processing device 130-1 may execute or use to implement the example methods of this specification. Storage devices 130-2 and 120-2 may be configured to store a first type of model parameters (e.g., model parameters corresponding to non-private data) and a second type of model parameters (e.g., model parameters corresponding to private data); associated instructions may also be stored that instruct the first terminal and the second terminal to perform operations. In some embodiments, the storage device may include mass storage, removable storage, volatile read-write memory, read-only memory (ROM), and the like, or any combination thereof.
The system in one or more embodiments of the present specification may be composed of a data receiving module and a plurality of data processing modules.
In some embodiments, in a system having the second terminal as an execution subject, the data receiving module includes a first data receiving module; the data processing module can comprise a first type accumulated data determining module, an accumulated loss value determining module and a model parameter updating module. The modules described above are implemented in a computing system as described in the application scenario, and each module includes respective instructions that may be stored on a storage medium and executed in a processor. The different modules may be located on the same device or on different devices. Data may be transferred between them via a program interface, a network, etc., and data may be read from or written to the storage device.
The first data receiving module may be configured to receive data of a first type from at least a first terminal and a second terminal.
The first type accumulated data determination module may be configured to calculate the first type accumulated data based on the received first type data.
The accumulated loss value determining module can be used for acquiring the accumulated loss value in a multi-party safety calculation mode; the cumulative loss value is determined by at least the first terminal and the second terminal based on the first type of cumulative data, the second type of data, and the sample label.
And the model parameter updating module can participate in the joint training of the models at least based on the first terminal and the second terminal to obtain a parameter updated model. In some embodiments, the model parameter update module may be further operable to: acquiring a second type of accumulated gradient in a multi-party safety calculation mode; the second type accumulated gradient is at least determined by the first terminal and the second terminal based on the loss value of the first terminal and the second type data; updating a second type of model parameters based on the second type of cumulative gradient. In some embodiments, the model parameter update module may be further operable to: calculating a first type cumulative gradient based on the cumulative loss value and first type data corresponding to at least the first terminal and the second terminal; updating a first type of model parameters based on the first type of cumulative gradient.
In some embodiments, in a system with a first terminal as an execution subject, the data receiving module includes a first data receiving module and a first type data transmission module; the data processing module can comprise a first loss value calculation module and a model parameter updating module.
The first data receiving module can be used for receiving the first type accumulated data and the second type model parameters from a third party. The first type data transmission module may be configured to transmit the first type data to a third party. The first loss value calculation module may be configured to calculate a first loss value based on the training data of the first loss value and the sample label. The model parameter updating module may be configured to participate in joint training of the first loss value based on at least models of the first terminal and the second terminal in a multi-party security computation manner, so as to obtain a parameter updated model.
It should be appreciated that the system and its modules in one or more implementations of the present description may be implemented in a variety of ways. For example, in some embodiments, the system and its modules may be implemented in hardware, software, or a combination of software and hardware. Wherein the hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory for execution by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the methods and systems described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided, for example, on a carrier medium such as a diskette, CD-or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The system and its modules of the present application may be implemented not only by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but also by software executed by various types of processors, for example, or by a combination of the above hardware circuits and software (e.g., firmware).
It should be noted that the above description of the processing device and its modules is for convenience of description only and is not intended to limit the present application to the scope of the embodiments illustrated. It will be appreciated by those skilled in the art that, given the teachings of the present system, any combination of modules or sub-system configurations may be used to connect to other modules without departing from such teachings.
FIG. 2 is an exemplary flow diagram of a method for model training based on private data, shown in accordance with some embodiments of the present description.
The variable names and formulas in the specification are only used for better understanding of the method described in the specification. In applying the present specification, based on common operation principles and machine learning principles, various immaterial transformations may be made to processes, variable names, formulas, such as transforming the order of rows or columns, transforming to an equivalent form during matrix multiplication, representing the same calculation in other calculation forms, and the like.
In this specification, convention is expressed as follows:
the training data of the joint training model comprises M data samples, each sample data comprises n-dimensional features, wherein the non-private features have p dimensions, the private features have q dimensions, and p + q is equal to n. The M sample data are held by at least two data owners. For convenience of illustration, some embodiments of the present description are described in detail with reference to three-party data owners, and A, B, C represents the first-party data owner, the second-party data owner, and the third-party data owner, respectively, and D represents the server. The first party data owner may also be referred to as a first terminal or a party a, the second party data owner may also be referred to as a second terminal or a party B, and the third party data owner may also be referred to as a third terminal or a party C.
In the description of the present specification, A, B, C, D will be described before the description, where a is a sample size Ma, B is a sample size Mb, and C is a sample size Mc, where M is Ma+Mb+Mc. First party data owner A owns MaData (X) corresponding to n-dimensional features in individual samplesa) Wherein Xa includes data (X) corresponding to the p-dimensional non-privacy featurea 0) Data (X) corresponding to said q-dimensional privacy featurea 1) (ii) a Second party data owner B owns MbData (X) corresponding to n-dimensional features in individual samplesb) Wherein Xb includes data (X) corresponding to the p-dimensional non-privacy featureb 0) Data (X) corresponding to said q-dimensional privacy featureb 1). Third party data owner C owns McData (X) corresponding to n-dimensional features in individual samplesc) Wherein Xc comprises data (X) corresponding to the p-dimensional non-privacy featurec 0) Data (X) corresponding to said q-dimensional privacy featurec 1). The server possesses first model parameters (W0) corresponding to the p-dimensional features and second model parameters (W1) corresponding to the q-dimensional features. In this specification, the model parameters may also be simply referred to as a model. The first model parameter W0 is a matrix of p rows and 1 columns, and the second model parameter W1 is a matrix of q rows and 1 columns.
In this specification, the multi-party data owner A, B, C has tags ya, yb, yc belonging to it, respectively, and M, respectivelyaRow 1 column, MbRow 1 column, McA column vector of row 1 and column. In the description, X representsaIs MaA matrix of samples, each sample being a row vector of 1 row and n columns, i.e. XaIs a MaA matrix of rows and columns, data X corresponding to the p-dimensional non-privacy featurea 0Is a MaA matrix of rows and columns, data X corresponding to said q-dimensional privacy featurea 1Is a MaA matrix of rows and q columns. Corresponding to, XbIs a MbMatrix of rows and columns, Xb 0Is a MbMatrix of rows and columns, Xb 1Is a MbA matrix of rows and q columns; xcIs a McMatrix of rows and columns, Xc 0Is a McMatrix of rows and columns, Xc 1Is a McA matrix of rows and q columns.
In this specification, for the sake of simplicity, a data column with a constant value of 1 added to sample data in linear regression or logistic regression calculation and a constant 1 added to a label are not specifically described, and n +1 in matrix calculation are not distinguished. This simplification has no substantial effect on the process described in this specification.
In some embodiments, server D holds co-trained model parameters. In the present embodiment, the jointly trained model parameters include model parameters W0 corresponding to non-privacy features and model parameters W1 corresponding to privacy features. As one of the explanations, in one or more embodiments of the present specification, the model parameter of the privacy feature, the model parameter of the privacy data, the model parameter of the privacy feature data, and the privacy model parameter may be understood as meaning, that is, the model parameter corresponding to the privacy data.
The above-identified expressions, variable names, and formulas and other expressions presented in this specification are only intended to provide a better understanding of the methods described in this specification. When the method is applied, various insubstantial transformations can be made on representation methods, variable names, formulas, calculation methods and the like based on common operation principles, technical principles and technical methods without influencing the essence and the corresponding technical effect of the representation methods, the variable names, the formulas, the calculation methods and the like. Such as, but not limited to, transposing the order of rows or columns, transforming to an equivalent form during matrix multiplication, representing the same calculation in other forms of calculation, etc.
At step 210, the server initializes model parameters.
Since the parameter update of the model is performed on the server D, the server D holds the model parameters, including non-privacy model parameters W0 corresponding to non-privacy data and privacy model parameters W1 corresponding to privacy data. The server initializes W0 and W1 respectively and prepares for model joint training.
The data owners A, B, C respectively have the non-private data Xa 0、Xb 0、Xc 0Sent to server D over network 140.
In step 230, the server calculates the cumulative value of non-private data and sends it to A, B, C together with the model parameters of the private data.
The non-privacy data integrated value Z0 is a calculation result calculated based on the non-privacy features and the non-privacy model parameters of each party, and in the representation given in this specification, the server calculates Z0 ═ X0 × W0. Where X0 corresponds to the result of the integration of the non-private data of the parties. The integration may be understood as a matrix concatenation operation of multiple non-privacy features. In some embodiments, the third party needs to tag the serial number and source of the sample in the first type of data at the time of integration. In some embodiments, the indicia may also be provided by the data provider, i.e., the first terminal and the second terminal.
In particular, the server may be on non-private data Xa 0、Xb 0、Xc 0The integration is performed to form a non-private data set X0. Due to non-private data Xa 0、Xb 0、Xc 0Are respectively MaMatrix of rows and columns, MbMatrix of rows and columns, McMatrix of p rows and columns, pair Xa 0、Xb 0、Xc 0The matrix concatenation is performed on the basis that the number of columns is not changed and the number of rows is increased, so that the matrix concatenation result X0 is a matrix with M rows and p columns, and Ma + Mb + Mc is M.
Based on the non-private data set X0 and the first model parameter W0, the server obtains a non-private data accumulated value Z0 ═ X0 × W0 through calculation, and thus the calculated Z0 is a matrix of M rows and 1 columns.
After the computation is completed, the server sends the non-private data rollup value Z0 and the latest second model parameters W1 to the data owner A, B, C.
Steps 240, A, B, C respectively calculate the loss value and the corresponding gradient value of the privacy data.
The data owner A, B, C may be based on accumulated non-private data Z0 and respective private data Xa 1、Xb 1、Xc 1Respective loss values are calculated.
The data owner A, B, C is based on the newly obtained second model parameters W1 and the privacy characteristics X owned by each digital model ownera 1、Xb 1、Xc 1Obtaining a first calculation result Z by product calculationa 1、Zb 1、Zc 1。
In the expression agreed in the present specification, the first calculation result Za 1＝Xa 1*W1，Zb 1＝Xb 1*W1；Zc 1＝Xc 1W1. Z thus obtaineda 1、Zb 1、Zc 1Are each MaRow 1 column, MbRow 1 column, McRow 1 column matrix.
The data owner A, B, C extracts the part of the computation results of the non-private data participating in the computation in the non-private data accumulated value Z0 based on the non-private data accumulated value Z0 received from the server D, and performs matrix summation of the extracted results and the first computation result to obtain a second computation result Za、Zb、Zc. The extraction means that the data owner A, B, C can obtain a part of Z0 of the respective non-private data participating in the operation. In some embodiments, the extraction process may be implemented in dependence on the server marking the sample as integrated in step 230.
The non-private data accumulated value Z0 is a matrix of M rows and 1 columns (i.e., (M is)a+Mb+Mc) A matrix of row 1 and column), the data owner A, B, C may extract the portion of the non-private data cumulative value Z0 that corresponds to the non-private data of each data owner for calculation. The portion extracted by the data owner A, B, C is represented by Z0a, Z0b and Z0c, wherein Z0a, Z0b and Z0c are respectively MaMatrix of rows and columns, MbMatrix of rows and columns, McRow 1 column matrix.
In the expression agreed in the present specification, the second calculation result Za＝Z0a+Za1；Zb＝Z0b+Zb1；Zc＝Z0c+Zc1. Z thus obtainedaIs a MaRow 1 column matrix. By the same token, can obtain ZbIs a MbMatrix of rows and columns, ZcIs a McRow 1 column matrix.
The data owner A, B, C converts the second calculation result Za、Zb、ZcSubstituting into sigmoid function to calculate A, B, C respective loss value
In the convention in this specification,
based on the respective labels ya, yb, yc of the data owner A, B, C, the respective loss values can be calculated
The data owner A, B, C may also be based on respective loss values da, db, dc and respective owned private data Xa 1、Xb 1、Xc 1Calculating gradient values G corresponding to respective private dataa 1、Gb 1、Gc 1。
Substituting the data owner A, B, C into the gradient calculation formula using the respective owned private data:
respectively obtain gradients G corresponding to the data owners A, B, C based on the private dataa 1、Gb 1、Gc 1。
In step 250, the server determines the cumulative privacy gradient value and the cumulative loss value through a multi-party security calculation mode.
The server obtains a cumulative privacy gradient value SW1 corresponding to the multi-party privacy data based on the multi-party security summation. The cumulative privacy gradient value SW1 corresponding to the multi-party privacy data is the summary result of the gradient values corresponding to the respective privacy features of the data owner A, B, C. The summary can be understood as operations such as vector summation or averaging of the respective privacy gradient values to reflect a common trend of multi-party data analysis. In the convention of this specification, the formula for calculation of SW1 is understood to mean that SW1 is equal to the gradient value G corresponding to private dataa 1、Gb 1、Gc 1The results are summarized.
The server obtains a cumulative loss value SE1 corresponding to the multi-party privacy data based on the multi-party security summation. The cumulative loss value SE1 corresponding to the multi-party data is the result of summing the respective loss values da, db, dc of the data owner A, B, C. The summary is understood with reference to the above description.
In some embodiments, the third party may obtain A, B, C the accumulated loss value and the accumulated privacy gradient value corresponding to the third party by way of a multi-party security calculation.
In step 260, the server updates the model parameters of the privacy data based on the accumulated privacy gradient values.
And the server performs product operation on the accumulated privacy gradient value corresponding to the multi-party privacy data and the leaving rate, and updates the privacy model parameter W1.
In the presentation as agreed upon in the present specification,
the server calculates update W1 — W1-learning _ rate × SW 1. In the present specification, the learning _ rate denotes a parameter that affects the magnitude of the decrease in the gradient descent method.
In step 270, the server calculates the cumulative non-privacy gradient value based on the cumulative loss value corresponding to the multi-party data, and updates the model parameters of the non-privacy data.
The server calculates a gradient value SW0 corresponding to the multi-party non-private data based on the cumulative loss value SE1 corresponding to the multi-party data.
And the server performs product operation on the non-privacy gradient value corresponding to the multi-party non-privacy data and the leaving rate, and updates the non-privacy model parameter W0.
In the representation agreed upon in this specification, the server computation SW0 ═ XTSE 1. The server calculates update W0 — W0-learning _ rate × SW 0.
The above process illustrates a gradient descent process, and steps 230 to 270 may be iterated until convergence, completing the training of the model. In this process, no more than one party can know the data held by any other party.
Fig. 3 is an exemplary flow diagram of a method of processing dialog information, shown in accordance with some embodiments of the present description.
In some embodiments, one or more steps of method 300 may be implemented in system 100 shown in FIG. 1. For example, one or more steps of method 300 may be stored as instructions in storage device 120 and invoked and/or executed by processing device 110.
At step 310, a third party receives data of a first type from at least a first terminal and a second terminal. In some embodiments, step 310 may be performed by a first type of data receiving module.
In some embodiments, the third party may be a terminal processing device or a server. The terminal processing device comprises a processor and a storage device, such as an iPad, a desktop computer, a notebook computer and the like. In some embodiments, the first terminal may be the data owner a depicted in part in fig. 2, and the second terminal may be the data owner B depicted in part in fig. 2. A more detailed understanding of the first terminal, the second terminal, and the third party may be found elsewhere in the specification.
In some embodiments, the first terminal holds first training data and corresponding sample labels, the first training data including a first class of data and a second class of data. The second terminal holds second training data and corresponding sample labels. The first training data and the second training data correspond to the same features of different samples. For example, the first training data may have MAA plurality of samples, each sample having n features; the second training data may have MBSamples, each sample having the same n features.
In some embodiments, at least one of the first training data and the second training data comprises a first class of data and a second class of data. The first type of data and the second type of data correspond to different privacy levels. In some embodiments, the second type of data has a higher privacy level than the first type of data, for example, the first type of data belongs to non-private data (e.g., hometown, city, etc. of the user in the training sample), and the second type of data belongs to private data (e.g., income, gender, income level, etc. of the user in the training sample), as will be explained with reference to fig. 2. In some embodiments, all terminal data includes private and non-private features; in some embodiments, some of the terminals include non-privacy features and privacy features, some of the terminals include only privacy features and no non-privacy features, and some of the terminals include only non-privacy features and no privacy features.
The third party receives at least the first type of data from the first terminal and the second terminal, as described in detail with reference to step 220 of fig. 2.
In some embodiments, the sample data held by the data owner may be user attribute information in at least one of insurance, banking, and medical fields. For example, a bank has identity information, running information, credit investigation information and the like of the bank client; the insurance company has the client identity information, historical insurance purchasing information, historical claim settlement information, health information, vehicle condition information and the like of the company; the medical institution has patient identity information, historical medical records and the like of the institution. In some embodiments, the user attribute information includes an image, text, or voice, among others.
In some embodiments, the model owned by the data owner may make predictions based on characteristics of the sample data. For example, a bank may predict the annual deposit growth rate of the bank based on characteristics of data such as quarterly user growth, increased user identity, and bank addition policy. In some embodiments, the model may also be used to confirm identity information of the user, which may include, but is not limited to, a credit rating for the user.
In some embodiments, the private data (e.g., the first private data and the second private data) in one or more embodiments of the present description may include private data related to the entity. In some embodiments, the entity may be understood as a subject of the visualization, which may include, but is not limited to, a user, a merchant, and the like. In some embodiments, the privacy data may include image data, text data, or sound data. For example, the image data in the privacy data may be a face image of the user, a logo image of the merchant, a two-dimensional code image capable of reflecting information of the user or the merchant, and the like. For example, the text data in the privacy data may be text data of the gender, age, academic calendar, income and the like of the user, or text data of the type of commodity traded by the merchant, the time of commodity trading by the merchant and the price interval of the commodity and the like. For example, the sound data of the privacy data may be related voice content containing user personal information or user feedback, and the corresponding user personal information or user feedback information may be obtained by parsing the voice content.
In some embodiments, a third party holds first class model parameters W1 and second class model parameters W2 of a joint training model, the first class model parameters and the second class model parameters corresponding to different features. Wherein the first type model data parameters correspond to the first type data X0 of the joint training model; the second type of model data parameters correspond to the second type of data X1 of the joint training model.
In some embodiments, a third party initializes the first type model parameters W1 and the second type model parameters W2 when new first type data is obtained.
At step 320, the third party calculates a first type of accumulated data based on the received first type of data. In some embodiments, step 320 may be performed by a first type cumulative data determination module.
In some embodiments, the third party may calculate the first type cumulative data Z0 based on the received first type data. For an understanding of Z0 and the specific calculation process, see step 230 of FIG. 2.
In step 330, the third party transmits the first type accumulated data and the second type model parameters at least to the first terminal and the second terminal.
In some embodiments, the third party may transmit the first type accumulated data and the second type model parameters to at least the first terminal and the second terminal, so that the first terminal and the second terminal perform subsequent calculations based on the latest first type accumulated data and the latest second type model parameters. The specific process can be seen in step 230 of fig. 2.
In some embodiments, step 340 may be performed by the cumulative loss value determination module.
In some embodiments, in order to ensure the security of the data, the loss value may be calculated by sharing or secret sharing, and the calculation result corresponding to the private data or the private data of each data party is involved in calculation to obtain the accumulated loss value corresponding to each data party without leakage.
In some embodiments, the loss value may be used to reflect the gap between the training model prediction value and the sample data truth. In some embodiments, the loss value may reflect a difference between the default value and the actual value by participating in the calculation. The related operation formulas of different training models are different, and the operation formulas corresponding to different parameter optimization algorithms are also different when the training models are the same. For example, in the embodiment shown in FIG. 2 of the present specification, the loss value is calculated by the formula
However, the operation formula for determining the loss value is not limited in one or more embodiments of the present disclosure. Some embodiments of this step are described in detail below, taking the first terminal as an example.
In some embodiments, the first terminal may calculate a first loss value of the joint training model, for example, da in fig. 2, based on the first-class accumulated data and the second-class model parameters, and the sample label y corresponding to the first terminal. The third party obtains the accumulated loss value SE1 summarized by the loss values da, db, and dc of each data terminal in a secure calculation manner, and the specific calculation process is described in detail in step 240 in fig. 2.
In some embodiments, the jointly trained model may comprise a linear regression model; logistic regression models may also be included.
In some embodiments, when the jointly trained model comprises a logistic regression model, a sigmoid function is required to calculate the loss value d, as described in detail with reference to step 240 of fig. 2.
In some embodiments, when the jointly trained model is a linear regression model, a linear function may be used to calculate the predicted value
Specifically, taking the linear function y as wx + b as an example, a formula for calculating the self-loss value of each data party is obtained:
the method by which each party calculates the respective loss value according to the above formula may refer to step 240 of fig. 2.
And 350, the third party participates the acquired accumulated loss value in the joint training of the models at least based on the first terminal and the second terminal to obtain a model with updated parameters. In some embodiments, step 350 may be performed by the model parameter update module.
In some embodiments, after the third party obtains the accumulated loss value SE1, a first type accumulated gradient SW0 corresponding to the first type data or non-private data on each data terminal may be determined according to the accumulated loss value SE1 and the first type accumulated data X0, and then the model parameter W0 of the non-private data may be updated based on the first type accumulated gradient SW 0. The detailed process is shown in step 270 of fig. 2.
The updating of the privacy data model parameters W1 corresponding to the privacy data of each data terminal may adopt the following method: and the third party acquires the second type of accumulated gradient corresponding to each data terminal in a safe calculation mode, and then updates the corresponding model parameter W1 based on the second type of accumulated gradient.
And the second type of accumulated gradient is the summary of gradient values corresponding to the privacy data of each data terminal based on the data terminal. For example, the second type of accumulated gradients SW1 are generated by each data terminal A, B, C based on the gradient value G corresponding to each private dataa 1、Gb 1、Gc 1The results are summarized. Second oneThe calculation process of the class accumulation gradient SW1 and the updating process of the privacy model parameters W1 can be seen in steps 240 and 250 of FIG. 2.
In some embodiments, a gradient descent method may be used to obtain a model that completes one parameter update. Specifically, the obtained encryption loss value can be calculated to obtain an encryption gradient value to participate in model training, and the above process is repeated until the iteration number reaches a predefined iteration number upper limit value or an error obtained by calculation after substituting the loss value is smaller than a predefined numerical value, so that a model with updated parameters is obtained.
In other embodiments, other parameter optimization methods may be used instead of the gradient descent method, such as newton descent method, and the like, and one or more embodiments of the present disclosure are not limited thereto.
It should be noted that the above description related to the flow 300 is only for illustration and explanation, and does not limit the applicable scope of the present application. Various modifications and changes to flow 300 will be apparent to those skilled in the art in light of this disclosure. However, such modifications and variations are intended to be within the scope of the present application.
The beneficial effects that may be brought by the embodiments of the present application include, but are not limited to: (1) multi-party data combined training is adopted, so that the utilization rate of data is improved, and the accuracy of a prediction model is improved; (2) in the optional privacy data protection scene, the privacy data and the non-privacy data are respectively and correspondingly processed according to different requirements, so that the security of the privacy data in the training data is improved; (3) in the selectable privacy data protection scene, only the data needing privacy protection in the training data is subjected to security processing, so that the computational efficiency of security calculation is improved. (4) In the optional privacy data protection scene, a third party is introduced to summarize non-privacy data and update model parameters, so that data transmission is not generated by each data party, and the security of multi-party data cooperation is improved. It is to be noted that different embodiments may produce different advantages, and in different embodiments, any one or combination of the above advantages may be produced, or any other advantages may be obtained.
Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing detailed disclosure is to be considered merely illustrative and not restrictive of the broad application. Various modifications, improvements and adaptations to the present application may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the present application and thus fall within the spirit and scope of the exemplary embodiments of the present application.
Also, this application uses specific language to describe embodiments of the application. Reference to "one embodiment," "an embodiment," and/or "some embodiments" means a feature, structure, or characteristic described in connection with at least one embodiment of the application. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the present application may be combined as appropriate.
Moreover, those skilled in the art will appreciate that aspects of the present application may be illustrated and described in terms of several patentable species or situations, including any new and useful combination of processes, machines, manufacture, or materials, or any new and useful improvement thereon. Accordingly, various aspects of the present application may be embodied entirely in hardware, entirely in software (including firmware, resident software, micro-code, etc.) or in a combination of hardware and software. The above hardware or software may be referred to as "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the present application may be represented as a computer product, including computer readable program code, embodied in one or more computer readable media.
The computer storage medium may comprise a propagated data signal with the computer program code embodied therewith, for example, on baseband or as part of a carrier wave. The propagated signal may take any of a variety of forms, including electromagnetic, optical, etc., or any suitable combination. A computer storage medium may be any computer-readable medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code located on a computer storage medium may be propagated over any suitable medium, including radio, cable, fiber optic cable, RF, or the like, or any combination of the preceding.
Computer program code required for the operation of various portions of the present application may be written in any one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C + +, C #, VB.NET, Python, and the like, a conventional programming language such as C, Visualbasic, Fortran2003, Perl, COBOL2002, PHP, ABAP, a dynamic programming language such as Python, Ruby, and Groovy, or other programming languages, and the like. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or processing device. In the latter scenario, the remote computer may be connected to the user's computer through any form of network, such as a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet), or in a cloud computing environment, or as a service using, for example, software as a service (SaaS).
Additionally, the order in which elements and sequences of the processes described herein are processed, the use of alphanumeric characters, or the use of other designations, is not intended to limit the order of the processes and methods described herein, unless explicitly claimed. While various presently contemplated embodiments of the invention have been discussed in the foregoing disclosure by way of example, it is to be understood that such detail is solely for that purpose and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover all modifications and equivalent arrangements that are within the spirit and scope of the embodiments herein. For example, although the system components described above may be implemented by hardware devices, they may also be implemented by software-only solutions, such as installing the described system on an existing processing device or mobile device.
Similarly, it should be noted that in the preceding description of embodiments of the application, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the embodiments. This method of disclosure, however, is not intended to require more features than are expressly recited in the claims. Indeed, the embodiments may be characterized as having less than all of the features of a single embodiment disclosed above.
Numerals describing the number of components, attributes, etc. are used in some embodiments, it being understood that such numerals used in the description of the embodiments are modified in some instances by the use of the modifier "about", "approximately" or "substantially". Unless otherwise indicated, "about", "approximately" or "substantially" indicates that the number allows a variation of ± 20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximations that may vary depending upon the desired properties of the individual embodiments. In some embodiments, the numerical parameter should take into account the specified significant digits and employ a general digit preserving approach. Notwithstanding that the numerical ranges and parameters setting forth the broad scope of the range are approximations, in the specific examples, such numerical values are set forth as precisely as possible within the scope of the application.
The entire contents of each patent, patent application publication, and other material cited in this application, such as articles, books, specifications, publications, documents, and the like, are hereby incorporated by reference into this application. Except where the application history document is inconsistent or conflicting with the present application as to the extent of the present claims, which are now or later appended to this application. It is noted that the descriptions, definitions and/or use of terms in this application shall control if they are inconsistent or contrary to the statements and/or uses of the present application in the material attached to this application.
Finally, it should be understood that the embodiments described herein are merely illustrative of the principles of embodiments of the present application. Other variations are also possible within the scope of the present application. Thus, by way of example, and not limitation, alternative configurations of the embodiments of the present application may be viewed as being consistent with the teachings of the present application. Accordingly, the embodiments of the present application are not limited to only those embodiments explicitly described and depicted herein.
Claims (20)
1. A method for model training based on selectable privacy data; the method comprises the following steps:
a third party receives data of a first type at least from a first terminal and a second terminal; calculating first-class accumulated data based on the received first-class data and the corresponding model parameters;
a third party acquires an accumulated loss value in a multi-party security calculation mode; the accumulated loss value is determined at least by the first terminal and the second terminal based on the first type of accumulated data, the second type of data and the sample label;
the third party participates the accumulated loss value in the calculation of a first type of accumulated gradient and a second type of accumulated gradient, and the first type of accumulated gradient and the second type of accumulated gradient are used for updating parameters of a joint training model at least based on the first terminal and the second terminal;
the first terminal and the second terminal respectively hold first training data and second training data, and the first training data and the second training data correspond to the same characteristics of different samples;
at least one of the first training data and the second training data comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same training samples.
2. The method of claim 1, the jointly trained model comprising a linear regression model or a logistic regression model.
3. The method of claim 1, wherein the third party participating in the calculation of the first type of cumulative gradient and the second type of cumulative gradient in the cumulative loss value comprises:
the third party obtains a second type of accumulated gradient in a multi-party safe calculation mode; the second type accumulated gradient is at least determined by the first terminal and the second terminal based on the loss value of the first terminal and the second type data;
and the third party updates the second type of model parameters based on the second type of accumulated gradient.
4. The method of claim 3, wherein the third party participating in the calculation of the first type of cumulative gradient and the second type of cumulative gradient with the cumulative loss value comprises:
the third party calculates a first type accumulated gradient based on the accumulated loss value and first type data at least corresponding to the first terminal and the second terminal;
the third party updates the first type model parameters based on the first type cumulative gradient.
5. The method of claim 1, the first training data and the second training data comprising image data, text data, or sound data related to an entity.
6. The method of claim 1, the manner of multi-party secure computation comprising and sharing.
7. A system for model training based on selectable privacy data; the system comprises:
the first-class data receiving module is used for receiving first-class data at least from a first terminal and a second terminal;
the first-class accumulated data determining module is used for calculating first-class accumulated data based on the received first-class data and the corresponding model parameters thereof;
the accumulated loss value determining module is used for acquiring an accumulated loss value in a multi-party safety calculation mode; the accumulated loss value is determined at least by the first terminal and the second terminal based on the first type of accumulated data, the second type of data and the sample label;
the model parameter updating module is used for enabling the accumulated loss value to participate in calculation of a first type of accumulated gradient and a second type of accumulated gradient, and the first type of accumulated gradient and the second type of accumulated gradient are used for updating parameters of a joint training model at least based on models of the first terminal and the second terminal;
the method comprises the steps that a first terminal and a second terminal respectively hold first training data and second training data, and the first training data and the second training data correspond to the same characteristics of different samples;
at least one of the first training data and the second training data comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same training samples.
8. The system of claim 7, the jointly trained model comprising a linear regression model or a logistic regression model.
9. The system of claim 7, the model parameter update module further to:
acquiring a second type of accumulated gradient in a multi-party safety calculation mode; the second type accumulated gradient is at least determined by the first terminal and the second terminal based on the loss value of the first terminal and the second type data;
updating a second type of model parameters based on the second type of cumulative gradient.
10. The system of claim 9, the model parameter update module further to:
calculating a first type cumulative gradient based on the cumulative loss value and first type data corresponding to at least the first terminal and the second terminal;
updating a first type of model parameters based on the first type of cumulative gradient.
11. The system of claim 7, the first training data and the second training data comprising image data, text data, or sound data related to an entity.
12. The system of claim 7, the manner of multi-party secure computing comprising and sharing.
13. An apparatus for model training based on selectable privacy data, the apparatus comprising a processor and a memory; the memory is used for storing instructions, and the processor is used for executing the instructions to realize the corresponding operation of the model training method based on the optional privacy data according to any one of claims 1 to 6.
14. A method for model training based on selectable privacy data; the method comprises the following steps:
the first terminal transmits the first type of data to a third party;
the method comprises the steps that a first terminal receives first-class accumulated data and second-class model parameters from a third party;
the first terminal calculates a first loss value based on self training data and a sample label;
the first terminal participates in the joint training of the first loss value at least based on the models of the first terminal and the second terminal in a multi-party safety calculation mode to obtain a model with updated parameters;
the training data of the first terminal comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to different characteristics of the same sample.
15. The method of claim 14, wherein the first terminal participates in joint training of the first loss value based on at least models of the first terminal and the second terminal by way of multi-party security computation, and obtaining a model with updated parameters comprises:
the first terminal participates the first loss value in calculating the accumulated loss value in a multi-party safety calculation mode; the cumulative loss value corresponds to at least the training data of the first terminal and the second terminal themselves and the sample label.
16. The method of claim 14, wherein the first terminal participates in joint training of the first loss value based on at least models of the first terminal and the second terminal by way of multi-party security computation, and obtaining a model with updated parameters comprises:
the first terminal calculates the second gradient of the first terminal based on the second data of the first terminal and the first loss value;
the first terminal participates the second type gradient of the first terminal in calculating a second type accumulated gradient in a multi-party safe calculation mode; the second type cumulative gradient corresponds to at least the loss values of the first terminal and the second type data.
17. A system for model training based on selectable privacy data; the system comprises:
the first-class data transmission module is used for transmitting the first-class data to a third party;
the first data receiving module is used for receiving the first type accumulated data and the second type model parameters from a third party;
the first loss value calculating module is used for calculating a first loss value based on self training data and the sample label;
the model parameter updating module is used for participating the first loss value in joint training at least based on models of the first terminal and the second terminal in a multi-party safety calculation mode to obtain a parameter updating model;
the training data of the first terminal comprises first class data and second class data corresponding to different privacy levels; the first type of data and the second type of data correspond to the same characteristics of different samples.
18. The system of claim 17, the model parameter update module further to:
participating the first loss value in calculating a cumulative loss value in a multi-party safety calculation mode; the cumulative loss value corresponds to at least the training data of the first terminal and the second terminal themselves and the sample label.
19. The system of claim 17, the model parameter update module further to:
calculating the second type gradient of the self based on the second type data of the self and the first loss value;
the second type gradient of the self is participated in calculating the second type accumulated gradient in a multi-party safe calculation mode; the second type cumulative gradient corresponds to at least the loss values of the first terminal and the second type data.
20. An apparatus for model training based on selectable privacy data, the apparatus comprising a processor and a memory; the memory is used for storing instructions, and the processor is used for executing the instructions to realize the corresponding operation of the model training method based on the optional privacy data according to any one of claims 14 to 16.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329551.1A CN111062492B (en) | 2019-12-20 | 2019-12-20 | Method and system for model training based on optional private data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911329551.1A CN111062492B (en) | 2019-12-20 | 2019-12-20 | Method and system for model training based on optional private data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111062492A CN111062492A (en) | 2020-04-24 |
| CN111062492B true CN111062492B (en) | 2022-05-17 |
Family
ID=70302636
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911329551.1A Active CN111062492B (en) | 2019-12-20 | 2019-12-20 | Method and system for model training based on optional private data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111062492B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112084520B (en) * | 2020-09-18 | 2021-03-23 | 支付宝(杭州)信息技术有限公司 | Method and device for protecting business prediction model of data privacy through joint training of two parties |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681749A (en) * | 2018-05-21 | 2018-10-19 | 中国科学院计算技术研究所 | Privacy information discriminating method based on network social intercourse platform |
| CN110135193A (en) * | 2019-05-15 | 2019-08-16 | 广东工业大学 | A kind of data desensitization method, device, equipment and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109388661B (en) * | 2017-08-02 | 2020-04-21 | 创新先进技术有限公司 | Model training method and device based on shared data |
| US11475350B2 (en) * | 2018-01-22 | 2022-10-18 | Google Llc | Training user-level differentially private machine-learned models |
-
2019
- 2019-12-20 CN CN201911329551.1A patent/CN111062492B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681749A (en) * | 2018-05-21 | 2018-10-19 | 中国科学院计算技术研究所 | Privacy information discriminating method based on network social intercourse platform |
| CN110135193A (en) * | 2019-05-15 | 2019-08-16 | 广东工业大学 | A kind of data desensitization method, device, equipment and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 面向数据共享交换的联邦学习技术发展综述;王亚珅;《无人系统技术》;20191115(第06期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111062492A (en) | 2020-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111125735B (en) | Method and system for model training based on private data | |
| CN111178547B (en) | Method and system for model training based on private data | |
| US11636136B2 (en) | Method and system for self-aggregation of personal data and control thereof | |
| CN111931216B (en) | Method and system for obtaining joint training model based on privacy protection | |
| CN111143878B (en) | Method and system for model training based on private data | |
| CN111931950B (en) | Method and system for updating model parameters based on federal learning | |
| US20210042645A1 (en) | Tensor Exchange for Federated Cloud Learning | |
| CN111143894B (en) | Method and system for improving safe multi-party computing efficiency | |
| CN113011587B (en) | Privacy protection model training method and system | |
| CN111460528B (en) | Multi-party combined training method and system based on Adam optimization algorithm | |
| CN110910041A (en) | Risk control method, system and device | |
| CN113032835A (en) | Privacy protection model training method, system and device | |
| Treleaven et al. | Federated learning: The pioneering distributed machine learning and privacy-preserving data technology | |
| CN111079947B (en) | Method and system for model training based on optional private data | |
| CN111062492B (en) | Method and system for model training based on optional private data | |
| Dubey et al. | Smart Education based on Blockchain Technology | |
| Edijala et al. | Applications of artificial intelligence in public procurement—case study of Nigeria | |
| Jaroszkowski et al. | Valuation of European options under an uncertain market price of volatility risk | |
| CN111931947A (en) | Training sample recombination method and system for distributed model training | |
| Fischer et al. | pTAS distributions with application to risk management | |
| Wu et al. | The total return swap pricing model under fuzzy random environments | |
| Ko et al. | A study on intention of use e-procurement systems for public institutions user | |
| Borisov | Regime tracking in markets with Markov switching | |
| İncegil et al. | The New Era: Transforming Healthcare Quality with Artificial Intelligence | |
| Houser et al. | Convergence of sums of dependent Bernoulli random variables: an application from portfolio theory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40028132 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |