CN117291258A - Neural network training reasoning method and system based on function secret sharing - Google Patents

Neural network training reasoning method and system based on function secret sharing Download PDF

Info

Publication number
CN117291258A
CN117291258A CN202311340908.2A CN202311340908A CN117291258A CN 117291258 A CN117291258 A CN 117291258A CN 202311340908 A CN202311340908 A CN 202311340908A CN 117291258 A CN117291258 A CN 117291258A
Authority
CN
China
Prior art keywords
model
data
client
secret sharing
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311340908.2A
Other languages
Chinese (zh)
Inventor
关志涛
刘苏漫
李轩
安宁钰
梁潇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Smart Grid Research Institute Co ltd
State Grid Corp of China SGCC
North China Electric Power University
Original Assignee
State Grid Smart Grid Research Institute Co ltd
State Grid Corp of China SGCC
North China Electric Power University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Smart Grid Research Institute Co ltd, State Grid Corp of China SGCC, North China Electric Power University filed Critical State Grid Smart Grid Research Institute Co ltd
Priority to CN202311340908.2A priority Critical patent/CN117291258A/en
Publication of CN117291258A publication Critical patent/CN117291258A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • G06N5/043—Distributed expert systems; Blackboards
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
    • H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/06—Network architectures or network communication protocols for network security for supporting key management in a packet data network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
    • H04L9/065—Encryption by serially and continuously modifying data stream elements, e.g. stream cipher systems, RC4, SEAL or A5/3
    • H04L9/0656—Pseudorandom key sequence combined element-for-element with data sequence, e.g. one-time-pad [OTP] or Vernam's cipher
    • H04L9/0662—Pseudorandom key sequence combined element-for-element with data sequence, e.g. one-time-pad [OTP] or Vernam's cipher with particular pseudorandom sequence generator
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
    • H04L9/0816—Key establishment, i.e. cryptographic processes or cryptographic protocols whereby a shared secret becomes available to two or more parties, for subsequent use
    • H04L9/085—Secret sharing or secret splitting, e.g. threshold schemes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/08—Key distribution or management, e.g. generation, sharing or updating, of cryptographic keys or passwords
    • H04L9/0861—Generation of secret information including derivation or calculation of cryptographic keys or passwords
    • H04L9/0869—Generation of secret information including derivation or calculation of cryptographic keys or passwords involving random numbers or seeds
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/40—Network security protocols
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/08—Randomization, e.g. dummy operations or using noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/46—Secure multiparty computation, e.g. millionaire problem

Abstract

The invention discloses a neural network training reasoning method and system based on function secret sharing, which decompose the training and reasoning process of machine learning into an online stage related to input and an offline stage unrelated to input by utilizing cryptographic technologies such as function secret sharing, and transfer heavy encryption calculation to the offline stage by generating related random values required by subsequent calculation during preprocessing, so that not only can all linear operations of the online stage be directly executed on secret sharing data without calling heavy encryption tools such as homomorphic encryption or frequent interaction processes, but also the communication cost of the online stage of machine learning in calculating complex nonlinear functions such as ReLU functions is greatly reduced, and the online cost of secure multi-party calculation can be transferred to the offline stage and local calculation; at the same time, expensive conversion operations when mixed use of garbled circuits and additive secret sharing or homomorphic encryption are avoided, which is also very advantageous for performance improvement.

Description

Neural network training reasoning method and system based on function secret sharing
Technical Field
The invention relates to a neural network training reasoning method and system based on function secret sharing, and belongs to the technical field of machine learning and privacy calculation.
Background
In the information age of today, machine learning has become an integral part of various fields, and data and models are used as core elements of machine learning, which play a vital role in the training and reasoning process of models. However, in practical applications, the use of data and models also causes a series of privacy problems, especially in outsourcing computing scenarios, the training and reasoning process of machine learning may lead to serious privacy leakage risks, which is a challenge to be solved. In outsourcing computing, the training and reasoning process of the model often requires that sensitive data or model parameters be sent to other parties, which may lead to the risk of leakage of personal privacy data of the user and model information of the model owner during transmission, especially in sensitive fields related to medical data, financial data and the like, as data leakage may bring about serious legal and ethical problems.
In order to solve this problem, many technical solutions have been proposed to protect privacy of machine learning processes by introducing cryptography techniques, especially homomorphic encryption and secure multiparty computing techniques.
Homomorphic encryption techniques allow computation in the encrypted state to achieve the goal of not exposing the original data and model parameters. However, the computational cost of homomorphic encryption increases dramatically with the increase in circuit depth supported by the solution, and neural network training itself is a computationally intensive task, and the use of homomorphic encryption further increases the computational burden, resulting in significantly longer training time, making it practically difficult to implement, and therefore, training and reasoning for neural network models is not considered to be implemented using homomorphic encryption techniques.
Secure multiparty computing allows multiple parties to complete collaborative computing without sharing private data, and each party can only see the final computing results without knowing the private data of the other parties. Current secure multi-party computing schemes often use garbled circuit techniques to implement private computation of nonlinear functions in neural networks, while these schemes achieve some success in terms of security, there are many drawbacks, such as high computational cost, the need to decompose nonlinear functions into binary gates when computing them in garbled circuits, and perform encryption computation on a bit-by-bit basis, which makes them very unfriendly to devices with limited computational and storage resources; a large amount of communication interaction is required in the outsourcing calculation process, which increases the overall delay and burden.
Therefore, the invention uses the Function Secret Sharing (FSS) to calculate the nonlinear activation function in the neural network so as to replace the garbled circuit technology, thereby improving the overall performance of the training and reasoning process of the neural network while ensuring the safety of the scheme.
Function secret sharing is directed to a specific class of computing functions, wherein parameter information of the functions is secret, and data input to the functions is public, and privacy protection of distributed data is achieved by applying the public input data to the secret-shared functions. The function secret sharing greatly reduces the communication cost when the complex function is calculated in the online stage by generating a related random value independent of input in the offline stage, so that the online cost of the secure multi-party calculation can be transferred to the offline stage and the local calculation. Therefore, it is needed to provide a neural network training and reasoning method based on function secret sharing, so as to reduce the online calculation and communication overhead during model training and data evaluation while guaranteeing the safety of machine learning.
Disclosure of Invention
In order to solve the problems, the invention provides a lightweight neural network training and reasoning method based on function secret sharing, so as to realize safe and low-cost machine learning training and reasoning.
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a neural network training method based on function secret sharing allows a client to delegate a calculation task to a third party for processing on the premise of protecting data privacy, and comprises an offline stage and an online stage:
offline stage:
the auxiliary party pre-generates a random value required for subsequent online calculation in an offline stage;
on-line stage:
the client prepares and preprocesses training data, and distributes the preprocessed training data to the server in a form of copying secret sharing;
the server receives the preprocessed training data, performs a training process of machine learning by using a secure multiparty calculation method, and hides a calculated intermediate result by using a random value;
after training, the server sends the trained model back to the client, and the client reconstructs the model to obtain a complete neural network model, so that the safety of the model is ensured, and the subsequent reasoning and using processes are convenient.
The client preprocesses the original training data through a data preprocessing technology so as to ensure the quality and usability of the data.
According to the neural network training method based on function secret sharing, in order to improve the overall efficiency of a privacy protection machine learning system, the training process of machine learning is divided into an offline stage independent of data and an online stage dependent on the data, and some heavy encryption operations are transferred to the offline stage, so that the huge expenditure for calculating a nonlinear function is remarkably reduced. Wherein the offline stage is used to generate preprocessing materials required by the online protocol, i.e., to generate relevant random values of various security operators (e.g., reLU functions), so as to facilitate various linear or nonlinear operation calculations using the assigned random values in the online stage.
In order to improve the performance of the model, the neural network training method based on function secret sharing further comprises the following steps: the server side evaluates the neural network model by using a verification set or a test set, and ensures that the model after outsourcing training reaches an expected performance level.
The invention mainly focuses on deep learning models such as Convolutional Neural Networks (CNNs), and the general structure of the CNNs comprises components such as a convolutional layer, a pooling layer, an activation function, a full connection layer and the like. The convolution layer is a core component of the CNN, and uses convolution operations to perform feature extraction on an input image. The convolution operation involves performing a dot product calculation of a convolution kernel matrix with a neighborhood of elements of the input data, and progressively traversing the entire input data by sliding the convolution kernel within a local region, thereby detecting various features in the image. Assuming that the sizes of the input matrix X and the convolution kernel matrix W are n×n and w×w, respectively, and U is used to represent the result of the convolution operation, the size of the result matrix U is (n-w+1) × (n-w+1), and the element calculation manner of the two-dimensional convolution can be expressed as follows:where j, k=0, …, n-w. After the convolutional layer, a nonlinear activation function is typically applied to increase the nonlinear capability of the network, e.g., the commonly used ReLU activation function f (x) =max (x, 0). Subsequently, the pooling layer is used to reduce the size of the feature map, reduce the number of parameters and the amount of computation to be learned, such as the maximum pooling operation MaxPool (x 0 ,…,x k-1 )＝max k (x 0 ,…,x k-1 ) It uses the maximum value selected from the local area as the pooling result. Finally, after a series of convolution, activation and pooling operations, the CNN will apply several fully connected layers, converting the feature map of the previous convolution and pooling layers into scores or probability distributions for certain specific categories. This transformation process can be expressed as y=wx+b, where W is the weight matrix and b is the bias vector.
On a macroscopic level, the forward propagation process of neural network model training consists of alternating linear and nonlinear operations. The linear operation comprises matrix multiplication operation of a full connection layer and convolution operation of a convolution layer, and the essence of the linear operation is the product of weight and input; the nonlinear operation comprises various activation functions and pooling operations, wherein the bottom operations of nonlinear functions such as ReLU and Maxpool are comparison operations. The back propagation process then completes the updating of the weights mainly by using the derivative of the activation function and the matrix multiplication operation.
A training process for performing machine learning using a series of secure multiparty computing techniques, comprising the following basic operations: matrix multiplication, convolution operations, reLU activation functions, maxpool functions, and the like. For matrix multiplication and convolution operations, secure multi-party computing utilizes the property of 2-out-of-3 duplicate secret sharing to implement secure multiplication computation. The participants divide the data into multiple shares, each held by a different participant, through a secret sharing protocol. Wherein, the auxiliary party uses a pseudo random number generator (PRG) to generate random values, and the weight matrix or convolution kernel is hidden, thereby realizing the multiplication calculation process of privacy protection. For nonlinear functions such as ReLU and Maxpool and derivatives thereof, the secure multiparty calculation utilizes a function secret sharing technology to realize privacy comparison operation. The auxiliary party generates an FSS key and a related random value in an offline stage, utilizes the random value to hide input data, and then uses an FSS evaluation function in an online stage to realize safe comparison calculation.
The server side provides computing services by three computing servers, namely P 0 、P 1 、P 2 The client divides the preprocessed training data into three shares, and distributes the three shares to three stations in the form of copying secret sharingA server to protect sensitive private information during subsequent transmission and processing of the data; and assuming that the computing servers execute a model training process of machine learning in a non-collusion mode according to the data of the data holder, after training is finished, the three computing servers send the trained model back to the client, and the client locally holds the model to ensure the control right of the original data and prevent privacy leakage.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
A neural network training system based on functional secret sharing, comprising: the system comprises a client, a server and an auxiliary party;
The client, namely the data owner, is an individual, organization or entity with specific data sets, the specific data sets can be used for training a machine learning model and used for preparing and preprocessing training data, the preprocessed training data is distributed to a server training model in a copying secret sharing mode, and after training is finished, the client receives the trained model sent back by the server and carries out reconstruction to obtain a complete neural network model;
the server side, namely the computing server, is responsible for receiving the preprocessed training data distributed by the client side in the form of copying secret sharing, executing the training process of machine learning by utilizing a secure multiparty computing method, hiding the computed intermediate result by utilizing a random value, and after training, sending the trained model back to the client side by the server side;
and the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe mode. The secondary party is assumed to be trusted and may be an entity, such as a computing node.
The client may be composed of N entities, and in the model training process, it is responsible for preparing training data, including features and labels, and when performing model training, each client needs to distribute its own private data to three computing servers in the form of duplicate secret sharing. It should be noted that the data owners may act as computing servers, but for ease of illustration, the present invention describes them separately.
Auxiliary party, i.e. P 3 The distribution of the random value may be accomplished through encrypted communications or other secure communication mechanisms to prevent the random value from being compromised or tampered with during delivery.
The server side is also used to evaluate the neural network model using a validation set or test set in order to ensure that the outsourced trained model reaches the desired level of performance.
The server side provides computing services by three computing servers, namely P 0 、P 1 、P 2 The client divides the preprocessed training data into three shares and distributes the shares to three servers in a form of copying secret sharing so as to protect sensitive private information in the process of transmitting and processing the data later; and assuming that the computing servers execute a model training process of machine learning in a non-collusion mode according to the data of the data holder, after training is finished, the three computing servers send the trained model back to the client, and the client locally holds the model to ensure the control right of the original data and prevent privacy leakage.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
A neural network reasoning method based on function secret sharing is similar to a model training process, the machine learning reasoning process is divided into an online stage related to data and an offline stage unrelated to the data, and some preprocessing processes are carried out in the offline stage, so that the overall efficiency of the privacy protection machine learning reasoning process is improved. Wherein the off-line phase is used to generate the preprocessing material required by the on-line protocol, i.e. to generate the relevant random values of the various security protocols, so that the on-line phase can perform various linear or nonlinear operations by using the assigned random values.
The invention mainly focuses on deep learning models such as convolutional neural networks, on a macroscopic level, the neural network model reasoning process is alternately composed of linear operation and nonlinear operation, the linear operation comprises matrix multiplication operation of a full-connection layer and convolutional operation of a convolutional layer, the essence of the linear operation is the product of weight and activation value, and the nonlinear operation comprises various activation functions such as ReLU and the like and Maxpool functions and the like.
A neural network reasoning method based on function secret sharing allows a client to obtain a prediction result of a model without exposing sensitive data and/or model details, and comprises an offline stage and an online stage:
Offline stage:
the auxiliary party pre-generates a random value required for subsequent online calculation in an offline stage;
on-line stage:
the client side prepares data needing to be subjected to reasoning prediction, a model owner selects a machine learning model suitable for a task, prepares model parameters, and deploys the machine learning model to the environment of the server side;
the client distributes the data to be subjected to reasoning prediction to the server in a form of copying secret sharing so as to protect sensitive information of a user in the process of transmitting and processing the data;
the server side uses a secure multiparty calculation method to infer the received data and uses a random value to hide the calculated intermediate result;
and the server returns the obtained reasoning result share secret to the client, and the client carries out reconstruction decryption on the reasoning result, so that a final reasoning result is obtained.
The server side is lifted by three computing serversFor computing services, i.e. P 0 、P 1 、P 2 The client side divides data needing to be subjected to reasoning prediction into three shares, and distributes the three shares to three servers in a copying secret sharing mode so as to protect sensitive private information in the process of subsequently transmitting and processing the data; after the reasoning is finished, the three computing servers return the obtained reasoning result share secret to the client, and the client locally stores the reasoning result share secret so as to ensure the control right on the original data and prevent privacy disclosure.
The server side provides an environment for reasoning calculation and comprises a server, hardware equipment and the like. It should be noted that the client and model owner may act as a computing server, but for ease of illustration, the invention is described separately.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
The data is inferred by a secure multiparty computing method, which comprises the following basic operations: matrix multiplication and convolution operations are implemented using the feature of 2-out-of-3 duplicate secret sharing, with random values generated by the secondary party using a pseudo-random number generator (PRG) to effect concealment of the multiplication result shares; the nonlinear activation functions such as ReLU and Maxpool are realized by using a function secret sharing technology, and the bottom layers of the nonlinear activation functions are subjected to comparison calculation and can be realized by using the FSS scheme of the distributed comparison function.
A neural network reasoning system based on functional secret sharing, comprising: the system comprises a client, a model owner, a server and an auxiliary party;
the client, namely the data owner, is an entity needing to use a model for reasoning, usually a person, an organization or an application, and is used for preparing data needing to be subjected to reasoning prediction, distributing the data needing to be subjected to reasoning prediction to the server in a form of copying secret sharing, receiving a reasoning result share returned by the secret of the server by the client after the reasoning is completed by the server, and reconstructing and decrypting the reasoning result so as to obtain a final reasoning result;
the model owner is a person, a group or an organization with a trained machine learning model, and is used for selecting the machine learning model suitable for a task, preparing model parameters, and deploying the machine learning model into a server-side environment to provide a machine learning prediction service, but not necessarily participate in specific reasoning calculation;
the server side, i.e., the computing service provider, is the entity responsible for performing the secure model inference computations, typically the cloud service provider or the outsourced computing service provider. The server side is used for receiving data which is copied by the client side and needs to be subjected to reasoning prediction, reasoning the received data by utilizing a secure multiparty calculation method, hiding the calculated intermediate result by utilizing a random value, and finally returning the obtained reasoning result share secret to the client side without exposing details of a model;
And the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe manner. The secondary party is assumed to be trusted and may be an entity, such as a computing node.
The client has the data to be input into the model, hopes to obtain the prediction result of the model through outsourcing reasoning, and does not want to share the original data. The client transmits the data that needs reasoning to the calculator, but does not directly access the model.
The model owner is responsible for the security and accuracy of the model and needs to ensure that the model is not misused or compromised.
Auxiliary party, i.e. P 3 The distribution process may be implemented by encrypted communications or other secure communication mechanisms to prevent random values from being compromised or tampered with during delivery.
The server side provides computing services by three computing servers, namely P 0 、P 1 、P 2 The client side divides data needing to be subjected to reasoning prediction into three shares, and distributes the three shares to three servers in a copying secret sharing mode so as to protect sensitive private information in the process of subsequently transmitting and processing the data; after the reasoning is finished, the three computing servers return the obtained reasoning result share secret to the client, and the client locally stores the reasoning result share secret so as to ensure the control right on the original data and prevent privacy disclosure.
The server side provides an environment for reasoning calculation and comprises a server, hardware equipment and the like. It should be noted that the client and model owner may act as a computing server, but for ease of illustration, the invention is described separately.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
The technology not mentioned in the present invention refers to the prior art.
According to the neural network training reasoning method and system based on function secret sharing, the training and reasoning process of machine learning is decomposed into an online stage related to input and an offline stage unrelated to input by utilizing cryptographic technologies such as function secret sharing, and heavy encryption calculation is transferred to the offline stage by generating related random values required by subsequent calculation during preprocessing, so that not only can all linear operations of the online stage be directly executed on secret sharing data without calling heavy encryption tools such as homomorphic encryption or carrying out frequent interaction processes, but also the communication cost of the online stage of machine learning in calculating complex nonlinear functions such as ReLU functions is greatly reduced, and the online cost of safe multi-party calculation can be transferred to the offline stage and local calculation; at the same time, the invention avoids the expensive conversion operation when mixed use of a garbled circuit and addition secret sharing or full homomorphic encryption, which is also very beneficial to the improvement of performance.
Drawings
FIG. 1 is a system model block diagram of a neural network training method based on function secret sharing of the present invention;
FIG. 2 is a system model structure diagram of a neural network reasoning method based on function secret sharing of the invention;
FIG. 3 is a flow chart of a neural network training method based on function secret sharing in an embodiment of the invention;
FIG. 4 is a flow chart of a neural network reasoning method based on function secret sharing in an embodiment of the invention;
Detailed Description
For a better understanding of the present invention, the following examples are further illustrated, but are not limited to the following examples.
Example 1
As shown in fig. 3, a neural network training method based on function secret sharing allows a data owner to delegate a calculation task to a third party for processing on the premise of protecting data privacy. The invention will be described with reference to the accompanying drawings, wherein the method comprises the following steps:
(1) Offline stage: the auxiliary party generates random values required by subsequent online calculation, and the security calculation protocols of matrix multiplication, reLU and other nonlinear functions mask the data by using the related random values.
Illustratively, the auxiliary party pre-calculates some random values during the offline phase that will be used in the later online calculations. When performing matrix multiplication or non-linear functions such as ReLU, three data shares are respectively obtained by the party P 0 、P 1 、P 2 (three computing servers). To perform the computation, the helper generates random values with a pseudo-random number generator (PRG) and masks the different shares with the random values to help hide intermediate results of the computation, thereby increasing the privacy of the computation process, the random values generated by the helper confuse the data, making the othersThe participants cannot directly obtain the calculation result. In this process, to reduce the overhead of communication, the secondary party may not need to send a complete list of random numbers to each of the participants, but rather create and send a random PRG seed for each of the participants so that the participants can use this seed to locally generate the same random number as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
Functional secret sharing allows a plurality of participants to cooperatively calculate the result of one specific function f in a secret sharing state, consisting of a key generation function KeyGen () and an evaluation function Eval (). First, keyGen () is called to generate three different keys, respectively called k 0 、k 1 And k 2 These keys are then distributed to three different servers P 0 、P 1 、P 2 . Each participant then calculates the result of the function share on the common input x using the evaluation function Eval (), i.eAndfinally, the calculation result of the original function f on the input x is obtained through the reconstruction operations such as exclusive OR or addition, namely, f (x) =f k0 (x)+f k1 (x)+f k2 (x) A. The invention relates to a method for producing a fibre-reinforced plastic composite In this embodiment, an auxiliary party (commonly referred to as P 3 ) Assistance of the auxiliary party P 3 Random generation of r in offline phase 0 、r 1 、r 2 Facilitating a key generation function, wherein r = r 0 +r 1 +r 2 For masking the original data in a nonlinear function such as ReLU.
The 2-out-of-3 duplicate secret sharing may be represented as [ x ]]＝(x 0 ,x 1 ,x 2 ) Where x=x 0 +x 1 +x 2 Participant P 0 Hold (x) 0 ,x 1 )，P 1 Hold (x) 1 ,x 2 )，P 2 Hold (x) 2 ,x 0 ). Implementing the multiplication operation in the form of 2-out-of-3 duplicate secret sharing requires the aid of a related random value, the auxiliary party P 3 Randomly generating alpha 0 、α 1 、α 2 Let alpha be 0 +α 1 +α 2 =0, and will α i Sent to the corresponding P i I ε {0,1,2}. Suppose two secret sharing values x are calculated]＝(x 0 ,x 1 ,x 2 ) And [ y ]]＝(y 0 ,y 1 ,y 2 ) Then the three servers compute z locally 0 ＝x 0 y 0 +x 1 y 0 +x 0 y 1 、z 1 ＝x 1 y 1 +x 2 y 1 +x 1 y 2 And z 2 ＝x 2 y 2 +x 0 y 2 +x 2 y 0 Thus z 0 、z 1 And z 2 Form [ z=x·y]3-out-of-3 secret sharing of (c). The three parties then perform re-sharing, i.e. using the associated random value (alpha 0 ,α 1 ,α 2 ) Will be alpha i +z i Send to P i-1 To realize z]2-out-of-3 sharing of (2-out-of-3), wherein { alpha } i 3-out-of-3 secret sharing where 0 is.
(2) On-line stage:
1) Initializing a system: the client prepares training data, including features and labels, and possibly processes the raw data by data preprocessing techniques to ensure the quality and usability of the data. Meanwhile, the three servers initialize the same machine learning model locally, and the same random initialization parameters or the pre-training model can be used.
Illustratively, three servers P 0 、P 1 、P 2 Initializing the same neural network model, the auxiliary party may randomly generate the weight parameter W 0 And bias parameter b 0 These parameters will become the initial state of the model, randomly generated on each server, ensuring that their initial values on different servers are the same. Alternatively, the server may use pre-provisioningThe model is trained on other data as an initial model. The pre-training model may be public, generic, or previously trained on similar tasks. Each server loads the same pre-training model as a starting point of model training, and then performs fine adjustment on the basis of the pre-training model according to respective training data shares so as to adapt to specific tasks, so that the characteristic representation capability of the pre-training model can be utilized to accelerate the convergence process of the model.
2) Input data: the client divides the preprocessed training data into three shares and distributes the shares to the three servers in the form of copy secret sharing so as to protect sensitive private information in the process of transmitting and processing the data.
Illustratively, to protect data privacy, the client divides the prepared training data x into three shares x in a duplicate secret sharing manner 0 、x 1 、x 2 And respectively sends each share to three servers P 0 、P 1 、P 2 Two of (a) P 0 Hold (x) 0 ,x 1 )，P 1 Hold (x) 1 ,x 2 )，P 2 Hold (x) 2 ,x 0 ). Based on the security of secret sharing, different shares of each data are distributed to different servers, no one server alone having complete data. Therefore, in the process of data transmission and processing, any one server cannot obtain the complete information of the original data, and the complete data can be rebuilt only under the condition of multiparty cooperation, so that the sensitive private information is effectively protected, and the data cannot be leaked in the transmission and processing process.
3) Model training: the server receives the secret share of the training data, performs a training process of machine learning using a secure multiparty computing technique, comprising the basic operations of: matrix multiplication and convolution operations are implemented using the feature of 2-out-of-3 duplicate secret sharing, with random values generated by the secondary party using a pseudo-random number generator (PRG) to effect concealment of the multiplication result shares; the nonlinear activation functions such as ReLU and Maxpool and derivatives thereof are realized by using a function secret sharing technology, and the bottom layers of the nonlinear activation functions are subjected to comparison calculation and can be realized by using an FSS scheme of a distributed comparison function.
Illustratively, three servers P 0 、P 1 、P 2 The secret shares of the training data are received separately, i.e. P 0 Receive (x) 0 ,x 1 )，P 1 Receive (x) 1 ,x 2 )，P 2 Receive (x) 2 ,x 0 ) These servers may perform a training process of machine learning using secure multiparty computing technology. Wherein the matrix multiplication operation can be obtained by expanding the multiplication operation, and the convolution operation can be regarded as a matrix multiplication operation with higher dimension, and the execution of the matrix multiplication operation requires an auxiliary party P 3 Participation in P 3 Random value (alpha) generated using a pseudo random number generator (PRG) 0 ,α 1 ,α 2 ) The random values play a role in masking and protecting private data in calculation, so that the safety of calculation is enhanced.
When nonlinear functions such as ReLU and Maxpool are executed, the data size is compared and calculated, and a distributed comparison function is used for realizing the operation. The comparison function is expressed asThe output result is b when the input value x satisfies 0.ltoreq.x < a, and 0 when x.ltoreq.a. The FSS scheme of the comparison function of the present embodiment is composed of +.>And->Composition is prepared. At the position ofIn the function, three keys k are generated by random values provided by the auxiliary party 0 、k 1 、k 2 Each key generated can be considered to be a key having 2 n Individual leaf nodesWherein each node is marked with a 0 or a 1 and n is the bit length of the input value x. In particular, the path labeled a from the root node to the leaf node is referred to as a special path. By traversing the binary tree formed by each key, starting from the most significant bit of x, comparing the bit with the special path, and then summing the traversing results of each key tree, the +.>As a result of (a). Variations of the compare function are invoked when calculating the ReLU functionAnd->And the method can be realized. For the Maxpool function, the underlying algorithm is to calculate d elements x 1 、x 2 、…、x d The maximum value of max ([ x) i ]，[x j ])＝ReLU([x i ]-[x j ])+[x j ]Simplifying this as an evaluation of the ReLU.
4) And (3) outputting a model: after training, three calculation servers send the trained model back to the client, and the client reconstructs the model to obtain a complete neural network model so as to ensure the safety of the model and facilitate the subsequent reasoning and use process.
Illustratively, after training, each participant at the server side holds a part of the trained neural network model, and in order to ensure the security of the model, three calculation servers send respective model parameters back to the client side in a secret sharing mode. After receiving the model parameters sent by the three calculation servers, the client can combine the parameters to reconstruct a complete neural network model so as to apply the model to perform tasks such as prediction, classification and the like in the subsequent reasoning and use processes.
In order to improve the performance of the model, the neural network training method based on function secret sharing further comprises the following steps: 5) The server side evaluates the aggregated model, measures performance by using a verification set or a test set, and ensures that the model after outsourcing training reaches an expected performance level.
Example 2
As shown in fig. 1, a neural network training system based on function secret sharing includes: the system comprises a client, a server and an auxiliary party;
the client, namely the data owner, is an individual, organization or entity with specific data sets, the specific data sets can be used for training a machine learning model and used for preparing and preprocessing training data, the preprocessed training data is distributed to a server training model in a copying secret sharing mode, and after training is finished, the client receives the trained model sent back by the server and carries out reconstruction to obtain a complete neural network model;
the server side, namely the computing server, is responsible for receiving the preprocessed training data distributed by the client side in the form of copying secret sharing, executing the training process of machine learning by utilizing a secure multiparty computing method, hiding the computed intermediate result by utilizing a random value, and after training, sending the trained model back to the client side by the server side;
And the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe mode. The secondary party is assumed to be trusted and may be an entity, such as a computing node.
The client may be composed of N entities, and in the model training process, it is responsible for preparing training data, including features and labels, and when performing model training, each client needs to distribute its own private data to three computing servers in the form of duplicate secret sharing. Auxiliary party, i.e. P 3 The distribution of the random value may be accomplished through encrypted communications or other secure communication mechanisms to prevent the random value from being compromised or tampered with during delivery.
The server side is also used to evaluate the neural network model using a validation set or test set in order to ensure that the outsourced trained model reaches the desired level of performance.
The server side provides computing services by three computing servers, namely P 0 、P 1 、P 2 The client divides the preprocessed training data into three shares and distributes the shares to three servers in a form of copying secret sharing so as to protect sensitive private information in the process of transmitting and processing the data later; and assuming that the computing servers execute a model training process of machine learning in a non-collusion mode according to the data of the data holder, after training is finished, the three computing servers send the trained model back to the client, and the client locally holds the model to ensure the control right of the original data and prevent privacy leakage.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
Example 3
As shown in fig. 4, a neural network reasoning method based on function secret sharing allows a client to obtain a prediction result of a model without exposing sensitive data or model details, and the present invention describes a scheme described in this embodiment with reference to the accompanying drawings, where the method includes the following steps:
(1) Offline stage: the auxiliary party generates random values required by subsequent online calculation, and the security calculation protocols of matrix multiplication, reLU and other nonlinear functions mask the data by using the related random values.
Illustratively, the auxiliary party pre-generates random values during the offline phase, which values are used in subsequent online calculations. These random values are generated by a pseudo random number generator (PRG) for masking different data shares during the computation, increasing the privacy of the computation process. When performing matrix multiplication and non-linear functions such as ReLU, three data shares are obtained by different parties P 0 、P 1 、P 2 The auxiliary party masks the shares by using a pre-generated random value to help hide intermediate calculation results and protect the privacy of calculation. To reduce the overhead of communication, the secondary party may not need to send a complete list of random numbers to each of the participants, but rather create and send a random PRG seed for each of the participants so that the participants can use this seed to locally generate the same random number as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
The function secret share is composed of a key generation function KeyGen () and an evaluation function Eval (). In the present embodiment, the auxiliary party P 3 Generating three random values r in offline phase 0 、r 1 、r 2 Satisfy r=r 0 +r 1 +r 2 The method is used for generating a key in the FSS, wherein r is a random value masking original data in a non-linear function such as ReLU and the like. While implementing the multiplication operation in the form of 2-out-of-3 duplicate secret sharing requires the helper to generate the random value alpha 0 、α 1 、α 2 Satisfy alpha 0 +α 1 +α 2 =0, and will α i Sent to the corresponding P i I ε {0,1,2}. The calculation result of the multiplication operation can be realized through the steps of local calculation of each participant and re-sharing of random value assistance.
(2) On-line stage:
1) Initializing a system: the client is ready for data that requires inference predictions. The model owner selects a machine learning model that is appropriate for the task and prepares model parameters and deploys the machine learning model in a secure manner into the computing service provider's environment.
The model owner illustratively splits its own model parameters, i.e., weight W and parameter b, into three shares, which are shared using secret sharing techniques to obtain three sets of weight shares and three sets of bias shares, respectively (W 0 ,w 1 ,w 2 ) And (b) 0 ,b 1 ,b 2 ). The model owner sends the corresponding shares of each set of parameters to three computing servers P, respectively 0 、P 1 、P 2 Each server only receives own parameter shares, and cannot directly obtain complete model parameters.
2) Input data: the client divides the original data into three shares and distributes it in the form of duplicate secret shares to three computing servers in order to protect the sensitive information of the user during the transmission and processing of the data.
Illustratively, the private data x owned by the client is divided into three shares x by copying secret sharing 0 、x 1 、x 2 And respectively sends each share to three servers P 0 、P 1 、P 2 Two of (a) P 0 Hold (x) 0 ,x 1 )，P 1 Hold (x) 1 ,x 2 )，P 2 Hold (x) 2 ,x 0 )。
3) Inference prediction: the server side receives the secret share of the user data, and utilizes a secure multiparty computing technology to infer the data, and the method comprises the following basic operations: matrix multiplication and convolution operations are implemented using the feature of 2-out-of-3 duplicate secret sharing, with random values generated by the secondary party using a pseudo-random number generator (PRG) to effect concealment of the multiplication result shares; the nonlinear activation functions such as ReLU and Maxpool are realized by using a function secret sharing technology, and the bottom layers of the nonlinear activation functions are subjected to comparison calculation and can be realized by using an FSS scheme of a distributed comparison function.
Illustratively, three servers P 0 、P 1 、P 2 The secret shares of the predicted data are received separately, i.e. P 0 Receive (x) 0 ,x 1 )，P 1 Receive (x) 1 ,x 2 )，P 2 Receive (x) 2 ,x 0 ) These servers may perform a training process of machine learning using secure multiparty computing technology. Wherein the matrix multiplication operation can be obtained by expanding the multiplication operation, and the convolution operation can be regarded as a matrix multiplication operation with higher dimension, and the execution of the matrix multiplication operation requires an auxiliary party P 3 Participation in P 3 Random value (alpha) generated using a pseudo random number generator (PRG) 0 ,α 1 ,α 2 ) The random values play a role in masking and protecting private data in calculation, so that the safety of calculation is enhanced.
When nonlinear functions such as ReLU and Maxpool are executed, the nonlinear functions are realized by means of a distributed comparison function. The specific form of the comparison function isB is output when x is more than or equal to 0 and less than a, and 0 is output when x is more than or equal to a. The FSS scheme of the comparison function of the present embodiment consists ofAnd->Composition (S)/(S)>The function generates three keys k using random values provided by the helper 0 、k 1 、k 2 Each generated key is considered to be a key having 2 n The binary tree of each leaf is obtained by traversing each binary tree composed of keys, comparing the most significant bit of x with a special path with leaf node as a according to the bit, and finally adding and summing the traversing results of each key tree>As a result of (a). Variant of calling a comparison function->Andto calculate the ReLU function, the Maxpool function is then calculated by max ([ x) i ]，[x j ])＝ReLU([x i ]-[x j ])+[x j ]Simplifying itFor evaluation of ReLU.
4) Outputting a result: and the server returns the obtained reasoning result share secret to the client, and the client carries out reconstruction decryption on the prediction result, so that a final reasoning result is obtained.
Illustratively, three servers send the resulting inference results y0, y1, y2, respectively, back to the client in the form of a duplicate secret share. And the client performs secret reconstruction on the received result to obtain a final reasoning result y.
Example 4
As shown in fig. 2, a neural network reasoning system based on function secret sharing includes: the system comprises a client, a model owner, a server and an auxiliary party;
the client, namely the data owner, is an entity needing to use a model for reasoning, usually a person, an organization or an application, and is used for preparing data needing to be subjected to reasoning prediction, distributing the data needing to be subjected to reasoning prediction to the server in a form of copying secret sharing, receiving a reasoning result share returned by the secret of the server by the client after the reasoning is completed by the server, and reconstructing and decrypting the reasoning result so as to obtain a final reasoning result;
the model owner is a person, a group or an organization with a trained machine learning model, and is used for selecting the machine learning model suitable for a task, preparing model parameters, and deploying the machine learning model into a server-side environment to provide a machine learning prediction service, but not necessarily participate in specific reasoning calculation;
the server side, i.e., the computing service provider, is the entity responsible for performing the secure model inference computations, typically the cloud service provider or the outsourced computing service provider. The server side is used for receiving data which is copied by the client side and needs to be subjected to reasoning prediction, reasoning the received data by utilizing a secure multiparty calculation method, hiding the calculated intermediate result by utilizing a random value, and finally returning the obtained reasoning result share secret to the client side without exposing details of a model;
And the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe manner. The secondary party is assumed to be trusted and may be an entity, such as a computing node.
The client has the data to be input into the model, hopes to obtain the prediction result of the model through outsourcing reasoning, and does not want to share the original data. The client transmits the data that needs reasoning to the calculator, but does not directly access the model. The model owner is responsible for the security and accuracy of the model and needs to ensure that the model is not misused or compromised. Auxiliary party, i.e. P 3 The distribution process may be implemented by encrypted communications or other secure communication mechanisms to prevent random values from being compromised or tampered with during delivery.
The server side provides computing services by three computing servers, namely P 0 、P 1 、P 2 The client side divides data needing to be subjected to reasoning prediction into three shares, and distributes the three shares to three servers in a copying secret sharing mode so as to protect sensitive private information in the process of subsequently transmitting and processing the data; after the reasoning is finished, the three computing servers return the obtained reasoning result share secret to the client, and the client locally stores the reasoning result share secret so as to ensure the control right on the original data and prevent privacy disclosure.
The server side provides an environment for reasoning calculation and comprises a server, hardware equipment and the like. It should be noted that the client and model owner may act as a computing server, but for ease of illustration, the invention is described separately.
To reduce data transmission, the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party. This approach avoids a large number of random number transmissions while ensuring that the same random value can be generated between the secondary party and each participant, respectively, to mask and protect the data in the computation.
According to the neural network training reasoning method and system based on function secret sharing, the training and reasoning process of machine learning is decomposed into an online stage related to input and an offline stage unrelated to input by utilizing cryptographic technologies such as function secret sharing, and heavy encryption calculation is transferred to the offline stage by generating related random values required by subsequent calculation during preprocessing, so that not only can all linear operations of the online stage be directly executed on secret sharing data without calling heavy encryption tools such as homomorphic encryption or carrying out frequent interaction processes, but also the communication cost of the online stage of machine learning in calculating complex nonlinear functions such as ReLU functions is greatly reduced, and the online cost of secure multi-party calculation can be transferred to the offline stage and local calculation; at the same time, the invention avoids the expensive conversion operation when mixed use of a garbled circuit and addition secret sharing or full homomorphic encryption, which is also very beneficial to the improvement of performance.
It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein. The scheme in the embodiment of the invention can be realized by adopting various computer languages, such as object-oriented programming language Java, an transliteration script language JavaScript and the like.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims (10)

1. A neural network training method based on function secret sharing is characterized in that: on the premise of protecting data privacy, the client is allowed to delegate the calculation task to a third party for processing, and the method comprises an offline stage and an online stage:
offline stage:
the auxiliary party pre-generates a random value required for subsequent online calculation in an offline stage;
on-line stage:
the client prepares and preprocesses training data, and distributes the preprocessed training data to the server in a form of copying secret sharing;
the server receives the preprocessed training data, performs a training process of machine learning by using a secure multiparty calculation method, and hides a calculated intermediate result by using a random value;
after training, the server sends the trained model back to the client, and the client reconstructs the model to obtain a complete neural network model.
2. The neural network training method based on function secret sharing as claimed in claim 1, wherein: the online phase further comprises: the server side evaluates the neural network model using the validation set or the test set.
3. A neural network training method based on functional secret sharing as claimed in claim 1 or 2, wherein: the server side provides computing services by three computing servers, and the client side divides the preprocessed training data into three shares and distributes the three shares to the three servers in a form of copying secret sharing; after training, the three calculation servers send the trained model back to the client, and the client reconstructs the model to obtain a complete neural network model.
4. A neural network training method based on function secret sharing as claimed in claim 3, wherein: the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party.
5. A neural network training method based on functional secret sharing as claimed in claim 1 or 2, wherein: the training process of machine learning is performed by using a secure multipartite computing method, and comprises a forward propagation process and a backward propagation process, wherein the forward propagation process is formed by alternately performing linear operation and nonlinear operation, the linear operation comprises matrix multiplication operation of a full-connection layer and convolution operation of a convolution layer, the nonlinear operation comprises an activation function, and the backward propagation process completes updating of the weight by using derivative of the activation function and the matrix multiplication operation.
6. A neural network training system based on function secret sharing, characterized in that: comprising the following steps: the system comprises a client, a server and an auxiliary party;
the client, namely the data owner, is used for preparing and preprocessing training data, distributing the preprocessed training data to the server-side training model in a copying secret sharing mode, and after training, receiving the trained model sent back by the server-side and reconstructing the model to obtain a complete neural network model;
the server side, namely the computing server, is responsible for receiving the preprocessed training data distributed by the client side in the form of copying secret sharing, executing the training process of machine learning by utilizing a secure multiparty computing method, hiding the computed intermediate result by utilizing a random value, and after training, sending the trained model back to the client side by the server side;
and the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe mode.
7. A neural network training system based on functional secret sharing as defined in claim 6, wherein: the server side is also configured to evaluate the neural network model using the validation set or the test set.
8. A neural network training system based on functional secret sharing as claimed in claim 6 or 7, wherein: the server side provides calculation services by three calculation servers, the client side divides the preprocessed training data into three shares, the three shares are distributed to the three servers in a mode of copying secret sharing, after training is finished, the three calculation servers send the trained model back to the client side, and the client side carries out reconstruction to obtain a complete neural network model;
the secondary party creates and transmits a random PRG seed for each computing server, which uses the received PRG seed to locally generate the same random value as the secondary party.
9. A neural network reasoning method based on function secret sharing is characterized in that: allowing clients to obtain predictions of a model without exposing sensitive data and/or model details, including offline and online phases:
offline stage:
the auxiliary party pre-generates a random value required for subsequent online calculation in an offline stage;
on-line stage:
the client side prepares data needing to be subjected to reasoning prediction, a model owner selects a machine learning model suitable for a task, prepares model parameters, and deploys the machine learning model to the environment of the server side;
The client distributes the data needing to be subjected to reasoning prediction to the server in a form of copying secret sharing;
the server side uses a secure multiparty calculation method to infer the received data and uses a random value to hide the calculated intermediate result;
and the server side returns the obtained reasoning result secret to the client side, and the client side carries out reconstruction decryption on the reasoning result, so that a final reasoning result is obtained.
10. A neural network reasoning system based on function secret sharing is characterized in that: comprising the following steps: the system comprises a client, a model owner, a server and an auxiliary party;
the client, namely the data owner, is used for preparing the data needing to be subjected to reasoning prediction, distributing the data needing to be subjected to reasoning prediction to the server in a copying secret sharing mode, receiving the reasoning result share returned by the server secret by the client after the reasoning is completed by the server, and reconstructing and decrypting the reasoning result so as to obtain a final reasoning result;
the model owner is used for selecting a machine learning model suitable for a task, preparing model parameters and deploying the machine learning model into the environment of the server side;
the server side, namely the computing service provider, is used for receiving the data which is shared by the client side and needs to be subjected to reasoning prediction, reasoning the received data by utilizing a secure multiparty computing method, hiding the intermediate result of the computation by utilizing a random value, and finally returning the obtained reasoning result secret to the client side without exposing the details of the model;
And the auxiliary party is used for generating random values required by subsequent online calculation in advance in an offline stage and distributing the generated random values to the server side in a safe manner.
CN202311340908.2A 2023-10-17 2023-10-17 Neural network training reasoning method and system based on function secret sharing Pending CN117291258A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311340908.2A CN117291258A (en) 2023-10-17 2023-10-17 Neural network training reasoning method and system based on function secret sharing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311340908.2A CN117291258A (en) 2023-10-17 2023-10-17 Neural network training reasoning method and system based on function secret sharing

Publications (1)

Publication Number Publication Date
CN117291258A true CN117291258A (en) 2023-12-26

Family

ID=89251692

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311340908.2A Pending CN117291258A (en) 2023-10-17 2023-10-17 Neural network training reasoning method and system based on function secret sharing

Country Status (1)

Country Link
CN (1) CN117291258A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117592527A (en) * 2024-01-18 2024-02-23 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Privacy protection neural network training method and device based on function secret sharing

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117592527A (en) * 2024-01-18 2024-02-23 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) Privacy protection neural network training method and device based on function secret sharing

Similar Documents

Publication Publication Date Title
Wang et al. Authenticated garbling and efficient maliciously secure two-party computation
Giacomelli et al. Privacy-preserving ridge regression with only linearly-homomorphic encryption
CN112183730B (en) Neural network model training method based on shared learning
CN112989368B (en) Method and device for processing private data by combining multiple parties
CN112822005B (en) Secure transfer learning system based on homomorphic encryption
Chandran et al. {SIMC}:{ML} inference secure against malicious clients at {Semi-Honest} cost
CN114595835B (en) Model training method and device based on federal learning, equipment and storage medium
CN117291258A (en) Neural network training reasoning method and system based on function secret sharing
CN113065145A (en) Privacy protection linear regression method based on secret sharing and random disturbance
US20240013034A1 (en) Neural network prediction system for privacy preservation
CN112865953A (en) Safe multi-party computing method, device and system based on auxiliary server
CN115333726A (en) Fixed point number secure multiplication method based on vector space secret sharing
Panzade et al. Towards faster functional encryption for privacy-preserving machine learning
Akimoto et al. Privformer: Privacy-preserving transformer with mpc
CN115314211A (en) Privacy protection machine learning training and reasoning method and system based on heterogeneous computing
Zhu et al. Securebinn: 3-party secure computation for binarized neural network inference
Liu et al. DHSA: efficient doubly homomorphic secure aggregation for cross-silo federated learning
CN117592527A (en) Privacy protection neural network training method and device based on function secret sharing
Sharma et al. Privacy-preserving deep learning with SPDZ
CN113591160B (en) State digital signature method and system based on symmetric passwords
CN115941351A (en) Trusted privacy computing system based on cloud service and encryption technology
CN115130568A (en) Longitudinal federated Softmax regression method and system supporting multiple parties
Panzade et al. FENet: Privacy-preserving Neural Network Training with Functional Encryption
Yang et al. Efficient Privacy-preserving Inference Outsourcing for Convolutional Neural Networks
Zhang et al. Joint Linear and Nonlinear Computation across Functions for Efficient Privacy-Preserving Neural Network Inference

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination