WO2021184346A1 - 隐私机器学习模型生成、训练方法、装置及电子设备 - Google Patents
隐私机器学习模型生成、训练方法、装置及电子设备 Download PDFInfo
- Publication number
- WO2021184346A1 WO2021184346A1 PCT/CN2020/080391 CN2020080391W WO2021184346A1 WO 2021184346 A1 WO2021184346 A1 WO 2021184346A1 CN 2020080391 W CN2020080391 W CN 2020080391W WO 2021184346 A1 WO2021184346 A1 WO 2021184346A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- machine learning
- learning model
- operator
- model
- plaintext
- Prior art date
Links
- 238000010801 machine learning Methods 0.000 title claims abstract description 253
- 238000000034 method Methods 0.000 title claims abstract description 61
- 238000012549 training Methods 0.000 title claims abstract description 36
- 230000003068 static effect Effects 0.000 claims abstract description 119
- 230000006870 function Effects 0.000 claims description 78
- 238000004590 computer program Methods 0.000 claims description 23
- 238000012360 testing method Methods 0.000 claims description 19
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000012795 verification Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 description 15
- 238000010586 diagram Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 10
- 238000003860 storage Methods 0.000 description 10
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000005457 optimization Methods 0.000 description 7
- 239000008186 active pharmaceutical agent Substances 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 108020004705 Codon Proteins 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000011478 gradient descent method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Definitions
- This specification relates to the field of machine learning technology, and in particular to a method, device and electronic device for generating and training a privacy machine learning model.
- Machine Learning has been applied in many fields.
- machine learning requires the use of a large amount of sample data
- two or more data holders will collaborate in machine learning modeling.
- privacy machine learning frameworks In order to solve this problem, various encryption machine learning-based frameworks (hereinafter referred to as privacy machine learning frameworks) have emerged, such as: TF-Encrypted, PySyft, etc.
- these privacy machine learning frameworks make use of the ease of use of the application programming interface (API, Application Programming Interface) of plaintext machine learning frameworks (such as Tensorflow, PyTorch, etc.), and at the same time train encrypted data through a variety of cryptographic algorithms And predictions, so that users can use them without expertise in cryptography, distributed systems, or high-performance computing.
- API Application Programming Interface
- plaintext machine learning frameworks such as Tensorflow, PyTorch, etc.
- the purpose of the implementation of this specification is to provide a privacy machine learning model generation and training method, device and electronic equipment, so as to reduce the implementation cost of the privacy machine learning model.
- the implementation of this specification provides a method for generating a private machine learning model, including:
- the static optimizer is created according to a cryptographic operator optimizer class
- the cryptographic operator optimizer class is derived from the native optimizer class of the plaintext machine learning framework
- the plaintext machine learning model is generated based on the plaintext machine learning framework .
- the implementation of this specification also provides a privacy machine learning model generation device, including:
- the acquisition module is used to acquire the static optimizer in the plaintext machine learning model
- An execution module configured to execute the static optimizer to replace native plaintext operators in the plaintext machine learning model with cryptographic operators, thereby generating a privacy machine learning model
- the static optimizer is created according to a cryptographic operator optimizer class
- the cryptographic operator optimizer class is derived from the native optimizer class of the plaintext machine learning framework
- the plaintext machine learning model is generated based on the plaintext machine learning framework .
- the implementation of this specification also provides an electronic device, including a memory, a processor, and a computer program stored on the memory.
- the computer program executes the aforementioned privacy machine when run by the processor. Learn how to generate models.
- the implementation of this specification also provides a privacy machine learning model training method, which includes the following steps:
- the first model When the first model satisfies the preset condition, the first model is used as a data prediction model.
- the implementation of this specification also provides a privacy machine learning model training device, including:
- the model training module is configured to train a private machine learning model based on a training sample set to generate a first model; the private machine learning model is generated by using the above-mentioned private machine learning model generation method;
- the model verification module is configured to test whether the first model meets a preset condition based on a test sample set
- the model determination module is configured to use the first model as a data prediction model when the first model satisfies the preset condition.
- the implementation of this specification also provides another electronic device, including a memory, a processor, and a computer program stored on the memory, and the computer program executes the following steps when the computer program is run by the processor:
- the first model When the first model satisfies the preset condition, the first model is used as a data prediction model.
- the implementations of this specification can also implement a static optimizer into the plaintext machine learning model; because the static optimizer is optimized based on the cryptographic operator
- the cipher operator class is created, and the cipher operator optimizer class is derived from the native optimizer class and cipher operator of the plaintext machine learning framework; in this way, when the plaintext machine learning model is overloaded, the static optimizer can be executed
- the original plaintext operator in the plaintext machine learning model is replaced with a cryptographic operator, thereby realizing a private machine learning model by reusing the existing plaintext machine learning model, thereby avoiding the unique use of a private machine learning framework in the prior art
- Figure 1 is a flowchart of a method for generating a private machine learning model in some embodiments of this specification
- Figure 2 is a calculation diagram of an exemplary implementation of this specification when a static optimizer is not imported into the plaintext machine learning model
- FIG. 3 is a calculation diagram after a static optimizer is imported into a plaintext machine learning model in an exemplary embodiment of this specification, and the static optimizer is executed;
- Figure 4 is a processing flow chart of the static optimizer in an embodiment of this specification
- FIG. 5 is a schematic diagram of a structure in which multiple data holders jointly perform machine learning in an exemplary embodiment of this specification
- Fig. 6 is a flowchart of a privacy machine learning model training method in some embodiments of this specification.
- Fig. 7 is a structural block diagram of a privacy machine learning model generating device in some embodiments of this specification.
- FIG. 8 is a structural block diagram of a privacy machine learning model training device in some embodiments of this specification.
- Fig. 9 is a structural block diagram of an electronic device in some embodiments of this specification.
- the privacy machine learning model generation method of some embodiments of this specification may include the following steps:
- the static optimizer is created according to a cryptographic operator optimizer class
- the cryptographic operator optimizer class is derived from the native optimizer class of the plaintext machine learning framework
- the plaintext machine learning model is generated based on the plaintext machine learning framework .
- the implementation of this specification can also implement a static optimizer into the plaintext machine learning model; because the static optimizer is created based on the cryptographic operator optimizer class, and the cryptographic operator
- the optimizer class is derived from the native optimizer class and cryptographic operator of the plaintext machine learning framework; in this way, when the plaintext machine learning model is overloaded, the static optimizer can be used to convert the plaintext machine learning model
- the original plaintext operator is replaced with a cryptographic operator, thereby realizing a private machine learning model by reusing the existing plaintext machine learning model, thereby avoiding the existing technology due to the use of private machine learning framework-specific APIs and private data types to re-encode
- the implementation of this specification reduces the implementation cost of the privacy machine learning model.
- the plaintext machine learning framework can be any existing plaintext machine learning framework, such as TensorFlow, PyTorch, or Caffe. Therefore, this specification does not limit the specific plaintext machine learning framework used to generate the plaintext machine learning model, and it can be selected according to actual needs.
- the cryptographic operator can be implemented by a developer through a static language (such as C, C++, etc.) programming in advance to improve efficiency.
- the password operator generally also includes a password gradient operator.
- these cryptographic operators should have a one-to-one correspondence with the native plaintext operators in the plaintext machine learning model generated based on the plaintext machine learning framework, so as to facilitate subsequent corresponding replacements.
- developers can register it in the plaintext machine learning framework to facilitate the use of the plaintext machine learning model.
- the cryptographic operator is any password that can provide privacy protection for the input data of all parties in the scenario of two or more data holders jointly (or collaboratively) performing machine learning modeling and prediction.
- the cryptographic operator may be a Secure Multi-Party Computation (MPC) operator, a homomorphic encryption (Homomorphic Encryption, HE) operator, or a zero-knowledge proof (Zero-knowledge) operator. Proof, ZKP) operator, etc.
- MPC Secure Multi-Party Computation
- HE homomorphic Encryption
- ZKP zero-knowledge proof
- this specification does not limit the specific codon operators used, which can be selected according to actual needs.
- the plaintext machine learning model when the plaintext machine learning model is generated by TensorFlow, and the cryptographic operator adopts the MPC operator, after importing the static optimizer into the plaintext machine learning model and executing the static optimizer, the plaintext machine learning model
- the native plaintext operator (TF Native Ops) in the learning model is replaced with an MPC operator (MPC Ops), so that the plaintext machine learning model can be turned into a machine learning model supported by MPC privacy protection.
- MPC Ops MPC operator
- Ops is the abbreviation of Operations.
- the native plaintext operator of the plaintext machine learning model is: matrix multiplication of the input variables X and Y (For example, as shown in the MatMul function in Figure 2), and provide the return value. Since the variables X and Y are processed under the plaintext machine learning model, the privacy and security of the variables X and Y are difficult to guarantee.
- the plaintext machine learning model is transformed into a privacy machine learning model.
- the corresponding MPC operator is: will be affected by MPC privacy
- the encrypted variables X and Y formed by the protection are matrix multiplied (for example, as shown in the MPCMatMul function in Figure 3), and the return value is provided.
- the plaintext machine learning framework is generally configured with a native optimizer class.
- developers can derive a derived class from the native optimizer class of the plaintext machine learning framework as the base class in advance, and add new cryptographic operators related to the derived class Members, thus forming the class of cryptographic operator optimizers. Therefore, the cryptographic operator optimizer class not only has the characteristics of a cryptographic operator, but also has all the features of the native optimizer class of the plaintext machine learning framework.
- the cryptographic operator optimizer class, cryptographic operator, and cryptographic static graph mentioned in the embodiments of this specification refer to the optimizer class, operator, and static graph under privacy protection, respectively.
- the developer can create a static optimizer according to the password operator optimizer class in advance.
- a static optimizer can be created according to the MPC Optimizer minimize function in the MPC operator optimizer class.
- users can obtain the static optimizer and import it into the plaintext machine learning model.
- the optimization algorithm of the static optimizer may adopt any suitable optimization algorithm.
- the optimization algorithm used may include, but is not limited to, gradient descent methods (such as standard gradient descent (GD), stochastic gradient descent, SGD), Batch gradient descent method (Batch Gradient Descent, BGD), momentum optimization method (such as Momentum, Nesterov accelerated gradient (Nesterov accelerated gradient, NAG), etc.), or adaptive learning rate optimization algorithm (such as AdaGrad algorithm, RMSProp algorithm, Adam Algorithm and AdaDelta algorithm) and so on.
- gradient descent methods such as standard gradient descent (GD), stochastic gradient descent, SGD), Batch gradient descent method (Batch Gradient Descent, BGD), momentum optimization method (such as Momentum, Nesterov accelerated gradient (Nesterov accelerated gradient, NAG), etc.
- momentum optimization method such as Momentum, Nesterov accelerated gradient (Nesterov accelerated gradient, NAG), etc.
- adaptive learning rate optimization algorithm such as AdaGrad algorithm, RMSProp algorithm, Adam Algorithm and AdaDelta algorithm
- the general principle for replacing the native plaintext operators in the plaintext machine learning model with cryptographic operators is: all native plaintext operators that affect data privacy protection need to be replaced with corresponding passwords Operators to ensure the privacy and security of the input data; for the original plaintext operators that do not affect the protection of data privacy, try not to replace them, so as to improve the reuse rate of the plaintext machine learning model, thereby helping to reduce the realization of the privacy machine learning model cost.
- a plaintext machine learning model that uses a computational graph (or called a directed acyclic graph) to represent processing logic (for example, a plaintext machine learning model generated based on Tensorflow), when each native plaintext operator in the plaintext machine learning model (If necessary, after replacing it with a corresponding cryptographic operator, the original static graph in the plaintext machine learning model is replaced with a cryptographic static graph.
- processing logic for example, a plaintext machine learning model generated based on Tensorflow
- replacing the native static image in the plaintext machine learning model with a cryptographic static image may include the following steps:
- the loss function is a quantitative representation of the difference in probability distribution between model output and test results, and is the objective function of model optimization.
- the process of model training or optimization is the process of minimizing the loss function. Therefore, when replacing the native static graph in the plaintext machine learning model with the cryptographic static graph, it is necessary to obtain the variable name returned by the loss function to facilitate subsequent replacement.
- a user when a user generates a plaintext machine learning model based on a plaintext machine learning framework, he can select a loss function from the set of loss functions provided by the plaintext machine learning framework.
- variable name returned by the loss function when the variable name returned by the loss function is in the variable set of the target model (the target model refers to the privacy machine learning model), there is no need to replace it, and it can be used in the static state of the target model.
- a tensor is obtained and returned according to the variable name returned by the loss function, so as to fully reuse the plaintext machine learning model, thereby further reducing the implementation cost of the privacy machine learning model.
- computational graphs (including static graphs) contain nodes and edges with directions. Nodes generally represent data operators, and edges generally represent the flow of tensors (usually a tensor represents an n-dimensional matrix).
- the node MPCMatMul function is the operator of the data
- the two input edges corresponding to the node MPCMatMul function represent the tensor composed of the data of the variables X and Y (such as ) Data flow direction.
- variable name returned by the loss function when the variable name returned by the loss function is not in the variable set of the target model, it is necessary to further confirm the type of the variable returned by the loss function;
- the original static image in the plaintext machine learning model is replaced with a cryptographic static image.
- the tensor name of the tensor can be obtained, and then it can be confirmed whether the tensor name is a placeholder, so that Can be selectively replaced later.
- the edge corresponding to the operator (generally a data edge, in special cases, it can also include a control edge) Recursive replacement to realize the privacy protection of the input data, and return the edge corresponding to the new operator formed after the replacement; then confirm whether the data stream corresponding to the operator contains the model private data of the target model.
- the edge corresponding to the new operator can be used as a parameter to call
- the cipher operator creation function creates a corresponding cipher operator and returns the cipher operator to realize data privacy protection.
- the operator (native plaintext operator) of the plaintext machine learning model can be deeply copied to the target In the model, and return the operator of the deep copy, in order to fully reuse the plaintext machine learning model, which is conducive to further reducing the implementation cost of the privacy machine learning model.
- the type of the variable returned by the loss function is usually one of the tensor type and the operator type.
- the type of the variable returned by the loss function is neither a tensor type nor an operator type, an error can be thrown.
- the cryptographic static graph may include a forward graph (ie, a static calculation graph corresponding to forward propagation) and a reverse graph (ie, a static calculation graph corresponding to back propagation); wherein, the reverse graph can be automatically created based on the forward graph.
- this specification provides a private machine learning model training method based on the private machine learning model.
- the method for training a private machine learning model may include the following steps:
- S602 Test whether the first model meets a preset condition based on the test sample set.
- each data holder cooperates to perform machine learning modeling.
- the terminal of each data holder may be configured with a private machine learning model generated based on the above-mentioned private machine learning model generating method, and the cryptographic operator of the private machine learning model is an MPC operator.
- MPC mobile phone
- the data of any one of the three data holders will not be known by the other two parties, thus achieving the improvement of the value of data utilization while taking into account To ensure the security of their own private data.
- each data holder can train a data prediction model.
- the aforementioned privacy machine learning model training method can be applied to any machine learning scenario, for example, it can include, but is not limited to, computer vision, natural language processing, robot decision-making (such as autonomous driving), and network security detection (such as spam detection, malicious programs ⁇ flow detection), medical fitting prediction, financial high-frequency trading, Internet data mining, association recommendation and other scenarios.
- machine learning scenario for example, it can include, but is not limited to, computer vision, natural language processing, robot decision-making (such as autonomous driving), and network security detection (such as spam detection, malicious programs ⁇ flow detection), medical fitting prediction, financial high-frequency trading, Internet data mining, association recommendation and other scenarios.
- the privacy machine learning model generation device may include:
- the obtaining module 71 may be used to obtain the static optimizer in the plaintext machine learning model
- the execution module 72 may be used to execute the static optimizer to replace native plaintext operators in the plaintext machine learning model with cryptographic operators, thereby generating a privacy machine learning model;
- the static optimizer is created according to a cryptographic operator optimizer class
- the cryptographic operator optimizer class is derived from the native optimizer class of the plaintext machine learning framework
- the plaintext machine learning model is generated based on the plaintext machine learning framework .
- the cryptographic operator may include any one of the following:
- the replacing the native plaintext operator in the plaintext machine learning model with a cryptographic operator includes:
- the replacement of the native static image in the plaintext machine learning model with the password static image may include:
- variable name returned by the loss function is in the variable set of the target model
- a tensor is obtained and returned in the static graph of the target model according to the variable name returned by the loss function.
- the replacement of the native static image in the plaintext machine learning model with the password static image may also include:
- the native static graph in the plaintext machine learning model is replaced with a cryptographic static graph.
- the replacement of the native static image in the plaintext machine learning model with the cryptographic static image according to the type of the variable returned by the loss function may include:
- the replacement of the native static graph in the plaintext machine learning model with the cryptographic static graph according to the type of the variable returned by the loss function may also include :
- the replacement of the native static graph in the plaintext machine learning model with the cryptographic static graph according to the type of the variable returned by the loss function may also include :
- the type of the variable returned by the loss function is an operator type, recursively replace the edge corresponding to the operator, and return the edge corresponding to the new operator formed after the replacement;
- the edge corresponding to the new operator is used as a parameter, the cipher operator creation function is called to create the corresponding cipher operator, and the cipher operator is returned .
- the replacement of the native static graph in the plaintext machine learning model with the cryptographic static graph according to the type of the variable returned by the loss function may also include :
- the operator of the plaintext machine learning model is deeply copied to the target model, and the operator of the deep copy is returned.
- the replacement of the native static graph in the plaintext machine learning model with the cryptographic static graph according to the type of the variable returned by the loss function may also include :
- the privacy machine learning model training device may include:
- the model training module 81 may be used to train a private machine learning model based on the training sample set to generate the first model
- the model verification module 82 may be used to test whether the first model meets a preset condition based on a test sample set
- the model determination module 83 may be configured to use the first model as a data prediction model when the first model satisfies the predetermined condition.
- the electronic device may include a memory, a processor, and a computer program stored on the memory, and the computer program may be executed when the processor is run. Corresponding method steps.
- the computer program stored on the memory is the instruction set corresponding to the above-mentioned privacy machine learning model generation method
- the computer program when the computer program is run by the processor, the following steps can be performed :
- the static optimizer is executed to replace native plaintext operators in the plaintext machine learning model with cryptographic operators, thereby generating a privacy machine learning model; the plaintext machine learning model is generated based on the plaintext machine learning framework.
- the computer program stored on the memory when the computer program stored on the memory is the instruction set corresponding to the above-mentioned privacy machine learning model training method, when the computer program is run by the processor, it can execute as follows step:
- the first model When the first model satisfies the preset condition, the first model is used as a data prediction model.
- These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
- the device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
- These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
- the instructions provide steps for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
- the computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
- processors CPUs
- input/output interfaces network interfaces
- memory volatile and non-volatile memory
- the memory may include non-permanent memory in a computer readable medium, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.
- RAM random access memory
- ROM read-only memory
- flash RAM flash memory
- Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
- the information can be computer-readable instructions, data structures, program modules, or other data.
- Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.
- the implementation of this specification can be provided as a method, a system or a computer program product. Therefore, the implementation of this specification may adopt the form of a complete hardware implementation, a complete software implementation, or an implementation combining software and hardware. Moreover, the implementation of this specification may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
- computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
- program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types.
- the embodiments of this specification can also be practiced in distributed computing environments. In these distributed computing environments, tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (24)
- 一种隐私机器学习模型生成方法,其特征在于,包括:获取明文机器学习模型中的静态优化器;执行所述静态优化器,以将所述明文机器学习模型中的原生明文算子替换为密码算子,从而生成隐私机器学习模型;其中,所述静态优化器根据密码算子优化器类创建,所述密码算子优化器类派生于所述明文机器学习框架的原生优化器类,所述明文机器学习模型基于明文机器学习框架生成。
- 如权利要求1所述的隐私机器学习模型生成方法,其特征在于,所述密码算子包括以下中的任意一种:安全多方计算算子;同态加密算子;零知识证明算子。
- 如权利要求1所述的隐私机器学习模型生成方法,其特征在于,所述将所述明文机器学习模型中的原生明文算子替换为密码算子,包括:将明文机器学习模型中的原生静态图替换为密码静态图。
- 如权利要求3所述的隐私机器学习模型生成方法,其特征在于,所述将明文机器学习模型中的原生静态图替换为密码静态图,包括:从明文机器学习模型的变量集合中获取损失函数返回的变量名称;确认所述损失函数返回的变量名称是否位于目标模型的变量集合中;当所述损失函数返回的变量名称位于目标模型的变量集合中时,在所述目标模型的静态图中根据所述损失函数返回的变量名称获取张量并返回。
- 如权利要求4所述的隐私机器学习模型生成方法,其特征在于,所述将明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数返回的变量名称未位于目标模型的变量集合中时,确认所述损失函数所返回变量的类型;根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图。
- 如权利要求5所述的隐私机器学习模型生成方法,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态 图,包括:当所述损失函数所返回变量的类型为张量类型时,获取该张量的张量名称;确认所述张量名称是否为占位符;当所述张量名称是占位符时,返回该张量。
- 如权利要求6所述的隐私机器学习模型生成方法,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述张量名称不是占位符时,对该张量所属的原生算子进行递归替换,并返回替换后形成的新算子;获取该张量的输出索引值,并从所述新算子中使用所述输出索引值获取输出边张量并返回。
- 如权利要求5所述的隐私机器学习模型生成方法,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数所返回变量的类型为算子类型时,对该算子对应的边进行递归替换,并返回替换后形成的新算子对应的边;确认该算子对应的数据流中是否包含目标模型的模型私有数据;当该算子对应的数据流中包含目标模型的模型私有数据时,以所述新算子对应的边作为参数,调用密码算子创建函数创建对应的密码算子,并返回所述密码算子。
- 如权利要求8所述的隐私机器学习模型生成方法,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当该算子对应的数据流中未包含目标模型的模型私有数据时,将所述明文机器学习模型的算子深度拷贝至所述目标模型中,并返回深度拷贝的算子。
- 如权利要求5所述的隐私机器学习模型生成方法,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数所返回变量的类型不是张量类型也不是算子类型时,抛出错误。
- 一种隐私机器学习模型生成装置,其特征在于,包括:获取模块,用于获取明文机器学习模型中的静态优化器;执行模块,用于执行所述静态优化器,以将所述明文机器学习模型中的原生明文算子替换为密码算子,从而生成隐私机器学习模型;其中,所述静态优化器根据密码算子优化器类创建,所述密码算子优化器类派生于所述明文机器学习框架的原生优化器类,所述明文机器学习模型基于明文机器学习框架生成。
- 如权利要求11所述的隐私机器学习模型生成装置,其特征在于,所述密码算子包括以下中的任意一种:安全多方计算算子;同态加密算子;零知识证明算子。
- 如权利要求11所述的隐私机器学习模型生成装置,其特征在于,所述将所述明文机器学习模型中的原生明文算子替换为密码算子,包括:将明文机器学习模型中的原生静态图替换为密码静态图。
- 如权利要求13所述的隐私机器学习模型生成装置,其特征在于,所述将明文机器学习模型中的原生静态图替换为密码静态图,包括:从明文机器学习模型的变量集合中获取损失函数返回的变量名称;确认所述损失函数返回的变量名称是否位于目标模型的变量集合中;当所述损失函数返回的变量名称位于目标模型的变量集合中时,在所述目标模型的静态图中根据所述损失函数返回的变量名称获取张量并返回。
- 如权利要求14所述的隐私机器学习模型生成装置,其特征在于,所述将明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数返回的变量名称未位于目标模型的变量集合中时,确认所述损失函数所返回变量的类型;根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图。
- 如权利要求15所述的隐私机器学习模型生成装置,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,包括:当所述损失函数所返回变量的类型为张量类型时,获取该张量的张量名称;确认所述张量名称是否为占位符;当所述张量名称是占位符时,返回该张量。
- 如权利要求16所述的隐私机器学习模型生成装置,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述张量名称不是占位符时,对该张量所属的原生算子进行递归替换,并返回替换后形成的新算子;获取该张量的输出索引值,并从所述新算子中使用所述输出索引值获取输出边张量并返回。
- 如权利要求15所述的隐私机器学习模型生成装置,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数所返回变量的类型为算子类型时,对该算子对应的边进行递归替换,并返回替换后形成的新算子对应的边;确认该算子对应的数据流中是否包含目标模型的模型私有数据;当该算子对应的数据流中包含目标模型的模型私有数据时,以所述新算子对应的边作为参数,调用密码算子创建函数创建对应的密码算子,并返回所述密码算子。
- 如权利要求18所述的隐私机器学习模型生成装置,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当该算子对应的数据流中未包含目标模型的模型私有数据时,将所述明文机器学习模型的算子深度拷贝至所述目标模型中,并返回深度拷贝的算子。
- 如权利要求15所述的隐私机器学习模型生成装置,其特征在于,所述根据所述损失函数所返回变量的类型,将所述明文机器学习模型中的原生静态图替换为密码静态图,还包括:当所述损失函数所返回变量的类型不是张量类型也不是算子类型时,抛出错误。
- 一种电子设备,包括存储器、处理器、以及存储在所述存储器上的计算机程序,其特征在于,所述计算机程序被所述处理器运行时执行权利要求1-10任意一项所述的方法。
- 一种隐私机器学习模型训练方法,其特征在于,包括如下步骤:基于训练样本集训练隐私机器学习模型,以生成第一模型;所述隐私机器学习模型 利用权利要求1-10任意一项所述的方法生成;基于测试样本集测试所述第一模型是否满足预设条件;当所述第一模型满足所述预设条件时,将所述第一模型作为数据预测模型。
- 一种隐私机器学习模型训练装置,其特征在于,包括:模型训练模块,用于基于训练样本集训练隐私机器学习模型,以生成第一模型;所述隐私机器学习模型利用权利要求1-10任意一项所述的方法生成;模型验证模块,用于基于测试样本集测试所述第一模型是否满足预设条件;模型确定模块,用于当所述第一模型满足所述预设条件时,将所述第一模型作为数据预测模型。
- 一种电子设备,包括存储器、处理器、以及存储在所述存储器上的计算机程序,其特征在于,所述计算机程序被所述处理器运行时执行如下步骤:基于训练样本集训练隐私机器学习模型,以生成第一模型;所述隐私机器学习模型利用权利要求1-10任意一项所述的方法生成;基于测试样本集测试所述第一模型是否满足预设条件;当所述第一模型满足所述预设条件时,将所述第一模型作为数据预测模型。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/080391 WO2021184346A1 (zh) | 2020-03-20 | 2020-03-20 | 隐私机器学习模型生成、训练方法、装置及电子设备 |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2020/080391 WO2021184346A1 (zh) | 2020-03-20 | 2020-03-20 | 隐私机器学习模型生成、训练方法、装置及电子设备 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021184346A1 true WO2021184346A1 (zh) | 2021-09-23 |
Family
ID=77767967
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/080391 WO2021184346A1 (zh) | 2020-03-20 | 2020-03-20 | 隐私机器学习模型生成、训练方法、装置及电子设备 |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2021184346A1 (zh) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114691148A (zh) * | 2022-04-11 | 2022-07-01 | 北京百度网讯科技有限公司 | 模型推理加速方法、装置、电子设备及存储介质 |
CN114721913A (zh) * | 2022-05-12 | 2022-07-08 | 华控清交信息科技(北京)有限公司 | 一种生成数据流图的方法、装置和用于生成数据流图的装置 |
CN115358400A (zh) * | 2022-08-24 | 2022-11-18 | 上海人工智能创新中心 | 深度学习模型接口的使用方法 |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108520181A (zh) * | 2018-03-26 | 2018-09-11 | 联想(北京)有限公司 | 数据模型训练方法和装置 |
CN108521326A (zh) * | 2018-04-10 | 2018-09-11 | 电子科技大学 | 一种基于向量同态加密的隐私保护的线性svm模型训练算法 |
CN108717514A (zh) * | 2018-05-21 | 2018-10-30 | 中国人民大学 | 一种机器学习中的数据隐私保护方法和系统 |
CN108959958A (zh) * | 2018-06-14 | 2018-12-07 | 中国人民解放军战略支援部队航天工程大学 | 一种关联大数据的隐私保护方法及系统 |
CN109426861A (zh) * | 2017-08-16 | 2019-03-05 | 阿里巴巴集团控股有限公司 | 数据加密、机器学习模型训练方法、装置及电子设备 |
US20190325995A1 (en) * | 2018-04-20 | 2019-10-24 | NEC Laboratories Europe GmbH | Method and system for predicting patient outcomes using multi-modal input with missing data modalities |
CN110765473A (zh) * | 2019-10-11 | 2020-02-07 | 矩阵元技术(深圳)有限公司 | 数据处理方法、装置、计算机设备和存储介质 |
-
2020
- 2020-03-20 WO PCT/CN2020/080391 patent/WO2021184346A1/zh active Application Filing
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109426861A (zh) * | 2017-08-16 | 2019-03-05 | 阿里巴巴集团控股有限公司 | 数据加密、机器学习模型训练方法、装置及电子设备 |
CN108520181A (zh) * | 2018-03-26 | 2018-09-11 | 联想(北京)有限公司 | 数据模型训练方法和装置 |
CN108521326A (zh) * | 2018-04-10 | 2018-09-11 | 电子科技大学 | 一种基于向量同态加密的隐私保护的线性svm模型训练算法 |
US20190325995A1 (en) * | 2018-04-20 | 2019-10-24 | NEC Laboratories Europe GmbH | Method and system for predicting patient outcomes using multi-modal input with missing data modalities |
CN108717514A (zh) * | 2018-05-21 | 2018-10-30 | 中国人民大学 | 一种机器学习中的数据隐私保护方法和系统 |
CN108959958A (zh) * | 2018-06-14 | 2018-12-07 | 中国人民解放军战略支援部队航天工程大学 | 一种关联大数据的隐私保护方法及系统 |
CN110765473A (zh) * | 2019-10-11 | 2020-02-07 | 矩阵元技术(深圳)有限公司 | 数据处理方法、装置、计算机设备和存储介质 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114691148A (zh) * | 2022-04-11 | 2022-07-01 | 北京百度网讯科技有限公司 | 模型推理加速方法、装置、电子设备及存储介质 |
CN114721913A (zh) * | 2022-05-12 | 2022-07-08 | 华控清交信息科技(北京)有限公司 | 一种生成数据流图的方法、装置和用于生成数据流图的装置 |
CN115358400A (zh) * | 2022-08-24 | 2022-11-18 | 上海人工智能创新中心 | 深度学习模型接口的使用方法 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111415013B (zh) | 隐私机器学习模型生成、训练方法、装置及电子设备 | |
WO2021184346A1 (zh) | 隐私机器学习模型生成、训练方法、装置及电子设备 | |
WO2022206510A1 (zh) | 联邦学习的模型训练方法、装置、设备及存储介质 | |
WO2022089256A1 (zh) | 联邦神经网络模型的训练方法、装置、设备、计算机程序产品及计算机可读存储介质 | |
US11100427B2 (en) | Multi-party computation system for learning a classifier | |
Tang et al. | Non-interactive privacy-preserving truth discovery in crowd sensing applications | |
TWI730622B (zh) | 資料處理方法、裝置和電子設備 | |
CN113239404B (zh) | 一种基于差分隐私和混沌加密的联邦学习方法 | |
US20180204284A1 (en) | Cryptographically secure financial instruments | |
Li et al. | Privacy-preserving feature selection with secure multiparty computation | |
US11410081B2 (en) | Machine learning with differently masked data in secure multi-party computing | |
CN111340614A (zh) | 基于联邦学习的样本采样方法、设备及可读存储介质 | |
TW202103154A (zh) | 資料處理方法、裝置和電子設備 | |
CN111428880A (zh) | 隐私机器学习实现方法、装置、设备及存储介质 | |
US20210042640A1 (en) | Determining model parameters using secret sharing | |
WO2021184347A1 (zh) | 实现隐私保护的数据处理方法和装置 | |
Papadimitriou et al. | DStress: Efficient differentially private computations on distributed data | |
CN110969264A (zh) | 模型训练方法、分布式预测方法及其系统 | |
US20200184081A1 (en) | Generation of a model parameter | |
US20230186049A1 (en) | Training method and apparatus for a neural network model, device and storage medium | |
US20220006614A1 (en) | Secret sigmoid function calculation system, secret logistic regression calculation system, secret sigmoid function calculation apparatus, secret logistic regression calculation apparatus, secret sigmoid function calculation method, secret logistic regression calculation method, and program | |
Bitansky et al. | A note on perfect correctness by derandomization | |
Fang et al. | CostCO: An automatic cost modeling framework for secure multi-party computation | |
CN112150279A (zh) | 一种基于多方计算的金融风险预测方法及预测系统 | |
WO2020211075A1 (zh) | 去中心化多方安全数据处理方法、装置及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20926172 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20926172 Country of ref document: EP Kind code of ref document: A1 |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.03.2023) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20926172 Country of ref document: EP Kind code of ref document: A1 |