WO2021120854A1 - 模型训练方法、成员探测装置的训练方法及其系统 - Google Patents

模型训练方法、成员探测装置的训练方法及其系统 Download PDF

Info

Publication number
WO2021120854A1
WO2021120854A1 PCT/CN2020/123760 CN2020123760W WO2021120854A1 WO 2021120854 A1 WO2021120854 A1 WO 2021120854A1 CN 2020123760 W CN2020123760 W CN 2020123760W WO 2021120854 A1 WO2021120854 A1 WO 2021120854A1
Authority
WO
WIPO (PCT)
Prior art keywords
training
model
data
training sample
output
Prior art date
Application number
PCT/CN2020/123760
Other languages
English (en)
French (fr)
Inventor
周俊
Original Assignee
支付宝(杭州)信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 支付宝(杭州)信息技术有限公司 filed Critical 支付宝(杭州)信息技术有限公司
Publication of WO2021120854A1 publication Critical patent/WO2021120854A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data

Definitions

  • the embodiments of this specification relate to the field of information technology, in particular to a model training method, a training method of a member detection device, and a system thereof.
  • One of the embodiments of this specification provides a model training method, which includes: training a model based on a training set; for any training sample in the training set, acquiring feature data, label data, and all data of the member detection device based on the training sample
  • the model is based on the output value of the output result of the training sample, and the output value reflects the probability of correctly inferring that the training sample belongs to the training set based on the output result; adjust the parameters and/or structure of the model to Decrease the output value.
  • One of the embodiments of this specification provides a model training system, which includes: a model training module for training a model based on a training set; a first acquisition module for acquiring member detection for any training sample in the training set The device is based on the feature data of the training sample, the label data, and the output value obtained by the model based on the output result of the training sample, the output value reflecting the probability of correctly inferring that the training sample belongs to the training set based on the output result ; An adjustment module for adjusting the parameters and/or structure of the model to reduce the output value.
  • One of the embodiments of this specification provides a model training device, which includes at least one processor and at least one storage device.
  • the storage device is used to store instructions.
  • One of the embodiments of this specification provides a training method for a member detection device, which includes: for any training sample in a training set of a model, acquiring feature data and label data of the training sample and the model based on the training sample The output result is used as training input data, and the information value reflecting the prediction performance of the model for the training sample is obtained as training output reference data; the training input data and training output reference data determined based on each training sample in the training set are used to train the member detection Device.
  • One of the embodiments of the present specification provides a training system for a member detection device, which includes: a second acquisition module for acquiring any training sample in the training set of the model to acquire feature data, label data, and all data of the training sample.
  • the model is based on the output result of the training sample as the training input data, and the information value reflecting the prediction performance of the model for the training sample is obtained as the training output reference data; the device training module is used to use the training sample determined based on the training set Training input data and training output reference data train the member detection device.
  • One of the embodiments of this specification provides a training device for a member detection device, which includes at least one processor and at least one storage device.
  • the storage device is used to store instructions.
  • the implementation is as follows: The training method of the member detection device described in any embodiment of the specification.
  • Fig. 1 is a schematic diagram of an application scenario of a machine learning system according to some embodiments of this specification
  • Fig. 2 is an exemplary flowchart of a model training method according to some embodiments of the present specification
  • Fig. 3 is an exemplary flowchart of a training method of a member detection device according to some embodiments of the present specification
  • Fig. 4 is an exemplary block diagram of a model training system according to some embodiments of the present specification.
  • Fig. 5 is an exemplary block diagram of a training system of a member detection device according to some embodiments of the present specification.
  • system means for distinguishing different components, elements, parts, parts, or assemblies of different levels.
  • device means for distinguishing different components, elements, parts, parts, or assemblies of different levels.
  • unit means for distinguishing different components, elements, parts, parts, or assemblies of different levels.
  • the words can be replaced by other expressions.
  • one party can put the trained model online to provide data prediction services for other parties.
  • other parties may input the feature data of the object to be predicted into the model to obtain the prediction result.
  • the model’s predictive performance on training samples is generally better than that of non-training samples
  • an attacker can determine whether the object belongs to the model based on the model’s predictive performance on an object (for example, user ID) Training set. That is, if the model shows better predictive performance for an object, it indicates that the object is more likely to belong to the training set of the model.
  • the object may be an entity such as a user, a merchant, an individual, a group, an institution, an organization, a company, or a school
  • the characteristic data may be based on one of text data, image data, and sound data of the entity or Multiple access.
  • the text data, image data, sound data and other data of the entity are private data to be protected.
  • the image data may include the user's face data, fingerprint data, etc.
  • the text data may include the user's gender, age, height, etc.
  • the sound data may include the user's recordings, etc.
  • an attacker can obtain characteristic data and label data of an object, input the characteristic data of the object into the model, and obtain the prediction result of the object output by the model. Furthermore, the attacker can judge whether the object belongs to the training set of the model by comparing the prediction result of the object with the label data of the object. If the difference between the prediction result output by the model and the label data of the object is small enough, that is, the model has a high prediction accuracy for the object, the object is considered to belong to the training set of the model. Wherein, whether the difference between the prediction result output by the model and the label data of the object is small enough depends on the magnitude of the label data. Just as an example, if the prediction result output by the model and the label data of the object are 0.99 and 1.00, respectively, the object can be considered to belong to the training set of the model.
  • an attacker can obtain feature data and label data of an object, input the feature data of the object (hereinafter referred to as original feature data) into the model, and obtain the prediction result of the object output by the model (hereinafter referred to as the original prediction result) .
  • some changes are made to the feature data of the object (for example, noise is added), and the result of the change is input to the model to obtain a prediction result output by the model based on the result of the change.
  • the attacker can add noise to the face image data.
  • the noise can be expressed in the face image as moles, beards, imprints, accessories, etc., or the face image can be rotated to a certain angle (0 ⁇ 360°), or the face image can be translated a certain distance.
  • the object can be considered Belongs to the training set of the model.
  • a model training method may be provided, so that the trained model can effectively prevent member attacks.
  • Fig. 1 is an application scenario diagram of a machine learning system 100 according to some embodiments of this specification.
  • the system 100 may include a server 110, a user 130, and a network 120.
  • the server 110 may be a computing device or a group of computing devices.
  • the computing device may be a mobile device, a tablet computer, a laptop computer, a desktop computer, etc., or any combination thereof.
  • the group of computing devices may be centralized or distributed.
  • the server 110 may be local or remote.
  • the server 110 may access information and/or data stored on the user terminal 130 or other network nodes on the network 120 via the network 120.
  • the server 110 may be implemented on a cloud platform.
  • the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.
  • the server 110 can use the training samples to complete the training of the machine model, and will launch the model to provide model prediction services to other users in the network 120.
  • the server 110 can train the model by means of a member detection device, so that it is difficult for an attacker to perform a member attack on the model, that is, it is difficult to detect the model through the feature data of the object to be predicted and the known label data. Whether the object to be predicted is in the training set of the model.
  • the server 110 may also implement training for the member detection device.
  • the client 130 may also be a computing device or a group of computing devices.
  • the computing device may be a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, a desktop computer 130-4, etc., or any combination thereof.
  • the group of computing devices may be centralized or distributed.
  • the client 130 may be local or remote.
  • the client 130 may access information and/or data stored on the server 110 or other network nodes on the network 120 via the network 120.
  • the client 130 may be implemented on a cloud platform.
  • the cloud platform may include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an internal cloud, a multi-layer cloud, etc., or any combination thereof.
  • the client 130 may use its own characteristic data and use the model provided by the server 110 to complete the prediction. That is, the model prediction service provided by the server 110 is used.
  • the client 130 may be owned by the attacker. The attacker can input the characteristic data of the object to be predicted with known label data into the model, and test the to be predicted based on the predictive performance of the model on the characteristic data. Whether the object is in the training set of the model. Among them, the prediction performance may include the prediction accuracy or robustness of the model for the set of feature data.
  • the network 120 may facilitate the exchange of data and/or information.
  • one or more components in the machine learning system 100 eg, the server 110 and the user 130
  • the client 130 may send the characteristic data to the server 110 via the network 120.
  • the server 110 may send the prediction result of the object to be predicted to the user 130 via the network 120.
  • the network 120 may be any type of wired or wireless network.
  • the network 120 may include a cable network, a wired network, an optical fiber network, a telecommunication network, an internal network, an Internet network, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), and a metropolitan area network (MAN) , Public Switched Telephone Network (PSTN), Bluetooth network, ZigBee network, Near Field Communication (NFC) network, etc. or any combination of the above.
  • the network 120 may include one or more network entry and exit points.
  • the network 120 may include wired and/or wireless network access points, such as base stations and/or multiple Internet exchange points 120-1, 120-2, through which one or more components of the machine learning system 100 can be Connect to the network 120 to exchange data and/or information.
  • Fig. 2 is an exemplary flowchart of a model training method according to some embodiments of the present specification.
  • the process 200 may include step 210 to step 230.
  • Step 210 Train a model based on the training set.
  • step 210 may be implemented by the model training module 410.
  • the model involved in this specification can be used in any prediction scenario, and the training samples in the corresponding training set can also be the feature data and label values of any object to be predicted.
  • the training sample may include private data of the entity, which may include multiple types such as text data, image data, or sound data.
  • the model may be a model trained on the training set to a required accuracy, or may be a model that has undergone preliminary training and the accuracy needs to be further improved.
  • the models mentioned in this specification include but are not limited to neural networks.
  • the model can be trained to meet the required accuracy by updating the model parameters.
  • the parameter update method includes but is not limited to the back propagation method.
  • step 210 needs to be used in subsequent steps to strengthen its ability to prevent member detection.
  • Step 220 For any training sample in the training set, obtain the feature data and label data of the member detection device based on the training sample and the output value of the model based on the output result of the training sample.
  • the output value reflects the output value based on the training sample.
  • the output result correctly infers the probability that the training sample belongs to the training set.
  • step 220 may be implemented by the first obtaining module 420.
  • a member detection device may be introduced for intensive training of the model.
  • the member detection device can simulate the process of the attacker's member detection. For example, it can fit the probability of an attacker successfully detecting whether the feature data belongs to the model training set based on the feature data, label data, and the prediction result of the model.
  • the member detection device may be implemented by one or more of machine learning models, other mathematical models, algorithms, and the like.
  • the member detection device can be pre-trained.
  • the member detection device obtains the feature data, label data and model output results of any training sample based on the training sample as input.
  • the output can reflect the probability of correctly inferring that the training sample belongs to the training set (can be It is called the output value of the member detection success rate of the training sample.
  • the output value may include a positive information value and a negative information value, where the positive information value may reflect a higher success rate of member detection of the training sample, and the negative information value may reflect a higher success rate of member detection of the training sample. low.
  • the positive information value and the negative information value may be 1, 0, or +1, -1, respectively.
  • the output value may be a score value, and the size of the score value may correspond to the size of the member's detection success rate. For example, the larger the score, the greater the member's detection success rate, and conversely, the smaller the score, the smaller the member's detection success rate.
  • Step 230 Adjust the parameters and/or structure of the model to reduce the output value.
  • step 230 may be implemented by the adjustment module 430.
  • the adjustment module 430 can reduce the output value corresponding to the training sample by adjusting the parameters and/or structure of the model, so as to target all the training samples in the training set. Strengthen the model's ability to prevent member detection.
  • the reduction of member detection success rate can be determined by the specific form of the output value. For example, when the output value includes a positive information value and a negative information value, if the output value corresponding to a certain training sample is a positive information value (that is, the member of the training sample has a higher detection success rate), the model can be adjusted. Parameters and/or structure so that the output value corresponding to the training sample changes from a positive information value to a negative information value. For another example, when the output value is a score and the score increases as the member detection success rate increases, the parameters and/or structure of the model can be adjusted so that the output value corresponding to any training sample is lower than Set the threshold.
  • the threshold may be set based on a range of scores. For example, when the range of scores is 0 to 1, the threshold may be set to 0.4, 0.1, 0.05, 0.01, and so on.
  • the attacker can infer whether the object belongs to the training set of the model based on the predictive performance of the model for an object. Based on this, to reduce the member detection success rate of any training sample, the adjustment module 430 needs to appropriately sacrifice (ie reduce) the prediction performance of the model for the training sample.
  • the following are examples of several specific methods to reduce the prediction performance of the model for training samples.
  • the adjustment module 430 can make the corresponding output of one or more nodes of the model uncertain under the condition of a given input value, so that the prediction result of the model based on the same feature data output has uncertainty. Uncertainty. In this way, the prediction accuracy of the model for the same training sample can be made uncertain, thereby reducing the success rate of the attacker's detection of members of a certain training sample.
  • the one or more nodes may be one or more neurons in the model, for example only, such as a model output layer node.
  • the manner of making the corresponding output of the one or more nodes uncertain under the condition of a given input value may include: adding noise to the output of the one or more nodes.
  • the added noise may include one or more of Gaussian noise, Laplacian noise, white noise, and the like.
  • the magnitude of the added noise depends on the magnitude of the output of the corresponding node, that is, the magnitude of the added noise is lower than the magnitude of the output of the corresponding node.
  • the numerical range of the output of a certain node is 10-20, and the numerical range of the noise added to the node can be -1 ⁇ 1.
  • the adjustment module 430 can shield one or more nodes in the model, thereby reducing the prediction accuracy of the model, which can reduce the success rate of the attacker's member detection of a certain training sample.
  • the adjustment module 430 may make one or more nodes of the hidden layer in the model output zero.
  • step 210 may be repeatedly performed. That is, after adjusting the parameters or structure of the model, the training set is used to train the model again. The two methods can be used for multiple rounds alternately to reduce the membership of the training sample under the premise that the accuracy of the model meets the required accuracy. Detection success rate.
  • Fig. 3 is an exemplary flowchart of a training method of a member detection device according to some embodiments of the present specification.
  • the process 300 may include step 310 to step 320.
  • Step 310 For any training sample in the training set of the model, obtain the feature data, label data, and the output result of the model based on the training sample as training input data, and obtain the information reflecting the model's response to the training sample.
  • the information value of the predicted performance is used as the training output reference data.
  • step 310 may be implemented by the second acquisition module 510.
  • the model may be any model that uses the member detection device to enhance its ability to prevent member detection, such as the model in the process 200.
  • the training input data is equivalent to the feature data of the training sample of the member detection device
  • the training output reference data is equivalent to the label data of the training sample of the member detection device
  • the training input data and the training output reference data constitute the member Training sample data of the detection device. It can be understood that the training samples of the member detection device are generated based on the training samples of the model.
  • the success rate of the attacker's member detection of any training sample can depend on the predictive performance of the model on the training sample. Therefore, the information value can reflect the prediction performance of the model on the training sample, and can also reflect the success rate of the attacker's detection of any training sample member. That is, the information value and the output value reflect the same content, so the second acquisition module 510 can use the information value as the label data of the training sample of the member detection device.
  • the information value can be It is positively correlated with the prediction accuracy of the model for the training sample; when the prediction result output by the model based on one or more changes of any training sample is sufficiently small compared to the prediction result or label data of the training sample, the attacker can It is considered that the training sample belongs to the training set of the model to which it belongs. Therefore, in some embodiments, the information value may be compared with the prediction result output by the model based on one or more changes in the feature data of the training sample relative to the training sample. Differences in prediction results or label data are negatively correlated.
  • the information value can be constructed based on at least one of the above two relationships and other relationships.
  • the information value may be in a discrete form. For example, when the prediction accuracy of the model for a certain training sample is greater than a set threshold, the information value is 1, otherwise, the information value is 0.
  • the prediction accuracy may be the similarity between the model prediction result and the label data, and the set threshold may be 0.85, 0.9, 0.95, and so on.
  • the feature data of a certain training sample can be deformed, such as adding noise or twisting the feature data, and then outputting the change result to the model to obtain the prediction result.
  • the prediction result corresponding to the changed feature data can be compared with The original prediction results (that is, the prediction results corresponding to the feature data) are compared. When the difference between the two is less than the set threshold, the confidence value is 1, and vice versa.
  • the set threshold may be 0.2, 0.1, 0.05, and so on.
  • the information value can also be a continuous value directly.
  • step 320 the member detection device is trained using the training input data determined based on the training samples in the training set and the training output reference data.
  • step 320 may be implemented by the device training module 520.
  • the member detection device can be trained with the model. For example, after the model is trained to a certain degree, the prediction result is obtained based on the model and the information value is constructed based on its prediction performance to train the member detection device. When the member detection device is trained to a certain level, the model is adjusted based on the output result of the member detection device.
  • the model training process alternates with the device training process to obtain a model that meets the requirements.
  • Fig. 4 is an exemplary block diagram of a model training system according to some embodiments of the present specification.
  • the system 400 may include a model training module 410, a first acquisition module 420, and an adjustment module 430.
  • the model training module 410 may be used to train a model based on a training set.
  • the first obtaining module 420 may be used to obtain, for any training sample in the training set, the feature data and label data of the member detection device based on the training sample, and the output result of the model based on the training sample.
  • the obtained output value reflects the probability of correctly inferring that the training sample belongs to the training set based on the output result.
  • the training sample may include private data of the entity, and the private data may include one or more of text data, image data, and sound data.
  • the adjustment module 430 may be used to adjust the parameters and/or structure of the model to reduce the output value. In some embodiments, the adjustment module 430 may be further used to make the corresponding output of one or more nodes of the model uncertain under a given input condition. In some embodiments, the adjustment module 430 may add noise to the output of the one or more nodes, so that the corresponding output of the one or more nodes under a given input condition has uncertainty. In some embodiments, the adjustment module 430 may be further used to shield one or more nodes in the model.
  • the model training system 400 may further include a member detection device training module 440.
  • the member detection device training module 440 may be used to: for any training sample in the training set of the model, obtain the feature data, label data of the training sample, and the output result of the model based on the training sample as training Input data, obtain the information value reflecting the prediction performance of the model for the training sample as training output reference data; train the member detection device by using the training input data and training output reference data determined based on each training sample in the training set.
  • the member detection device training module 440 may be implemented by the member detection device training system 500 disclosed in FIG. 5. That is, for more details about the member detection device training module 440, please refer to FIG. 5 and related descriptions.
  • Fig. 5 is an exemplary block diagram of a training system of a member detection device according to some embodiments of the present specification.
  • the system 500 may include a second acquisition module 510 and a device training module 520.
  • the second acquisition module 510 may be used for any training sample in the training set of the model to acquire feature data, label data of the training sample, and the output result of the model based on the training sample as training input data. Obtain the information value reflecting the prediction performance of the model for the training sample as the training output reference data. In some embodiments, the information value may be positively correlated with the prediction accuracy of the model for the training sample. In some embodiments, the information value may be negatively related to the difference between the prediction result output by the model based on one or more change results of the characteristic data of the training sample with respect to the prediction result or label data of the training sample.
  • the training sample may include private data of the entity, and the private data may include one or more of text data, image data, and sound data.
  • the device training module 520 may be used to train the member detection device using training input data and training output reference data determined based on each training sample in the training set.
  • system and its modules shown in FIG. 4 and FIG. 5 can be implemented in various ways.
  • the system and its modules may be implemented by hardware, software, or a combination of software and hardware.
  • the hardware part can be implemented using dedicated logic;
  • the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware.
  • processor control codes for example on a carrier medium such as a disk, CD or DVD-ROM, such as a read-only memory (firmware Such codes are provided on a programmable memory or a data carrier such as an optical or electronic signal carrier.
  • the system and its modules in this specification can not only be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc. It can also be implemented by, for example, software executed by various types of processors, or can be implemented by a combination of the foregoing hardware circuit and software (for example, firmware).
  • the first acquisition module 420 and the adjustment module 430 disclosed in FIG. 4 may be different modules in a system, or one module may realize the functions of the two modules.
  • the member detection device training module 440 disclosed in FIG. 4 may be implemented by the member detection device training system 500 shown in FIG. 5.
  • the first acquisition module 420/model training module 410 disclosed in FIG. 4 and the second acquisition module 510/device training module 520 disclosed in FIG. 5 may be two modules, or may be combined into One module. Such deformations are all within the protection scope of this specification.
  • the possible beneficial effects of the embodiments of this specification include, but are not limited to: (1) The member detection device can evaluate that the attacker correctly infers that the training sample belongs to the training set of the model based on the feature data, label data, and prediction result of any training sample With the assistance of the member detection device, the parameters and/or structure of the model can be adjusted to strengthen the model’s ability to prevent member detection; (2) By repeating the relevant steps or procedures, the accuracy of the model can still meet the requirements. It should be noted that different embodiments may have different beneficial effects. In different embodiments, the possible beneficial effects may be any one or a combination of the above, or any other beneficial effects that may be obtained.
  • the computer storage medium may contain a propagated data signal containing a computer program code, for example on a baseband or as part of a carrier wave.
  • the propagated signal may have multiple manifestations, including electromagnetic forms, optical forms, etc., or a suitable combination.
  • the computer storage medium may be any computer readable medium other than the computer readable storage medium, and the medium may be connected to an instruction execution system, device, or device to realize communication, propagation, or transmission of the program for use.
  • the program code located on the computer storage medium can be transmitted through any suitable medium, including radio, cable, fiber optic cable, RF, or similar medium, or any combination of the above medium.
  • the computer program codes required for the operation of each part of the embodiments of this specification can be written in any one or more programming languages, including object-oriented programming languages such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET , Python, etc., conventional programming languages such as C language, VisualBasic, Fortran2003, Perl, COBOL2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages, etc.
  • the program code can run entirely on the user's computer, or as an independent software package on the user's computer, or partly on the user's computer and partly on a remote computer, or entirely on the remote computer or processing equipment.
  • the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (for example, via the Internet), or in a cloud computing environment, or as a service Use software as a service (SaaS).
  • LAN local area network
  • WAN wide area network
  • SaaS service Use software as a service

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Security & Cryptography (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

模型训练方法、成员探测装置的训练方法及其系统，以防范成员攻击造成的数据隐私泄露。其中，成员探测装置可以基于特征数据、标签数据和模型的预测结果评估攻击者基于这些数据成功探测出该特征数据是否属于模型训练集的概率。进而，模型训练系统可以通过调节所训练模型的参数和/或结构来降低该概率。

Description

模型训练方法、成员探测装置的训练方法及其系统 技术领域
本说明书实施例涉及信息技术领域，特别涉及模型训练方法、成员探测装置的训练方法及其系统。
背景技术
如今，机器学习模型在金融、医疗等领域得到了广泛的应用。在模型预测阶段，攻击者通过一定手段可以推断出某用户ID是否属于模型的训练集，这种攻击方式可称为“成员探测”或“成员攻击”。当训练集来源于疾病数据库、黑名单、人脸数据库等敏感数据集时，会暴露该用户ID是否属于敏感数据集，从而造成用户隐私的泄露。
目前，希望提供一种能够防范成员探测的机器学习方案。
发明内容
本说明书实施例之一提供一种模型训练方法，其中，包括：基于训练集训练模型；对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果得到的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率；调节所述模型的参数和/或结构，以降低所述输出值。
本说明书实施例之一提供一种模型训练系统，其中，包括：模型训练模块，用于基于训练集训练模型；第一获取模块，用于对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果得到的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率；调节模块，用于调节所述模型的参数和/或结构，以降低所述输出值。
本说明书实施例之一提供一种模型训练装置，其中，包括至少一个处理器和至少一个存储设备，所述存储设备用于存储指令，当至少一个处理器执行指令时，实现如本说明书任一实施例所述的模型训练方法。
本说明书实施例之一提供一种成员探测装置的训练方法，其中，包括：对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该 训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据；利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
本说明书实施例之一提供一种成员探测装置的训练系统，其中，包括：第二获取模块，用于对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据；装置训练模块，用于利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
本说明书实施例之一提供一种成员探测装置的训练装置，其中，包括至少一个处理器和至少一个存储设备，所述存储设备用于存储指令，当至少一个处理器执行指令时，实现如本说明书任一实施例所述的成员探测装置的训练方法。
附图说明
本说明书将以示例性实施例的方式进一步说明，这些示例性实施例将通过附图进行详细描述。这些实施例并非限制性的，在这些实施例中，相同的编号表示相同的结构。
图1根据本说明书一些实施例所示的机器学习系统的应用场景示意图；
图2为根据本说明书一些实施例所示的模型训练方法的示例性流程图；
图3为根据本说明书一些实施例所示的成员探测装置的训练方法的示例性流程图；
图4为根据本说明书一些实施例所示的模型训练系统的示例性框图；
图5为根据本说明书一些实施例所示的成员探测装置的训练系统的示例性框图。
具体实施方式
为了更清楚地说明本说明书实施例的技术方案，下面将对实施例描述中所需要使用的附图作简单的介绍。显而易见地，下面描述中的附图仅仅是本说明书的一些示例或实施例，对于本领域的普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图将本说明书应用于其它类似情景。除非从语言环境中显而易见或另做说明，图中相同标号代表相同结构或操作。
应当理解，本文使用的“系统”、“装置”、“单元”和/或“模组”是用于区分不同级别的 不同组件、元件、部件、部分或装配的一种方法。然而，如果其他词语可实现相同的目的，则可通过其他表达来替换所述词语。
如本说明书和权利要求书中所示，除非上下文明确提示例外情形，“一”、“一个”、“一种”和/或“该”等词并非特指单数，也可包括复数。一般说来，术语“包括”与“包含”仅提示包括已明确标识的步骤和元素，而这些步骤和元素不构成一个排它性的罗列，方法或者设备也可能包含其它的步骤或元素。
本说明书中使用了流程图用来说明根据本说明书的实施例的系统所执行的操作。应当理解的是，前面或后面操作不一定按照顺序来精确地执行。相反，可以按照倒序或同时处理各个步骤。同时，也可以将其他操作添加到这些过程中，或从这些过程移除某一步或数步操作。
在一些应用场景中，某一方可以将训练好的模型上线为其他方提供数据预测服务。具体地，其他方可以将待预测对象的特征数据输入到所述模型中，得到预测结果。由于模型对训练样本表现出的预测性能相对非训练样本通常要好，在一些成员探测的情景中，攻击者可基于模型对于某对象(例如，用户ID)的预测性能来判断该对象是否属于该模型的训练集。即，若模型对于某对象表现出较好的预测性能，则表明该对象属于该模型的训练集的可能性更大。在一些实施例中，所述对象可以是用户、商户、个人、团体、机构、组织、公司、学校等实体，所述特征数据可以基于实体的文本数据、图像数据、声音数据中的一种或多种获取。在一些实施例中，所述实体的文本数据、图像数据、声音数据等数据为有待保护的隐私数据。仅作为示例，图像数据可以包括用户的人脸数据、指纹数据等等，文本数据可以包括用户的性别、年龄、身高等等，声音数据可以包括用户的录音等等。
例如，攻击者可以获取某对象的特征数据和标签数据，将该对象的特征数据输入模型，获得模型输出的该对象的预测结果。进而，攻击者可通过比较该对象的预测结果与该对象的标签数据来判断该对象是否属于该模型的训练集。若模型输出的预测结果与该对象的标签数据的差异足够小，即模型对该对象的预测精度较高，则认为该对象属于该模型的训练集。其中，模型输出的预测结果与该对象的标签数据的差异是否足够小可视标签数据的量级而定。仅作为示例，若模型输出的预测结果与该对象的标签数据分别为0.99和1.00，则可认为该对象属于模型的训练集。
又如，攻击者可以获取某对象的特征数据和标签数据，将该对象的特征数据(以下称为原特征数据)输入模型，获得模型输出的该对象的预测结果(以下称为原预测结 果)。另外，对该对象的特征数据做某种变化(例如，添加噪声)，并将变化结果输入到模型中，获得模型基于该变化结果输出的预测结果。以特征数据为人脸图像数据为例，攻击者可以在人脸图像数据中添加噪声，该噪声在人脸图像中可表现为痣、胡子、印记、饰品等，或者可以将人脸图像旋转一定角度(0～360°)，又或者可以将人脸图像平移一定距离。当发现模型基于原特征数据的一个或多个变化结果输出的预测结果相对标签数据或原预测结果的差距足够小时，即模型对该对象的具有良好的鲁棒性，攻击者可认为该对象属于模型的训练集。其中，模型基于原特征数据的一个或多个变化结果输出的预测结果相对原预测结果的变化是否足够小可视原预测结果或标签数据的量级而定。仅作为示例，若该对象的原预测结果为7.00，模型基于原特征数据的原特征数据的3个变化结果输出的预测结果分别为6.97、7.01、6.99或者均仍为7.00，则可认为该对象属于模型的训练集。
因此，在一些实施例中，可以提供一种模型训练方法，以使训练好的模型能够有效的防止成员攻击。
图1为根据本说明书一些实施例所示的机器学习系统100的应用场景图。系统100可以包括服务端110、用户端130以及网络120。
在一些实施例中，服务端110可以是一台计算设备或计算设备组。所述计算设备可以是移动设备、平板计算机、膝上型计算机、台式计算机等或其任意组合。所述计算设备组可以是集中式或分布式的。在一些实施例中，服务端110可以是本地的，也可以是远程的。例如，服务端110可以经由网络120访问存储于用户端130或网络120上其他网络节点的信息和/或数据。在一些实施例中服务端110可以在云平台上实施。仅作为示例，所述云平台可以包括私有云、公共云、混合云、社区云、分布云、内部云、多层云等或其任意组合。
在一些实施例中，服务端110可以利用训练样本完成机器模型的训练，并将将模型上线以为网络120中的其他用户提供模型预测服务。在一些实施例中，服务端110可以借助成员探测装置训练所述模型，使攻击者难以对所述模型进行成员攻击，即难以通过待预测对象的特征数据以及已知的标签数据试探出所述待预测对象是否位于该模型的训练集中。在一些实施例中，服务端110还可以实现对所述成员探测装置的训练。
在一些实施例中，用户端130也可以是一台计算设备或计算设备组。所述计算设备可以是移动设备130-1、平板计算机130-2、膝上型计算机130-3、台式计算机130-4等或其任意组合。所述计算设备组可以是集中式或分布式的。在一些实施例中，用户端 130可以是本地的，也可以是远程的。例如，用户端130可以经由网络120访问存储于服务端110或网络120上其他网络节点的信息和/或数据。在一些实施例中用户端130可以在云平台上实施。仅作为示例，所述云平台可以包括私有云、公共云、混合云、社区云、分布云、内部云、多层云等或其任意组合。
在一些实施例中，用户端130可以使用自身拥有的特征数据，并借助服务端110提供的模型完成预测。即使用服务端110提供的模型预测服务。在一些实施例中，用户端130可能为攻击者所有，攻击者可以将已知标签数据的待预测对象的特征数据输入到所述模型中，基于模型对该特征数据的预测性能试探该待预测对象是否位于所述模型的训练集中。其中，预测性能可以包括模型对该组特征数据的预测精度或鲁棒性等。
网络120可促进数据和/或信息的交换。在一些实施例中，机器学习系统100中的一个或多个组件(如，服务端110和用户端130)可通过网络120传输数据和/或信息至机器学习系统100中的其他组件。例如，用户端130可通过网络120将特征数据发送给服务端110。又如，服务端110可通过网络120将待预测对象的预测结果发送给用户端130。在一些实施例中，网络120可是任意类型的有线或无线网络。例如，网络120可包括缆线网络、有线网络、光纤网络、电信网络、内部网络、网际网络、区域网络(LAN)、广域网络(WAN)、无线区域网络(WLAN)、都会区域网络(MAN)、公共电话交换网络(PSTN)、蓝牙网络、ZigBee网络、近场通讯(NFC)网络等或以上任意组合。在一些实施例中，网络120可包括一个或多个网络进出点。例如，网络120可包含有线和/或无线网络进出点，如基站和/或多个网际网络交换点120-1、120-2，通过这些进出点，机器学习系统100的一个或多个组件可连接到网络120上以交换数据和/或信息。
图2为根据本说明书一些实施例所示的模型训练方法的示例性流程图。流程200可以包括步骤210～步骤230。
步骤210，基于训练集训练模型。在一些实施例中，步骤210可以由模型训练模块410实现。
本说明书涉及的模型可以用于任意预测场景，相应的训练集中的训练样本也可以是关于任意待预测对象的特征数据以及标签值。在一些实施例中，所述训练样本可以包括实体的隐私数据，其可以包括文本数据、图像数据或声音数据等多种类型。在一些实施例中，所述模型可以是基于训练集训练到精度达到要求的模型，也可以是经过初步训练，精度还有待进一步提高的模型。
在一些实施例中，本说明书提及的模型包括但不限于神经网络。在一些实施例中，可以通过更新模型参数来训练出满足所需精度的模型。在一些实施例中，参数更新的方法包括但不限于反向传播法。
应当理解，步骤210中训练出的模型有待通过后续步骤来加强其防范成员探测的能力。
步骤220，对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率。在一些实施例中，步骤220可以由第一获取模块420实现。
由前述的成员探测情景可知，攻击者可基于某对象的特征数据、标签数据和模型基于该对象的预测结果确定模型对于该对象的预测性能，进而推断出该对象是否属于该模型的训练集。为了提供所述模型防范成员探测的能力，在一些实施例中，可以引入成员探测装置进行模型的强化训练。在一些实施例中，成员探测装置可以模拟攻击者进行成员探测的过程。如，其可以基于特征数据、标签数据和模型的预测结果拟合攻击者基于这些数据成功探测出该特征数据是否属于模型训练集的概率。在一些实施例中，成员探测装置可以是机器学习模型或其他数学模型、算法等方式中的一种或多种来实现。所述成员探测装置可以预先训练好。在模型训练过程中，成员探测装置获得任一训练样本的特征数据、标签数据和模型基于该训练样本的输出结果作为输入，输出可反映正确推断出该训练样本属于所述训练集的概率(可称为该训练样本的成员探测成功率)的所述输出值。
在一些实施例中，所述输出值可以包括正信息值和负信息值，其中，正信息值可反映训练样本的成员探测成功率较高，负信息值可反映训练样本的成员探测成功率较低。例如，正信息值和负信息值可分别为1、0，或者+1、-1。在一些实施例中，所述输出值可以为分值，分值的大小可与成员探测成功率的大小对应。例如，分值越大则表示成员探测成功率越大，反过来分值越小则表示成员探测成功率越小。
关于成员探测装置的实现方式，可以参考图3及其相关描述。
步骤230，调节所述模型的参数和/或结构，以降低所述输出值。在一些实施例中，步骤230可以由调节模块430实现。
降低任一训练样本对应的所述输出值意味着：针对该训练样本，调节后的模型防 范成员探测的能力得到加强。在一些实施例中，对于训练集中的每个训练样本，调节模块430可通过调节所述模型的参数和/或结构，来减小该训练样本对应的输出值，从而针对训练集中的所有训练样本加强模型防范成员探测的能力。
成员探测成功率的降低可以由所述输出值的具体形式决定。例如，当所述输出值包括正信息值和负信息值时，若某训练样本对应的输出值为正信息值(即该训练样本的成员探测成功率较高)，则可以调节所述模型的参数和/或结构，以使该训练样本对应的输出值由正信息值变为负信息值。又如，当所述输出值为分值且分值随成员探测成功率增大而增大时，可以调节所述模型的参数和/或结构，以使任一训练样本对应的输出值低于设定阈值。在一些实施例中，该阈值可基于分值的范围设定，例如，分值的范围为0～1时，该阈值可设定为0.4、0.1、0.05、0.01等等。
根据成员探测的相关内容，攻击者可基于模型对于某对象的预测性能推断出该对象是否属于该模型的训练集。基于此，若要降低任一训练样本的成员探测成功率，调节模块430需要适当牺牲(即降低)模型对于该训练样本的预测性能。下面示例几种降低模型对于训练样本的预测性能的具体方法。
在一些实施例中，调节模块430可以使所述模型的一个或多个节点在给定输入值的条件下对应的输出具有不确定性，进而使得所述模型基于同一特征数据输出的预测结果具有不确定性。如此，可以使所述模型对于同一训练样本的预测精度具有不确定性，从而可降低攻击者对某训练样本的成员探测成功率。在具体的实施例中，所述一个或多个节点可以模型中的一个或多个神经元，仅作为示例，如模型输出层节点。
在一些实施例中，使所述一个或多个节点在给定输入值的条件下对应的输出具有不确定性的方式可以包括：在所述一个或多个节点的输出中添加噪声。在一些实施例中，添加的噪声可以包括高斯噪声、拉普拉斯噪声、白噪声等中的一种或多种。在一些实施例中，所添加的噪声的量级视相应节点的输出的量级而定，即，所添加的噪声的量级低于相应节点的输出的量级。例如，某节点的输出的数值范围为10～20，添加至该节点的噪声的数值范围可为-1～1。
在一些实施例中，调节模块430可以屏蔽所述模型中的一个或多个节点，进而降低所述模型的预测精度，即可降低攻击者对某训练样本的成员探测成功率。仅作为示例，调节模块430可以使模型中隐藏层的一个或多个节点输出为0。
值得说明的是，调整所述模型的参数和/或结构可能会导致所述模型的精度无法满 足所需精度。为此，在一些实施例中，可重复执行步骤210。即，对模型的参数或结构进行调节后，再次利用训练集训练所述模型，两种手段可以交替进行多轮，以在所述模型的精度满足所需精度的前提下，降低训练样本的成员探测成功率。
图3为根据本说明书一些实施例所示的成员探测装置的训练方法的示例性流程图。流程300可以包括步骤310～步骤320。
步骤310，对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据。在一些实施例中，步骤310可以由第二获取模块510实现。
其中，所述模型可以是借助成员探测装置加强其防范成员探测能力的任一模型，如流程200中的模型。所述训练输入数据相当于成员探测装置的训练样本的特征数据，所述训练输出参考数据相当于成员探测装置的训练样本的标签数据，所述训练输入数据和所述训练输出参考数据即构成成员探测装置的训练样本数据。可以理解，成员探测装置的训练样本是基于所述模型的训练样本生成的。
根据成员探测的相关内容，攻击者对任一训练样本的成员探测成功率可取决于模型对该训练样本的预测性能。因此，所述信息值可以反映模型对训练样本的预测性能，也就可以反映攻击者对任一训练样本的成员探测成功率。即，所述信息值和所述输出值反映一致的内容，从而第二获取模块510可以将所述信息值作为成员探测装置的训练样本的标签数据。
进一步地，根据成员探测的相关内容：当模型对于任一训练样本的预测精度较高时，攻击者可认为该训练样本属于所属模型的训练集，因此在一些实施例中，所述信息值可以与所述模型对于该训练样本的预测精度正相关；当模型基于任一训练样本的一个或多个变化结果输出的预测结果相对该训练样本的预测结果或标签数据的差异足够小时，攻击者可认为该训练样本属于所属模型的训练集，因此在一些实施例中，所述信息值可以与所述模型基于该训练样本的特征数据的一个或多个变化结果输出的预测结果相对该训练样本的预测结果或标签数据的差异负相关。在一些实施例中，所述信息值可依据上述两种关系以及其他关系中的至少一种来构建。在一些实施例中，信息值可以是离散形式，例如，当模型对于某一训练样本的预测精度大于设定阈值时，信息值为1，反之，信息值为0。所述预测精度可以是模型预测结果与标签数据的相似度，所述设定阈值可以是0.85、0.9、0.95等等。又例如，可以对某一训练样本的特征数据进行变形处 理，如加入噪声或扭转特征数据，再将变化结果输出到所述模型中得到预测结果，可以将变化后的特征数据对应的预测结果与原始预测结果(即特征数据对应的预测结果)进行比较，当两者差异小于设定阈值时，信心值为1，反之为-1。所述设定阈值可以是0.2、0.1、0.05等。在一些实施例中，信息值也可以直接为连续数值。
步骤320，利用基于训练集中各训练样本确定的训练输入数据以及和训练输出参考数据训练所述成员探测装置。在一些实施例中，步骤320可以由装置训练模块520实现。
在一些实施例中，成员探测装置可以与所述模型一起训练。例如，可以待模型训练一定程度后，基于该模型获取预测结果以及基于其预测性能构造信息值，对成员探测装置进行训练。当成员探测装置训练到一定程度后，再基于成员探测装置的输出结果调节所述模型。模型训练过程与装置训练过程交替进行，从而得到满足要求的模型。
应当注意的是，上述有关流程200和300的描述仅仅是为了示例和说明，而不限定本说明书的适用范围。对于本领域技术人员来说，在本说明书的指导下可以对流程200和300进行各种修正和改变。然而，这些修正和改变仍在本说明书的范围之内。
图4为根据本说明书一些实施例所示的模型训练系统的示例性框图。系统400可以包括模型训练模块410、第一获取模块420和调节模块430。
在一些实施例中，模型训练模块410可以用于基于训练集训练模型。
在一些实施例中，第一获取模块420可以用于对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果得到的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率。所述训练样本可以包括实体的隐私数据，所述隐私数据可以包括文本数据、图像数据、声音数据中的一种或多种。
在一些实施例中，调节模块430可以用于调节所述模型的参数和/或结构，以降低所述输出值。在一些实施例中，调节模块430可以进一步用于使所述模型的一个或多个节点在给定输入的条件下对应的输出具有不确定性。在一些实施例中，调节模块430可以在所述一个或多个节点的输出中添加噪声，以使所述一个或多个节点在给定输入的条件下对应的输出具有不确定性。在一些实施例中，调节模块430可以进一步用于屏蔽所述模型中的一个或多个节点。
参考前述内容，成员探测装置可以与所述模型一起训练。相应地，模型训练系统400还可以包括成员探测装置训练模块440。在一些实施例中，成员探测装置训练模块 440可以用于：对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据；利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练成员探测装置。应当理解，在一些实施例中，成员探测装置训练模块440可以由图5披露的成员探测装置的训练系统500实现。即，关于成员探测装置训练模块440的更多细节，可以参考图5及其相关描述。
图5为根据本说明书一些实施例所示的成员探测装置的训练系统的示例性框图。系统500可以包括第二获取模块510和装置训练模块520。
在一些实施例中，第二获取模块510可以用于对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据。在一些实施例中，所述信息值可以与所述模型对于该训练样本的预测精度正相关。在一些实施例中，所述信息值可以与所述模型基于该训练样本的特征数据的一个或多个变化结果输出的预测结果相对该训练样本的预测结果或标签数据的差异负相关。在一些实施例中，所述训练样本可以包括实体的隐私数据，所述隐私数据可以包括文本数据、图像数据、声音数据中的一种或多种。
在一些实施例中，装置训练模块520可以用于利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练成员探测装置。
关于图4和图5所示的系统及其模块的更多细节，还可以参考图2和图3所示的流程及其步骤的相关描述。
应当理解，图4和图5所示的系统及其模块可以利用各种方式来实现。例如，在一些实施例中，系统及其模块可以通过硬件、软件或者软件和硬件的结合来实现。其中，硬件部分可以利用专用逻辑来实现；软件部分则可以存储在存储器中，由适当的指令执行系统，例如微处理器或者专用设计硬件来执行。本领域技术人员可以理解上述的方法和系统可以使用计算机可执行指令和/或包含在处理器控制代码中来实现，例如在诸如磁盘、CD或DVD-ROM的载体介质、诸如只读存储器(固件)的可编程的存储器或者诸如光学或电子信号载体的数据载体上提供了这样的代码。本说明书的系统及其模块不仅可以有诸如超大规模集成电路或门阵列、诸如逻辑芯片、晶体管等的半导体、或者诸如现场可编程门阵列、可编程逻辑设备等的可编程硬件设备的硬件电路实现，也可以用例如由各种类型的处理器所执行的软件实现，还可以由上述硬件电路和软件的结合(例如， 固件)来实现。
需要注意的是，以上对于系统及其模块的描述，仅为描述方便，并不能把本说明书限制在所举实施例范围之内。可以理解，对于本领域的技术人员来说，在了解系统的原理后，可能在不背离这一原理的情况下，对各个模块进行任意组合，或者构成子系统与其他模块连接。例如，在一些实施例中，图4中披露的第一获取模块420和调节模块430可以是一个系统中的不同模块，也可以是一个模块实现这两个模块的功能。又如，在一些实施例中，图4中披露的成员探测装置训练模块440可以由图5所示的成员探测装置的训练系统500实现。又如，在一些实施例中，图4中披露的第一获取模块420/模型训练模块410和图5中披露的第二获取模块510/装置训练模块520可以是两个模块，也可以合并为一个模块。诸如此类的变形，均在本说明书的保护范围之内。
本说明书实施例可能带来的有益效果包括但不限于：(1)成员探测装置可评估攻击者基于任一训练样本的特征数据、标签数据和预测结果正确推断出该训练样本属于模型的训练集的概率，在成员探测装置的协助下可调节模型的参数和/或结构，使得模型防范成员探测的能力得到加强；(2)通过重复执行相关步骤或流程，可保证模型精度仍满足需求。需要说明的是，不同实施例可能产生的有益效果不同，在不同的实施例里，可能产生的有益效果可以是以上任意一种或几种的组合，也可以是其他任何可能获得的有益效果。
上文已对基本概念做了描述，显然，对于本领域技术人员来说，上述详细披露仅仅作为示例，而并不构成对本说明书实施例的限定。虽然此处并没有明确说明，本领域技术人员可能会对本说明书实施例进行各种修改、改进和修正。该类修改、改进和修正在本说明书实施例中被建议，所以该类修改、改进、修正仍属于本说明书示范实施例的精神和范围。
同时，本说明书使用了特定词语来描述本说明书的实施例。如“一个实施例”、“一实施例”、和/或“一些实施例”意指与本说明书至少一个实施例相关的某一特征、结构或特点。因此，应强调并注意的是，本说明书中在不同位置两次或多次提及的“一实施例”或“一个实施例”或“一个替代性实施例”并不一定是指同一实施例。此外，本说明书的一个或多个实施例中的某些特征、结构或特点可以进行适当的组合。
此外，本领域技术人员可以理解，本说明书实施例的各方面可以通过若干具有可专利性的种类或情况进行说明和描述，包括任何新的和有用的工序、机器、产品或物质的组合，或对他们的任何新的和有用的改进。相应地，本说明书实施例的各个方面可以 完全由硬件执行、可以完全由软件(包括固件、常驻软件、微码等)执行、也可以由硬件和软件组合执行。以上硬件或软件均可被称为“数据块”、“模块”、“引擎”、“单元”、“组件”或“系统”。此外，本说明书实施例的各方面可能表现为位于一个或多个计算机可读介质中的计算机产品，该产品包括计算机可读程序编码。
计算机存储介质可能包含一个内含有计算机程序编码的传播数据信号，例如在基带上或作为载波的一部分。该传播信号可能有多种表现形式，包括电磁形式、光形式等，或合适的组合形式。计算机存储介质可以是除计算机可读存储介质之外的任何计算机可读介质，该介质可以通过连接至一个指令执行系统、装置或设备以实现通讯、传播或传输供使用的程序。位于计算机存储介质上的程序编码可以通过任何合适的介质进行传播，包括无线电、电缆、光纤电缆、RF、或类似介质，或任何上述介质的组合。
本说明书实施例各部分操作所需的计算机程序编码可以用任意一种或多种程序语言编写，包括面向对象编程语言如Java、Scala、Smalltalk、Eiffel、JADE、Emerald、C++、C#、VB.NET、Python等，常规程序化编程语言如C语言、VisualBasic、Fortran2003、Perl、COBOL2002、PHP、ABAP，动态编程语言如Python、Ruby和Groovy，或其他编程语言等。该程序编码可以完全在用户计算机上运行、或作为独立的软件包在用户计算机上运行、或部分在用户计算机上运行部分在远程计算机运行、或完全在远程计算机或处理设备上运行。在后种情况下，远程计算机可以通过任何网络形式与用户计算机连接，比如局域网(LAN)或广域网(WAN)，或连接至外部计算机(例如通过因特网)，或在云计算环境中，或作为服务使用如软件即服务(SaaS)。
此外，除非权利要求中明确说明，本说明书实施例所述处理元素和序列的顺序、数字字母的使用、或其他名称的使用，并非用于限定本说明书实施例流程和方法的顺序。尽管上述披露中通过各种示例讨论了一些目前认为有用的发明实施例，但应当理解的是，该类细节仅起到说明的目的，附加的权利要求并不仅限于披露的实施例，相反，权利要求旨在覆盖所有符合本说明书实施例实质和范围的修正和等价组合。例如，虽然以上所描述的系统组件可以通过硬件设备实现，但是也可以只通过软件的解决方案得以实现，如在现有的处理设备或移动设备上安装所描述的系统。
同理，应当注意的是，为了简化本说明书实施例披露的表述，从而帮助对一个或多个发明实施例的理解，前文对本说明书实施例的描述中，有时会将多种特征归并至一个实施例、附图或对其的描述中。但是，这种披露方法并不意味着本说明书实施例对象所需要的特征比权利要求中提及的特征多。实际上，实施例的特征要少于上述披露的单 个实施例的全部特征。
针对本说明书引用的每个专利、专利申请、专利申请公开物和其他材料，如文章、书籍、说明书、出版物、文档等，特此将其全部内容并入本说明书作为参考。与本说明书内容不一致或产生冲突的申请历史文件除外，对本申请权利要求最广范围有限制的文件(当前或之后附加于本说明书中的)也除外。需要说明的是，如果本说明书附属材料中的描述、定义、和/或术语的使用与本说明书所述内容有不一致或冲突的地方，以本说明书的描述、定义和/或术语的使用为准。
最后，应当理解的是，本说明书中所述实施例仅用以说明本说明书实施例的原则。其他的变形也可能属于本说明书实施例的范围。因此，作为示例而非限制，本说明书实施例的替代配置可视为与本说明书的教导一致。相应地，本说明书的实施例不仅限于本说明书明确介绍和描述的实施例。

Claims (22)

  1. 一种保护数据隐私的模型训练方法，其中，包括：
    基于训练集训练模型；
    对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果得到的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率；
    调节所述模型的参数和/或结构，以降低所述输出值。
  2. 根据权利要求1所述的方法，其中，所述训练样本包括实体的隐私数据，所述隐私数据包括文本数据、图像数据、声音数据中的一种或多种。
  3. 根据权利要求1所述的方法，其中，所述调节所述模型的参数和/或结构，包括：
    使所述模型的一个或多个节点在给定输入的条件下对应的输出具有不确定性。
  4. 根据权利要求3所述的方法，其中，所述使所述模型的一个或多个节点在给定输入的条件下对应的输出具有不确定性，包括：
    在所述一个或多个节点的输出中添加噪声。
  5. 根据权利要求1所述的方法，其中，所述调节所述模型的参数和/或结构，包括：
    屏蔽所述模型中的一个或多个节点。
  6. 根据权利要求1所述的方法，其中，还包括：
    对于模型的训练集中的任一训练样本：
    获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果，作为训练输入数据；
    获取反映所述模型对于该训练样本的预测性能的信息值，作为训练输出参考数据；
    利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
  7. 一种保护数据隐私的模型训练系统，其中，包括：
    模型训练模块，用于基于训练集训练模型；
    第一获取模块，用于对于所述训练集中的任一训练样本，获取成员探测装置基于该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果得到的输出值，所述输出值反映基于所述输出结果正确推断出该训练样本属于所述训练集的概率；
    调节模块，用于调节所述模型的参数和/或结构，以降低所述输出值。
  8. 根据权利要求7所述的系统，其中，所述训练样本包括实体的隐私数据，所述隐私数据包括文本数据、图像数据、声音数据中的一种或多种。
  9. 根据权利要求7所述的系统，其中，所述调节模块进一步用于使所述模型的一个或多个节点在给定输入的条件下对应的输出具有不确定性。
  10. 根据权利要求9所述的系统，其中，所述调节模块进一步用于在所述一个或多个节点的输出中添加噪声。
  11. 根据权利要求7所述的系统，其中，所述调节模块进一步用于屏蔽所述模型中的一个或多个节点。
  12. 根据权利要求7所述的系统，其中，还包括成员探测装置训练模块，所述成员探测装置训练模块用于：
    对于模型的训练集中的任一训练样本：
    获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果，作为训练输入数据；
    获取反映所述模型对于该训练样本的预测性能的信息值，作为训练输出参考数据；
    利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
  13. 一种保护数据隐私的模型训练装置，其中，包括至少一个处理器和至少一个存储设备，所述存储设备用于存储指令，当至少一个处理器执行指令时，实现如权利要求1～6中任一项所述的方法。
  14. 一种成员探测装置的训练方法，其中，包括：
    对于模型的训练集中的任一训练样本：
    获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果，作为训练输入数据；
    获取反映所述模型对于该训练样本的预测性能的信息值，作为训练输出参考数据；
    利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
  15. 根据权利要求14所述的方法，其中，所述训练样本包括实体的隐私数据，所述隐私数据包括文本数据、图像数据、声音数据中的一种或多种。
  16. 根据权利要求14所述的方法，其中，所述信息值与所述模型对于该训练样本的预测精度正相关。
  17. 根据权利要求14所述的方法，其中，所述信息值与所述模型基于该训练样本的特征数据的一个或多个变化结果输出的预测结果相对该训练样本的预测结果或标签数据的差异负相关。
  18. 一种成员探测装置的训练系统，其中，包括：
    第二获取模块，用于对于模型的训练集中的任一训练样本，获取该训练样本的特征数据、标签数据和所述模型基于该训练样本的输出结果作为训练输入数据，获取反映所述模型对于该训练样本的预测性能的信息值作为训练输出参考数据；
    装置训练模块，用于利用基于训练集中各训练样本确定的训练输入数据和训练输出参考数据训练所述成员探测装置。
  19. 根据权利要求18所述的系统，其中，所述训练样本包括实体的隐私数据，所述隐私数据包括文本数据、图像数据、声音数据中的一种或多种。
  20. 根据权利要求18所述的系统，其中，所述信息值与所述模型对于该训练样本的预测精度正相关。
  21. 根据权利要求18所述的系统，其中，所述信息值与所述模型基于该训练样本的特征数据的一个或多个变化结果输出的预测结果相对该训练样本的预测结果或标签数据的差异负相关。
  22. 一种成员探测装置的训练装置，其中，包括至少一个处理器和至少一个存储设备，所述存储设备用于存储指令，当至少一个处理器执行指令时，实现如权利要求14～17中任一项所述的方法。
PCT/CN2020/123760 2019-12-20 2020-10-26 模型训练方法、成员探测装置的训练方法及其系统 WO2021120854A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911333307.2A CN111079946A (zh) 2019-12-20 2019-12-20 模型训练方法、成员探测装置的训练方法及其系统
CN201911333307.2 2019-12-20

Publications (1)

Publication Number Publication Date
WO2021120854A1 true WO2021120854A1 (zh) 2021-06-24

Family

ID=70316652

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/123760 WO2021120854A1 (zh) 2019-12-20 2020-10-26 模型训练方法、成员探测装置的训练方法及其系统

Country Status (2)

Country Link
CN (1) CN111079946A (zh)
WO (1) WO2021120854A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111079946A (zh) * 2019-12-20 2020-04-28 支付宝(杭州)信息技术有限公司 模型训练方法、成员探测装置的训练方法及其系统
CN111401483B (zh) * 2020-05-15 2022-05-17 支付宝(杭州)信息技术有限公司 样本数据处理方法、装置及多方模型训练系统
CN111475852B (zh) * 2020-06-19 2020-09-15 支付宝(杭州)信息技术有限公司 基于隐私保护针对业务模型进行数据预处理的方法及装置
CN112100623B (zh) * 2020-08-21 2023-12-22 百度在线网络技术（北京）有限公司 机器学习模型的风险评估方法、装置、设备及存储介质
CN112784990A (zh) * 2021-01-22 2021-05-11 支付宝(杭州)信息技术有限公司 一种成员推断模型的训练方法
CN112750535B (zh) * 2021-01-30 2024-03-12 云知声智能科技股份有限公司 用于衡量模型不确定性的方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109952582A (zh) * 2018-09-29 2019-06-28 区链通网络有限公司 一种强化学习模型的训练方法、节点、系统及存储介质
CN110008696A (zh) * 2019-03-29 2019-07-12 武汉大学 一种面向深度联邦学习的用户数据重建攻击方法
CN111079946A (zh) * 2019-12-20 2020-04-28 支付宝(杭州)信息技术有限公司 模型训练方法、成员探测装置的训练方法及其系统

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10963319B2 (en) * 2016-01-06 2021-03-30 International Business Machines Corporation Enhancing privacy of sensor data from devices using ephemeral cohorts
CN107368752B (zh) * 2017-07-25 2019-06-28 北京工商大学 一种基于生成式对抗网络的深度差分隐私保护方法
US11138333B2 (en) * 2018-03-07 2021-10-05 Private Identity Llc Systems and methods for privacy-enabled biometric processing
US11062036B2 (en) * 2018-05-11 2021-07-13 Paypal, Inc. System and method for generating privacy data containment and reporting
CN109255444B (zh) * 2018-08-10 2022-03-29 深圳前海微众银行股份有限公司 基于迁移学习的联邦建模方法、设备及可读存储介质
CN109934004A (zh) * 2019-03-14 2019-06-25 中国科学技术大学 一种机器学习服务系统中保护隐私的方法
CN110516812A (zh) * 2019-07-19 2019-11-29 南京航空航天大学 基于对抗样本的抗成员推理攻击的ai模型隐私保护方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109952582A (zh) * 2018-09-29 2019-06-28 区链通网络有限公司 一种强化学习模型的训练方法、节点、系统及存储介质
CN110008696A (zh) * 2019-03-29 2019-07-12 武汉大学 一种面向深度联邦学习的用户数据重建攻击方法
CN111079946A (zh) * 2019-12-20 2020-04-28 支付宝(杭州)信息技术有限公司 模型训练方法、成员探测装置的训练方法及其系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
NASR MILAD MILAD@CS.UMASS.EDU; SHOKRI REZA REZA@COMP.NUS.EDU.SG; HOUMANSADR AMIR AMIR@CS.UMASS.EDU: "Machine Learning with Membership Privacy using Adversarial Regularization", COMPUTER AND COMMUNICATIONS SECURITY, ACM, 2 PENN PLAZA, SUITE 701NEW YORKNY10121-0701USA, 15 January 2018 (2018-01-15) - 19 October 2018 (2018-10-19), 2 Penn Plaza, Suite 701New YorkNY10121-0701USA, pages 634 - 646, XP058449200, ISBN: 978-1-4503-5693-0, DOI: 10.1145/3243734.3243855 *

Also Published As

Publication number Publication date
CN111079946A (zh) 2020-04-28

Similar Documents

Publication Publication Date Title
WO2021120854A1 (zh) 模型训练方法、成员探测装置的训练方法及其系统
US20200366671A1 (en) Identity verification and management system
US20210004718A1 (en) Method and device for training a model based on federated learning
US11556730B2 (en) Methods and apparatus for distributed use of a machine learning model
US11526745B2 (en) Methods and apparatus for federated training of a neural network using trusted edge devices
US10169717B2 (en) System and method for defining and using different levels of ground truth
WO2020082572A1 (zh) 生成式对抗网络的训练方法、相关设备及介质
US10997717B2 (en) Method and system for generating a confidence score using deep learning model
US20200194008A1 (en) Electronic apparatus and controlling method thereof
US11366980B2 (en) Privacy enhanced machine learning
US20170308620A1 (en) Making graph pattern queries bounded in big graphs
US11663625B2 (en) Intercepting inadvertent conversational disclosure of personal information
US11989268B2 (en) Dummy class framework for continual supervised learning applications
CN114298202A (zh) 一种图像加密方法、装置、电子设备及存储介质
CN117176368A (zh) 终端侧的隐私风险评估方法及装置、介质、电子设备
CN115964706A (zh) 联邦学习场景下的训练数据毒化防御方法
CN112784990A (zh) 一种成员推断模型的训练方法
Bharadwaj et al. Reliable human authentication using AI-based multibiometric image sensor fusion: Assessment of performance in information security
WO2023019970A1 (zh) 一种攻击检测方法及装置
Hui et al. Horizontal Federated Learning and Secure Distributed Training for Recommendation System with Intel SGX
US20240171675A1 (en) System and method for recognizing undesirable calls
US20240146734A1 (en) Large language model-based authentication
WO2024051364A1 (zh) 活体检测模型训练方法、装置、存储介质以及终端
US20230196195A1 (en) Identifying, or checking integrity of, a machine-learning classification model
EP4373031A1 (en) System and method for recognizing undersirable calls

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20902788

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20902788

Country of ref document: EP

Kind code of ref document: A1