WO2021098255A1

WO2021098255A1 - Training method and device for neural network model for protecting privacy and security

Info

Publication number: WO2021098255A1
Application number: PCT/CN2020/103605
Authority: WO
Inventors: 翁海琴
Original assignee: 支付宝(杭州)信息技术有限公司
Priority date: 2019-11-19
Filing date: 2020-07-22
Publication date: 2021-05-27
Also published as: CN110874471B; TW202121263A; TWI745958B; CN110874471A

Abstract

Provided in the embodiments of the present description are a training method and device for a neural network model for protecting privacy and security. The method comprises: acquiring a preliminarily trained target neural network model and a training dataset, the target neutral network model comprising multiple intermediary layers, the training dataset comprising a first number of member samples; determining a decision-making critical layer and a decision-making irrelevant layer, the degree of influence of the decision-making critical layer on a decision result being greater than the degree of influence of the decision-making irrelevant layer on the decision result; retraining the target neural network model on the basis of the member samples in the training dataset, the retraining fixing a parameter of the decision-making irrelevant layer of the target neural network model, thus allowing some neurons of the decision-making critical layer to stop working at a certain probability so as to adjust a parameter of the decision-making critical layer. This prevents an attacker from detecting training data of the neural network model.

Description

Training method and device of neural network model for protecting privacy and safety

Technical field

One or more embodiments of this specification relate to the field of artificial intelligence, and in particular to a method and device for training a neural network model that protects privacy and security.

Background technique

In the field of artificial intelligence, neural network models are widely deployed in various practical scenarios, such as face detection, product recommendation, etc. While achieving high effectiveness and accuracy, the neural network model also over-memorizes the data information in the training set, which will be detected by the attacker through certain specific techniques (such as member inference attacks and model stealing attacks). This leads to the leakage of training data. These training data may involve user privacy information.

Therefore, it is hoped that there will be an improved scheme that can provide a method of training a neural network model that protects privacy and security, so as to prevent an attacker from detecting the training data of the neural network model.

Summary of the invention

One or more embodiments of this specification describe a method and device for training a neural network model that protects privacy and security, which can prevent an attacker from detecting the training data of the neural network model.

In a first aspect, a method for training a neural network model for protecting privacy and security is provided. The method includes: obtaining a preliminary trained target neural network model and a training data set, the target neural network model including a plurality of intermediate layers, and the training The data set includes a first number of member samples; determining important decision-making layers and decision-independent layers in the plurality of intermediate layers, and the degree of influence of the important decision-making layers on the decision results is greater than the degree of influence of the decision-independent layers on the decision results ; According to each member sample in the training data set, the target neural network model is retrained, and the retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons in the important decision-making layer are A certain probability of stopping work to adjust the parameters of the important layer of decision-making.

In a possible implementation manner, the preliminary training adjusts the parameters of each intermediate layer in the target neural network model.

In a possible implementation manner, the determining the important decision-making layer and the decision-independent layer in the plurality of intermediate layers includes: using the member samples and non-member samples as evaluation samples to form an evaluation data set; The evaluation sample is input to the target neural network model, and each intermediate layer of the target neural network model outputs the characteristics of each intermediate layer of the evaluation sample; according to the characteristics of each intermediate layer of the evaluation sample, and whether the evaluation sample is a member sample , To determine the important decision-making layer and decision-independent layer in each intermediate layer.

Further, the forming an evaluation data set by using the member samples and non-member samples as evaluation samples includes: extracting a second number of member samples from the first number of member samples; and the second number is smaller than the The first quantity; a third quantity of non-member samples is obtained, and the distribution of the non-member samples is the same as the distribution of the member samples;

The second number of member samples and the third number of non-member samples are used as evaluation samples to form an evaluation data set.

Further, the determining the important decision-making layers and decision-independent layers in each intermediate layer according to the characteristics of each intermediate layer of the evaluation sample and whether the evaluation sample is a member sample includes: reducing the dimensionality of each intermediate layer feature of the evaluation sample After processing, it is used as the sample feature of the interpretable classifier, and whether the evaluation sample is a member sample is used as the sample label, and the interpretable classifier is trained; according to the trained interpretable classifier, it is determined that the decision in each intermediate layer is important Layer has nothing to do with decision-making.

Further, said performing dimensionality reduction processing on each intermediate layer feature of the evaluation sample includes: training an autoencoder for each intermediate layer; using the autoencoder corresponding to each intermediate layer to analyze the middle layer of the evaluation sample. Layer features are processed for dimensionality reduction.

Further, the interpretable classifier is a tree model or a logistic regression model.

In a possible implementation manner, the method further includes: repeating the re-training after replacing the part of the neurons.

In a possible implementation, the certain probability is 50%.

In a second aspect, a training device for a neural network model that protects privacy and security is provided. The device includes: an acquisition unit for acquiring a preliminary training target neural network model and a training data set, the target neural network model including a plurality of intermediate The training data set includes the first number of member samples; the determining unit is used to determine the important decision-making layer and the decision-independent layer among the plurality of intermediate layers acquired by the acquiring unit, and the important decision-making layer is relevant to the decision-making The degree of influence of the result is greater than the degree of influence of the decision-independent layer on the decision result; a training unit for retraining the target neural network model according to each member sample in the training data set obtained by the obtaining unit, The retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons of the decision-making important layer stop working with a certain probability to adjust the parameters of the decision-making important layer.

In a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

In a fourth aspect, a computing device is provided, including a memory and a processor, the memory stores executable code, and the processor implements the method of the first aspect when the executable code is executed by the processor.

Through the method and device provided in the embodiments of this specification, first obtain a preliminary trained target neural network model and a training data set, the target neural network model includes a plurality of intermediate layers, and the training data set includes a first number of member samples; Then determine the important decision-making layer and the decision-independent layer in the plurality of intermediate layers. The important decision-making layer has a greater influence on the decision result than the decision-independent layer has on the decision result; finally, according to the training data set Each member sample retrains the target neural network model. The retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons of the important decision-making layer stop working with a certain probability to adjust the important decision-making layer Parameters. It can be seen from the above that, in the embodiment of this specification, after the initial training of the target neural network model, based on the performance of each intermediate layer, the decision important layer and the decision irrelevant layer are determined, and different adjustment parameters are adopted for the decision important layer and the decision irrelevant layer. The strategy retrains the target neural network model, so that some neurons in the important decision-making layer will stop working with a certain probability to adjust the parameters of the important decision-making layer, so as to prevent the attacker from detecting the training data of the neural network model.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

Figure 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification;

Fig. 2 shows a flowchart of a training method of a neural network model for protecting privacy and security according to an embodiment;

Fig. 3 shows a schematic block diagram of a training device for a neural network model for protecting privacy and security according to an embodiment.

Detailed ways

The following describes the solutions provided in this specification with reference to the accompanying drawings.

Fig. 1 is a schematic diagram of an implementation scenario of an embodiment disclosed in this specification. This implementation scenario involves the training of a neural network model to protect privacy. Specifically, based on a white box scenario, it can be understood that the model structure and parameters of the neural network model can be known in the white box scenario. 1, the embodiment of this specification uses conventional methods to conduct preliminary training on the target neural network model based on each member sample in the training data set, and then conducts a security review on the target neural network model based on the evaluation sample. The evaluation sample is a member sample or Non-member sample. Among them, extract each intermediate layer feature of the target neural network model corresponding to the evaluation sample, train the interpretable classifier based on each intermediate layer feature and whether the evaluation sample is a member sample, and analyze the interpretable classifier to determine that the decision is important Layer and decision-independent layer, and then based on the analysis results to make targeted fine-tuning of the target neural network model, so as to prevent the model from leaking private information.

FIG. 2 shows a flowchart of a training method of a neural network model for protecting privacy and security according to an embodiment, and the method may be based on the implementation scenario shown in FIG. 1. As shown in Figure 2, the method for training a neural network model for privacy protection in this embodiment includes the following steps: Step 21: Obtain a preliminary trained target neural network model and a training data set. The target neural network model includes a plurality of intermediate Layer, the training data set includes a first number of member samples; step 22, determining important decision-making layers and decision-independent layers in the plurality of intermediate layers, the decision-making important layer has a greater influence on the decision result than the decision The degree of influence of the irrelevant layer on the decision result; step 23, according to each member sample in the training data set, the target neural network model is retrained, and the retraining fixes the decision-making irrelevant layer of the target neural network model Parameters, make some neurons in the important decision-making layer stop working with a certain probability to adjust the parameters of the important decision-making layer. The following describes the specific implementation of each of the above steps.

First, in step 21, a preliminary training target neural network model and a training data set are obtained. The target neural network model includes a plurality of intermediate layers, and the training data set includes a first number of member samples. It is understandable that the above-mentioned preliminary training can adopt a conventional training method, and the above-mentioned training data set can be used to train the target neural network model.

In an example, the preliminary training adjusts the parameters of each intermediate layer in the target neural network model.

Then in step 22, the decision important layer and the decision irrelevant layer among the plurality of intermediate layers are determined, and the degree of influence of the important decision layer on the decision result is greater than the degree of influence of the decision irrelevant layer on the decision result. It is understandable that the important decision-making layer can be one or more layers, and the decision-independent layer can also be one or more layers.

In one example, the member samples and non-member samples are used as evaluation samples to form an evaluation data set; any evaluation sample is input into the target neural network model to obtain the output of each intermediate layer of the target neural network model. The characteristics of each intermediate layer of the evaluation sample; according to the characteristics of each intermediate layer of the evaluation sample and whether the evaluation sample is a member sample, determine the important decision-making layer and the decision-independent layer in each intermediate layer.

It is understandable that if the target neural network model contains n intermediate layers, each evaluation sample corresponds to n intermediate layer features, that is, n feature maps, denoted as M_1, M_2,..., M_n.

It is understandable that the member samples refer to the samples in the training data set. Non-member samples refer to samples outside the training data set.

Further, the evaluation data set can be constructed in the following manner: extract a second number of member samples from the first number of member samples; the second number is less than the first number; obtain a third number of non-member samples The distribution of the non-member samples is the same as the distribution of the member samples; the second number of member samples and the third number of non-member samples are used as evaluation samples to form an evaluation data set.

Further, the dimensionality reduction processing of each middle layer feature of the evaluation sample is used as the sample feature of the interpretable classifier, and whether the evaluation sample is a member sample is used as the sample label, and the interpretable classifier is trained; The interpretable classifier determines the important decision-making layers and decision-independent layers in each intermediate layer.

Further, for each intermediate layer, an autoencoder is trained; the autoencoders corresponding to each intermediate layer are used to perform dimensionality reduction processing on the intermediate layer features of the intermediate layer of the evaluation sample. Optionally, after the dimensionality reduction processing, the dimension of the middle layer feature is 1. The intermediate output result of the autoencoder is used as the feature after dimensionality reduction.

Among them, the autoencoder is a type of artificial neural network used in unsupervised learning, and its function is to perform characterization learning on the input information by taking the input information as the learning target.

In the embodiments of this specification, other methods other than the autoencoder may be used to perform dimensionality reduction processing on the features of the middle layer, for example, a principal component analysis (PCA) method.

Among them, the interpretable classifier is an interpretable model, and the interpretable model refers to the decision-making method of the model that can be understood by human experts.

In one example, the interpretable classifier is a tree model or a logistic regression model. The above-mentioned tree model is for example the Xgboost classifier. The Xgboost classifier is a boosted tree model that integrates multiple tree models to form a powerful classifier. The Xgboost classifier can find the features that play an important role in its decision-making process, and this feature corresponds to the middle layer features of the target neural network model, so that you can know which middle layer features the Xgboost classifier uses to determine whether the evaluation sample is a member Samples, and these middle-layer features will leak the private data of the model. The intermediate layer that the Xgboost classifier mainly relies on for decision-making can be defined as an important decision-making layer, and the remaining intermediate layers can be defined as decision-independent layers.

Finally, in step 23, the target neural network model is retrained according to each member sample in the training data set, and the retraining fixes the parameters of the decision-independent layer of the target neural network model to make the decision-making important layer Some neurons stop working with a certain probability to adjust the parameters of important decision-making layers. It is understandable that the above process of retraining the model is equivalent to fine-tuning the model.

The embodiments of this specification are based on the principle of Dropout. Dropout is a neural network regularization technology that prevents the model from overfitting by preventing the number of trainings from performing complex adaptations.

In one example, after replacing some of the neurons, the retraining is repeated.

In an example, the certain probability is 50 percent.

Through the method provided in the embodiments of this specification, first obtain a preliminary training target neural network model and a training data set, the target neural network model includes a plurality of intermediate layers, and the training data set includes a first number of member samples; and then determine The decision-making important layer and the decision-independent layer in the plurality of intermediate layers, the degree of influence of the important decision-making layer on the decision result is greater than the degree of influence of the decision-independent layer on the decision result; finally according to each member in the training data set Sample, the target neural network model is retrained, and the retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons of the important decision-making layer stop working with a certain probability to adjust the parameters of the important decision-making layer . It can be seen from the above that, in the embodiment of this specification, after the initial training of the target neural network model, based on the performance of each intermediate layer, the decision important layer and the decision irrelevant layer are determined, and different adjustment parameters are adopted for the decision important layer and the decision irrelevant layer. The strategy retrains the target neural network model, so that some neurons in the important decision-making layer will stop working with a certain probability to adjust the parameters of the important decision-making layer, so as to prevent the attacker from detecting the training data of the neural network model.

According to another embodiment, there is also provided a training device for a neural network model that protects privacy and security, and the device is used to execute the method for training a neural network model that protects privacy and security provided in the embodiments of this specification. Fig. 3 shows a schematic block diagram of a training device for a neural network model for protecting privacy and security according to an embodiment. As shown in FIG. 3, the device 300 includes: an obtaining unit 31, configured to obtain a preliminary training target neural network model and a training data set, the target neural network model includes a plurality of intermediate layers, and the training data set includes a first A number of member samples; a determining unit 32, configured to determine important decision-making layers and decision-independent layers among the plurality of intermediate layers obtained by the obtaining unit 31, and the decision-making important layer has a greater influence on the decision result than the decision The degree of influence of the irrelevant layer on the decision result; the training unit 33 is used for retraining the target neural network model according to each member sample in the training data set acquired by the acquiring unit 31, and the retraining fixed station The parameters of the decision-independent layer of the target neural network model make some neurons of the important decision-making layer stop working with a certain probability to adjust the parameters of the important decision-making layer.

Optionally, as an embodiment, the preliminary training adjusts the parameters of each intermediate layer in the target neural network model.

Optionally, as an embodiment, the determining unit 32 includes: a constructing subunit for composing an evaluation data set using the member samples and non-member samples as evaluation samples; and a feature extraction subunit for combining the Any evaluation sample obtained by constructing a subunit is input into the target neural network model, and each intermediate layer of the target neural network model outputs the characteristics of each intermediate layer of the evaluation sample respectively; the subunit is determined to be used according to the characteristics Extract the characteristics of each intermediate layer of the evaluation sample obtained by the subunit, and whether the evaluation sample is a member sample, and determine the important decision-making layer and the decision-independent layer in each intermediate layer.

Further, the construction subunit is specifically configured to: extract a second number of member samples from the first number of member samples; the second number is less than the first number; and obtain a third number of non-member samples The distribution of the non-member samples is the same as the distribution of the member samples; the second number of member samples and the third number of non-member samples are used as evaluation samples to form an evaluation data set.

Further, the determining subunit is specifically used to: perform dimensionality reduction processing on each intermediate layer feature of the evaluation sample as the sample feature of the interpretable classifier, and use whether the evaluation sample is a member sample as a sample label, and to determine whether the evaluation sample is a member sample. The interpretable classifier is trained; according to the trained interpretable classifier, the decision-making important layer and the decision-independent layer in each intermediate layer are determined.

Optionally, as an embodiment, the device further includes: an update unit, configured to make the training unit 33 repeat the re-training after replacing the part of the neurons.

Optionally, as an embodiment, the certain probability is 50%.

With the device provided in the embodiment of this specification, first the obtaining unit 31 obtains a preliminary training target neural network model and a training data set, the target neural network model includes a plurality of intermediate layers, and the training data set includes a first number of member samples ; Then the determining unit 32 determines the important decision-making layer and the decision-independent layer in the plurality of intermediate layers, and the decision-making important layer has a greater degree of influence on the decision result than the decision-independent layer has on the decision result; finally the training unit 33 According to each member sample in the training data set, the target neural network model is retrained. The retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons in the important decision-making layer are set to a certain value. The probability of stopping work adjusts the parameters of the important layer of decision-making. It can be seen from the above that, in the embodiment of this specification, after the initial training of the target neural network model, based on the performance of each intermediate layer, the decision important layer and the decision irrelevant layer are determined, and different adjustment parameters are adopted for the decision important layer and the decision irrelevant layer. The strategy retrains the target neural network model, so that some neurons in the important decision-making layer will stop working with a certain probability to adjust the parameters of the important decision-making layer, so as to prevent the attacker from detecting the training data of the neural network model.

According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method described in conjunction with FIG. 2.

According to an embodiment of still another aspect, there is also provided a computing device, including a memory and a processor, the memory stores executable code, and when the processor executes the executable code, it implements what is described in conjunction with FIG. 2 method.

Those skilled in the art should be aware that, in one or more of the foregoing examples, the functions described in the present invention can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on the computer-readable medium.

The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. The protection scope, any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the protection scope of the present invention.

Claims

A method for training a neural network model for protecting privacy and security, the method comprising:

Acquiring a preliminary trained target neural network model and a training data set, the target neural network model including a plurality of intermediate layers, and the training data set including a first number of member samples;

Determining an important decision-making layer and a decision-independent layer among the plurality of intermediate layers, where the degree of influence of the important decision-making layer on decision results is greater than the degree of influence of the decision-independent layer on the decision results;

According to each member sample in the training data set, the target neural network model is retrained. The retraining fixes the parameters of the decision-independent layer of the target neural network model, so that some neurons in the important decision-making layer are set to a certain value. Probability of stopping work adjusts the parameters of important layers of decision-making
The method of claim 1, wherein the preliminary training adjusts the parameters of each intermediate layer in the target neural network model.
The method according to claim 1, wherein the determining the important decision-making layer and the decision-independent layer among the plurality of intermediate layers comprises:

Use the member sample and the non-member sample as the evaluation sample to form an evaluation data set;

Inputting any evaluation sample into the target neural network model to obtain each intermediate layer feature of the evaluation sample respectively output by each intermediate layer of the target neural network model;

According to the characteristics of each intermediate layer of the evaluation sample and whether the evaluation sample is a member sample, determine the important decision-making layer and the decision-independent layer in each intermediate layer.
The method according to claim 3, wherein said using the member samples and non-member samples as evaluation samples to form an evaluation data set comprises:

Extract a second number of member samples from the first number of member samples; the second number is less than the first number;

Obtaining a third number of non-member samples, where the distribution of the non-member samples is the same as the distribution of the member samples;

The second number of member samples and the third number of non-member samples are used as evaluation samples to form an evaluation data set.
The method according to claim 3, wherein the determining the important decision-making layer and the decision-independent layer in each intermediate layer according to the characteristics of each intermediate layer of the evaluation sample and whether the evaluation sample is a member sample comprises:

Performing dimensionality reduction processing on each middle layer feature of the evaluation sample as the sample feature of the interpretable classifier, and training the interpretable classifier whether the evaluation sample is a member sample as a sample label;

According to the interpretable classifier after training, determine the important decision-making layer and decision-independent layer in each intermediate layer.
The method according to claim 5, wherein said performing dimensionality reduction processing on the features of each middle layer of the evaluation sample comprises:

For each intermediate layer, train an autoencoder;

The self-encoder corresponding to each intermediate layer is used to perform dimensionality reduction processing on the intermediate layer features of the intermediate layer of the evaluation sample.
The method of claim 5, wherein the interpretable classifier is a tree model or a logistic regression model.
The method of claim 1, wherein the method further comprises:

After replacing some of the neurons, repeat the training again.
The method of claim 1, wherein the certain probability is 50 percent.
A training device for a neural network model that protects privacy and safety, the device includes:

An obtaining unit, configured to obtain a preliminary training target neural network model and a training data set, the target neural network model includes a plurality of intermediate layers, and the training data set includes a first number of member samples;

The determining unit is configured to determine the important decision-making layer and the decision-independent layer among the plurality of intermediate layers acquired by the acquiring unit, and the degree of influence of the important decision-making layer on the decision result is greater than that of the decision-independent layer degree;

The training unit is configured to retrain the target neural network model according to the member samples in the training data set acquired by the acquisition unit, and the retraining fixes the parameters of the decision-independent layer of the target neural network model , To make some neurons of the important decision-making layer stop working with a certain probability to adjust the parameters of the important decision-making layer.
The device of claim 10, wherein the preliminary training adjusts the parameters of each intermediate layer in the target neural network model.
The apparatus according to claim 10, wherein the determining unit comprises:

Constructing a sub-unit for forming an evaluation data set using the member samples and non-member samples as evaluation samples;

The feature extraction subunit is configured to input any evaluation sample obtained by the constructing subunit into the target neural network model to obtain each intermediate layer feature of the evaluation sample respectively output by each intermediate layer of the target neural network model;

The determining subunit is used to determine the important decision-making layer and the decision-independent layer in each intermediate layer according to the features of each intermediate layer of the evaluation sample obtained by the feature extraction subunit, and whether the evaluation sample is a member sample.
The device according to claim 12, wherein the construction subunit is specifically used for:

Extract a second number of member samples from the first number of member samples; the second number is less than the first number;

Obtaining a third number of non-member samples, where the distribution of the non-member samples is the same as the distribution of the member samples;

The second number of member samples and the third number of non-member samples are used as evaluation samples to form an evaluation data set.
The device according to claim 12, wherein the determining subunit is specifically configured to:

Performing dimensionality reduction processing on each middle layer feature of the evaluation sample as the sample feature of the interpretable classifier, and training the interpretable classifier whether the evaluation sample is a member sample as a sample label;

According to the interpretable classifier after training, determine the important decision-making layer and decision-independent layer in each intermediate layer.
The device according to claim 14, wherein said performing dimensionality reduction processing on each intermediate layer feature of the evaluation sample comprises:

For each intermediate layer, train an autoencoder;

The self-encoder corresponding to each intermediate layer is used to perform dimensionality reduction processing on the intermediate layer features of the intermediate layer of the evaluation sample.
The device of claim 14, wherein the interpretable classifier is a tree model or a logistic regression model.
The device of claim 10, wherein the device further comprises:

The update unit is used to make the training unit repeat the re-training after replacing the part of the neurons.
The apparatus of claim 10, wherein the certain probability is 50 percent.
A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed in a computer, the computer is caused to execute the method according to any one of claims 1-9.
A computing device includes a memory and a processor, the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1-9 is implemented.