CN115454949A - Shared data determination method and device, electronic equipment and storage medium - Google Patents

Abstract

The present application provides a shared data determination method, device, electronic equipment and storage medium. The method includes: receiving generated record data and sensitive record data of the current batch; using a local discriminator network to update the local discriminator network according to the generated record data and sensitive record data of the current batch; using the updated local discriminator network to construct a local The discriminator responds, and uses the data sharing platform to update the generator network according to the pre-acquired real integrated record training data, synthesized integrated record training data and discriminator response training relation discriminator; input the pre-acquired random vector into the updated The generator network is used to obtain the generated record data group; the generated record data group includes: multiple generated record data; and the target shared data is constructed according to the weight of each generated record data. Realize the data sharing that can achieve vertical segmentation while ensuring the avoidance of privacy leakage, and ensure the high availability of shared data.

Description

Shared data determination method, device, electronic equipment and storage medium

technical field

The present application relates to the technical field of data processing, and in particular to a method, device, electronic device and storage medium for determining shared data.

Background technique

In related technologies, in order to achieve vertical split data sharing, each data owner usually constructs a generative model using its own local data set, and then uses the learned generative model to generate data, and finally the data generated by each party Integration to form a shared data set. However, there is a problem in related technologies that the availability of shared data is poor due to ID mismatch between various partial data sets.

Contents of the invention

In view of this, the purpose of the present application is to propose a method, device, electronic device and storage medium for determining shared data.

Based on the purpose, in the first aspect, the present application provides a method for determining shared data, including:

S1: Receive the current batch of generated record data and sensitive record data;

S2: Using the local discriminator network to update the local discriminator network according to the generated record data and sensitive record data of the current batch;

S3: Use the updated local discriminator network to construct the local discriminator response, and use the data sharing platform to update and generate server network;

S4: Input the pre-collected random vector into the updated generator network to obtain a generated record data set; the generated record data set includes: a plurality of generated record data;

S5: Construct target shared data according to the weight of each generated record data.

In a possible implementation, the receiving the current batch of generated record data and sensitive record data includes:

receiving said production record data for a current batch from said network of producers;

The current batch of sensitive record data is obtained according to sampling of the pre-acquired sensitive data set.

In a possible implementation manner, the updating the local discriminator network according to the generated record data and sensitive record data of the current batch by using the local discriminator network includes:

using the local discriminator network to determine a current batch loss function based on the generated record data and sensitive record data of the current batch;

determining gradient information of the local discriminator network according to the current batch loss function, and performing pruning processing on the gradient information;

According to the adaptive noise generation technology, the Gaussian noise sampled from the Gaussian distribution is used to perturb the pruned gradient information to determine the update parameters;

The local discriminator network is updated according to the update parameters.

In a possible implementation manner, the determining the loss function of the current batch according to the generated record data and sensitive record data of the current batch by using the local discriminator network further includes:

和二阶动量估计

Initialize the parameters of the discriminator network; the parameters of the discriminator network include: first-order momentum estimation

and the second-order momentum estimate

In a possible implementation manner, the Gaussian noise sampled from a Gaussian distribution is used to perturb the pruned gradient information to determine update parameters, including:

和二阶动量估计

进行更新以确定所述更新参数；其中，所述更新公式，表示为The first-order momentum is estimated according to the update formula

and the second-order momentum estimate

Perform an update to determine the update parameters; wherein, the update formula is expressed as

表示更新后的一阶动量估计，

表示更新后的二阶动量估计，β₁表示第一衰减速率，β₂表示第二衰减速率，

表示第t-1轮的一阶动量估计，

表示第t-1轮的二阶动量估计，

表示判别器第t轮的梯度向量。in,

represents the updated first-order momentum estimate,

represents the updated second-order momentum estimate, β1 represents the _first decay rate, and β2 represents the _second decay rate,

Denotes the first-order momentum estimate for round t-1,

Denotes the second-order momentum estimate for round t-1,

Indicates the gradient vector of discriminator round t.

In a possible implementation, the inputting the pre-collected random vector into the updated generator network to obtain the generated record data set includes:

Inputting the pre-collected random vector into the updated generator network, and using the updated generator network to determine to generate record data;

Steps S1-S3 are repeatedly executed until the number of iterations reaches a threshold, and the generated record data determined according to the generator network in each iteration are determined to obtain the generated record data set.

In a possible implementation manner, the constructing target shared data according to the weight of each generated record data further includes:

Save the weight of the generated record data determined by the generator network in each iteration;

inputting hidden vectors extracted according to the prior distribution into the generator network in each iteration to determine a plurality of synthetic record data;

Assign weights for each of the synthetic record data; wherein, the update formula for updating the weights in each iteration is expressed as

表示合成记录数据，

表示第r个生成器的生成记录，R表示选择的生成器网络的数量，d_j表示距离函数，M表示特征数量。Among them, w _ri represents the weight,

represents synthetic record data,

Indicates the generation record of the rth generator, R indicates the number of selected generator networks, d _j indicates the distance function, and M indicates the number of features.

The update formula for updating the synthetic record data in each iteration is expressed as

表示第r个生成器的生成记录。in,

Indicates the generation record of the rth generator.

In a second aspect, the present application provides a device for determining shared data, including:

a receiving module configured to receive the generated record data and sensitive record data of the current batch;

A first update module configured to update the local discriminator network according to the generated record data and sensitive record data of the current batch by using the local discriminator network;

The second update module is configured to use the updated local discriminator network to construct a local discriminator response, and use the data sharing platform to record training data according to the pre-acquired real ensemble, synthesize the ensemble record training data and the discriminator response training relational discriminator to update the generator network;

The determination module is configured to input the pre-collected random vector into the updated generator network to obtain the generated record data set; the generated record data set includes: a plurality of generated record data;

A construction module configured to construct target shared data according to the weight of each generated record data.

In a third aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the program, the computer program described in the first aspect is implemented. The shared data determination method described above.

In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method described in the first aspect Shared data determination method.

From the above, it can be seen that the shared data determination method, device, electronic equipment and storage medium provided by the present application receive the generated record data and sensitive record data of the current batch; Generate record data and sensitive record data for the second time to update the local discriminator network; use the updated local discriminator network to construct a local discriminator response, and use the data sharing platform to synthesize integrated records based on the pre-acquired real integrated record training data The training data and the discriminator respond to the training relationship discriminator to update the generator network; the random vectors collected in advance are input to the updated generator network to obtain the generated record data set; the generated record data set includes: Generate record data; construct target shared data according to the weight of each generated record data. It realizes the data sharing that can achieve vertical segmentation while ensuring the avoidance of privacy leakage, and then ensures that the final shared data has high availability.

Description of drawings

In order to more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments or related technologies. Obviously, the accompanying drawings in the following description are only for this application Embodiments, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

FIG. 1 shows a schematic diagram of a scenario of multi-party data sharing in the related art.

Fig. 2 shows an exemplary flow chart of a method for determining shared data provided by an embodiment of the present application.

FIG. 3 shows a schematic diagram of a multi-party data sharing algorithm satisfying differential privacy according to an embodiment of the present application.

Fig. 4(a) shows a comparison chart of IS scores matching the first MNIST data set according to various algorithms according to the embodiment of the present application.

Fig. 4(b) shows a comparison chart of FID scores matching the first MNIST data set according to various algorithms according to the embodiment of the present application.

Fig. 4(c) shows a comparison chart of IS scores matching the second MNIST data set according to various algorithms according to the embodiment of the present application.

FIG. 4( d ) shows a comparison chart of FID scores matching the second MNIST data set according to various algorithms of the embodiments of the present application.

Fig. 5(a) shows a comparison chart of the first experimental results of various algorithms under different privacy budgets according to the embodiment of the present application.

Fig. 5(b) shows a comparison chart of the results of the second experiment of each algorithm under different privacy budgets according to the embodiment of the present application.

FIG. 6( a ) shows a comparison chart of the results of the first experiment of each algorithm under different numbers of participants according to the embodiment of the present application.

Fig. 6(b) shows a comparison chart of the results of the second experiment of each algorithm under different numbers of participants according to the embodiment of the present application.

Fig. 7 shows an exemplary structural diagram of an apparatus for determining shared data provided by an embodiment of the present application.

Fig. 8 shows an exemplary structural diagram of an electronic device provided by an embodiment of the present application.

detailed description

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application shall have the usual meanings understood by those skilled in the art to which the present application belongs. "First", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Comprising" or "comprising" and similar words mean that the elements or items appearing before the word include the elements or items listed after the word and their equivalents, without excluding other elements or items. Words such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "Down", "Left", "Right" and so on are only used to indicate the relative positional relationship. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

As mentioned in the background technology section, data sharing helps to stimulate the hidden economic value in data. McKinsey & Company reported that if the current seven major industries (business, finance, healthcare, education, transportation, electricity, and oil and gas) ) The data in ) are open to each other, which will bring a lot of economic benefits. Data sharing helps to mine the knowledge contained in the data, for example: after sharing the data of hospitals and CDCs, it can be used to analyze the mode of disease transmission and improve the level of public medical care; after sharing the data of multiple shopping platforms, it can be used for personal Optimized product recommendation to improve consumer shopping experience; sharing data from multiple banks can better evaluate customer credit and monitor multi-party lending and financial fraud. However, under normal circumstances, data is often distributed in the hands of multiple data owners (for example, hospitals have residents’ medical records, banks have customer account records), and contain a lot of sensitive information. If the data of different data owners is directly Sharing will bring serious privacy leakage problems.

Referring to Figure 1, this scenario mainly involves three roles: data sharing platform, data owner and data user. Among them, each data owner holds a local sensitive data set about different attributes of the same group of users. The sharing platform assists in the sharing of local sensitive data sets, and builds a data sharing model. The new shared data generated by the data sharing model has the same statistical distribution characteristics as the integrated data sets, and at the same time, there is no direct sharing of local sensitive data sets. , which protects the privacy of each data to a certain extent. Data users can use the shared data to carry out various data analysis and mining tasks.

It can be seen from the above processing that in the process of multi-party data sharing, it is possible to prevent the data sharing model and the final data user from obtaining the privacy of the data owner, but before the multi-party data sharing model is formed, the private data of each data owner It could still be leaked. Specifically, for each local sensitive data set, there are the following three roles that may pose a threat to privacy: 1) data sharing platform; 2) data owners participating in data sharing; 3) data users or others who may obtain The ultimate shared data potential attacker.

The privacy-preserving multi-party data sharing technology provides a feasible solution to the privacy leakage problem caused by multi-party data sharing. The Differential Privacy (DP) technology proposed in recent years provides a feasible solution to the privacy leakage problem caused by data sharing. Different from traditional anonymity-based privacy models (eg, k-anonymity [1] and l-diversity, etc.), differential privacy provides a strict, quantifiable privacy protection method for sensitive data. By adding an appropriate amount of noise to the statistical results to ensure that modifying a record in the data set will not have a significant impact on the statistical results, so as to achieve the requirement of privacy protection.

Through research, the applicant found that in related technologies, an intuitive method for vertically splitting data sharing is: each data owner uses its own local data set to build a generative model, such as GAN or BayesNetwork, and then uses the learned The generative model generates data, and finally integrates the data generated by each party to form a shared data set. However, it is difficult to guarantee the usability of the resulting data due to the ID mismatch between the various partial datasets.

For this reason, a shared data determination method, device, electronic device and storage medium provided by the present application receive the generated record data and sensitive record data of the current batch; data and sensitive record data to update the local discriminator network; use the updated local discriminator network to construct the local discriminator response, and use the data sharing platform to obtain the pre-acquired real integrated record training data, synthesize integrated record training data and all The discriminator responds to the training relationship discriminator to update the generator network; the random vectors collected in advance are input to the updated generator network to obtain the generated record data set; the generated record data set includes: a plurality of generated record data ; Construct target shared data according to the weight of each generated record data. It realizes the data sharing that can achieve vertical segmentation while ensuring the avoidance of privacy leakage, and then ensures that the final shared data has high availability.

The method for determining shared data provided by the embodiment of the present application will be specifically described below through specific embodiments.

Referring to Fig. 2, a method for determining shared data provided in the embodiment of the present application specifically includes the following steps:

S1: Receive the current batch of generated record data and sensitive record data;

S2: Using the local discriminator network to update the local discriminator network according to the generated record data and sensitive record data of the current batch;

S3: Use the updated local discriminator network to construct the local discriminator response, and use the data sharing platform to update and generate server network;

S4: Input the pre-collected random vector into the updated generator network to obtain a generated record data set; the generated record data set includes: a plurality of generated record data;

S5: Construct target shared data according to the weight of each generated record data.

In some embodiments, this application can be applied to the scenario of multiple data owners and data sharing platforms. Taking this as an example, each data owner k holds a discriminator network, and the data sharing platform holds a generating A network G and two relational discriminators.

In some embodiments, the latent variable z (usually random noise subject to Gaussian distribution) generates samples through the generator network G. For the discriminator D, this is a binary classification problem, and V(D,G) is two Common cross-entropy loss in classification problems. In order to ensure that V(D,G) achieves the maximum value, the discriminator can be trained for k iterations, and then the generator can be iterated once. The specific training process is as follows:

Initialize the parameters of the two networks of generator G and discriminator D. Draw n samples from the training set, and the generator generates n samples using the defined noise distribution. Fix the generator G and train the discriminator D to make it distinguish between true and false as much as possible. After cyclically updating the discriminator D for k times, update the generator G once, so that the discriminator cannot distinguish between true and false as much as possible. After multiple update iterations, ideally, the final discriminator D cannot distinguish whether the picture comes from the real training sample set or the sample generated by the generator G. At this time, the discrimination probability is 0.5, and the training is completed.

In some embodiments, the DPGDAN algorithm proposed in this application involves two stages. In stage 1, the data sharing platform conducts 1-to-K training with K data owners. In particular, the discriminator and generator are trained alternately. Among them, each data owner uses the adaptive gradient perturbation method to update the parameters of the discriminator held by it; and the data sharing platform uses the discriminator feedback that satisfies differential privacy to update the parameters of the generator network held by itself. The parameters are updated. In stage 2, a random vector is sampled from a certain prior distribution, and the random vector is sent to the generator network obtained during the training process to construct a synthetic record, and the synthetic record is aggregated to obtain the final shared data record.

In some embodiments, the DPGDAN algorithm mentioned in this application involves the training of four neural networks, namely, the local discriminator network held by each data owner, the generator network held by the data sharing platform, and the relational discriminator network. In particular, the local discriminator network is used to distinguish local sensitive data records from those generated by the generator, while the generator network is dedicated to generating synthetic records that are considered "real" by these K discriminator networks. Given the above different objectives, the objective function of the generator can be written as follows:

表示真实记录关系型判别器，

表示合成记录关系型判别器，

和

表示关系型判器，

表示生成器的整合记录，

表示生成器关于A的局部记录，

表示生成器关于B的局部记录。where m represents the batch size,

Denotes a ground-truth relational discriminator,

Denotes a synthetic record relational discriminator,

with

Represents a relational discriminator,

represents the integration record for the generator,

represents the generator's local record of A,

Represents the generator's local record about B.

The objective function of the synthetic record relation discriminator is as follows:

表示真实记录关系型判别器，

表示合成记录关系型判别器，

和

表示关系型判器，

表示混淆操作后的整合记录，

表示对整合记录的梯度，

表示生成器生成的整合记录。where m represents the batch size,

Denotes a ground-truth relational discriminator,

Denotes a synthetic record relational discriminator,

with

Represents a relational discriminator,

Indicates the integrated record after the obfuscation operation,

Denotes the gradient to the integrated record,

Represents the consolidated record produced by the generator.

The objective function of the real record relation discriminator is as follows:

表示真实记录关系型判别器，

表示合成记录关系型判别器，

和

表示关系型判器，

表示整合记录数据，

表示混淆后的整合记录，x_i表示整合记录。where m represents the batch size,

Denotes a ground-truth relational discriminator,

Denotes a synthetic record relational discriminator,

with

Represents a relational discriminator,

Indicates the consolidated record data,

Indicates the integrated record after obfuscation, and _xi indicates the integrated record.

和

用于区分真实的局部记录数据和生成的局部记录数据，其目标函数分别如下：Two local discriminators

with

It is used to distinguish real partial record data from generated partial record data, and its objective functions are as follows:

Among them, x _iA and x _iB represent partial recording data.

In some embodiments, the gradient update of the generator network needs to perform gradient descent on its objective function, however, this contains the sensitive information of the feedback of the discriminator. In order to update the generator gradient without revealing the privacy of each data owner, the gradient calculation of the generator network is first decomposed, and then the sensitive information related to the discriminator network is determined, and then desensitized.

Based on the chain rule of calculus, the gradient calculation of the generator can be decomposed as follows:

表示判别器网络的响应，其中包含了敏感信息，

表示非敏感的计算因子。Among them, represents the objective function, K represents the number of local discriminators,

Represents the response of the discriminator network, which contains sensitive information,

Indicates an insensitive calculation factor.

After the above decomposition of the gradient calculation, the iterative process of the following steps can be used to desensitize the parameters of the generator network and generate privacy-preserving discriminator network feedback.

和二阶动量估计

In some embodiments, the determining the loss function of the current batch according to the generated record data and the sensitive record data of the current batch by using the local discriminator network further includes: initializing parameters of the discriminator network; The parameters of the discriminator network are described, including: first-order momentum estimation

and the second-order momentum estimate

In some embodiments, the receiving the current batch of generated record data and sensitive record data includes: receiving the current batch of generated record data from the generator network; obtaining according to pre-acquired sensitive data set sampling The sensitive record data of the current batch.

In some embodiments, the updating the local discriminator network according to the generated record data and the sensitive record data of the current batch by using the local discriminator network includes: using the local discriminator network according to the current batch The generated record data and sensitive record data determine the current batch loss function; determine the gradient information of the local discriminator network according to the current batch loss function, and perform pruning processing on the gradient information; generate The technique uses Gaussian noise sampled from a Gaussian distribution to perturb the pruned gradient information to determine update parameters; and updates the local discriminator network according to the update parameters.

After the above decomposition of the gradient calculation, the iterative process of the following steps can be used to desensitize the parameters of the generator network and generate privacy-preserving discriminator network feedback.

二阶动量估计

以及噪音规模σ₀。Step 1: Initialize discriminator network parameters, first-order momentum estimation

second order momentum estimation

and the noise scale σ ₀ .

Step 2: Each data owner receives a batch of synthetic record data from the generator

Step 3: Each data owner extracts a batch of sensitive data record data locally

Step 4: Input the data records obtained in steps 1 and 2 into the generator network, and calculate its loss function:

表示均值，D为判别器网络，G为生成器网络，z为隐向量，x_k为真实记录数据。in,

Indicates the mean value, D is the discriminator network, G is the generator network, z is the hidden vector, and x _k is the real recorded data.

Step 5: Calculate the gradient information of the discriminator network according to the loss function.

Step 6: According to the adaptive noise generation method, Gaussian noise is sampled from the Gaussian distribution to perturb the gradient:

二阶动量估计

进行更新：Step 7: Estimating the first-order momentum

second order momentum estimation

Make an update:

表示更新后的一阶动量估计，

表示更新后的二阶动量估计，β₁表示第一衰减速率，β₂表示第二衰减速率，

表示第t-1轮的一阶动量估计，

表示第t-1轮的二阶动量估计，

表示判别器第t轮的梯度向量。in,

represents the updated first-order momentum estimate,

represents the updated second-order momentum estimate, β1 represents the _first decay rate, and β2 represents the _second decay rate,

Denotes the first-order momentum estimate for round t-1,

Denotes the second-order momentum estimate for round t-1,

Indicates the gradient vector of discriminator round t.

Step 8: Update the weights of the discriminator network.

In some embodiments, the inputting the pre-collected random vectors into the updated generator network to obtain the generated record data set includes: inputting the pre-collected random vectors into the updated generator network, and using The updated generator network determines the generated record data; repeat steps S1-S3 until the number of iterations reaches a threshold, and determine the generated record data determined according to the generator network in each iteration to obtain the generated record data set.

In some embodiments, the specific steps of snapshot aggregation include:

Save the weight information of the generator network obtained by different iterations;

Feed the hidden vector extracted from the prior distribution into the generator network to obtain the corresponding synthetic record data;

assigning a weight to each of said synthetic record data;

The weight and shared data are updated according to the update formula.

Specifically, the construction of the target shared data according to the weight of each generated record data also includes: saving the weight of the generated record data determined according to the generator network in each round of iteration; A vector is input into the generator network in each round of iteration to determine a plurality of synthetic record data; weights are assigned to each of the synthetic record data; wherein, the update formula for updating the weight in each round of iteration is expressed as

表示合成记录数据，

表示第r个生成器的生成记录，R表示选择的生成器网络的数量，d_j表示距离函数，M表示特征数量。Among them, w _ri represents the weight,

represents synthetic record data,

Indicates the generation record of the rth generator, R indicates the number of selected generator networks, _dj indicates the distance function, and M indicates the number of features.

The update formula for updating the synthetic record data in each iteration is expressed as

表示第r个生成器的生成记录。in,

Indicates the generation record of the rth generator.

The snapshot aggregation method of this application is based on the idea that, ideally, a trained generator can reproduce the real data distribution. Synthesis of shared records is done by following the standard practice in GANs, i.e., feeding a trained generator network with latent vectors z_i sampled from a prior distribution, and then taking the output of the generator as shared data. However, the generator and discriminator networks may not be trained long enough due to the limited privacy budget. Therefore, only using a trained generator network to generate shared records will ignore useful information about the generator during training. To this end, this application proposes a snapshot aggregation method that leverages the generator network obtained during training to improve the utility of the final shared data.

In the comparison, the widely used real data sets Adult and Bank are used for experimental verification. Among them, Table 2 lists the hyperparameter values in the experiment. The Adult data set contains 48,842 US census data records; the Bank data set contains 45,211 account records of Portuguese banking institutions. The statistics of the two data sets are shown in Table 1 below.

Table 1 Statistical analysis of Adult and Bank datasets

Table 2 Hyperparameter settings

The method is evaluated following standard practice for data sharing tasks, i.e. measuring the utility of shared data through the effectiveness of machine learning. In particular, we first train a predictive model using shared data, and then test the trained predictive model on a real test set. The higher the accuracy of the predictive model, the better the usefulness of the data.

The effect and performance of the method of the present application will be described below by comparing the Accuracy experimental results of the method of the present application with the existing method. Referring to FIG. 3 , it shows the effectiveness of the shared data determination algorithm provided by the present application. Experimental results refer to Fig. 4 (a)-Fig. 4 (d), Fig. 5 (a)-Fig. 5 (b) and Fig. 6 (a)-Fig. 6 (b), wherein DPGDAN algorithm represents that the application embodiment provides algorithm, Nonprivat algorithm and Nosplit algorithm are algorithms in the related art. From Fig. 4(a)-Fig. 4(d), Fig. 5(a)-Fig. 5(b) and Fig. 6(a)-Fig. Under the privacy budget and the number of different participants, the performance of the comparison algorithm is close to that of the comparison algorithm, and the application can better preserve the original distribution characteristics of the original data under the given privacy budget. It should be pointed out that the comparison algorithm used in this application is only to indicate the upper limit of performance, and cannot be directly used for privacy-preserving vertical split data sharing tasks.

From Figure 4(a)-Figure 4(d), it can be seen that the shared data generated by this application can better support various data prediction or machine learning tasks. From Fig. 5(a)-Fig. 5(b), we can see that our application maintains good performance under different privacy budgets, because the snapshot aggregation method ensures a better balance between privacy and data utility. In addition to trade-offs, the learning paradigm of this application can also use data from various parties to guide model updates. Furthermore, the 1-versus-K design of the present application makes the generator network more likely to receive the informative signal from the discriminator network, thus better guiding the generator to approximate the real data distribution.

From Fig. 6(a)-Fig. 6(b), we can see that the accuracy drop of our application is small compared with the baseline. The reason is that increasing the number of discriminators reduces the variance of the min-max objective function over the entire run and speeds up the convergence.

From the above, it can be seen that the shared data determination method, device, electronic equipment and storage medium provided by the present application receive the generated record data and sensitive record data of the current batch; Generate record data and sensitive record data for the second time to update the local discriminator network; use the updated local discriminator network to construct a local discriminator response, and use the data sharing platform to synthesize integrated records based on the pre-acquired real integrated record training data The training data and the discriminator respond to the training relationship discriminator to update the generator network; the random vectors collected in advance are input to the updated generator network to obtain the generated record data set; the generated record data set includes: Generate record data; construct target shared data according to the weight of each generated record data. It realizes the data sharing that can achieve vertical segmentation while ensuring the avoidance of privacy leakage, and then ensures that the final shared data has high availability.

In order to solve the problem that the unilateral data publishing method satisfying differential privacy mentioned in the background technology cannot be directly applied to vertically split data sharing, the applicant proposed a multi-party data sharing method satisfying differential privacy (DPGDAN algorithm). The main idea of this method is that each data owner and data sharing platform jointly train a customized Generative Adversarial Network (GAN) to extract the distribution information of each local sensitive data set. In particular, each data owner holds a discriminator, uses the local sensitive data set to train it, and then desensitizes the feedback information of the discriminator, uploads it to the data sharing platform, and the data sharing platform utilizes the collected The feedback information from the discriminator and the feedback from the relational discriminator update the parameters of the generator. After training, the generator generates shared data with the assistance of the discriminator. Since feedback information from various data owners is used in the update process of the generator, the final shared data can have higher utility.

It should be noted that the method in the embodiment of the present application may be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario, and is completed by cooperation of multiple devices. In the case of such a distributed scenario, one of the multiple devices may only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete all described method.

It should be noted that the above describes some embodiments of the present application. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that of the described embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Fig. 7 shows an exemplary structural diagram of an apparatus for determining shared data provided by an embodiment of the present application.

Based on the same inventive concept, the present application further provides a device for determining shared data corresponding to the method in any of the embodiments.

Referring to FIG. 7 , the device for determining shared data includes: a receiving module, a first updating module, a second updating module, a determining module, and a building module; wherein,

a receiving module configured to receive the generated record data and sensitive record data of the current batch;

A first update module configured to update the local discriminator network according to the generated record data and sensitive record data of the current batch by using the local discriminator network;

The second update module is configured to use the updated local discriminator network to construct a local discriminator response, and use the data sharing platform to record training data according to the pre-acquired real ensemble, synthesize the ensemble record training data and the discriminator response training relational discriminator to update the generator network;

The determination module is configured to input the pre-collected random vector into the updated generator network to obtain the generated record data set; the generated record data set includes: a plurality of generated record data;

A construction module configured to construct target shared data according to the weight of each generated record data.

In a possible implementation manner, the receiving module is further configured to:

receiving said production record data for a current batch from said network of producers;

The current batch of sensitive record data is obtained according to sampling of the pre-acquired sensitive data set.

In a possible implementation manner, the first update module is further configured to:

using the local discriminator network to determine a current batch loss function based on the generated record data and sensitive record data of the current batch;

determining gradient information of the local discriminator network according to the current batch loss function, and performing pruning processing on the gradient information;

According to the adaptive noise generation technology, the Gaussian noise sampled from the Gaussian distribution is used to perturb the pruned gradient information to determine the update parameters;

The local discriminator network is updated according to the update parameters.

In a possible implementation manner, the device further includes: an initialization module;

The initialization module is further configured to:

和二阶动量估计

Initialize the parameters of the discriminator network; the parameters of the discriminator network include: first-order momentum estimation

and the second-order momentum estimate

In a possible implementation manner, the second update module is further configured to:

和二阶动量估计

进行更新以确定所述更新参数；其中，所述更新公式，表示为The first-order momentum is estimated according to the update formula

and the second-order momentum estimate

Perform an update to determine the update parameters; wherein, the update formula is expressed as

表示更新后的一阶动量估计，

表示更新后的二阶动量估计，β₁表示第一衰减速率，β₂表示第二衰减速率，

表示第t-1轮的一阶动量估计，

表示第t-1轮的二阶动量估计，

表示判别器第t轮的梯度向量。in,

represents the updated first-order momentum estimate,

represents the updated second-order momentum estimate, β1 represents the _first decay rate, and β2 represents the _second decay rate,

Denotes the first-order momentum estimate for round t-1,

Denotes the second-order momentum estimate for round t-1,

Indicates the gradient vector of discriminator round t.

In a possible implementation manner, the determination module is further configured to:

Inputting the pre-collected random vector into the updated generator network, and using the updated generator network to determine to generate record data;

Steps S1-S3 are repeatedly executed until the number of iterations reaches a threshold, and the generated record data determined according to the generator network in each iteration are determined to obtain the generated record data set.

In a possible implementation manner, the device further includes: a third update module;

The third update module is further configured to:

Save the weight of the generated record data determined by the generator network in each iteration;

inputting hidden vectors extracted according to the prior distribution into the generator network in each iteration to determine a plurality of synthetic record data;

Assign weights for each of the synthetic record data; wherein, the update formula for updating the weights in each iteration is expressed as

表示合成记录数据，

表示第r个生成器的生成记录，，R表示选择的生成器网络的数量，d_j表示距离函数，M表示特征数量。Among them, w _ri represents the weight,

represents synthetic record data,

Denotes the generation record of the rth generator, R represents the number of selected generator networks, _dj represents the distance function, and M represents the number of features.

The update formula for updating the synthetic record data in each iteration is expressed as

表示第r个生成器的生成记录。in,

Indicates the generation record of the rth generator.

For the convenience of description, when describing the above devices, functions are divided into various modules and described separately. Of course, when implementing the present application, the functions of each module can be implemented in one or more pieces of software and/or hardware.

The device in the embodiment is used to implement the corresponding method for determining shared data in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiment, so details are not repeated here.

Fig. 8 shows an exemplary structural diagram of an electronic device provided by an embodiment of the present application.

Based on the same inventive concept, and corresponding to the method in any of the above embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, the processing The method for determining shared data described in any one of the above embodiments is implemented when the program is executed by the computer. FIG. 8 shows a schematic diagram of a more specific hardware structure of an electronic device provided by this embodiment. The device may include: a processor 810 , a memory 820 , an input/output interface 830 , a communication interface 840 and a bus 850 . The processor 810 , the memory 820 , the input/output interface 830 and the communication interface 840 are connected to each other within the device through the bus 850 .

The processor 810 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of this specification.

The memory 820 may be implemented in the form of ROM (Read Only Memory, read only memory), RAM (Random Access Memory, random access memory), static storage device, dynamic storage device, and the like. The memory 820 can store operating systems and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 820 and invoked by the processor 810 for execution.

The input/output interface 830 is used to connect the input/output module to realize information input and output. The input/output module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 840 is used to connect a communication module (not shown in the figure), so as to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.), and can also realize communication through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

Bus 850 includes a path for carrying information between the various components of the device (eg, processor 810, memory 820, input/output interface 830, and communication interface 840).

It should be noted that although the device only shows a processor 810, a memory 820, an input/output interface 830, a communication interface 840, and a bus 850, in a specific implementation process, the device may also include of other components. In addition, those skilled in the art can understand that the device may only include components necessary to implement the solutions of the embodiments of this specification, and does not necessarily include all the components shown in the figure.

The electronic device in this embodiment is used to implement the corresponding method for determining shared data in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

Based on the same inventive concept, the present application also provides a non-transitory computer-readable storage medium corresponding to the method in any of the embodiments described above, the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions It is used to make the computer execute the method for determining shared data as described in any one of the above embodiments.

The computer-readable medium in this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The computer instructions stored in the storage medium of the embodiments are used to enable the computer to execute the method for determining shared data as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is exemplary only, and is not intended to imply that the scope of the application (including claims) is limited to these examples; under the thinking of the application, the above embodiments or Combinations of technical features in different embodiments are also possible, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the application as described above, which are not provided in details for the sake of brevity.

In addition, to simplify illustration and discussion, and so as not to obscure the embodiments of the present application, well-known power supply/connection circuits associated with integrated circuit (IC) chips and other components may or may not be shown in the provided figures. ground connection. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the embodiments of the present application, and this also takes into account the fact that details regarding the implementation of these block diagram devices are highly dependent on the implementation of the embodiments of the present application to be implemented. platform (ie, the details should be well within the purview of those skilled in the art). Where specific details (e.g., circuits) have been set forth to describe example embodiments of the present application, it will be apparent to those skilled in the art that reference may be made without or with variation from these specific details. Implement the embodiment of the present application below. Accordingly, these descriptions should be regarded as illustrative rather than restrictive.

Although the application has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations of those embodiments will be apparent to those of ordinary skill in the art from the foregoing description. For example, other memory architectures such as dynamic RAM (DRAM) may use the discussed embodiments.

The embodiments of the present application are intended to embrace all such alternatives, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent replacements, improvements, etc. within the spirit and principles of the embodiments of the present application shall be included within the protection scope of the present application.

Claims (10)

1. A method for determining shared data, comprising:

s1: receiving the generated record data and the sensitive record data of the current batch;

s2: updating the local arbiter network according to the generated record data and the sensitive record data of the current batch by using the local arbiter network;

s3: constructing a local discriminator response by using the updated local discriminator network, and updating a generator network by using a data sharing platform according to real integrated record training data, synthetic integrated record training data and the discriminator response training relation discriminator which are obtained in advance;

s4: inputting the random vector collected in advance into an updated generator network to obtain a generated record data set; the generating of the record data set includes: a plurality of generation record data;

s5: and constructing target shared data according to the weight of each generated record data.

2. The method of claim 1, wherein receiving production record data and sensitive record data for a current batch comprises:

receiving the generation record data for a current batch from the generator network;

and sampling according to a pre-acquired sensitive data set to obtain the sensitive record data of the current batch.

3. The method of claim 1, wherein said updating the local arbiter network from the generated log data and the sensitive log data of the current batch using the local arbiter network comprises:

determining a current batch loss function according to the generated record data and the sensitive record data of the current batch by using the local arbiter network;

determining gradient information of the local discriminator network according to the current batch loss function, and pruning the gradient information;

according to the self-adaptive noise generation technology, gaussian noise obtained by sampling from Gaussian distribution is used for disturbing the gradient information after pruning so as to determine updating parameters;

and updating the local arbiter network according to the updated parameters.

4. The method of claim 3, wherein determining a current lot loss function from the generated log data and the sensitive log data of the current lot using the local arbiter network further comprises:

initializing parameters of the discriminator network; the parameters of the discriminator network comprise: first order momentum estimation

And second order momentum estimation

5. The method of claim 4, wherein perturbing the pruned gradient information with Gaussian noise sampled from the Gaussian distribution to determine the update parameters comprises:

estimating the first order momentum according to an updated formula

And second order momentum estimation

Updating to determine the updated parameters; wherein the update formula is expressed as

Wherein,

representing the updated first-order momentum estimate,

representing the updated second-order momentum estimate, beta ₁ Denotes a first decay rate, beta ₂ Which is indicative of a second rate of decay,

representing the first order momentum estimate for the t-1 th round,

representing the second order momentum estimate for the t-1 th round,

the gradient vector for round t-1 is shown.

6. The method of claim 1, wherein inputting the pre-collected random vectors into an updated generator network to obtain the generated record data set comprises:

inputting a random vector collected in advance into an updated generator network, and determining to generate record data by using the updated generator network;

and (4) repeatedly executing the steps S1-S3 until the iteration times reach a threshold value, and determining the generated record data determined according to the generator network in each iteration so as to obtain the generated record data group.

7. The method of claim 6, wherein said constructing target shared data according to the weight of each of said generated record data further comprises:

storing the weight of the generated record data determined according to the generator network in each iteration;

inputting hidden vectors extracted according to a prior distribution into the generator network in each iteration to determine a plurality of synthetic record data;

assigning a weight to each of the synthetic record data; wherein the updating formula for updating the weight in each iteration is expressed as

Wherein w _ri The weight is represented by a weight that is,

it is indicated that the resultant recorded data,

representing the generation record of the R-th generator, R representing the number of generator networks selected, d _j Representing the distance function and M the number of features.

An update formula for updating the composite record data in each iteration, expressed as

Wherein,

the representation represents the generated record representing the r-th generator.

8. A shared data determining apparatus, comprising:

the receiving module is configured to receive the generated record data and the sensitive record data of the current batch;

a first update module configured to update the local arbiter network according to the generated record data and sensitive record data of the current batch using a local arbiter network;

the second updating module is configured to construct a local discriminator response by using the updated local discriminator network, and update the generator network by using the data sharing platform according to the real integrated record training data, the synthetic integrated record training data and the discriminator response training relation discriminator which are acquired in advance;

a determining module configured to input a pre-collected random vector to the updated generator network to obtain a generated record data set; the generating of the record data set includes: a plurality of generation record data;

a construction module configured to construct target shared data according to the weight of each of the generated record data.

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the program.

10. A non-transitory computer readable storage medium storing computer instructions for causing a computer to implement the method of any one of claims 1 to 7.

Images