CN113807536A

CN113807536A - Data processing method and device and electronic equipment

Info

Publication number: CN113807536A
Application number: CN202110357649.9A
Authority: CN
Inventors: 张一凡; 王虎; 李怡欣; 陈晓霖; 黄志翔; 彭南博; 程建波
Original assignee: Jingdong Technology Holding Co Ltd
Current assignee: Jingdong Technology Holding Co Ltd
Priority date: 2021-04-01
Filing date: 2021-04-01
Publication date: 2021-12-17
Anticipated expiration: 2041-04-01
Also published as: CN113807536B

Abstract

The application discloses a data processing method and device and electronic equipment. The method comprises the following steps: obtaining model scores respectively sent by a data party and a service party, wherein the model scores are generated by the data party and the service party according to respective trained generation models; and generating a label of the current user according to the model score and the trained collaborative model, so that in the data processing process, a data party and a service party only need to transmit the model score without encrypting the score to realize privacy protection, thereby avoiding technical problems caused by complex cryptology theoretical barriers, extremely high data communication and training cost and difficulty of parallel and asynchronous calculation while ensuring data safety and reducing the risk of data information leakage, and improving the efficiency in the data processing process.

Description

Data processing method and device and electronic equipment

Technical Field

The present application relates to the field of data processing technologies, and in particular, to a data processing method and apparatus, and an electronic device.

Background

In the scenes of modeling of Federal Learning (Federal Learning) and the like, due to the requirements of data and tag privacy protection on business parties, user privacy protection on data parties and the like, multiple parties cannot directly share data.

In the related art, data is usually encrypted and decrypted by adopting a cryptography technology, and feature engineering, univariate analysis, model training, reasoning and the like in a normal modeling process are completed by matching with a complex data interaction process. However, the cryptography theory has high technical barriers, high data communication and training costs, and difficulty in parallel and asynchronous computation, which inevitably causes technical problems of high cost and low efficiency in the existing data processing process. Therefore, how to improve the efficiency of the data processing process and reduce the cost consumed in the process has become one of the important research directions.

Disclosure of Invention

The present application is directed to solving, at least to some extent, one of the technical problems in the related art.

Therefore, a first objective of the present application is to provide a data processing method, which is used for solving the technical problems of high cost and low efficiency in the related data processing method process.

A second object of the invention is to propose another data processing method.

A third object of the present invention is to provide a data processing apparatus.

A fourth object of the invention is to propose another data processing device.

A fifth object of the invention is to propose an electronic device.

A sixth object of the present invention is to propose a computer-readable storage medium.

In order to achieve the above object, an embodiment of a first aspect of the present application provides a data processing method, where the method includes the following steps: obtaining model scores respectively sent by a data party and a service party, wherein the model scores are generated by the data party and the service party according to respective trained generation models; and generating a label of the current user according to the model score and the trained collaborative model.

According to the data processing method, a business party can obtain model scores sent by a data party and a business party respectively, the model scores are generated by the data party and the business party according to respective trained generation models, and then labels of current users are generated according to the model scores and the trained collaborative models so as to realize data processing, so that in the data processing process, the data party and the business party only need to transmit the model scores and can realize privacy protection without encrypting the scores, the data security is ensured, the data information leakage risk is reduced, meanwhile, the technical problems caused by complex cryptology theoretical barriers, extremely high data communication and training costs and the difficulty of parallel and asynchronous calculation are avoided, and the efficiency in the data processing process is improved.

In order to achieve the above object, a second aspect of the present application provides another data processing method, including: generating a model score according to a generated model trained by a data side; and sending the model score to a service party so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a trained generation model and the trained collaborative model.

In the second aspect of the present application, a data processing method is provided, in which a data side can generate a model score by a trained generative model according to the data side, further sending the model score to a service party, so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a generated model trained by the service party and the trained cooperative model, so as to realize data processing, and in the process of data processing, the data side and the service side only need to transmit the model score without encrypting the score to realize privacy protection, the method has the advantages that the data safety is ensured, the risk of data information leakage is reduced, meanwhile, the technical problems caused by complex cryptology theory barriers, extremely high data communication and training cost and difficulty of parallel and asynchronous calculation are avoided, and the efficiency in the data processing process is improved.

In order to achieve the above object, an embodiment of a third aspect of the present application provides a data processing apparatus, including: the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire model scores respectively sent by a data party and a service party, and the model scores are generated by the data party and the service party according to respective trained generation models; and the generation module is configured to generate the label of the current user according to the model score and the trained collaborative model.

The third aspect of the present application provides a data processing apparatus, a service party may obtain model scores sent by a data party and a service party respectively, where the model scores are generated by the data party and the service party according to respective trained generation models, and then generate a label of a current user according to the model scores and the trained collaborative models, so as to implement data processing, so that in a data processing process, the data party and the service party only need to transmit the model scores, privacy protection may be implemented without encrypting the scores, while ensuring data security and reducing a risk of data information leakage, technical problems caused by complex cryptography theoretical barriers, extremely high data communication and training costs, and difficulties in parallel and asynchronous computation are avoided, and efficiency in the data processing process is improved.

In order to achieve the above object, a fourth aspect of the present application provides another data processing apparatus, including: a first generation module configured to generate a model score from a trained generative model of a data party; and the second generation module is configured to send the model score to a service party, so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a trained generation model and the trained collaborative model.

In a fourth aspect of the present application, there is provided a data processing apparatus, wherein a data side may generate a model score by training a generative model according to the data side, further sending the model score to a service party, so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a generated model trained by the service party and the trained cooperative model, so as to realize data processing, and in the process of data processing, the data side and the service side only need to transmit the model score without encrypting the score to realize privacy protection, the method has the advantages that the data safety is ensured, the risk of data information leakage is reduced, meanwhile, the technical problems caused by complex cryptology theory barriers, extremely high data communication and training cost and difficulty of parallel and asynchronous calculation are avoided, and the efficiency in the data processing process is improved.

In order to achieve the above object, an embodiment of a fifth aspect of the present application provides an electronic device, including: the data processing method comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the data processing method according to any one of the embodiments of the first aspect or the second aspect of the present application.

In order to achieve the above object, a sixth aspect of the present application provides a computer-readable storage medium, which when executed by a processor, implements a data processing method as described in any one of the first or second aspect of the present application.

Drawings

FIG. 1 is a schematic illustration of a model training process disclosed in one embodiment of the present application;

FIG. 2 is a schematic diagram of a model inference process disclosed in one embodiment of the present application;

FIG. 3 is a schematic flow chart diagram illustrating a data processing method according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of a structure for generating a countermeasure network as disclosed in one embodiment of the present application;

FIG. 5 is a schematic diagram of a generator, an arbiter, and a sample distribution as disclosed in one embodiment of the present application;

FIG. 6 is a schematic flow chart diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 7 is a schematic flow chart diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 8 is a schematic flow chart diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 9 is a schematic flow chart diagram illustrating a data processing method according to another embodiment of the present disclosure;

FIG. 10 is a schematic flow chart diagram of a data processing method according to another embodiment of the present disclosure;

FIG. 11 is a schematic flow chart diagram illustrating a data processing method according to another embodiment of the present disclosure;

FIG. 12 is a schematic illustration of features disclosed in one embodiment of the present application;

FIG. 13 is a graph illustrating generative model penalties disclosed in one embodiment of the present application;

FIG. 14 is a schematic illustration of coordination model loss disclosed in one embodiment of the present application;

FIG. 15 is a schematic illustration of a model training process disclosed in one embodiment of the present application;

FIG. 16 is a schematic diagram of a model training process disclosed in another embodiment of the present application;

FIG. 17 is a block diagram of a data processing apparatus according to an embodiment of the present disclosure;

FIG. 18 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 19 is a schematic block diagram of a data processing apparatus according to another embodiment of the present disclosure;

FIG. 20 is a block diagram of a data processing apparatus according to another embodiment of the present disclosure;

fig. 21 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For a better understanding of the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

It should be understood that "and/or" referred to in the embodiments of the present application describes an association relationship of associated objects, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

It should be noted that, in practical applications, for various frameworks, algorithms related to federal learning modeling, such as Secure Boost, federal neural network, and the like, all implement data processing based on technologies such as homomorphic encryption and multi-party data interaction.

The following explains the training process of the federated modeling by taking SecureBoost as an example, as shown in fig. 1. For a wind control good or bad user prediction modeling application scenario, the SecureBoost training process may include the following steps:

optionally, the business party computes locally. Wherein, the business side with the user quality label calculates the gradient of each sample by using the residual error of the own label and the last tree, and the gradient is obtained after encryption<g₁>,<h₁>),...,(<g_n>,<h_n>And sending the data to a data side.

Further, gradient sum is calculated by data side box cipher text. Wherein, the data side classifies the owned characteristics according to a classification method, and then calculates the gradient sum of samples in each box by adding homomorphism

And

and then returns to the service side.

Further, the service side decrypts the calculation. After receiving the result of the data side, the service side decrypts to obtain each characteristic and G corresponding to the box combination_L、G_R、H_L、H_RFrom which the characteristic information gain in SecureBoost is calculated. After the service side obtains the decrypted data side box gradient summation result, a summation item in the following formula is calculated:

further, a maximum gain is selected. Wherein each threshold L of each feature is compared_splitSelecting the maximum value, if the corresponding characteristic ID and the threshold value ID are data sides, sending the data sides to the data sides, and segmenting the sample space by using the characteristics returned by the data sides; and if the business side is adopted, the segmentation is directly carried out.

Further, loop until convergence or termination. The method comprises the steps of obtaining a node in a tree model, and executing the steps until training is finished.

It should be noted that, as shown in fig. 2, the trained Secure Boost model marks the party to which each node belongs, and the inference sample x is split according to the party where the current node is located, and is transmitted to the next node according to the split result to continue splitting until the result of the leaf node is obtained. Obviously, the current federal modeling process has certain technical thresholds, and some problems also exist in practical application.

Thus, the data processing method in the related art has the following technical problems:

1. complex theoretical barriers to cryptography: the federal modeling technology in the related framework heavily depends on cryptology theories such as differential privacy and homomorphic encryption to complete the protection of data privacy, and for most model practitioners, the cryptology theory has certain thresholds, so that the cryptology theory is not beneficial to understanding and optimizing in the actual modeling process, and meanwhile, the situation of federal learning innovation under the traditional machine learning model thinking is limited.

2. Extremely high data communication and training costs: as data needs to be transmitted among multiple parties in each splitting process of model training, requirements on communication bandwidth and communication time are provided, and modeling cost and time consumption are improved. In the actual modeling process, at least 30 hours are consumed for training a federal model under 10-kilo-magnitude data, which is far longer than the normal training time of plaintext data, and additional cost problems such as model tuning and later maintenance are brought.

3. Difficulties with parallel and asynchronous computations: related federal modeling processes lack the feasibility of parallelism and asynchrony, taking secure boost as an example, because each new tree of the Boosting method needs to depend on the result of the last tree, parallel computation cannot be really realized, and the construction process of a single tree depends on interaction of multi-party data, so that the difficulty of asynchronous training also exists.

Therefore, the data processing method provided by the application can be based on the theory of counterstudy modeling, so that each participant does not need to encrypt and interactively transmit information such as samples or gradients, and the purpose of federal modeling is achieved through cooperative training of model components. Further, training and communication costs of the model can be greatly simplified by transmitting only the scores generated by the parties and the residuals returned by the business parties possessing the tags. Further, parallel and asynchronous computations for machine learning may be adapted.

A data processing method, a data processing apparatus, and an electronic device according to embodiments of the present application are described below with reference to the accompanying drawings.

Fig. 3 is a schematic flow chart of a data processing method according to an embodiment of the present application.

As shown in fig. 3, the data processing method provided in the embodiment of the present application is executed by a service side, and specifically includes the following steps:

s101, model scores sent by a data party and a service party are obtained, and the model scores are generated by the data party and the service party according to the trained generation models.

In the embodiment of the application, when each iteration is started, all data parties and business parties respectively generate a score according to the trained generation models. Furthermore, the business party generates a model score according to the corresponding trained generative model, and can receive the model score sent by the data party after the data party generates the model score.

For example, model scores may be generated for all data parties

Where i represents different data parties, and i ∈ [1, n ]](ii) a Model scores and f may be generated for a business party_B. That is to say that the position of the first electrode,

the trained generation model may be any parameterized Machine Learning (ML) model, for example, a Decision Tree (Decision Tree), a neural network, or the like.

And S102, generating a label of the current user according to the model score and the trained collaborative model.

Optionally, after the business party obtains the model scores of the data party and the business party, the model scores of the data party and the business party can be input into the trained collaborative model, and the current user label is generated through collaborative model integration.

Therefore, according to the data processing method, the business party can obtain the model scores respectively sent by the data party and the business party, the model scores are generated by the data party and the business party according to the trained generation models respectively, and then the label of the current user is generated according to the model scores and the trained collaborative model so as to realize data processing.

In the present application, the label of the current user may be generated according to a trained generation model and a trained collaborative model based on a generated countermeasure Network (GAN for short).

The following is a brief description of the structure and training procedure of GAN.

As shown in fig. 4, the basic structure of GAN may include: a Generator (Generator)2-1 and a Discriminator (Discriminator) 2-2.

The generator may receive an input vector with or without real samples 2-3 for generating a false-true picture/vector or values 2-4, etc. Alternatively, if there are no real samples, then random inputs 2-5 may be used that satisfy a Distribution, such as a sample mean Distribution, a Gaussian Distribution, or the like.

It should be noted that the generator may be any model that can output values, such as a neural network, a tree model, a simple linear or non-linear mapping, and so on. In this case, the generated false samples 2-6 are input to the discriminator together with the true samples 2-3 in the training data.

The discriminator may be any model that has an input in a real sample format (picture/vector or numerical value, etc.) and an output as a picture authenticity label. As with the generator, the arbiter can be any model, such as a neural network, a tree model, a simple linear or non-linear mapping, and the like.

Therefore, the GAN has strong expansibility, and the training idea in the GAN can be used for scenes such as multiple sample generation, counterstudy and the like.

As shown in FIG. 5, 3-1 to 3-4 represent the distribution of real samples, 3-5 to 3-8 represent the distribution of discrimination probabilities of discriminators, 3-9 to 3-12 represent the distribution of generated samples, Z represents noise input, and Z to x represent the mapping of the distribution after passing through the generator.

In this case, the difference between the distribution generated by the generator and the true distribution is large in the state shown in fig. 5(a), and the probability of discriminating a sample is not stable because the discriminator has not been trained sufficiently, so that it is necessary to train the k-round discriminator first. Further, after the training is completed, the discriminator reaches the state shown in fig. 5(b), and the discrimination effect of the discriminator is improved. Further, after 1 round of training the generator after lifting, the generator can reach the state shown in fig. 5(c), and the sample distribution generated by the generator is already close.

Further, by repeatedly iterating the three states of fig. 5(a), 5(b), and 5(c), the state shown in fig. 5(d) is finally reached, that is, the generator can generate false samples that are false or spurious, the discriminator cannot distinguish the generated samples from the true samples, and at this time, the discrimination probability is 50%, and the training purpose is achieved.

In summary, the countermeasure thought of the generator and the arbiter can be applied to the modeling scene of the federal study, and is converted into the thought of generation and cooperation according to different purposes. Alternatively, a generative model and a collaborative model may be built based on generative and collaborative ideas.

It should be noted that the purpose of the generator in the GAN is to generate false samples that can be falsified, and the discriminator needs to distinguish these false samples from the true samples, so as to complete the training of the model in the countermeasure. And each party of the federal modeling is opposite to the thought of the GAN, the generated models generate model scores respectively, and the cooperative model is used for integrating the model scores of each party to achieve the optimal modeling effect, so that the model training is essentially completed in cooperation.

The following explains the training process for the generative model and the collaborative model.

As a possible implementation manner, as shown in fig. 6, the method specifically includes the following steps:

s601, obtaining sample model scores respectively sent by a data party and a service party, wherein the sample model scores are generated by the data party and the service party according to generation models to be trained respectively.

Optionally, the service party generates a model score according to the corresponding trained generative model, and after the data party generates the model score, the service party can receive the model sent by the data partyA type score. In this case, the business side obtains the sample model scores of the data side and the business side, namely

And S602, performing model training for preset times according to the sample model scores, the real labels of the sample users stored by the business party and the collaborative model to be trained to obtain the collaborative model to be optimized.

The preset number of times may be set according to actual conditions, for example, k times.

S603, generating a total label difference value of the sample user corresponding to the collaborative model to be optimized and label difference values of the sample users corresponding to the data side and the business side according to the sample model score, the real labels of the sample users and the collaborative model to be optimized, so that the data side and the business side can calculate respective losses according to the label difference values of the sample users, and optimizing the model to be trained according to the respective losses.

As a possible implementation manner, as shown in fig. 7, in the above step, a process of generating a total tag difference value of the sample user corresponding to the collaborative model to be optimized according to the sample model score, the real tag of the sample user, and the collaborative model to be optimized specifically includes the following steps:

s701, inputting sample model scores corresponding to a data party and a service party into a collaborative model to be optimized to obtain a total prediction label of a sample user.

S702, calculating a difference value between the real label of the sample user and the predicted total label of the sample user to obtain a total label difference value of the sample user.

For example, the total difference in labels for the sample user may be generated by the following formula:

where y represents the true label of the sample user, δ_ALLIndicating the total difference in labels.

As a possible implementation manner, as shown in fig. 8, in the above step, a process of generating a label difference value of the sample user corresponding to each of the data party and the service party according to the sample model score, the real label of the sample user, and the collaborative model to be optimized specifically includes the following steps:

s801, inputting the sample model scores corresponding to other data parties or business parties except the current data party or business party into the collaborative model to be optimized, and obtaining the prediction labels of the sample users corresponding to the current data party or business party.

S802, calculating a difference value between the real label of the sample user and the predicted total label of the sample user corresponding to the current data party or service party to obtain a label difference value of the sample user corresponding to the current data party or service party.

For example, the label difference value for a sample user may be generated by the following formula:

wherein the content of the first and second substances,

and delta_BAnd the label difference value represents the sample user corresponding to the data party and the service party respectively.

To sum up, the business party B may train the collaborative model for a preset number of times, for example, K times, by using all data party model scores and the model scores generated by the business party as inputs according to a method of training a machine learning classification model, and then calculate the total difference δ of the collaborators according to the following formula_ALLVariance values for backtransmission

And delta_ALL：

It should be noted that the design concept of the backhaul difference is as follows: in the iteration, after the collaborative model is trained, if the direction of optimization of each party needs to be guided by the difference, the model scores generated by each party need to be removed, the scores of other parties are input into the collaborative model for training, and the difference between the output of the collaborative model and the real label, which is the direction of optimization of each party, is obtained.

Further, after the difference value calculation of each party is completed, each party can locally calculate the loss of each generation model and use the loss for the optimization of the generation model, and after the optimization of the generation model is completed, a new round of model score is regenerated.

The Loss (Loss) of each generative model can be calculated by the following formula:

therein, Loss_BAnd

representing the loss of the data side and the business side respectively.

And S604, returning to the step of obtaining the sample model scores respectively sent by the data party and the service party to perform the next iteration until the total label difference value of the sample user is smaller than the preset total label difference value threshold value.

Optionally, the step of obtaining the sample model scores sent by the data party and the service party respectively may be returned, the sample model scores are transmitted to the service party, and a new iteration is started.

Further, with a certain number of iterations, for example, 1-3 iterations, when the difference between the two total differences is smaller than a certain epsilon value, for example, 1 e-6.

S605, respectively determining the generated model to be trained and the collaborative model to be optimized which are obtained at the last time as a trained generated model and a trained collaborative model.

It should be noted that before trying to determine the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model respectively, training may be performed until the total loss satisfies the condition.

As a possible implementation manner, as shown in fig. 9, the method specifically includes the following steps:

and S901, updating the real label of the sample user into a difference value between the real label of the sample user and a preset error label of the sample user.

And S902, returning to the step of obtaining the sample model scores respectively sent by the data party and the service party for retraining until the total loss is less than a preset total loss threshold value, wherein the total loss is calculated by the service party according to the total label difference value of the sample user.

Alternatively, the idea of Boosting can be adopted, which will

And converting into a new label, and retraining until the total loss is less than a certain value, for example, the error contrast between the model trained by the full data is less than epsilon, and finishing the training.

It should be noted that, after the training is finished, each of the data side and the business side has a generation model, and the business side has a cooperation model.

Therefore, the data processing method provided by the application can flexibly apply the relevant knowledge in the field of machine learning to the Federal learning scene, so that the cryptography technologies such as homomorphic encryption are not needed, various machine learning models can be used by each modeling participant when the models are generated in the actual application, complex ciphertext data interaction is not needed, and the actual model effect can be greatly improved.

Fig. 10 is a schematic flow chart of another data processing method according to an embodiment of the present application.

As shown in fig. 10, the data processing method provided in the embodiment of the present application is executed by a data side, and specifically includes the following steps:

s1001, generating a model score according to the generated model trained by the data side.

S1002, the model scores are sent to a business party, so that the business party can generate the label of the current user according to the model scores sent by the data party, the model scores generated by the business party according to the generated model trained by the business party and the trained collaborative model.

In the embodiment of the application, when each iteration is started, all data parties and business parties respectively generate a score according to the trained generation models. Further, after the data side generates the model score, the model score can be sent to the business side, so that the business side can generate the label of the current user according to the model score sent by the data side, the model score generated by the business side according to the generated model trained by the business side and the trained collaborative model.

Therefore, according to the data processing method, the data side can generate the model score according to the generation model trained by the data side, and then the model score is sent to the business side, so that the business side can generate the label of the current user according to the model score sent by the data side, the model score generated by the business side according to the generation model trained by the business side and the trained collaborative model, and data processing is realized.

It should be noted that, in the present application, the generative models generate model scores respectively, and the collaborative model is used to integrate the model scores of each party to achieve the optimal modeling effect.

As a possible implementation manner, as shown in fig. 11, the method specifically includes the following steps:

s1101, generating a sample model score according to a generation model to be trained by a data side.

S1102, sending the sample model scores to a business party, so that the business party performs model training for preset times according to the sample model scores sent by a data party, the sample model scores generated by the business party according to a generation model to be trained, real labels of sample users stored by the business party and a collaborative model to be trained, obtaining the collaborative model to be optimized, generating total label differences of the sample users corresponding to the collaborative model to be optimized and label differences of the sample users corresponding to the data party and the business party according to the sample model scores, the real labels of the sample users and the collaborative model to be optimized, calculating the loss of the business party according to the label differences of the sample users of the business party, and optimizing the generation model to be trained according to the loss of the business party.

It should be noted that after the data side sends the sample model scores to the service side, the service side obtains the sample model scores of the data side and the service side, that is, the sample model scores of the data side and the service side are obtained

S1103, calculating the loss of the data side according to the label difference value of the sample user of the data side, and optimizing the generated model to be trained according to the loss of the data side.

The business party B can adopt all data party model scores and the model scores generated by the business party as input according to a mode of training a machine learning classification model, train a preset number of times, such as K times of cooperative models, by combining label information owned by the business party, and then respectively calculate the total difference value delta of the cooperative device according to the following formula_ALLVariance values for backtransmission

And delta_ALL：

therein, Loss_BAnd

representing the loss of the data side and the business side respectively.

And S1104, returning to the step of generating the sample model score according to the generation model to be trained of the data side, and performing the next iteration until the total label difference value of the sample user is smaller than the preset total label difference value threshold value.

And the business party inputs the sample model scores corresponding to the data party and the business party into the collaborative model to be optimized to obtain a predicted total label of the sample user, and calculates the difference between the real label of the sample user and the predicted total label of the sample user to obtain the total label difference of the sample user.

The service party inputs the sample model scores corresponding to other data parties or service parties except the current data party or service party into the collaborative model to be optimized, the prediction label of the sample user corresponding to the current data party or service party is obtained, and the difference between the real label of the sample user and the total prediction label of the sample user corresponding to the current data party or service party is calculated to obtain the difference.

S1105, respectively determining the generated model to be trained and the collaborative model to be optimized which are obtained at the last time as the trained generated model and the trained collaborative model.

Optionally, after the service side updates the real label of the sample user to the difference value between the real label of the sample user and the preset error label of the sample user, the step of generating the sample model score according to the generation model to be trained of the data side is returned to perform retraining until the total loss is smaller than the preset total loss threshold value, and the total loss is calculated by the service side according to the total label difference value of the sample user.

It should be noted that the data processing method provided by the present application can be applied to various application scenarios.

For the processing scenario of the feature fusion data, if the magnitude of the test data is 27640, the feature dimension is 47 dimensions, and the feature display is shown in fig. 12.

Optionally, the data may be roughly equally divided into three parts according to the characteristic dimension to simulate a three-part collaborative modeling scenario. Taking a neural network model as a generation model and a cooperation model for generating each side score and performing cooperation training at the same time as an example, in this case, the generation model and the cooperation model both adopt networks with structures [ X,3,4,5, Y ] to generate cooperation scores and perform cooperation, a hidden layer activation function of the generation model selects a Linear rectification function (called ReLU for short), an output function selects a Sigmoid function (also called S-type growth curve), and the hidden layer of the cooperation model and the generation model are set to be the same.

Thus, the loss curves of 3 generative models shown in fig. 13(a) - (d) can be obtained, and it can be seen that the general trends of the 3 curves are the same, but oscillation synergy exists for a period of time in each lifting process; in addition, a loss curve of the cooperative model shown in fig. 14 can be obtained, and it can be seen that the total loss is steadily decreasing, and there is a short oscillation in each lifting process.

In this case, it is obvious that the data processing method proposed in the present application has at least the following beneficial effects:

1. antagonism generation translates into integrated synergy. The method and the device use the optimization theory to disassemble and recombine the antagonism generation process in the GAN, and input the result of the generation process into the cooperative process, so that the final optimization targets of the antagonism generation process and the cooperative process are the same, and the effect of cooperative training is achieved.

2. Nonlinear feature integration of the synergists: the application adopts the idea of nonlinear transformation of the collaborative model, including but not limited to linear integration of generating model scores by adopting a neural network, the idea of training an integrated model by adopting a label, and the like. In addition, a space Embedding (Embedding) method is used, the respective characteristics of the generated models are reduced, a collaborative model is adopted for secondary Embedding and supervised training, and the effect of multi-party collaborative supervised modeling under privacy protection is achieved.

3. Generator optimization form: the application provides an optimization form of a generative model, which comprises interpolation of how to adopt a collaborative model to obtain the generative model for optimization, an optimization method in an iterative process and the like. In addition, the direction in the non-convex optimization is utilized, the mode that the collaborative model returns the residual error to the generated model is innovated to guide the iterative process of the generated model, so that the optimization direction of each generated model is the direction of loss reduction of the collaborative model, and the interpretable efficient optimization purpose is achieved.

4. The cooperative lifting method comprises the following steps: the application describes a lifting method used in a collaborative modeling scene, including how to control an iterative process to start the next round of collaborative training based on residual errors and the like. In addition, a Boosting method for effectively landing in the integrated model is applied, information of generating model samples is mined in a mode of iteration and residual error fitting, and model effect improvement under privacy protection is achieved.

5. The collaborative model reasoning method comprises the following steps: the application describes the reasoning process of the collaborative model, including how to adopt the generative model and the collaborative model to finish reasoning, etc. In addition, the method is applied to inference integration after modeling is completed, and the scores obtained by local inference of each generation model can be directly output after the cooperation models are integrated, so that the effect of efficient cooperative inference of all parties is achieved.

In summary, as shown in fig. 15 to 16, the data processing method provided by the present application is directed to the data side a₁～A_nAnd the business party B with the label y can realize the training of the federal learning coordination gradient promotion tree through the processes of generating a model score by a data party/business party, integrating by the business party, updating an integration model, returning gradients of all parties and the like.

Based on the same application concept, the embodiment of the application also provides a device corresponding to the data processing method.

Fig. 17 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.

As shown in fig. 17, the data processing apparatus 1000 includes: a first acquisition module 110 and a generation module 120. Wherein the content of the first and second substances,

a first obtaining module 110, configured to obtain model scores sent by a data party and a service party respectively, where the model scores are generated by the data party and the service party according to respective trained generation models;

a generating module 120 configured to generate a label of the current user according to the model score and the trained collaborative model.

As shown in fig. 18, the data processing apparatus 1000 further includes:

a second obtaining module 130, configured to obtain sample model scores sent by the data party and the business party, respectively, where the sample model scores are generated by the data party and the business party according to respective generation models to be trained;

the training module 140 is configured to perform model training for a preset number of times according to the sample model scores, the real labels of the sample users stored by the business party, and the collaborative model to be trained, so as to obtain a collaborative model to be optimized;

an optimization module 150, configured to generate, according to the sample model score, the real label of the sample user, and the collaborative model to be optimized, a total label difference value of the sample user corresponding to the collaborative model to be optimized, and a label difference value of the sample user corresponding to each of the data party and the business party, so that the data party and the business party calculate respective losses according to the label difference value of the sample user, and optimize the generated model to be trained according to the respective losses;

the iteration module 160 is configured to return the step of obtaining the sample model scores respectively sent by the data party and the service party to perform the next iteration until the total difference value of the labels of the sample users is smaller than a preset total difference value threshold value of the labels;

a determining module 170 configured to determine the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model, respectively.

According to an embodiment of the application, the determining module 170 is further configured to:

updating the real label of the sample user to be a difference value between the real label of the sample user and a preset error label of the sample user;

and returning to the step of obtaining the sample model scores respectively sent by the data party and the service party for retraining until the total loss is less than a preset total loss threshold value, wherein the total loss is calculated by the service party according to the total label difference value of the sample user.

According to an embodiment of the application, the optimization module 150 is further configured to:

inputting the sample model scores corresponding to the data party and the service party into the collaborative model to be optimized to obtain a total predicted label of the sample user;

and calculating the difference between the real label of the sample user and the predicted total label of the sample user to obtain the total label difference of the sample user.

inputting the sample model scores corresponding to the other data parties or the business parties except the current data party or the business party into the collaborative model to be optimized to obtain the prediction labels of the sample users corresponding to the current data party or the business party;

and calculating the difference value between the real label of the sample user and the predicted total label of the sample user corresponding to the current data party or the current service party to obtain the label difference value of the sample user corresponding to the current data party or the current service party.

Therefore, according to the data processing device, a business party can obtain model scores sent by a data party and a business party respectively, the model scores are generated by the data party and the business party according to respective trained generation models, and then labels of current users are generated according to the model scores and the trained collaborative models so as to realize data processing, so that in the data processing process, the data party and the business party only need to transmit the model scores and do not need to encrypt the scores so as to realize privacy protection, the data security is ensured, the data information leakage risk is reduced, meanwhile, the technical problems caused by complex cryptology theoretical barriers, extremely high data communication and training costs and the difficulty of parallel and asynchronous calculation are avoided, and the efficiency in the data processing process is improved.

Fig. 19 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.

As shown in fig. 19, the data processing apparatus 2000 includes: a first generation module 210 and a second generation module 220. Wherein the content of the first and second substances,

a first generation module 210 configured to generate a model score from a trained generative model of a data party;

the second generating module 220 is configured to send the model score to a service party, so that the service party generates a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a trained generation model and the trained collaborative model.

As shown in fig. 20, the data processing apparatus 2000 further includes:

a third generating module 230 configured to generate a sample model score from a generative model to be trained by the data party;

a first optimization module 240 configured to send the sample model scores to the business parties, the business party performs model training for preset times according to the sample model scores sent by the data party, the sample model scores generated by the business party according to the generation model to be trained, the real labels of the sample users stored by the business party and the collaborative model to be trained to obtain the collaborative model to be optimized, generating a total label difference value of the sample user corresponding to the collaborative model to be optimized according to the sample model score, the real label of the sample user and the collaborative model to be optimized, and the label difference values of the sample users corresponding to the data party and the service party respectively, calculating the loss of the business party according to the label difference value of the sample user of the business party, and optimizing the generation model to be trained according to the loss of the business party;

a second optimization module 250, configured to calculate a loss of the data party according to the label difference of the sample user of the data party, and optimize the generation model to be trained according to the loss of the data party;

the iteration module 260 is configured to return the step of generating the sample model score according to the generation model to be trained by the data party to perform the next iteration until the total label difference of the sample user is smaller than a preset total label difference threshold;

a determining module 270 configured to determine the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model, respectively.

According to an embodiment of the application, the first optimization module 240 is further configured to:

and after the business party updates the real label of the sample user to be the difference value between the real label of the sample user and the preset error label of the sample user, returning to the step of generating the sample model score according to the generation model to be trained of the data party for retraining until the total loss is less than a preset total loss threshold value, wherein the total loss is calculated by the business party according to the total label difference value of the sample user.

According to an embodiment of the application, the total label difference of the sample user is obtained by inputting the sample model scores corresponding to the data party and the service party to the collaborative model to be optimized by the service party, obtaining a predicted total label of the sample user, and calculating a difference between a real label of the sample user and the predicted total label of the sample user.

According to an embodiment of the present application, the business party inputs the sample model scores corresponding to the data party or the business party other than the data party or the business party to the collaborative model to be optimized, so as to obtain the prediction label of the sample user corresponding to the data party or the business party, and calculates the difference between the real label of the sample user and the total prediction label of the sample user corresponding to the data party or the business party.

Therefore, according to the data processing device, the data side can generate the model score according to the generation model trained by the data side, and then the model score is sent to the business side, so that the business side can generate the label of the current user according to the model score sent by the data side, the model score generated by the business side according to the generation model trained by the business side and the trained collaborative model, and data processing is realized.

Based on the same application concept, the embodiment of the application also provides the electronic equipment.

Fig. 21 is a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in fig. 21, the electronic device 3000 includes a memory 310, a processor 320, and a computer program stored in the memory 310 and operable on the processor 320, and when the processor executes the computer program, the data processing method is implemented.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

While the preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all alterations and modifications as fall within the scope of the application.

It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims

1. A data processing method, comprising:

obtaining model scores respectively sent by a data party and a service party, wherein the model scores are generated by the data party and the service party according to respective trained generation models;

and generating a label of the current user according to the model score and the trained collaborative model.

2. The method of claim 1, further comprising:

acquiring sample model scores respectively sent by the data party and the service party, wherein the sample model scores are generated by the data party and the service party according to respective generation models to be trained;

performing model training for preset times according to the sample model scores, the real labels of the sample users stored by the business party and the collaborative model to be trained to obtain a collaborative model to be optimized;

generating a total label difference value of the sample user corresponding to the collaborative model to be optimized and label difference values of the sample users corresponding to the data side and the business side according to the sample model score, the real labels of the sample users and the collaborative model to be optimized, so that the data side and the business side calculate respective losses according to the label difference values of the sample users, and optimizing the generated model to be trained according to the respective losses;

returning to the step of obtaining the sample model scores respectively sent by the data party and the service party to perform the next iteration until the total label difference value of the sample user is smaller than a preset total label difference value threshold value;

and respectively determining the generated model to be trained and the collaborative model to be optimized which are obtained at the last time as the trained generated model and the trained collaborative model.

3. The method according to claim 2, wherein before determining the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model, respectively, further comprising:

4. The method according to claim 2, wherein the generating of the total label difference value of the sample user corresponding to the collaborative model to be optimized according to the sample model score, the real label of the sample user, and the collaborative model to be optimized comprises:

5. The method of claim 2, wherein generating the label difference values of the sample users corresponding to the data party and the business party respectively according to the sample model scores, the real labels of the sample users, and the collaborative model to be optimized comprises:

6. A data processing method, comprising:

generating a model score according to a generated model trained by a data side;

and sending the model score to a service party so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a trained generation model and the trained collaborative model.

7. The method of claim 6, further comprising:

generating a sample model score according to the generation model to be trained of the data party;

sending the sample model scores to the service party so that the service party performs model training for preset times according to the sample model scores sent by the data party, the sample model scores generated by the service party according to a generated model to be trained, real labels of sample users stored by the service party and the collaborative model to be trained to obtain the collaborative model to be optimized, generating a total label difference value of the sample user corresponding to the collaborative model to be optimized according to the sample model score, the real label of the sample user and the collaborative model to be optimized, and the label difference values of the sample users corresponding to the data party and the service party respectively, calculating the loss of the business party according to the label difference value of the sample user of the business party, and optimizing the generation model to be trained according to the loss of the business party;

calculating the loss of the data party according to the label difference value of the sample user of the data party, and optimizing the generation model to be trained according to the loss of the data party;

returning to the step of generating the sample model score according to the generation model to be trained of the data side to perform the next iteration until the total label difference value of the sample user is smaller than a preset total label difference value threshold value;

8. The method according to claim 7, wherein before determining the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model, respectively, further comprising:

9. The method according to claim 7, wherein the total label difference of the sample user is obtained by the service party inputting the sample model scores corresponding to the data party and the service party to the collaborative model to be optimized to obtain a predicted total label of the sample user, and calculating a difference between the real label of the sample user and the predicted total label of the sample user.

10. The method according to claim 7, wherein the difference between the labels of the sample users corresponding to the data party and the service party is obtained by the service party inputting the sample model scores corresponding to the data party or the service party other than the data party or the service party to the collaborative model to be optimized, obtaining the predicted label of the sample user corresponding to the current data party or the service party, and calculating the difference between the real label of the sample user and the predicted total label of the sample user corresponding to the current data party or the service party.

11. A data processing apparatus, comprising:

the system comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is configured to acquire model scores respectively sent by a data party and a service party, and the model scores are generated by the data party and the service party according to respective trained generation models;

and the generation module is configured to generate the label of the current user according to the model score and the trained collaborative model.

12. The apparatus of claim 11, further comprising:

the second acquisition module is configured to acquire sample model scores respectively sent by the data party and the business party, wherein the sample model scores are generated by the data party and the business party according to respective generation models to be trained;

the training module is configured to perform model training for preset times according to the sample model scores, the real labels of the sample users stored by the business party and the collaborative model to be trained to obtain a collaborative model to be optimized;

the optimization module is configured to generate a total label difference value of the sample user corresponding to the collaborative model to be optimized and a label difference value of the sample user corresponding to each of the data party and the business party according to the sample model score, the real label of the sample user and the collaborative model to be optimized, so that the data party and the business party calculate respective losses according to the label difference value of the sample user and optimize the generated model to be trained according to the respective losses;

the iteration module is configured to return the step of obtaining the sample model scores respectively sent by the data party and the service party to perform the next iteration until the total label difference value of the sample user is smaller than a preset total label difference value threshold value;

a determining module configured to determine the generated model to be trained and the collaborative model to be optimized, which are obtained last time, as the trained generated model and the trained collaborative model, respectively.

13. The apparatus of claim 12, wherein the determination module is further configured to:

14. The apparatus of claim 12, wherein the optimization module is further configured to:

15. The apparatus of claim 12, wherein the optimization module is further configured to:

16. A data processing apparatus, comprising:

a first generation module configured to generate a model score from a trained generative model of a data party;

and the second generation module is configured to send the model score to a service party, so that the service party can generate a label of the current user according to the model score sent by the data party, the model score generated by the service party according to a trained generation model and the trained collaborative model.

17. The apparatus of claim 16, further comprising:

a third generation module configured to generate a sample model score from a generative model to be trained by the data party;

a first optimization module configured to send the sample model score to the service party, so that the service party performs model training for a preset number of times according to the sample model score sent by the data party, the sample model score generated by the service party according to a generated model to be trained, a real label of a sample user stored by the service party and a collaborative model to be trained, and obtains the collaborative model to be optimized, generating a total label difference value of the sample user corresponding to the collaborative model to be optimized according to the sample model score, the real label of the sample user and the collaborative model to be optimized, and the label difference values of the sample users corresponding to the data party and the service party respectively, calculating the loss of the business party according to the label difference value of the sample user of the business party, and optimizing the generation model to be trained according to the loss of the business party;

the second optimization module is configured to calculate the loss of the data party according to the label difference value of the sample user of the data party and optimize the generation model to be trained according to the loss of the data party;

the iteration module is configured to return the step of generating the sample model score according to the generation model to be trained of the data party to carry out the next iteration until the total label difference value of the sample user is smaller than a preset total label difference value threshold value;

18. The apparatus of claim 17, wherein the first optimization module is further configured to:

19. The apparatus of claim 17, wherein the total label difference of the sample user is obtained by the service party inputting the sample model scores corresponding to the data party and the service party to the collaborative model to be optimized, obtaining a predicted total label of the sample user, and calculating a difference between the real label of the sample user and the predicted total label of the sample user.

20. The apparatus of claim 17, wherein the difference between the labels of the sample users corresponding to the data party and the service party is obtained by the service party inputting the sample model scores corresponding to the data party or the service party other than the data party or the service party to the collaborative model to be optimized, obtaining the predicted label of the sample user corresponding to the current data party or the service party, and calculating a difference between the real label of the sample user and the predicted total label of the sample user corresponding to the current data party or the service party.

21. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the data processing method of any one of claims 1-5 or 6-8.

22. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the data processing method of any one of claims 1-5 or claims 6-8.