CN112418442A

CN112418442A - Data processing method, device, equipment and storage medium for federal transfer learning

Info

Publication number: CN112418442A
Application number: CN202011393120.4A
Authority: CN
Inventors: 康焱; 刘洋
Original assignee: WeBank Co Ltd
Current assignee: WeBank Co Ltd
Priority date: 2020-12-02
Filing date: 2020-12-02
Publication date: 2021-02-26

Abstract

The invention discloses a data processing method, a device, equipment and a storage medium for federated migration learning, wherein the method comprises the following steps: determining each first feature extraction model by the first participant based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene and a plurality of feature extraction models in the first participant; and determining a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to the first business scene in the second participant, the to-be-trained prediction model and the first feature extraction model. According to the method, the target model which gives consideration to model migration and model interpretability can be obtained through federal migration learning, the knowledge of the sample is migrated through the target feature extraction model, the sample data federal construction model based on different participants is achieved, the time consumption of the trained model for achieving the target performance is reduced, and the utilization rate of computer computing resources is improved.

Description

Data processing method, device, equipment and storage medium for federal transfer learning

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a method, a device, equipment and a storage medium for processing data of federated migration learning.

Background

Machine learning performs supervised training on a large amount of labeled data to achieve good performance and effect, however, a large labeled data set is limited in quantity and application field, and manually labeling a sufficient amount of training data is often costly. Aiming at the problem, a transfer learning method is generally adopted to solve the problem, namely, a discriminator is trained to adjust parameters of a transfer learning network, so that the distribution deviation between data in a source field and data in a target field is reduced under the transfer learning network after the parameters are adjusted, and the transfer learning network has a better effect when the target field is applied to finish a target task.

However, the lack of interpretability of deep learning models makes them difficult to use for migratory learning in applications that require model interpretability (e.g., financial risk control), and low complexity deep learning models have a weak ability to learn migratable knowledge from raw data and thus are not robust in migratory ability. This creates a contradiction, the deep learning model with strong migration capability lacks interpretability, and the deep learning model with strong interpretability has weak migration capability, so that the deep learning model cannot take account of both interpretability and migration capability. Meanwhile, different user data are currently stored in servers of different data holders or participants, and due to privacy protection of the user data, interaction of the user data cannot be directly performed between different servers, that is, the servers cannot share respective user data to perform joint modeling, so that each server can perform modeling only based on a small amount of user data, and therefore, training is required for a longer time to enable the trained model to reach a target performance, and further, a computer needs to consume a large amount of resources and computing power, and the utilization rate of computer computing resources is low.

The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.

Disclosure of Invention

The invention mainly aims to provide a data processing method, a device, equipment and a storage medium for federated migration learning, and aims to solve the technical problems that a deep learning model cannot give consideration to interpretability and migration capability, and the utilization rate of computer computing resources is low because time sequence data cannot be directly interacted between different participants.

In order to achieve the above object, the present invention provides a data processing method for federated migration learning, which includes the following steps:

determining each first feature extraction model by the first participant based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene and a plurality of feature extraction models in the first participant;

and determining a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to the first business scene in the second participant, the to-be-trained prediction model and the first feature extraction model.

Further, the step of determining, by the first participant, each first feature extraction model based on the first sample corresponding to the first business scenario, the second sample corresponding to the second business scenario, and the plurality of feature extraction models includes:

grouping the characteristics of the first samples by the first participant based on preset service requirements to obtain a preset number of first sample characteristic groups, and grouping the characteristics of the second samples based on the preset service requirements to obtain a preset number of second sample characteristic groups;

acquiring first feature representations corresponding to the first sample feature groups and second feature representations corresponding to the second sample feature groups through a preset number of feature extraction models by a first participant;

determining a plurality of first domain distinguishing loss values by a first participant based on each first feature characterization and a corresponding domain distinguishing model thereof, and determining a second feature extraction model based on each first domain distinguishing loss value and a corresponding feature extraction model;

and determining a plurality of second domain distinguishing loss values by the first participant based on the second feature characterizations and the corresponding domain distinguishing models thereof, and determining first feature extraction models based on the second domain distinguishing loss values and the corresponding second feature extraction models.

Further, the step of obtaining, by the first participant, the first feature characterization corresponding to each first sample feature group through a preset number of feature extraction models, and the step of obtaining the second feature characterization corresponding to each second sample feature group includes:

and inputting each first sample feature group into a corresponding preset number of feature extraction models through the first participant to obtain first feature representations, and inputting each second sample feature group into a corresponding preset number of feature extraction models to obtain second feature representations.

Further, the step of determining, by the first participant, a plurality of first domain discrimination loss values based on the respective first feature characterizations and the corresponding domain discrimination models thereof, and determining the second feature extraction model based on the respective first domain discrimination loss values and the corresponding feature extraction models comprises:

inputting each first feature characterization into a corresponding domain distinguishing model through a first participant to obtain a first domain distinguishing loss value corresponding to each first feature characterization;

and determining each updated domain distinguishing model based on each first domain distinguishing loss value through the first participant, and updating the corresponding feature extraction model through domain confrontation learning based on each first domain distinguishing loss value so as to obtain each second feature extraction model.

Further, the step of determining, by the first participant, a plurality of second domain discrimination loss values based on the respective second feature characterizations and the corresponding domain discrimination models thereof, and the step of determining the respective first feature extraction models based on the plurality of second domain discrimination loss values and the corresponding second feature extraction models comprises:

inputting each second feature representation into the corresponding updated domain distinguishing model through the first participant to obtain a second domain distinguishing loss value corresponding to each second feature representation;

and determining each target domain distinguishing model based on each second domain distinguishing loss value through the first participant, updating the corresponding second feature extraction model through domain confrontation learning based on each second domain distinguishing loss value, and obtaining each first feature extraction model.

Further, the step of determining a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to a first business scenario in a second participant, a prediction model to be trained, and the first feature extraction model includes:

obtaining the third sample in the second party through the second party, and obtaining a first characteristic representation corresponding to the first sample through the first party;

and calculating a prediction loss value of the prediction model to be trained through a longitudinal federated learning algorithm based on the first feature characterization and the third sample, and updating the prediction model to be trained and each first feature extraction model based on the prediction loss value respectively to obtain a target prediction model and each target feature extraction model.

Further, the step of calculating a prediction loss value of the prediction model to be trained through a longitudinal federal learning algorithm based on the first feature characterization and the third sample, and updating the prediction model to be trained and each first feature extraction model based on the prediction loss value respectively to obtain the target prediction model and each target feature extraction model includes:

performing model training on the to-be-trained prediction model through a longitudinal federated learning algorithm based on the first feature characterization and a first feature characterization set corresponding to the third sample to obtain a prediction loss value;

updating the prediction model to be trained through a longitudinal federal learning algorithm based on the prediction loss value to obtain a target prediction model;

and updating each first feature extraction model through a longitudinal federal learning algorithm based on the predicted loss value so as to obtain each target feature extraction model.

Further, the target prediction model is a fraud score prediction model; the first service scene comprises a credit scoring scene of a user, the second service scene comprises a fraud scoring scene of the user, the first sample comprises credit scoring data in a first participant, the second sample is fraud scoring data in the first participant, and the third sample is credit scoring data in a second participant;

the step of determining a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to a first business scenario in a second participant, a prediction model to be trained, and a first feature extraction model includes:

and determining a target prediction model and each target feature extraction model according to the credit score data of the first participant, the credit score data of the second participant, the prediction model to be trained and the first feature extraction model.

Further, after the step of determining the target prediction model and each target feature extraction model based on the first sample, the third sample corresponding to the first business scenario in the second participant, the prediction model to be trained, and the first feature extraction model, the method further includes:

the method comprises the steps of obtaining to-be-predicted feature characterizations corresponding to first to-be-predicted samples in a first participant, wherein the first participant groups features of the first to-be-predicted samples based on preset business requirements to obtain a preset number of to-be-predicted sample feature groups, and obtains to-be-predicted feature characterizations corresponding to the to-be-predicted sample feature groups based on a preset number of target feature extraction models, and the first to-be-predicted samples are second business scene samples;

acquiring a second sample to be predicted in a third participant, wherein the second sample to be predicted is a second business scene sample;

and calculating the prediction result of the target prediction model through a longitudinal federal learning algorithm based on the feature characterization to be predicted and the feature characterization set corresponding to the second sample to be predicted.

In addition, in order to achieve the above object, the present invention further provides a data processing apparatus for federal transfer learning, including:

the first model determining module is used for determining each first feature extraction model through a first participant based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene and a plurality of feature extraction models in the first participant;

and the second model determining module is used for determining a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to the first business scene in the second participant, the to-be-trained prediction model and the first feature extraction model.

In addition, in order to achieve the above object, the present invention further provides a data processing apparatus for federal transfer learning, including: the system comprises a memory, a processor and a federated transfer learning data processing program which is stored on the memory and can run on the processor, wherein the federated transfer learning data processing program realizes the steps of the federated transfer learning data processing method when being executed by the processor.

In addition, in order to achieve the above object, the present invention further provides a computer-readable storage medium, on which a data processing program for federal migration learning is stored, and when being executed by a processor, the data processing program for federal migration learning implements the aforementioned steps of the data processing method for federal migration learning.

In addition, to achieve the above object, the present invention further provides a computer program product, which when executed, implements the steps of the data processing method for federal transfer learning.

The method comprises the steps that a first participant determines each first feature extraction model based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene and a plurality of feature extraction models in the first participant; then, based on the first sample, a third sample corresponding to a first business scene in a second participant, a prediction model to be trained and a first feature extraction model, determining a target prediction model and each target feature extraction model, jointly establishing the target prediction model with excellent performance by using rich features owned by the first participant, class labels owned by the second participant and class labels owned by the third participant through federal learning, obtaining the target model with consideration of model migration and model interpretability through federal migration learning, migrating the knowledge of the sample through the target feature extraction model, predicting through the interpretable target prediction model, and explaining the contribution of each feature group to a prediction result, thereby achieving the purpose of considering both model migration and model interpretability, because the model is constructed based on federal sample data of different participants, the accuracy of carrying out fraud scoring and other predictions on the sample to be predicted is improved. Meanwhile, a model is constructed based on sample data federation of different participants, the defect that a large amount of computer computing resources are consumed in the prior art is overcome, the time consumption for the trained model to reach the target performance is reduced, and the utilization rate of the computer computing resources is improved.

Drawings

FIG. 1 is a schematic structural diagram of a data processing device for federated migration learning in a hardware operating environment according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a first embodiment of a federated transfer learning data processing method according to the present invention;

FIG. 3 is a sample diagram of a first service scenario in the data processing method for federated transfer learning according to the present invention;

FIG. 4 is a sample diagram of a second service scenario in the federated transfer learning data processing method of the present invention;

FIG. 5 is a schematic diagram of a training process of Federal transfer learning in the data processing method of Federal transfer learning according to the present invention;

FIG. 6 is a schematic diagram of a prediction process in the data processing method of Federal transfer learning according to the present invention;

fig. 7 is a functional module diagram of an embodiment of data processing for federated migration learning according to the present invention.

The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

As shown in fig. 1, fig. 1 is a schematic structural diagram of a data processing device for federal migration learning in a hardware operating environment according to an embodiment of the present invention.

The data processing device for federal migration learning in the embodiment of the present invention may be a PC, or may be a mobile terminal device having a display function, such as a smart phone, a tablet PC, an electronic book reader, an MP3(Moving Picture Experts Group Audio Layer III, motion video Experts compression standard Audio Layer 3) player, an MP4(Moving Picture Experts Group Audio Layer IV, motion video Experts compression standard Audio Layer 4) player, a portable computer, and the like.

As shown in fig. 1, the data processing device for federal migration learning may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (e.g., a magnetic disk memory). The memory 1005 may alternatively be a storage device separate from the processor 1001.

Optionally, the data processing device for federal migration learning may further include a camera, a Radio Frequency (RF) circuit, a sensor, an audio circuit, a WiFi module, and the like. Such as light sensors, motion sensors, and other sensors. Of course, the data processing device for federal migration learning may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and so on, which are not described herein again.

Those skilled in the art will appreciate that the terminal structure shown in fig. 1 does not constitute a limitation of the data processing apparatus for federal migration learning, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.

As shown in fig. 1, a memory 1005, which is a kind of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a data processing program of federal migration learning.

In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a backend server and performing data communication with the backend server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and processor 1001 may be used to invoke the federal migration learned data handler stored in memory 1005.

In this embodiment, the data processing device for federal migration learning includes: the system comprises a memory 1005, a processor 1001 and a data processing program of federal migration learning, wherein the data processing program of federal migration learning is stored on the memory 1005 and can run on the processor 1001, and when the processor 1001 calls the data processing program of federal migration learning stored in the memory 1005, the steps of the data processing method of federal migration learning in the following embodiments are executed.

The invention also provides a data processing method for federated migration learning, and referring to fig. 2, fig. 2 is a schematic flow diagram of a first embodiment of the data processing method for federated migration learning according to the present invention.

Step S101, determining each first feature extraction model by a first participant based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene and a plurality of feature extraction models in the first participant;

in this embodiment, the first service scenario and the second service scenario are two service scenarios with similar but different services, and the sample corresponding to the first service scenario and the sample corresponding to the second service scenario have the same feature space. For example, the first service scenario is credit score and fraud score, and the second service scenario is fraud score.

In the data processing method of federal migration learning according to this embodiment, at least three participants are included, such as a participant a (first participant), a participant B (second participant), and a participant C (third participant). Participant a and participant B have a large number of overlapping samples on the first business scenario s. Participant a and participant C have a certain number of overlapping samples in the second service scenario t.

The first sample is a sample corresponding to a first service scene in a first participant, that is, data corresponding to the first sample is data of the first service scene; the second sample is a sample corresponding to a second service scenario in the first participant, that is, data corresponding to the second sample is data of the second service scenario; the third sample is a sample corresponding to the first service scenario in the second participant, that is, data corresponding to the third sample is data of the first service scenario; the fourth sample is a sample corresponding to the second service scenario in the third participant, that is, the data corresponding to the fourth sample is the data of the second service scenario. The first sample and the third sample are respectively the overlapping samples of the participant a and the participant B in the first service scenario, and the first sample and the fourth sample are respectively the overlapping samples of the participant a and the participant C in the second service scenario, wherein the number of samples in the first sample (far) is larger than the number of samples in the second sample.

The data corresponding to the first sample, the data corresponding to the second sample, the data corresponding to the third sample and the data corresponding to the fourth sample can be table data, and each column in the table data represents a feature (such as age, sex, work, salary and the like); each row represents a sample (e.g., a user of a financial product).

The participant a provides different domain labels (or service labels) for the samples of different service scenarios, that is, each sample in the first sample has a different domain label from each sample in the second sample, each sample in the first sample has the same domain label, each sample in the second sample has the same domain label, for example, the domain label of each sample in the first sample is 1, and the domain label of each sample in the second sample is 0. Each overlapping sample of the participant B on the first service scenario s has a class label, that is, each sample in the third sample has a class label; a small number of overlapping samples of participant C on the traffic scene t have no category label.

In this embodiment, when performing federated migration learning, each first feature extraction model is determined by a first participant based on a first sample corresponding to a first service scenario, a second sample corresponding to a second service scenario, and a plurality of feature extraction models in the first participant. Specifically, a plurality of first feature characterizations corresponding to the first samples and a plurality of second feature characterizations corresponding to the second samples are determined through the feature extraction models, and since the samples in the first samples and the samples in the second samples both have domain labels, the first feature characterizations are input into the corresponding domain differentiation models respectively for model training to obtain updated domain differentiation models and a plurality of first domain differentiation loss values, and the corresponding feature extraction models are updated according to the first domain differentiation loss values to obtain the second feature extraction models, wherein each first feature characterization input corresponds to one domain differentiation model, and each first feature characterization input corresponds to one feature extraction model.

And then, inputting the second feature characterization into the corresponding updated domain distinguishing model for model training, determining a target domain distinguishing model and a plurality of second domain distinguishing loss values, and updating the corresponding second feature extraction model according to the second domain distinguishing loss values to obtain a first feature extraction model.

Step S102, a target prediction model and each target feature extraction model are determined based on the first sample, a third sample corresponding to a first business scene in a second participant, a prediction model to be trained and the first feature extraction model.

In this embodiment, after determining each first feature extraction model, a third sample corresponding to the first sample in the second participant is obtained, the third sample is an overlapped sample corresponding to the first sample in the samples of the second participant, the third sample belongs to the first service scene, and determining a target prediction model and each target feature extraction model according to the first sample, the third sample, the prediction model to be trained and the first feature extraction model, specifically, determining the first characteristic representation and the prediction loss value corresponding to the third sample through the prediction model to be trained, namely, the model training is carried out on the first characteristic representation and the third sample corresponding to the first sample through the prediction model to be trained to obtain a prediction loss value, and updating the prediction model to be trained and each first feature extraction model according to the prediction loss value so as to obtain a target prediction model and each target feature extraction model.

The prediction model to be trained is an interpretable machine learning model, for example, the prediction model to be trained is a linear regression model or a logistic regression model.

Referring to fig. 3, in fig. 3, a participant a is a first participant, a participant B is a second participant, and the participants a and B have a large number of overlapping samples on a first service scenario s. F_i ^A,sA first sample feature group corresponding to the first sample, f_j ^B,sThe corresponding monomer characteristic of the third sample.

Referring to fig. 4, in fig. 4, a participant a is a first participant, a participant C is a third participant, and the participants a and C have a large number of overlapping samples on a second service scenario t. F_i ^A,tAnd a second sample feature set corresponding to the second sample.

Preferably, the target prediction model is a fraud score prediction model; the first service scenario comprises a credit scoring scenario of the user, the second service scenario comprises a fraud scoring scenario of the user, the first sample comprises credit scoring data in a first participant, the second sample comprises fraud scoring data in the first participant, and the third sample comprises credit scoring data in a second participant.

Step S102 includes:

The credit scoring data comprises the age, sex, work, salary, school calendar, deposit information, fixed asset information, loan information, advance payment record, overdue payment record and the like of the user; fraud scoring data includes the user's age, gender, job, salary, school calendar, deposit information, fixed property information, loan information, advance payment record, overdue payment record, advance payment information, advance loan information, and the like.

Specifically, the first participant determines each first feature extraction model according to credit score data in the first participant, fraud score data in the first participant and a plurality of feature extraction models; and then inputting the prediction model to be trained according to the credit score data in the first participant and the credit score data in the second participant for federal training, and updating the prediction model to be trained and the first feature extraction model according to the training result to obtain a target prediction model and each target feature extraction model.

In the data processing method for federal transfer learning provided in this embodiment, each first feature extraction model is determined by a first participant based on a first sample corresponding to a first service scene, a second sample corresponding to a second service scene, and a plurality of feature extraction models in the first participant; then, based on the first sample, a third sample corresponding to a first business scene in a second participant, a prediction model to be trained and a first feature extraction model, determining a target prediction model and each target feature extraction model, jointly establishing the target prediction model with excellent performance by using rich features owned by the first participant, class labels owned by the second participant and class labels owned by the third participant through federal learning, obtaining the target model with consideration of model migration and model interpretability through federal migration learning, migrating the knowledge of the sample through the target feature extraction model, predicting through the interpretable target prediction model, and explaining the contribution of each feature group to a prediction result, thereby achieving the purpose of considering both model migration and model interpretability, because the model is constructed based on federal sample data of different participants, the accuracy of carrying out fraud scoring and other predictions on the sample to be predicted is improved. Meanwhile, a model is constructed based on sample data federation of different participants, the defect that a large amount of computer computing resources are consumed in the prior art is overcome, the time consumption for the trained model to reach the target performance is reduced, and the utilization rate of the computer computing resources is improved.

Based on the first embodiment, a second embodiment of the data processing method for federal migration learning according to the present invention is provided, in this embodiment, step S101 includes:

step S201, the first participant groups the characteristics of the first sample based on the preset service requirement to obtain a first sample characteristic group, and groups the characteristics of the second sample based on the preset service requirement to obtain a second sample characteristic group

Step S202, acquiring first feature representations corresponding to the first sample feature groups and second feature representations corresponding to the second sample feature groups through a preset number of feature extraction models by a first participant;

step S203, determining a plurality of first domain distinguishing loss values through a first participant based on a first feature characterization and a corresponding domain distinguishing model thereof, and determining a second feature extraction model based on each first domain distinguishing loss value and the corresponding feature extraction model;

step S204, a plurality of second domain distinguishing loss values are determined through the first participant based on the second feature characterization and the corresponding domain distinguishing model thereof, and each first feature extraction model is determined based on each second domain distinguishing loss value and the corresponding second feature extraction model.

In this embodiment, in order to make the number of the first feature representations the same as that of the second feature representations, for a first service scenario and a second service scenario, a part of features that are completely the same between the first service scenario and the second service scenario is determined as a preset service requirement, and then a first participant groups features of a first sample based on the preset service requirement to obtain a first sample feature group corresponding to the first sample, and groups features of a second sample based on the preset service requirement to obtain a second sample feature group corresponding to the second sample. And sequentially inputting the first sample feature group into the corresponding feature extraction models through the first participant to obtain a first feature characterization, and sequentially inputting the second sample feature group into the corresponding feature extraction models to obtain a second feature characterization.

Further, in an embodiment, step S202 includes:

Specifically, the first participant may number the first sample feature group, the second sample feature group, and the feature extraction model respectively, so as to implement one-to-one correspondence between the first sample feature group, the second sample feature group, and the feature extraction model according to the numbers, that is, each sample feature group in the first sample feature group (or the second sample feature group) corresponds to one feature extraction model respectively, and then the first participant inputs each sample feature group in the first sample feature group into the corresponding feature extraction model respectively to obtain a plurality of first feature characterizations, and the first participant inputs each sample feature group in the second sample feature group into the corresponding feature extraction model respectively to obtain a plurality of second feature characterizations.

Then, a plurality of first domain distinguishing loss values are determined by the first participant based on the first feature characterization and the domain distinguishing model, and a second feature extraction model is determined based on each first domain distinguishing loss value and the corresponding feature extraction model. The first feature representations correspond to the domain distinguishing models one by one, namely, each sample feature group in the first sample feature group (or the second sample feature group) corresponds to one of the domain distinguishing models respectively, the first participant inputs each first feature representation into the corresponding domain distinguishing model for model training to obtain a first domain distinguishing loss value corresponding to each first feature representation, the corresponding feature extraction model is updated based on each first domain distinguishing loss value to obtain a second feature extraction model, and meanwhile, the corresponding domain distinguishing model is updated based on each first domain distinguishing loss value to obtain the updated domain distinguishing model.

And finally, determining a plurality of second domain distinguishing loss values through the first participant based on the second feature characterization and the updated domain distinguishing model, and determining a first feature extraction model based on the second domain distinguishing loss values and the second feature extraction model. Specifically, the first participant inputs each second feature representation into the corresponding updated domain differentiation model for model training to obtain a second domain differentiation loss value corresponding to each second feature representation, updates the corresponding second feature extraction model based on each second domain differentiation loss value to obtain each first feature extraction model, and updates the corresponding updated domain differentiation model based on each second domain differentiation loss value to obtain the target domain differentiation model.

In the data processing method for federal transfer learning provided in this embodiment, the first participant groups the features of the first samples based on the preset service requirement to obtain a preset number of first sample feature groups, and groups the features of the second samples based on the preset service requirement to obtain a preset number of second sample feature groups; then, obtaining first feature representations corresponding to the first sample feature groups and second feature representations corresponding to the second sample feature groups through a preset number of feature extraction models by a first participant; determining a plurality of first domain distinguishing loss values through a first participant based on each first feature characterization and a corresponding domain distinguishing model thereof, and determining a second feature extraction model based on each first domain distinguishing loss value and a corresponding feature extraction model; and finally, determining a plurality of second domain distinguishing loss values by the first participant based on each second feature characterization and the corresponding domain distinguishing model thereof, and determining each first feature extraction model based on each second domain distinguishing loss value and the corresponding second feature extraction model, so that the updating of the domain distinguishing model and the feature extraction model is realized, and the accuracy of model training is improved.

Based on the second embodiment, a third embodiment of the data processing method for federal migration learning according to the present invention is provided, in this embodiment, step S203 includes:

step S301, inputting each first feature characterization into a corresponding domain distinguishing model through a first participant to obtain a first domain distinguishing loss value corresponding to each first feature characterization;

step S302, each updated domain distinguishing model is determined by the first participant based on each first domain distinguishing loss value, and the corresponding feature extraction model is updated by domain confrontation learning based on each first domain distinguishing loss value so as to obtain each second feature extraction model.

In this embodiment, when the first feature representations are obtained, the first participant inputs the corresponding domain distinguishing models to the first feature representations for model training to obtain first domain distinguishing loss values corresponding to the first feature representations, that is, the first participant inputs the first feature representations to the corresponding domain distinguishing models respectively to obtain first prediction domain labels corresponding to the first feature representations, and calculates the first domain distinguishing loss values corresponding to the first feature representations according to the first prediction domain labels and the domain labels corresponding to the first feature representations respectively.

And then, updating the corresponding feature extraction model through domain confrontation learning based on each first domain distinguishing loss value to obtain a second feature extraction model, and updating the corresponding domain distinguishing model according to each first domain distinguishing loss value to obtain an updated domain distinguishing model.

Further, in an embodiment, step S204 includes:

step S303, inputting each second feature representation into the corresponding updated domain distinguishing model through the first participant so as to obtain a second domain distinguishing loss value corresponding to each second feature representation;

step S304, determining each target domain distinguishing model based on each second domain distinguishing loss value through the first participant, updating the corresponding second feature extraction model through domain confrontation learning based on each second domain distinguishing loss value, and obtaining each first feature extraction model.

In this embodiment, each second feature representation is sequentially input to the corresponding updated domain distinguishing model for model training, so as to obtain a second domain distinguishing loss value corresponding to each second feature representation, that is, the second feature representations are respectively input to the corresponding updated domain distinguishing models, so as to obtain a second prediction domain label corresponding to each second feature representation, and the second domain distinguishing loss value corresponding to each second feature representation is calculated according to the second prediction domain label and the domain label corresponding to each second feature representation.

And then, updating the corresponding second feature extraction model through domain confrontation learning based on each second domain distinguishing loss value to obtain the first feature extraction model. And updating the corresponding updated domain distinguishing model according to each second domain distinguishing loss value to obtain a target domain distinguishing model, and taking the target domain distinguishing model as the domain distinguishing model in the iterative training process of subsequent model training.

It should be noted that the training times of the domain classification model, that is, the preset training times, may be preset, and when the target domain classification model is obtained, the training times corresponding to the domain classification model are updated, that is, the training times are increased by 1 to obtain new training times, and if the updated training times reach the preset training times, the training of the current feature extraction model is finished; and if the updated training times do not reach the preset training times, taking the target domain distinguishing model as a domain distinguishing model, taking the first feature extraction model as a feature extraction model, and returning to execute the step S301.

Referring to FIG. 5, in FIG. 5, the feature group F in the first sample feature group is passed₁ ^A,sAnd a feature group F in the second sample feature group₁ ^A,tTraining to obtain a target characteristic model R₀… … passing through feature set F in the first sample feature set_n ^A,sAnd a feature group F in the second sample feature group_n ^A,tTraining to obtain an updated feature model R_n-1，D₀……D₂For a domain differentiation model, L_d,0……L_d,3The loss values are distinguished for the domain.

In the data processing method for federal transfer learning provided in this embodiment, each first feature representation is input into a corresponding domain differentiation model through a first participant, so as to obtain a first domain differentiation loss value corresponding to each first feature representation; and then, determining each updated domain distinguishing model based on each first domain distinguishing loss value through the first participant, updating the corresponding feature extraction model through domain confrontation learning based on each first domain distinguishing loss value to obtain each second feature extraction model, updating the domain distinguishing models and the feature extraction models, and further improving the accuracy of model training.

Based on the first embodiment, a fourth embodiment of the data processing method for federal migration learning according to the present invention is provided, in this embodiment, step S102 includes:

step S401, a second participant obtains the third sample in the second participant, and a first participant obtains a first characteristic representation corresponding to the first sample;

and S402, determining a prediction loss value corresponding to the first feature characterization and the third sample through the to-be-trained prediction model based on longitudinal federal learning, and updating the to-be-trained prediction model and each first feature extraction model based on the prediction loss value respectively to obtain a target prediction model and each target feature extraction model.

In this embodiment, when the first feature extraction model is obtained, a third sample in the second party is obtained by the second party, and a first feature representation corresponding to the first sample is obtained by the first party. Wherein the third sample does not require grouping of features, each sample feature in the third sample being a single feature.

Then, based on the longitudinal federal learning, determining a prediction loss value corresponding to a first feature characterization and a third sample through a to-be-trained prediction model, respectively updating the to-be-trained prediction model and each first feature extraction model based on the prediction loss value to obtain a target prediction model and each target feature extraction model, namely, based on the longitudinal federal learning, inputting the first feature characterization and the third sample into the interpretable to-be-trained prediction model for model training to obtain the prediction loss value, specifically, determining a feature set corresponding to the first feature characterization and the third sample, and inputting the feature characterization set corresponding to the first feature characterization and the third sample into the interpretable to-be-trained prediction model for model training based on the longitudinal federal learning, wherein the feature set corresponding to the first feature characterization and the third sample is a data set comprising all the first feature characterization and the third sample, and updating the prediction model to be trained according to the prediction loss value to obtain a target prediction model, and updating the first feature extraction model according to the prediction loss value to obtain each target feature extraction model.

It should be noted that, in other embodiments, when the target feature extraction model is determined, the number of training rounds of the prediction model to be trained is accumulated, if the number of training rounds is smaller than a preset value, the target prediction model is used as the prediction model to be trained, the target feature extraction model is used as the first feature extraction model, and the step S402 is continuously executed.

Referring to FIG. 5, in FIG. 5, r in the first feature characterization corresponding to the first sample feature group_i ^A,sWith (r)₁ ^A,s、r₂ ^A,s、r₃ ^A,sEtc.) and monomer characteristics f_j ^B,s(f₁ ^B,s、f₂ ^B,s、f₃ ^B,sEtc.) input the prediction model to be trained to obtain a prediction loss value L_cls ^B,sFinally based on the predicted loss value L_cls ^B,sAnd updating each updated feature extraction model to obtain a target feature extraction model, and updating the to-be-trained prediction model to obtain a target prediction model G.

In the data processing method for federal transfer learning provided in this embodiment, the second participant obtains the third sample in the second participant, and the first participant obtains the first feature representation corresponding to the first sample; and then, determining a prediction loss value corresponding to the first feature characterization and the third sample through a to-be-trained prediction model based on longitudinal federal learning, updating the to-be-trained prediction model and each first feature extraction model based on the prediction loss value respectively to obtain a target prediction model and each target feature extraction model, and training according to the first feature characterization and the third sample to obtain a target model so as to further improve the accuracy of model training.

Based on the fourth embodiment, a fifth embodiment of the data processing method for federal migration learning according to the present invention is provided, in this embodiment, step S402 includes:

step S501, model training is carried out on the prediction model to be trained through a longitudinal federated learning algorithm based on the first feature characterization and a first feature characterization set corresponding to the third sample, so that a prediction loss value is obtained;

step S502, updating the prediction model to be trained through a longitudinal federal learning algorithm based on the prediction loss value to obtain a target prediction model;

and S503, updating each first feature extraction model through a longitudinal federal learning algorithm based on the predicted loss value to obtain a target feature extraction model.

In this embodiment, after the first feature extraction model is determined, feature sets corresponding to the first feature characterization and the third sample are determined, and based on the first feature characterization and the first feature set corresponding to the third sample, the prediction model to be trained is subjected to model training through a longitudinal federated learning algorithm to obtain a prediction loss value, that is, based on longitudinal federated learning, the feature characterization sets corresponding to the first feature characterization and the third sample are input into the interpretable prediction model to be trained to perform model training to obtain a corresponding prediction category label, and a prediction loss value is calculated according to the prediction category label and a category label (real category label) corresponding to the third sample, where the feature sets corresponding to the first feature characterization and the third sample are data sets including all the first feature characterization and the third sample.

And then, according to the prediction loss value, updating the prediction model to be trained through a longitudinal federal learning algorithm to obtain a target prediction model, and according to the prediction loss value, updating the first feature extraction model through the longitudinal federal learning algorithm to obtain a target feature extraction model.

According to the data processing method for federal transfer learning, model training is carried out on a to-be-trained prediction model through a longitudinal federal learning algorithm based on a first feature characterization and a first feature characterization set corresponding to a third sample, so that a prediction loss value is obtained; then updating the prediction model to be trained through a longitudinal federal learning algorithm based on the prediction loss value to obtain a target prediction model; and then updating each first feature extraction model through a longitudinal federated learning algorithm based on the prediction loss value to obtain a target feature extraction model, so that the prediction model to be trained and the first feature extraction model are updated according to the first feature representation and the prediction loss value corresponding to the third sample, and the accuracy of model training is further improved.

Based on the foregoing embodiments, a sixth embodiment of the data processing method for federated migration learning according to the present invention is provided, where in this embodiment, after step S102, the method further includes:

step S601, obtaining to-be-predicted feature characterizations corresponding to a first to-be-predicted sample in a first participant, wherein the first participant groups features of the first to-be-predicted sample based on a preset service requirement to obtain a to-be-predicted sample feature group, and obtains to-be-predicted feature characterizations corresponding to each to-be-predicted sample feature group based on a target feature extraction model, wherein the first to-be-predicted sample is a sample corresponding to a second service scene;

step S602, a second sample to be predicted in a third participant is obtained, and the second sample to be predicted is a second business scene sample;

and step S603, calculating the prediction result of the target prediction model through a longitudinal federal learning algorithm based on each feature characterization to be predicted and the feature characterization set corresponding to the second sample to be predicted.

After the target prediction model and the target feature extraction model are obtained, federal prediction can be performed on the second business scenario based on the first party and the third party through the target prediction model and the target feature extraction model.

In the embodiment, a feature characterization to be predicted corresponding to a first sample to be predicted in a first participant is obtained, wherein the first participant groups features of the first sample to be predicted based on a preset service requirement to obtain a sample feature group to be predicted, and obtains the feature characterization to be predicted corresponding to each sample feature group to be predicted based on a target feature extraction model, and the first sample to be predicted is a sample corresponding to a second service scene; specifically, the process of grouping the first to-be-predicted samples is similar to the process of grouping the first samples, and is not described herein again. Then, a second sample to be predicted in the third participant is obtained. The first sample to be predicted is a sample corresponding to a second service scene in the first participant, that is, data corresponding to the first sample to be predicted is data of the second service scene; the second sample to be predicted is a sample corresponding to a second service scenario in the third participant, that is, data corresponding to the second sample to be predicted is data of the second service scenario.

And then, determining each feature characterization to be predicted and a feature characterization set corresponding to the second sample to be predicted, specifically, splicing each feature characterization to be predicted and the second sample to be predicted to obtain the feature characterization set, wherein the feature characterization to be predicted and the feature characterization set corresponding to the second sample to be predicted include all features of each feature characterization to be predicted and the second sample to be predicted. And then, calculating a prediction result of the target prediction model through a longitudinal federated learning algorithm based on each feature characterization to be predicted and the feature set corresponding to the second sample to be predicted, inputting each feature characterization to be predicted and the feature set corresponding to the second sample to be predicted into the target prediction model based on longitudinal federated learning to obtain the prediction result, and accurately predicting the samples of the first participant and the third participant in a federated manner through the target prediction model.

Referring to fig. 4, in fig. 4, a participant a is a first participant, a participant C is a third participant, and the participants a and C have a large number of overlapping samples on a second service scenario t. F_i ^A,tFor the set of characteristics of the sample to be predicted corresponding to the first sample to be predicted, f_j ^C,tAnd the single feature to be predicted corresponding to the second sample to be predicted.

Referring to FIG. 6, in FIG. 6, F₁ ^A,t、F₂ ^A,t、F₃ ^A,tFor the set of characteristics of the sample to be predicted corresponding to the first sample to be predicted, f₁ ^C ^,t、f₂ ^C,t、f₃ ^C,t、f₄ ^C,tFor the monomer characteristic to be predicted corresponding to the second sample to be predicted, F₁ ^A,t、F₂ ^A,t、F₃ ^A,tRespectively inputting corresponding target feature extraction models R₁、R₂、R₃To obtain a plurality of characteristics r to be predicted₁ ^A,t、r₂ ^A,t、r₃ ^AtAnd then inputting the multiple characteristics to be predicted and the characteristics of the single body to be predicted into the target prediction model G to obtain a prediction result, namely a category label.

In the data processing method for federal transfer learning provided by this embodiment, to-be-predicted feature representations corresponding to first to-be-predicted samples in a first participant are obtained, where the first participant groups features of the first to-be-predicted samples based on preset service requirements to obtain a preset number of to-be-predicted sample feature groups, and obtains to-be-predicted feature representations corresponding to the to-be-predicted sample feature groups based on a preset number of target feature extraction models, where the first to-be-predicted samples are second service scene samples; then, a second sample to be predicted in a third participant is obtained, wherein the second sample to be predicted is a second business scene sample; and then based on each feature characterization to be predicted and the feature characterization set corresponding to the second sample to be predicted, calculating the prediction result of the target prediction model through a longitudinal federated learning algorithm, carrying out accurate federated prediction on the samples of the first participant and the third participant through the target prediction model, improving the accuracy of the prediction result through the target feature extraction model obtained by migration learning, and explaining the reason of obtaining the prediction result through the target prediction model so as to realize the consideration of model migration and model interpretability.

An embodiment of the present invention further provides a data processing apparatus for federated migration learning, where referring to fig. 7, fig. 7 is a schematic diagram of functional modules of an embodiment of the data processing apparatus for federated migration learning according to the present invention, where the data processing apparatus for federated migration learning includes:

a first model determining module 100, configured to determine, by a first participant, each first feature extraction model based on a first sample corresponding to a first service scenario, a second sample corresponding to a second service scenario, and a plurality of feature extraction models in the first participant;

a second model determining module 200, configured to determine a target prediction model and each target feature extraction model based on the first sample, a third sample corresponding to the first business scenario in the second participant, the prediction model to be trained, and the first feature extraction model.

Optionally, the first model determination module 100 is further configured to:

and determining a plurality of second domain distinguishing loss values by the first participant based on the second feature characterizations and the corresponding domain distinguishing models thereof, and determining the first feature extraction models based on the second domain distinguishing loss values and the corresponding second feature extraction models.

Optionally, the first model determination module 100 is further configured to:

Optionally, the second model determination module 200 is further configured to:

Optionally, the target prediction model is a fraud score prediction model; the first service scene comprises a credit scoring scene of a user, the second service scene comprises a fraud scoring scene of the user, the first sample comprises credit scoring data in a first participant, the second sample is fraud scoring data in the first participant, and the third sample is credit scoring data in a second participant;

the second model determination module 200 is further configured to:

Optionally, the data processing apparatus for federal migration learning further includes:

In addition, an embodiment of the present invention further provides a storage medium, where a data processing program for federal transfer learning is stored on the storage medium, and when being executed by a processor, the data processing program for federal transfer learning implements the steps of the data processing method for federal transfer learning as described above.

The method implemented when the data processing program for federal migration learning running on the processor is executed may refer to each embodiment of the data processing method for federal migration learning of the present invention, and is not described herein again.

In addition, an embodiment of the present invention further provides a computer program product, where the computer program product implements the steps of the data processing method for federal migration learning as described above when executed.

The method implemented when the computer program product executes the data processing program of the federal migration learning may refer to each embodiment of the data processing method of the federal migration learning of the present invention, and is not described herein again.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. The data processing method for federated transfer learning is characterized by comprising the following steps:

2. The federated transfer learning-based data processing method of claim 1, wherein the step of determining, by the first participant, each first feature extraction model based on a first sample corresponding to a first business scenario, a second sample corresponding to a second business scenario, and a plurality of feature extraction models in the first participant, comprises:

3. The federal migration learning data processing method as claimed in claim 2, wherein the step of obtaining, by the first participant, the first feature characterization corresponding to each first sample feature group through a preset number of feature extraction models, and the step of obtaining the second feature characterization corresponding to each second sample feature group includes:

4. The federal migration learning data processing method as claimed in claim 2, wherein the step of determining, by the first party, a plurality of first domain discriminant loss values based on each first feature characterization and its corresponding domain discriminant model, and the step of determining the second feature extraction model based on each first domain discriminant loss value and its corresponding feature extraction model comprises:

5. The federated transfer learning data processing method of claim 4, wherein the determining, by the first participant, a plurality of second domain discriminant loss values based on the respective second feature characterizations and their corresponding domain discriminant models, and wherein the determining, by the first participant, the respective first feature extraction models based on the plurality of second domain discriminant loss values and their corresponding second feature extraction models comprises:

6. The federal migration learning data processing method as claimed in claim 1, wherein the step of determining the target prediction model and each target feature extraction model based on the first sample, the third sample corresponding to the first business scenario in the second participant, the prediction model to be trained, and the first feature extraction model comprises:

7. The federal transfer learning data processing method as claimed in claim 6, wherein the step of calculating a prediction loss value of the prediction model to be trained by a longitudinal federal learning algorithm based on the first feature characterization and the third sample, and updating the prediction model to be trained and each first feature extraction model based on the prediction loss value respectively to obtain the target prediction model and each target feature extraction model comprises:

8. The federal migration learning data processing method of claim 1, wherein the objective prediction model is a fraud score prediction model; the first service scene comprises a credit scoring scene of a user, the second service scene comprises a fraud scoring scene of the user, the first sample comprises credit scoring data in a first participant, the second sample is fraud scoring data in the first participant, and the third sample is credit scoring data in a second participant;

9. The federal migration learning data processing method as claimed in any one of claims 1 to 8, wherein after the step of determining the target prediction model and each target feature extraction model based on the first sample, the third sample corresponding to the first business scenario in the second participant, the prediction model to be trained, and the first feature extraction model, the method further comprises:

10. The utility model provides a data processing apparatus of federated migratory learning, its characterized in that, data processing apparatus of federated migratory learning includes:

11. The utility model provides a data processing equipment of federal migration learning which characterized in that, data processing equipment of federal migration learning includes: a memory, a processor, and a federated transfer-learning data processing program stored on the memory and operable on the processor, the federated transfer-learning data processing program when executed by the processor implementing the steps of the federated transfer-learning data processing method of any of claims 1-9.

12. A computer-readable storage medium, having stored thereon a federated migration learning data processing program that, when executed by a processor, implements the steps of the federated migration learning data processing method of any of claims 1-9.

13. A computer program product, characterized in that it implements the steps of a data processing method of federal transfer learning as claimed in any one of claims 1 to 9 when executed.