CN112132676A

CN112132676A - Method and device for determining contribution degree of joint training target model and terminal equipment

Info

Publication number: CN112132676A
Application number: CN202010974527.XA
Authority: CN
Inventors: 霍昱光; 孙昊; 王雪; 权纯
Original assignee: CCB Finetech Co Ltd
Current assignee: CCB Finetech Co Ltd
Priority date: 2020-09-16
Filing date: 2020-09-16
Publication date: 2020-12-25
Anticipated expiration: 2040-09-16
Also published as: CN112132676B

Abstract

The specification provides a method and a device for determining contribution of a joint training target model and terminal equipment. Based on the method, corresponding data processing can be performed on the second terminal device based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal device for the target model training; determining a test result capable of measuring the marginal prediction effect improvement effect of a second data source introduced into the second terminal device on the target model training by using a preset test interval; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result. Therefore, the technical problem that the contribution degree of the data source participating in the joint training scene cannot be accurately and quantitatively evaluated in the existing method can be effectively solved.

Description

Method and device for determining contribution degree of joint training target model and terminal equipment

Technical Field

The specification belongs to the technical field of internet, and particularly relates to a method and a device for determining contribution degree of a joint training target model, and a terminal device.

Background

In some business scenarios, different data parties often own different data sources. Sometimes, it is necessary to use data sources owned by different data parties at the same time to build a required target model through joint training (for example, federal learning) without revealing data owned by the data parties to each other.

For example, in a financial business scenario, bank a owns the loan data of the bank platform of the residents in city B, while the shopping site C owns the loan data of the network platform of the residents in city B. At present, bank A hopes to be able to train to obtain a prediction model which can more comprehensively and accurately predict the credit and debit risk of residents in city B.

Aiming at the scene, bank A hopes to cooperate with shopping website C, and a prediction model for predicting the credit and debit risk of residents in city B is constructed through joint training by utilizing data sources owned by bank A. However, based on the existing related method, bank a cannot accurately determine how much the improvement effect of introducing the data source owned by the shopping website C on the marginal prediction effect of the model for training the loan credit risk is, and further cannot accurately determine the contribution degree of the shopping website C on the model training, so that a corresponding cooperation agreement with the shopping website C cannot be reasonably reached.

Therefore, a method capable of accurately evaluating the contribution degree of the newly added data source to the target training model in the combined training scenario is needed.

Disclosure of Invention

The specification provides a method, a device and a terminal device for determining the contribution degree of a joint training target model, so as to accurately evaluate the contribution degree of a newly added data source to the target training model in a joint training scene.

The method for determining the contribution degree of the joint training target model provided by the specification comprises the following steps:

acquiring a contrast model and a target model; the comparison model is a model obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal device and the second terminal device are subjected to joint training;

performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment for the target model training;

determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training;

and determining contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

In one embodiment, after determining the contribution of the second data source owned by the second terminal device to the joint training target model, the method further comprises:

determining reward data aiming at the second terminal equipment according to the contribution degree;

and sending corresponding reward data to the second terminal equipment.

In one embodiment, the matching processing rule includes: processing rules based on horizontal federated learning, processing rules based on vertical federated learning, or processing rules based on federated migratory learning.

In one embodiment, the matching processing rule is determined as follows:

determining a joint training type adopted by a first terminal device and a second terminal device when a target model is jointly trained; wherein the joint training types include: horizontal federal learning, vertical federal learning, or federal migratory learning;

and determining a matched processing rule according to the joint training type.

In one embodiment, in the case where the matching processing rule is determined to be a processing rule based on longitudinal federal learning, the target model includes a first model held by the first terminal device and a second model held by the second terminal device.

In an embodiment, in a case that it is determined that the matched processing rule is a processing rule based on longitudinal federal learning, according to the matched processing rule, based on the comparison model, the target model, and the test data, performing corresponding data processing with the second terminal device to determine a marginal prediction effect improvement parameter for target model training of a second data source owned by the second terminal device, the method includes:

processing the test data by using the comparison model to obtain a first group of prediction probabilities;

processing the test data by using the first model to obtain a first processing result;

obtaining a second processing result from the second terminal device; the second processing result is obtained by processing the test data by the second terminal equipment by using the second model;

determining a second set of prediction probabilities according to the first processing result and the second processing result;

and determining a marginal prediction effect promotion parameter of the second data source for the training of the target model according to the first group of prediction probabilities and the second group of prediction probabilities.

In one embodiment, the obtaining of the second processing result from the second terminal device includes:

receiving an encrypted second processing result; the second terminal equipment encrypts the second processing result based on a homomorphic encryption algorithm to obtain a corresponding encrypted second processing result;

and correspondingly decrypting the encrypted second processing result to obtain the second processing result.

In one embodiment, determining a marginal prediction effect improvement parameter of a second data source for the target model training according to the first set of prediction probabilities and the second set of prediction probabilities comprises:

calculating a first AUC parameter based on the control model and a second AUC parameter based on the target model according to the data label of the test data, the first set of prediction probabilities and the second set of prediction probabilities;

calculating a standard deviation according to the first AUC parameter and the second AUC parameter;

and calculating a marginal prediction effect promotion parameter of the second data source for the target model training according to the first AUC parameter, the second AUC parameter and the standard deviation.

In one embodiment, the predetermined test interval is configured according to the following equation:

wherein the content of the first and second substances,

is a first parameter of the AUC, which is,

is a second AUC parameter which is a function of,

is standard deviation, z_1-α/2Cumulatively calculating a probability density value equal to z for a standard normal distribution_1-α/2And (c) a z score of (a) is a preset confidence level.

In one embodiment, determining, according to the test result, a contribution degree of a second data source owned by a second terminal device to a joint training target model includes:

and according to the inspection result, determining that the contribution degree of a second data source owned by the second terminal equipment to the joint training target model meets the preset requirement under the condition that the marginal prediction effect promotion parameter value is determined to be outside the preset inspection interval.

In an embodiment, in a case that it is determined that the matched processing rule is a processing rule based on horizontal federal learning, the test data acquired by the first terminal device is first test data, and the test data acquired by the second terminal device is second test data.

In an embodiment, in a case that it is determined that the matched processing rule is a processing rule based on horizontal federal learning, according to the matched processing rule, based on the comparison model, the target model, and the test data, performing corresponding data processing with the second terminal device to determine a marginal prediction effect improvement parameter for target model training of a second data source owned by the second terminal device, the method includes:

processing the first test data by using the comparison model to obtain a first group of prediction probabilities;

processing the first test data by using the target model to obtain a second group of prediction probabilities;

performing local operation according to the first group of prediction probabilities and the second group of prediction probabilities to obtain first local group data;

according to the first group of prediction probabilities and the second group of prediction probabilities, performing interactive operation with second terminal equipment through encryption communication to obtain first interactive component data;

constructing a first data body according to the first local component data and the first interactive component data, and encrypting the first data body;

sending the encrypted first data volume to an intermediate server; and the intermediate server is used for determining a marginal prediction effect promotion parameter of the second data source for the target model training according to the encrypted first data body and the encrypted second data body from the second terminal device.

In one embodiment, the objective model includes a prediction model for predicting data tags to which the user belongs, the data tags including blacklist tags and whitelist tags.

In one embodiment, the first terminal device includes a plurality of sub-terminal devices each having a sub-data source.

In one embodiment, after determining, according to the test result, a degree of contribution of a second data source owned by a second terminal device to a joint training target model, the method further includes:

and determining whether to cooperate with the second terminal equipment to carry out joint training according to the contribution degree.

The present specification also provides an apparatus for determining a contribution of a joint training target model, including:

the acquisition module is used for acquiring a comparison model and a target model; the comparison model is a model obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal device and the second terminal device are subjected to joint training;

the processing module is used for performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule so as to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment on the target model training;

the inspection module is used for determining a corresponding inspection result according to the marginal prediction effect promotion parameter and a preset inspection interval; the detection result is used for measuring the marginal prediction effect improvement effect of introducing a second data source owned by second terminal equipment to target model training;

and the determining module is used for determining the contribution degree of a second data source owned by the second terminal equipment to the joint training target model according to the test result.

The specification also provides a terminal device, which comprises a processor and a memory for storing processor executable instructions, wherein the processor realizes acquisition of a contrast model and a target model when executing the instructions; the comparison model is a model obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal device and the second terminal device are subjected to joint training; performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment on the target model training; determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

The present specification also provides a computer readable storage medium having stored thereon computer instructions which, when executed, enable obtaining a reference model, a target model; the comparison model is obtained under the condition that the first terminal equipment and the second terminal equipment are not subjected to joint training, and the target model is obtained under the condition that the first terminal equipment and the second terminal equipment are subjected to joint training; performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment for the training of the target model; determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

According to the method, the device and the terminal equipment for determining the contribution degree of the joint training target model, different joint training types are distinguished according to matched processing rules, corresponding data processing is performed on the joint training types and the second terminal equipment based on the reference model, the target model and the test data in a targeted manner, and therefore marginal prediction effect promotion parameters of a second data source owned by the second terminal equipment for target model training are determined; determining a test result capable of measuring the marginal prediction effect improvement effect of a second data source introduced into the second terminal device on the target model training by using a preset test interval; and then determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result. Therefore, the technical problem that the contribution degree of the data source participating in the joint training scene cannot be accurately and quantitatively evaluated in the existing method can be effectively solved through statistical test, and the technical effect that the contribution degree of a newly-added data source to the target training model in the joint training scene can be accurately determined is achieved.

Drawings

In order to more clearly illustrate the embodiments of the present specification, the drawings needed to be used in the embodiments will be briefly described below, and the drawings in the following description are only some of the embodiments described in the present specification, and other drawings can be obtained by those skilled in the art without inventive labor.

FIG. 1 is a diagram illustrating an embodiment of a structural configuration of a system to which a method for determining a contribution degree of a joint training target model provided in an embodiment of the present disclosure is applied;

FIG. 2 is a schematic flow chart diagram illustrating a method for determining a contribution of a joint training target model according to an embodiment of the present disclosure;

FIG. 3 is a diagram illustrating an embodiment of a method for determining a contribution degree of a joint training target model according to an embodiment of the present disclosure;

FIG. 4 is a diagram illustrating an embodiment of a method for determining a contribution degree of a joint training target model according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a server according to an embodiment of the present disclosure;

fig. 6 is a schematic structural composition diagram of a device for determining a contribution degree of a joint training target model according to an embodiment of the present specification.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is obvious that the described embodiments are only a part of the embodiments of the present specification, and not all of the embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments in the present specification without any inventive step should fall within the scope of protection of the present specification.

Considering that the effect of a data source participating in joint training in a complex joint training scene such as horizontal federal learning on improving the marginal prediction effect of the target model training cannot be accurately determined based on the existing related method, and whether the contribution of the data source participating in the joint training scene meets the requirement cannot be quantitatively evaluated more accurately. For example, it cannot be evaluated quantitatively whether the promotion of a newly introduced data source to the training of the target model is statistically significant, which in turn results in an inability to effectively measure the data value of the newly introduced data source, and affects the relevant cooperation with the data party holding the data source.

For the root cause of the above problem, the present specification considers that a target model obtained by different joint training types can be distinguished by using data processing characteristics in different joint training type scenes, and a matched processing rule is selected and used in a targeted manner to perform corresponding data processing with the second terminal device based on the comparison model, the target model and the test data, so as to more accurately determine marginal prediction effect promotion parameters for target model training of the second data source owned by the second terminal device in a more complex joint training scene. Furthermore, a preset test interval is introduced, and a test result capable of measuring the magnitude of the lifting effect of a second data source owned by the introduced second terminal device on the marginal prediction effect of the target model training is determined by using the preset test interval; and then, according to the test result, the contribution degree of a second data source owned by the second terminal device to the joint training target model can be accurately determined. Therefore, the contribution degree of the newly added data source to the target training model in the joint training scene can be accurately determined through statistical test, and the technical problem that the contribution degree of the data source participating in the joint training scene cannot be accurately quantified and evaluated in the existing method is solved.

Referring to fig. 1, the method for determining the contribution of the joint training target model according to the embodiment of the present disclosure may be applied to a system including a plurality of terminal devices and an intermediate server. The intermediate server and the plurality of terminal devices can be connected in a wired or wireless manner to perform specific data interaction.

In an embodiment, the terminal device may be a device disposed at a side of a data party and connected with a data source owned by the data party. For example, the terminal device a may be a server disposed on the side of bank a, and possesses loan data of the bank platform of bank a about residents in city B. The terminal device C may be a server installed on one side of the C shopping site, and may have loan data on a network platform of a B city resident of the C shopping site.

The intermediate server may be a platform server in charge of data services in the data processing system and trusted by a plurality of terminal devices. In particular, the intermediate server may participate in or assist in data interaction and data processing between different devices in the data processing system. For example, the intermediate server may be a platform server of a data interaction platform.

In this embodiment, the intermediate server may specifically include a background server capable of implementing functions such as data transmission and data processing. Specifically, the intermediate server may be, for example, an electronic device having data operation, storage function and network interaction function. Alternatively, the intermediate server may also be a software program running in the electronic device and providing support for data processing, storage and network interaction. In this embodiment, the number of servers included in the intermediate server is not particularly limited. The intermediate server may be specifically one server, several servers, or a server cluster formed by several servers.

In this embodiment, the terminal device may specifically include a client device capable of implementing functions such as data acquisition and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a notebook computer, an intelligent mobile phone, and the like. Alternatively, the terminal device may be a software application capable of running in the electronic device. For example, it may be some APP running on a cell phone, etc.

In an embodiment, a data interaction platform may be specifically constructed by the data processing system, and based on the data interaction platform, data parties holding different data sources may perform specific data interaction through corresponding terminal devices. For example, a data party needing a joint training model may initiate a model training request on a terminal device platform to find a suitable data party as a partner, and obtain a required target model through joint training using data sources owned by both parties. For another example, a data party needing to share the data source may actively release the referral information of the owned data source on the platform through the terminal device.

The embodiment of the specification further provides a method for determining the contribution degree of the joint training target model. The method can be applied to the first terminal device side. As shown in fig. 2, the method may be implemented as follows.

S201: acquiring a contrast model and a target model; the comparison model is a model obtained under the condition that the first terminal equipment and the second terminal equipment are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal equipment and the second terminal equipment are subjected to joint training;

s202: performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment on the target model training;

s203: determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training;

s204: and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

In an embodiment, the first terminal device may be specifically understood as a terminal device of one data party initiating the joint training request, or terminal devices of existing multiple data parties initiating the joint training request.

The data source owned by the first terminal device may be denoted as a first data source. The first data source may include a large amount of data. Each data in the first data source may specifically be first characteristic data corresponding to the first data identity.

The first characteristic data may be parameter data describing one or more attributes of a data object. The first data identifier may specifically be identification information used to indicate the data object. Specifically, the first data identifier may be information such as a name and a number of the data object. Each first characteristic data corresponds to a first data identification.

The first feature data may be one feature data or a combination of a plurality of feature data corresponding to the same first data id.

Specifically, for example, one of the data in the first data source may be the income of the user M of 100 yuan. Here, the name M of the user may be understood as a first data identifier of the data, and the income 100 yuan may be understood as first feature data corresponding to M. For another example, one of the data in the first data source may be that the selling price of the item 002 is 200 yuan and the place of origin is S city. Here, the product number 002 may be understood as a first data identification of the data, a selling price of 200 yuan, and a place of origin S city may be understood as a first characteristic data corresponding to 002. Of course, the first data identification and the first characteristic data listed above are only schematic illustrations. In specific implementation, the first data identifier and the first feature data may also be other types of data according to a specific application scenario and a processing requirement. The present specification is not limited to these.

The second terminal device may be specifically understood as a terminal device of a partner performing joint training in cooperation with the first terminal device in response to the joint training request. The data source owned by the second terminal device may be denoted as a second data source. The second data source may specifically include a large amount of data. Each data in the second data source may specifically be second characteristic data corresponding to the second data identity.

In one embodiment, the above-mentioned comparison model may be specifically understood as a model obtained without introducing the second terminal device and without performing joint training with the second terminal device. The target model may be specifically understood as a model obtained in a case where the first terminal device and the second terminal device perform joint training.

In an embodiment, the matching processing rule may specifically include: processing rules based on horizontal federal learning, processing rules based on vertical federal learning, or processing rules based on federal migration learning, etc.

In an embodiment, the matching processing rule may specifically be determined as follows: determining a joint training type adopted by a first terminal device and a second terminal device when a target model is jointly trained; wherein the joint training types include: horizontal federal learning, vertical federal learning, or federal migratory learning; and determining the matched processing rule according to the joint training type.

Specifically, in the case that the type of the joint training is determined to be the horizontal federal learning, the matched processing rule may be determined to be a processing rule based on the horizontal federal learning. In the case where the type of joint training is determined to be longitudinal federal learning, the matching processing rule may be determined to be a processing rule based on longitudinal federal learning. In the case where the joint training type is determined to be federated migration learning, the matching processing rule may be determined to be a federated migration learning based processing rule.

In an embodiment, specifically, the federal learning may specifically mean that, in the process of model learning and training, each data party participating in joint training may perform joint modeling by using data of other data parties; and all data parties do not need to share data resources, namely, under the condition that data are not local, joint training is carried out to establish a shared machine learning model.

The longitudinal federated learning may specifically refer to a federated learning mode adopted in a case where data identifiers in data sources owned by data parties participating in the joint training overlap more and feature data overlap less. In this case, the data may be segmented according to the longitudinal direction (i.e. feature dimension), and then the data with the same data identifier but not identical features in each data source may be extracted for training. For example, there are two different institutions, one being a bank in one location and the other being an e-commerce in the same location. Their user population is likely to contain a large percentage of the inhabitants of the site, and therefore the intersection of users is large (i.e. there is more overlap of data identifications). However, since the bank records the user's balance and credit rating, and the e-commerce maintains the user's browsing and purchasing history, their user feature intersection is small (i.e., feature data overlap is less). In this case, these different features can be aggregated in an encrypted state through longitudinal federal learning to enhance the federal learning of model capabilities.

The above-mentioned horizontal federal learning may specifically refer to a federal learning mode adopted in a case where data identifications overlap less and feature data overlap more in data sources owned by data parties participating in joint training. In this case, the data set may be segmented according to the horizontal direction (i.e. the user dimension), and then the data with the same characteristics and the incompletely same data identifiers in each data source may be extracted for training. For example, there are two banks in different regions, and their user groups are respectively from the regions where they are located, and the intersection of each other is very small (i.e. the data identifications overlap less). However, since their services are very similar, the recorded user characteristics are mostly the same (i.e. there is more overlap of the characteristic data). At this point, the build model may be trained through horizontal federal learning.

The federated migration learning may specifically refer to a federated learning mode adopted in a case where data identifications in data sources owned by data parties participating in the joint training overlap less and feature data also overlap less. In this case, the data may not be segmented, but migration learning is used to overcome the deficiency of the data or the label. For example, there are two different institutions, one being a bank located in city B and the other being an e-commerce located in city Q. Due to regional constraints, the intersection of the user groups of the two organizations is small (i.e., the data identifications overlap less). Meanwhile, due to the difference of mechanism types, the data characteristics of the two data are only partially overlapped (namely, the overlapping of the characteristic data is less). Under the condition, migration learning can be introduced to solve the problems of small scale of unilateral data and few label samples so as to improve the effect of the model for effective federal learning.

In an embodiment, processing rules matched with each joint training type can be configured in advance according to data processing characteristics under different joint training types and requirements such as data privacy protection, so that different joint training types can be distinguished in the following, and corresponding data processing is performed by using the matched processing rules, so that marginal prediction effect improvement parameters of a second data source owned by a second terminal device under a joint training scene for target model training can be acquired more accurately, and data errors are reduced.

In an embodiment, in a case that the matched processing rule is determined to be a processing rule based on longitudinal federal learning, the target model may specifically include a first model held by the first terminal device and a second model held by the second terminal device.

In this embodiment, in a joint training scenario of longitudinal federal learning, the first terminal device and the second terminal device participating in joint training do not always hold a complete target model, but only hold one component model of the target model. Specifically, the first terminal device holds a first model in the target model, and the second terminal device holds a second model in the target model.

In this embodiment, in the joint training scenario of longitudinal federal learning, the data labels are often concentrated on one side. For example, the first terminal device may possess all data tags. In addition, under the combined training scene of longitudinal federal learning, most of the data of the first data source and the data of the second data source are data with high coincidence degree of data identification.

Based on the above characteristics, in a case that it is determined that the matched processing rule is a processing rule based on longitudinal federal learning, the above processing rule is performed with the second terminal device according to the matched processing rule, based on the comparison model, the target model, and the test data, so as to determine an edge prediction effect improvement parameter of the second data source owned by the second terminal device for the target model training, and in a specific implementation, the following contents may be included:

s1: processing the test data by using the comparison model to obtain a first group of prediction probabilities;

s2: processing the test data by using the first model to obtain a first processing result;

s3: obtaining a second processing result from the second terminal device; the second processing result is obtained by processing the test data by the second terminal equipment by using the second model;

s4: determining a second set of prediction probabilities according to the first processing result and the second processing result;

s5: and determining a marginal prediction effect promotion parameter of the second data source for the target model training according to the first group of prediction probabilities and the second group of prediction probabilities.

In one embodiment, when implemented, the first terminal device may input the test data into the comparison model and output probability values of the respective test data belonging to the respective data tags as the first set of prediction probabilities.

The first terminal device may input the test data into the first model, and output probability values of the respective test data belonging to the respective data tags as the first processing result. Meanwhile, the second terminal device may input the test data into the second model, and output probability values of the test data belonging to the data tags as a second processing result.

Further, the first terminal device needs to use the first processing result obtained based on the first model and the second processing result obtained based on the second model together to obtain the final second set of prediction probabilities.

In an embodiment, the obtaining of the second processing result from the second terminal device may include the following steps: receiving an encrypted second processing result; the second terminal equipment encrypts the second processing result based on a homomorphic encryption algorithm to obtain a corresponding encrypted second processing result; and correspondingly decrypting the encrypted second processing result to obtain the second processing result.

In this embodiment, in order to avoid that the intermediate server obtains the specific content of the second processing result, or that the second processing result is intercepted by another third party in the transmission process, the second terminal device may encrypt the second processing result after the second processing result is obtained based on the homomorphic encryption algorithm; and based on a homomorphic encryption algorithm, the encrypted second processing result is safely sent to the first terminal equipment through multiple encryption interaction with the first terminal equipment, so that the data safety can be effectively protected.

In this embodiment, after receiving the encrypted second processing result, the first terminal device may perform decryption processing based on a corresponding homomorphic encryption algorithm to obtain plaintext data of the second processing result.

In an embodiment, the determining, according to the first group of prediction probabilities and the second group of prediction probabilities, a marginal prediction effect improvement parameter of a second data source for training a target model may include the following steps:

s1: calculating a first AUC parameter based on the control model and a second AUC parameter based on the target model according to the data label of the test data, the first set of prediction probabilities and the second set of prediction probabilities;

s2: calculating a standard deviation according to the first AUC parameter and the second AUC parameter;

s3: and calculating a marginal prediction effect promotion parameter of the second data source for the target model training according to the first AUC parameter, the second AUC parameter and the standard deviation.

In an embodiment, the AUC (area under the Curve of ROC) parameter may be specifically understood as an area under a receiver operating characteristic Curve (ROC), and the accuracy of the established model may be evaluated according to the AUC parameter.

In an embodiment, in order to quantitatively introduce the marginal prediction effect improvement effect of the second data source owned by the second terminal device on the target model training through a statistical test, in a specific implementation, the preset test space may be configured according to the first AUC parameter, the second AUC parameter, the standard deviation, the preset confidence level α (which may also be referred to as a significance level), and the like.

In one embodiment, the preset check interval may be specifically configured according to the following equation:

wherein the content of the first and second substances,

is a first parameter of the AUC, which is,

is a second AUC parameter which is a function of,

is standard deviation, z_1-α/2Cumulatively calculating a probability density value equal to z for a standard normal distribution_1-α/2Z score of (c).

In one embodiment, the specific value of the preset confidence level may be flexibly set according to specific application scenarios and accuracy requirements.

In one embodiment, the magnitude of the promotion effect on the marginal prediction effect of the target model training after the introduction of the second data source owned by the second terminal device can be measured in a quantitative manner through the test result. For example, whether the improvement effect of the second data source owned by the second terminal device on the marginal prediction effect of the training target model is significant or not can be judged according to the above-mentioned check result, and whether the training target model of the first terminal device has a higher data value or not can be judged, so that whether the first terminal device cooperates with the second terminal device or not can be determined, and valuable guidance and reference can be provided for how to reasonably determine a specific cooperation mode and a cooperation protocol.

In an embodiment, the determining, according to the test result, a contribution degree of a second data source owned by a second terminal device to a joint training target model may include the following steps: and according to the inspection result, determining that the contribution degree of a second data source owned by the second terminal equipment to the joint training target model meets the preset requirement under the condition that the marginal prediction effect promotion parameter value is determined to be outside the preset inspection interval. After the contribution degree is determined to meet the preset requirement, it can be determined that the marginal prediction effect improvement effect of a second data source owned by the second terminal device on the training target model is large. Accordingly, the second data source is of relatively low value to the first terminal device.

And on the contrary, according to the inspection result, under the condition that the marginal prediction effect promotion parameter value is determined to be in the preset inspection interval, determining that the contribution degree of a second data source owned by the second terminal device to the joint training target model does not meet the preset requirement. After the contribution degree is determined to be not in accordance with the preset requirement, it can be determined that the marginal prediction effect improvement effect of the second data source owned by the second terminal device on the training target model is small. Accordingly, the second data source is of relatively low value to the first terminal device.

In one embodiment, it is contemplated that in a longitudinal federated learning-based (supervised learning) joint training (modeling) scenario, the tag data may be binary class tag data (i.e., containing two different types of tags) and owned by a single initiator or a single data party. Wherein the test data are respectively located at C₁And C₂Two tag groups corresponding to different data tags.

With reference to the algorithm code shown in fig. 3, in the case where no second terminal device participates (denoted as r ═ 0), the data labels of the test data belonging to the two groups can be predicted by using the comparison model, and the obtained corresponding prediction probability values are respectively denoted as

And

as a first set of prediction probabilities.

When the second terminal device participates (r is 1), the predicted probability values of the data tags of the test data to which the two groups belong, which are determined by integrating the first processing result and the second processing result, may be respectively referred to as:

and

as a second set of prediction probabilities.

Then, for the above two cases, the area AUC parameter under the corresponding receiver operating characteristic curve (ROC) can be calculated and recorded as

(i.e., first AUC parameter) and

(i.e., the second AUC parameter). Specifically, the calculation can be performed according to the following formula:

wherein the content of the first and second substances,

is a haverside step function (Heaviside step function), which has the following characteristics:

and II (·) is an indicative function (indicator function), specifically, if an event a occurs, the indicative function II (a) is 1, otherwise, the indicative function II (a) is 0.

Further, a vector L ═ (1, -1) may be defined^TAnd respectively calculating a covariance matrix S and a standard deviation based on the AUC according to the first AUC parameter and the second AUC parameter

Specifically, the X structure components in the two cases of the second terminal device can be calculated according to the first group of probability values and the second group of probability values respectively

(may be referred to as first component data) and a Y-structural component

(which may be referred to as second component data), the specific formula is as follows:

then, two 2 × 2 matrices are defined, respectively, as first matrices

And a second matrix

Wherein, the specific values of the specific elements in the two matrices, for example, the (r, s) -th element, can be calculated according to the following formula:

according to the above formula, each element in the matrix can be determined, and then the first matrix and the second matrix can be obtained.

Finally, let vector L be (1, -1)^TFirstly, calculating a covariance matrix S based on the first matrix and the second matrix; further calculating the corresponding standard deviation according to the covariance matrix

The formula used in the specific calculation is as follows:

and calculating a difference value according to the first AUC parameter and the second AUC parameter:

and construct a statistic based on the above differences

And the marginal prediction effect promotion parameter is used as a marginal prediction effect promotion parameter of the second data source for the target model training.

The significance level (or referred to as a preset confidence level) α may then be set according to the specific application scenario and accuracy requirements. And configuring a preset check interval according to the significance level and the marginal prediction effect promotion parameter of the second data source for the target model training, and recording the preset check interval as

Wherein z is_1-α/2Means that the standard normal distribution cumulative probability density value is equal to the z-score (z score) at 1- α/2.

And then, checking whether the marginal prediction effect promotion parameter of the second data source for the target model training is located in the preset check interval to obtain a corresponding check result.

According to the test result, if it is determined that the U value is in the interval

And the two AUC values are not different statistically, which means that the introduction of the second terminal equipment in federal learning has no significant effect on improving the prediction effect of the model.

On the contrary, according to the checking result, if it is determined that the U value is in the interval

Besides, the two AUC values are statistically significantly different, which means that the introduction of the second terminal device in federal learning has a significant effect on improving the model prediction effect.

Therefore, the corresponding test result can be accurately determined through the method, so that the marginal prediction effect improvement effect of a second data source owned by the second terminal device on the target model training and the data value of the second data source are introduced under the combined training scene of longitudinal federal learning, and the contribution degree of the second terminal device can be accurately judged.

In this embodiment, under a joint training scenario of horizontal federal learning, the first terminal device and the second terminal device participating in joint training may hold a complete target model. But data tags are often held separately by both parties. In addition, under the combined training scene of horizontal federal learning, most of the data of the first data source and the data of the second data source are data with high coincidence degree of feature data.

Based on the above characteristics, in a case that it is determined that the matched processing rule is a processing rule based on horizontal federal learning, the above processing rule is performed with the second terminal device according to the matched processing rule, based on the comparison model, the target model, and the test data, so as to determine an edge prediction effect improvement parameter of the second data source owned by the second terminal device for the target model training, and in a specific implementation, the following contents may be included:

s1: processing the first test data by using the comparison model to obtain a first group of prediction probabilities;

s2: processing the first test data by using the target model to obtain a second group of prediction probabilities;

s3: performing local operation according to the first group of prediction probabilities and the second group of prediction probabilities to obtain first local component data;

s4: according to the first group of prediction probabilities and the second group of prediction probabilities, performing interactive operation with second terminal equipment through encryption communication to obtain first interactive component data;

s5: constructing a first data body according to the first local component data and the first interactive component data, and encrypting the first data body;

s6: sending the encrypted first data volume to an intermediate server; and the intermediate server is used for determining a marginal prediction effect promotion parameter of the second data source for the target model training according to the encrypted first data body and the encrypted second data body from the second terminal device.

In one embodiment, in particular, it is contemplated that in a horizontal federally learned (supervised learning) joint training (modeling) scenario, test data may be located at C, respectively₁And C₂Two groups (e.g. C)₁Representing a breach of contract test data set corresponding to a breach of contract label, C₂Representing a non-default test data set, corresponding to a non-default label) are owned by a plurality of data parties, respectively. At this time, the owner of a single classification label data can only see the local model prediction result (i.e. the prediction probability of the data label of the belonged test data), but cannot see the prediction probabilities of other test data.

Based on the above, it is possible to participate in specific data processing by introducing an intermediate coordination arbitrator (e.g., intermediate server) based on an encryption algorithm (e.g., homomorphic encryption algorithm, etc.).

Specifically, it is assumed that the number of terminal devices having data tags participating in the joint training is Q, and the terminal device number set is Q. For a simpler example, the value of Q may be 2, and the corresponding terminal devices having data tags participating in the joint training may only include both the first terminal device and the second terminal device.

Wherein for an existing data party (e.g. the first terminal device) q, which owns the data tag, at C₁And C₂The number of test data of the two groups is m respectively_qAnd n_qThe corresponding test data set of numbers may be

And

in case of no participation of the second terminal device, for the first terminal device q possessing the data label, it can be C₁And C₂The data labels of the test data in the two groups are predicted, and the corresponding prediction probability values can be respectively recorded as:

and

(i.e., the first set of predicted probabilities). Under the condition that the second terminal equipment participates, the prediction probability values of the data labels of the test data belonging to the two groups are respectively recorded as:

and

(i.e., the second set of prediction probabilities). Here, C₁Group C and₂the total number of samples of a group can be individually counted as

And

C₁group C and₂the number sets of all test data of a group are respectively denoted by

And

the predicted probability values can be encrypted locally to obtain encrypted values

And

specifically, when the first AUC parameter and the second AUC parameter are calculated, the corresponding first AUC parameter and second AUC parameter can be calculated according to the following equations for two situations of whether the second terminal device participates according to the first group of prediction probabilities and the second group of predictions on different terminal devices, and the calculated first AUC parameter and second AUC parameter are recorded as

And

in addition, the X structural component in two situations of whether the second terminal equipment participates or not

And Y structural component

Can be calculated according to the following equation:

in the above formula, the first term of the X structural component

And the first term of the Y structural component

May be calculated locally on the data side and may be referred to as a local structure component (e.g., a first local component data calculated locally at a first terminal device, a second local component data calculated locally at a second terminal device), belonging to a local intermediate calculation result. And the second term of the structural component of X

And a second term of the Y structural component

The structural components of the interaction (e.g., first interaction component data obtained by a first terminal device through an interactive operation with a second terminal device, and second interaction component data obtained by the second terminal device through an interactive operation with the first terminal device) will be calculated through the communication interaction between the participants.

Referring to the algorithm code shown in fig. 4, in a first module (including steps 4 to 12) in the algorithm code, each terminal device participating in the joint training may calculate a local structural component as respective local component data.

In a second module (including the steps 13 to 23) of the algorithm code, each terminal device participating in the joint training carries out direct encryption communication interaction with other terminal devices respectively, and calculates interactive structural components to obtain respective interactive component data; finally, each terminal device may aggregate the respective local component data and the interactive component data (including steps 24 to 29).

In a third module (step 31 to step 33) of the algorithm code, each terminal device participating in the joint training may construct a corresponding data body according to the respective local component data and the interactive component data. For example, the first terminal device may construct a first data volume based on the first local component data and the first interaction component data. The second terminal device may construct the first data volume based on the second local component data and the second interactive component data.

In addition, each terminal device can encrypt respective data body and encrypt the data body

And

and sending to the intermediate server (including step 33). Thereby, the security of the data can be further improved.

The intermediate server decrypts the received encrypted data body, and calculates corresponding first AUC parameter and second AUC parameter respectively for two situations of whether a second terminal device participates, and records the first AUC parameter and the second AUC parameter as

And

then, the vector L is defined as (1-1), and the covariance matrix S and the standard deviation are calculated based on the above result

The specific formula is as follows:

and construct a statistic based on the above differences

The significance level (or preset confidence level) a can then be set according to the specific situation and accuracy requirements. And configuring a preset check interval according to the significance level and the marginal prediction effect promotion parameter of the second data source for the target model training, and recording the preset check interval as

Therefore, the corresponding test result can be accurately determined through the method, so that the marginal prediction effect improvement effect of a second data source owned by the second terminal device on the target model training and the data value of the second data source are introduced under the combined training scene of horizontal federal learning, and the contribution degree of the second terminal device can be accurately judged.

In conclusion, the method can be simultaneously applied to two different federal learning scenarios of longitudinal federal learning and transverse federal learning, and the effect of introducing the second terminal equipment to the marginal prediction effect improvement of the target model training is accurately quantified by calculating and utilizing the inspection result of the second terminal equipment to the marginal prediction effect improvement parameter, so that the contribution degree of the second terminal equipment can be accurately measured.

In an embodiment, under the condition that the matched processing rule is determined to be the processing rule based on the federal migration learning, according to the matched processing rule, according to the data processing characteristics in the combined training scenario of the federal migration learning, corresponding data processing is performed on the second terminal device based on the comparison model, the target model and the test data, so that the marginal prediction effect improvement parameter of the second data source owned by the second terminal device on the target model training in the combined training scenario of the federal migration learning is accurately determined, and errors are reduced. And then, configuring a preset inspection interval in a similar manner, and determining a corresponding inspection result according to the marginal prediction effect promotion parameter trained by the target model and the preset inspection interval.

In an embodiment, after determining the contribution degree of the second data source owned by the second terminal device to the joint training target model, when the method is implemented, the following may be further included: determining reward data aiming at the second terminal equipment according to the contribution degree; and sending corresponding reward data to the second terminal equipment.

In one embodiment, the intermediate server may further generate and send corresponding reward data to the second terminal device according to the contribution degree of the second terminal device, so as to encourage other terminal devices to actively use their own data sources to participate in joint training with other terminal devices.

In contrast, when detecting that the contribution degree of the second terminal device is low, the intermediate server may further generate and send penalty data to the second terminal device, so as to reversely encourage other terminal devices to actively use their own data sources to participate in joint training with other terminal devices.

In an embodiment, before determining the partner, the first terminal device may further obtain, in the manner of the above-described contribution, sample data of the first data source and sample data of a data source owned by the undetermined terminal device, and further may calculate, according to the sample data of the two parties, a test result of a marginal prediction effect improvement parameter, trained on the target model, of the data source owned by the undetermined terminal device as a value evaluation result of joint training of the undetermined terminal device. And then the first terminal equipment can accurately screen out the undetermined terminal equipment with a data source with higher data value and better effect from the plurality of undetermined terminal equipment according to the value evaluation result to serve as the second terminal equipment meeting the requirement for cooperation, so that blind searching of a partner is avoided, and the effect and accuracy of the target model obtained through joint training can be improved.

In one embodiment, the object model may specifically include a prediction model for predicting a data tag to which the user belongs in a financial business scenario. Correspondingly, the data tag may specifically include a blacklist tag and a whitelist tag. The target model may specifically further comprise a prediction model for predicting whether the consumer is a potential purchasing customer, which may be applied in an online shopping scenario, and the like. Of course, the above listed object models are only illustrative. In specific implementation, the target model may be other types of prediction models according to specific application scenarios and processing requirements. The present specification is not limited to these.

In an embodiment, the first terminal device may specifically include a plurality of sub terminal devices, and each of the plurality of sub terminal devices has a sub data source. That is, on the side of the first terminal device, a plurality of data parties having different sub-data sources may have been present in the past to cooperate with a plurality of sub-terminal devices to combine the joint training model. On the basis, whether a new data party, namely the second terminal device, needs to be introduced or not is determined through the determination method of the contribution degree, so that more data sources are integrated, and a target model with better effect and higher accuracy is obtained through combined training.

As can be seen from the above, in the method for determining the contribution of the joint training target model provided in the embodiment of the present specification, corresponding data processing is performed on the second terminal device based on the comparison model, the target model, and the test data according to the matching processing rule, so as to determine the marginal prediction effect improvement parameter of the second data source owned by the second terminal device for the target model training; determining a test result capable of measuring the marginal prediction effect improvement effect of a second data source introduced into the second terminal device on the target model training by using a preset test interval; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result. Therefore, the technical problem that the contribution degree of the data source participating in the joint training scene cannot be accurately and quantitatively evaluated in the existing method can be effectively solved. And the marginal prediction effect promotion parameters of the second data source owned by the second terminal equipment for the target model training are determined by distinguishing the joint training types adopted in the joint training of the target model and pertinently adopting matched processing rules, so that the marginal prediction effect promotion parameters under different joint training scene types can be determined more efficiently and safely, errors are reduced, and the accuracy of the determined contribution degree is improved.

An embodiment of the present specification further provides a terminal device, including a processor and a memory for storing processor-executable instructions, where the processor, when implemented specifically, may perform the following steps according to the instructions: acquiring a contrast model and a target model; the comparison model is a model obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal device and the second terminal device are subjected to joint training; performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment on the training of the target model; determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced into the second terminal device on the target model training; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

In order to more accurately complete the above instructions, referring to fig. 5, another specific terminal device is provided in the embodiments of the present specification, where the terminal device includes a network communication port 501, a processor 502, and a memory 503, and the above structures are connected by an internal cable, so that the structures may perform specific data interaction.

The network communication port 501 may be specifically configured to obtain a comparison model and a target model; the comparison model is obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is obtained under the condition that the first terminal device and the second terminal device are subjected to joint training.

The processor 502 may be specifically configured to perform corresponding data processing with the second terminal device based on the comparison model, the target model, and the test data according to the matched processing rule, so as to determine a marginal prediction effect improvement parameter of a second data source owned by the second terminal device, where the second data source is trained on the target model; determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

The memory 503 may be specifically configured to store a corresponding instruction program.

In this embodiment, the network communication port 501 may be a virtual port that is bound to different communication protocols, so that different data can be sent or received. For example, the network communication port may be a port responsible for web data communication, a port responsible for FTP data communication, or a port responsible for mail data communication. In addition, the network communication port can also be a communication interface or a communication chip of an entity. For example, it may be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it may also be a bluetooth chip.

In this embodiment, the processor 502 may be implemented in any suitable manner. For example, the processor may take the form of, for example, a microprocessor or processor and a computer-readable medium that stores computer-readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, an Application Specific Integrated Circuit (ASIC), a programmable logic controller, an embedded microcontroller, and so forth. The description is not intended to be limiting.

In this embodiment, the memory 503 may include multiple layers, and in a digital system, the memory may be any memory as long as it can store binary data; in an integrated circuit, a circuit without a physical form and with a storage function is also called a memory, such as a RAM, a FIFO and the like; in the system, the storage device in physical form is also called a memory, such as a memory bank, a TF card and the like.

The present specification further provides a computer storage medium based on the method for determining the contribution degree of the joint training target model, where the computer storage medium stores computer program instructions, and when the computer program instructions are executed, the computer program instructions implement: acquiring a contrast model and a target model; the comparison model is obtained under the condition that the first terminal equipment and the second terminal equipment are not subjected to joint training, and the target model is obtained under the condition that the first terminal equipment and the second terminal equipment are subjected to joint training; performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment for the training of the target model; determining a corresponding test result according to the marginal prediction effect promotion parameter and a preset test interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training; and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

In this embodiment, the storage medium includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Cache (Cache), a Hard Disk Drive (HDD), or a Memory Card (Memory Card). The memory may be used to store computer program instructions. The network communication unit may be an interface for performing network connection communication, which is set in accordance with a standard prescribed by a communication protocol.

In this embodiment, the functions and effects specifically realized by the program instructions stored in the computer storage medium can be explained by comparing with other embodiments, and are not described herein again.

Referring to fig. 6, on a software level, the embodiment of the present specification further provides a device for determining a contribution degree of a joint training target model, and the device may specifically include the following structural modules.

The obtaining module 601 may be specifically configured to obtain a comparison model and a target model; the comparison model is a model obtained under the condition that the first terminal device and the second terminal device are not subjected to joint training, and the target model is a model obtained under the condition that the first terminal device and the second terminal device are subjected to joint training;

the processing module 602 may be specifically configured to perform, according to a matched processing rule, corresponding data processing with the second terminal device based on the comparison model, the target model, and the test data, so as to determine a marginal prediction effect improvement parameter of a second data source owned by the second terminal device, where the second data source is trained on the target model;

the inspection module 603 is specifically configured to determine a corresponding inspection result according to the marginal prediction effect improvement parameter and a preset inspection interval; the detection result is used for measuring and introducing a marginal prediction effect improvement effect of a second data source owned by second terminal equipment on target model training;

the determining module 604 may be specifically configured to determine, according to the test result, a contribution degree of a second data source owned by the second terminal device to the joint training target model. It should be noted that, the units, devices, modules, etc. illustrated in the above embodiments may be implemented by a computer chip or an entity, or implemented by a product with certain functions. For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, and the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

As can be seen from the above, the device for determining the contribution of the joint training target model provided in the embodiments of the present specification can effectively solve the technical problem that the contribution of the data source participating in the joint training scene cannot be accurately quantitatively evaluated in the existing method

Although the present specification provides method steps as described in the examples or flowcharts, additional or fewer steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one manner of performing the steps in a great number of orders, and does not represent the only order of execution. When an apparatus or client product in practice executes, it may execute sequentially or in parallel (e.g., in a parallel processor or multithreaded processing environment, or even in a distributed data processing environment) according to the embodiments or methods shown in the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, article, or apparatus that comprises the recited elements is not excluded. The terms first, second, etc. are used to denote names, but not any particular order.

Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented entirely by logically programming method steps such as logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.

This description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

From the above description of the embodiments, it is clear to those skilled in the art that the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions in the present specification may be essentially embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments in the present specification.

The embodiments in the present specification are described in a progressive manner, and the same or similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The description is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.

While the specification has been described with examples, those skilled in the art will appreciate that there are numerous variations and permutations of the specification that fall within the spirit of the specification, and it is intended that the appended claims include such variations and modifications without departing from the spirit of the specification.

Claims

1. A method for determining contribution of a joint training target model is characterized by comprising the following steps:

performing corresponding data processing with the second terminal equipment based on the comparison model, the target model and the test data according to the matched processing rule to determine a marginal prediction effect promotion parameter of a second data source owned by the second terminal equipment on the target model training;

and determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

2. The method of claim 1, wherein after determining a contribution degree of a second data source owned by a second terminal device to the joint training target model, the method further comprises:

and sending corresponding reward data to the second terminal equipment.

3. The method of claim 1, wherein the matching processing rule comprises: processing rules based on horizontal federal learning, processing rules based on vertical federal learning, or processing rules based on federal migration learning.

4. A method according to claim 3, characterized in that the matching processing rule is determined in the following way:

and determining a matched processing rule according to the joint training type.

5. The method of claim 3, wherein in the case that the matching processing rule is determined to be a processing rule based on longitudinal federated learning, the target model comprises a first model held by a first terminal device and a second model held by a second terminal device.

6. The method according to claim 5, wherein in a case that it is determined that the matched processing rule is a processing rule based on longitudinal federal learning, performing corresponding data processing with the second terminal device according to the matched processing rule based on the comparison model, the target model, and the test data to determine a marginal prediction effect improvement parameter for target model training of a second data source owned by the second terminal device, includes:

and determining a marginal prediction effect promotion parameter of the second data source for the target model training according to the first group of prediction probabilities and the second group of prediction probabilities.

7. The method of claim 6, wherein obtaining the second processing result from the second terminal device comprises:

8. The method of claim 6, wherein determining the marginal predictive effect improvement parameter of the second data source for the target model training according to the first set of predictive probabilities and the second set of predictive probabilities comprises:

9. The method of claim 8, wherein the predetermined inspection interval is configured according to the following equation:

wherein the content of the first and second substances,

is a first parameter of the AUC, which is,

is a second AUC parameter which is a function of,

is standard deviation, z_1-α/2The cumulative probability density value is equal to z for the standard normal distribution_1-α/2And (c) a z score of (a) is a preset confidence level.

10. The method of claim 1, wherein determining, according to the test result, a contribution degree of a second data source owned by a second terminal device to the joint training target model comprises:

11. The method according to claim 3, wherein in a case where the matching processing rule is determined to be a processing rule based on horizontal federal learning, the test data acquired by the first terminal device is first test data, and the test data acquired by the second terminal device is second test data.

12. The method according to claim 11, wherein in a case that the matched processing rule is determined to be a processing rule based on horizontal federal learning, performing corresponding data processing with the second terminal device according to the matched processing rule based on the comparison model, the target model and the test data to determine a marginal prediction effect improvement parameter for target model training of a second data source owned by the second terminal device, includes:

performing local operation according to the first group of prediction probabilities and the second group of prediction probabilities to obtain first local component data;

13. The method of claim 1, wherein the goal model comprises a prediction model for predicting data tags to which the user belongs, the data tags comprising blacklist tags and whitelist tags.

14. The method of claim 1, wherein the first terminal device comprises a plurality of sub-terminal devices, each having a sub-data source.

15. The method of claim 1, wherein after determining, according to the test result, a contribution degree of a second data source owned by a second terminal device to a joint training target model, the method further comprises:

16. An apparatus for determining a contribution of a joint training target model, comprising:

the inspection module is used for determining a corresponding inspection result according to the marginal prediction effect promotion parameter and a preset inspection interval; the detection result is used for measuring the marginal prediction effect improvement effect of a second data source introduced to the second terminal device on the target model training;

and the determining module is used for determining the contribution degree of a second data source owned by the second terminal device to the joint training target model according to the test result.

17. A terminal device comprising a processor and a memory for storing processor-executable instructions which, when executed by the processor, implement the steps of the method of any one of claims 1 to 15.

18. A computer-readable storage medium having computer instructions stored thereon which, when executed, implement the steps of the method of any one of claims 1 to 15.