CN112288088A

CN112288088A - Business model training method, device and system

Info

Publication number: CN112288088A
Application number: CN202011585759.2A
Authority: CN
Inventors: 陈超超; 周俊; 王力
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2021-01-29
Anticipated expiration: 2040-12-29
Also published as: CN112288088B

Abstract

Embodiments of the present specification provide methods, apparatuses, and systems for training a business model via a plurality of member devices. The first and second member devices have first and second data, respectively, the first and second data constitute a training sample set for model training in a vertically sliced manner, and the first member device has label data of the training sample. And in each circulation, each member device is cooperated, and the current business model is trained by using the current training sample and the model prediction result of the current training sample is obtained. And determining a first training sample with the largest prediction error in the current training samples according to the model prediction result at the first member equipment, and sending the sample identification of the first training sample to each second member equipment. The member devices cooperate to select a second training sample, similar to the first training sample, from the unused training samples as a current training sample for the next round of processing.

Description

Business model training method, device and system

Technical Field

Embodiments of the present specification relate generally to the field of artificial intelligence, and more particularly, to a method, apparatus, and system for training a business model via first and second member devices.

Background

Machine learning techniques are widely applied in various business application scenarios. In a business application scenario, a machine learning model is used as a business model to perform various business prediction services, such as classification prediction, business risk prediction, and the like. In many cases, the business model requires model training using business data of multiple data owners. Multiple data owners (e.g., e-commerce companies, courier companies, and banks) each own a different portion of the feature data used to train the business model. The multiple data owners generally want to use each other's data together to train the business model uniformly, but do not want to provide their respective data to other data owners to prevent their own data from being leaked.

In view of the above situation, a business model training method capable of protecting data security is proposed, which can cooperate with a plurality of data owners to train a business model for the plurality of data owners to use while ensuring the data security of each of the plurality of data owners.

Disclosure of Invention

In view of the foregoing, embodiments of the present specification provide a method, apparatus and system for training a business model via a plurality of member devices. By using the method, the device and the system, the business model with good model performance can be trained by using the training sample set with less labeled data.

According to an aspect of embodiments herein, there is provided a method for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the method being performed by the first member device, the method comprising: the following processes are executed in a loop until a loop end condition is satisfied: the method comprises the steps that the method is cooperated with each second member device, a current business model is trained by using a current training sample, and a model prediction result of the current training sample is obtained according to the current business model; determining a first training sample according to the model prediction result, and sending a sample identifier of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample; and selecting, in cooperation with each second member device, a second training sample similar to the first training sample from unused training samples of the training sample set, wherein the second training sample is used as a current training sample for a next cycle process when the cycle end condition is not satisfied.

Optionally, in one example of the above aspect, the second training sample may be a training sample having a maximum sample similarity with the first training sample.

Optionally, in an example of the above aspect, the number of samples of similar training samples in the second training samples for each first training sample is balanced.

Optionally, in an example of the above aspect, in cooperation with each second member device, selecting a second training sample similar to the first training sample from the unused training samples of the training sample set includes: for each first training sample, calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with each second member device, wherein the first member device and the second member device respectively have similarity slicing data of the sample similarity of each unused training sample; and in cooperation with each second member device, according to the similarity slicing data of each unused training sample, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples by using a secret sharing-based comparison protocol, and using the predetermined number of training samples as second training samples corresponding to the first training sample.

Optionally, in one example of the above aspect, the sample similarity comprises one of a sample similarity based on euclidean distance, a sample similarity based on manhattan distance, a sample similarity based on chebyshev distance, a sample similarity based on minkowski distance, a sample similarity based on hamming distance, and a sample similarity based on cosine similarity.

Optionally, in one example of the above aspect, the first data and the second data comprise business data based on text data, image data, and/or voice data.

According to another aspect of embodiments of the present specification, there is provided a method for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the method being performed by the second member device, the method comprising: the following processes are executed in a loop until a loop end condition is satisfied: the method comprises the steps that a current training sample is used for training a current business model in cooperation with a first member device and other second member devices, and a model prediction result of the current training sample is obtained according to the current business model; receiving sample identifications of first training samples from a first member device, wherein the first training samples are a preset number of training samples with maximum prediction errors in the current training samples determined at the first member device according to the model prediction result; selecting, in cooperation with the first member device and the other second member devices, a second training sample similar to the first training sample from the unused training samples of the training sample set, wherein the second training sample is used as a current training sample for a next cycle process when the cycle end condition is not satisfied.

Optionally, in one example of the above aspect, the second training sample is the training sample having the greatest sample similarity to the first training sample.

Optionally, in an example of the above aspect, in cooperation with the first member device and the other second member devices, selecting a second training sample similar to the first training sample from the unused training samples of the training sample set comprises: for each first training sample, calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with first member equipment and other second member equipment, wherein the first member equipment and the second member equipment respectively have similarity segmentation data of the sample similarity of each unused training sample; and in cooperation with the first member device and other second member devices, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples as second training samples corresponding to the first training sample by using a secret sharing-based comparison protocol according to the similarity slicing data of each unused training sample.

According to another aspect of embodiments of the present specification, there is provided a method for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the method being performed jointly by the first and second member devices, the method comprising: the following processes are executed in a loop until a loop end condition is satisfied: the first member equipment cooperates with each second member equipment, a current business model is trained by using a current training sample, and a model prediction result of the current training sample is obtained according to the current business model; determining a first training sample according to the model prediction result at a first member device, and sending a sample identifier of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample; and the first member device and each second member device cooperate to select a second training sample similar to the first training sample from the unused training samples in the training sample set, wherein the second training sample is used as a current training sample of the next cycle process when the cycle end condition is not met.

According to another aspect of embodiments of the present specification, there is provided an apparatus for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the apparatus applied to the first member device, the apparatus comprising: at least one processor, a memory coupled with the at least one processor, and a computer program stored in the memory, the at least one processor executing the computer program to implement: the following processes are executed in a loop until a loop end condition is satisfied: the method comprises the steps that the method is cooperated with each second member device, a current business model is trained by using a current training sample, and a model prediction result of the current training sample is obtained according to the current business model; determining a first training sample according to the model prediction result, and sending a sample identifier of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample; and selecting, in cooperation with each second member device, a second training sample similar to the first training sample from unused training samples of the training sample set, wherein the second training sample is used as a current training sample for a next cycle process when the cycle end condition is not satisfied.

Optionally, in one example of the above aspect, the at least one processor executes the computer program to implement: for each first training sample, calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with each second member device, wherein the first member device and the second member device respectively have similarity slicing data of the sample similarity of each unused training sample; and in cooperation with each second member device, according to the similarity slicing data of each unused training sample, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples by using a secret sharing-based comparison protocol, and using the predetermined number of training samples as second training samples corresponding to the first training sample.

According to another aspect of embodiments of the present specification, there is provided an apparatus for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the apparatus being applied to the second member device, the apparatus comprising: at least one processor, a memory coupled with the at least one processor, and a computer program stored in the memory, the at least one processor executing the computer program to implement: the following processes are executed in a loop until a loop end condition is satisfied: the method comprises the steps that a current training sample is used for training a current business model in cooperation with a first member device and other second member devices, and a model prediction result of the current training sample is obtained according to the current business model; receiving sample identifications of first training samples from a first member device, wherein the first training samples are a preset number of training samples with maximum prediction errors in the current training samples determined at the first member device according to the model prediction result; selecting, in cooperation with the first member device and the other second member devices, a second training sample similar to the first training sample from the unused training samples of the training sample set, wherein the second training sample is used as a current training sample for a next cycle process when the cycle end condition is not satisfied.

Optionally, in one example of the above aspect, the at least one processor executes the computer program to implement: for each first training sample, calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with first member equipment and other second member equipment, wherein the first member equipment and the second member equipment respectively have similarity segmentation data of the sample similarity of each unused training sample; and in cooperation with the first member device and other second member devices, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples as second training samples corresponding to the first training sample by using a secret sharing-based comparison protocol according to the similarity slicing data of each unused training sample.

According to another aspect of embodiments herein, there is provided a system for training a business model via a plurality of member devices, comprising: a first member device comprising means for training a business model via a plurality of member devices as described above; and at least one second member device comprising the apparatus for training a business model via a plurality of member devices as described above, wherein the first member device has first data, the second member device has second data, the first and second data compose a training sample set for model training in a vertically sliced manner, and the first member device has label data of the training sample.

According to another aspect of embodiments of the present specification, there is provided a computer readable storage medium storing a computer program for execution by a processor to implement the method performed on the first member device side for training a business model via a plurality of member devices as described above.

According to another aspect of embodiments of the present description, there is provided a computer program product comprising a computer program for execution by a processor to implement the method performed on the first member device side for training a business model via a plurality of member devices as described above.

According to another aspect of embodiments of the present specification, there is provided a computer readable storage medium storing a computer program for execution by a processor to implement the method performed on the second member device side for training a business model via a plurality of member devices as described above.

According to another aspect of embodiments herein, there is provided a computer program product comprising a computer program for execution by a processor to implement the method for training a business model via a plurality of member devices performed on a second member device side as described above.

Drawings

A further understanding of the nature and advantages of the present disclosure may be realized by reference to the following drawings. In the drawings, similar components or features may have the same reference numerals.

FIG. 1 illustrates an example schematic of an architecture of a business model training system for training a business model via a plurality of member devices in accordance with an embodiment of the present description.

Fig. 2 shows a schematic diagram of an example of vertically sliced training sample data according to an embodiment of the present specification.

FIG. 3 illustrates a flow diagram of a business model training method for training a business model via a plurality of member devices in accordance with an embodiment of the present description.

FIG. 4 shows a flowchart of one example of a training sample selection process for business model training in accordance with an embodiment of the present description.

Fig. 5 shows a block diagram of a business model training apparatus applied to a first member device for training a business model via a plurality of member devices according to an embodiment of the present description.

Fig. 6 is a block diagram illustrating an example of an implementation of the training sample selection unit of fig. 5.

Fig. 7 shows a block diagram of a business model training apparatus applied to a second member device for training a business model via a plurality of member devices according to an embodiment of the present description.

FIG. 8 illustrates an example schematic diagram of a business model training apparatus based on a computer implementation on a first member device side in accordance with an embodiment of the present description.

FIG. 9 illustrates an example schematic diagram of a computer-implemented business model training apparatus on a second member device side in accordance with an embodiment of the present description.

Detailed Description

The subject matter described herein will now be discussed with reference to example embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and thereby implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as needed. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with respect to some examples may also be combined in other examples.

As used herein, the term "include" and its variants mean open-ended terms in the sense of "including, but not limited to. The term "based on" means "based at least in part on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first," "second," and the like may refer to different or the same object. Other definitions, whether explicit or implicit, may be included below. The definition of a term is consistent throughout the specification unless the context clearly dictates otherwise.

In this specification, the term "business model" refers to a machine learning model applied in a business scenario for business prediction services, such as machine learning models for classification prediction, business risk prediction, and the like. Examples of machine learning models may include, but are not limited to: linear regression models, logistic regression models, neural network models, decision tree models, support vector machines, and the like.

The specific implementation of the business model depends on the business scenario applied. For example, in an application scenario where the business model is applied to classify a user, the business model is implemented as a user classification model. Accordingly, the user characteristic data of the user to be classified can be subjected to user classification prediction according to the service model. In an application scenario where the business model is applied to business risk prediction for business transactions occurring on a business system, the business model is implemented as a business risk prediction model. Accordingly, business risk prediction can be performed on the business transaction characteristic data of the business transaction according to the business model.

Before business prediction service is carried out by using the business model, the business model training is carried out by using the mark sample. The larger the scale of the marked sample is, the better the model performance of the trained business model is. If the training sample is too large, the training time is too much. How to select a proper training sample from a training sample set with a large scale to perform business model training becomes a problem to be solved urgently.

In view of the foregoing, embodiments of the present specification provide a business model training scheme for training a business model via a plurality of member devices. In the business model training scheme, a business model training system includes a first member device and at least one second member device. The first and second member devices have first and second data, respectively, the first and second data constitute a training sample set for model training in a vertically sliced manner, and the first member device has label data of the training sample. And in each circulation, each member device is cooperated, and the current business model is trained by using the current training sample and the model prediction result of the current training sample is obtained. And determining a first training sample with the largest prediction error in the current training samples according to the model prediction result at the first member equipment, and sending the sample identification of the first training sample to each second member equipment. The member devices cooperatively use secret sharing-based multiparty security computation to select a second training sample similar to the first training sample from unused training samples as a current training sample for the next round of processing. By using the business model training scheme, during each cycle training, a training sample with a large prediction error in the current training sample is determined according to the trained current business model, and if the prediction error is large, the sample is not easy to learn by the model, so that a sample similar to the training sample with the large prediction error is selected from unused samples in a training sample set to serve as the current training sample of the next cycle, and the business model is made to learn the sample in the next cycle of model training, thereby improving the model training effect and the model training efficiency.

In embodiments of the present specification, the term "secret sharing" belongs to a cryptographic primitive. In the secret sharing process, original data is split and distributed in a random number mask mode, each piece of distributed data is held by different managers, and a single data holder or data holders with the number less than the protocol regulation number cannot recover the secret. Secret sharing technology is a basic technology for protecting information security and performing security calculation.

For example, if desired on raw data

Secret sharing is performed. First, the corresponding security level parameter is selected and the corresponding finite field is generated (e.g.

). Then, the random numbers are uniformly distributed in a finite field and recorded

. Then, calculate

Will be

To the data owner 1, and

and distributed to the data owner 2. In the following protocol description, data will be described

Is shared as

Wherein, in the step (A),

。

a business model training method, a business model training apparatus, and a business model training system for training a business model via a plurality of member devices according to embodiments of the present specification will be described in detail below with reference to the accompanying drawings.

FIG. 1 illustrates an example schematic of an architecture of a business model training system 100 for training a business model via a plurality of member devices in accordance with an embodiment of the present description.

As shown in FIG. 1, business model training system 100 includes a first member device 110, at least one second member device 120, and a network 130. In the example of FIG. 1, the at least one second member device 120 includes two second member devices 120-1 and 120-2. In other embodiments of the present description, the at least one second member device 120 may include one second member device or more than two second member devices.

In the example of FIG. 1, first member device 110 and second member devices 120-1 and 120-2 are communicatively coupled via network 130, thereby communicating data between each other. In other embodiments of the present description, business model training system 100 may not include network 130, and first member device 110 is directly communicatively coupled with second member devices 120-1 and 120-2.

First and second member devices 110, 120-1, 120-2, respectively, have local data, e.g., first member device 110 has first data and each second member device has second data. In this description, the local data of first member device 110 and second member devices 120-1, 120-2 may include traffic data collected locally by the respective member devices. The business data may include characteristic data of the business object. Examples of business objects may include, but are not limited to, users, goods, events, or relationships. Accordingly, the business data may include, for example, but is not limited to, locally collected user characteristic data, commodity characteristic data, event characteristic data, or relationship characteristic data, such as user characteristic data, business process data, financial transaction data, commodity transaction data, medical health data, and the like. Business data can be applied to business models for model prediction, model training, and other suitable multiparty data joint processing, for example. Further, first and second member devices 110, 120-1, 120-2 may have respective business model data, e.g., model parameters of respective business model structures.

In this specification, the service data may include service data based on text data, image data, and/or voice data. Accordingly, the business model may be applied to business risk identification, business classification, or business decision, etc., based on text data, image data, and/or voice data. For example, the local data may be medical data collected by a hospital, and the business model may be used to perform disease examinations or disease diagnoses. Alternatively, the collected local data may include user characteristic data. Accordingly, the business model may be applied to business risk identification, business classification, business recommendation or business decision, etc. based on user characteristic data. Examples of business models may include, but are not limited to, face recognition models, disease diagnosis models, business risk prediction models, service recommendation models, and so forth.

In an embodiment of the present specification, the first data and the respective second data constitute a training sample set for model training in a vertically sliced manner, and the first member device has label data of the training sample. Fig. 2 shows a schematic diagram of an example of vertically sliced training sample data according to an embodiment of the present specification.

In the example of fig. 2, 2 data parties Alice and Bob are shown, as are the multiple data parties. Each data party Alice and Bob possesses partial feature data of each training sample in all training samples in the training sample set, and for each training sample, the partial feature data possessed by the data party Alice and Bob are combined together to form the complete content of the training sample. For example, assume that the content of a training sample includes label data

And characteristic data

、

After vertical segmentation, the data owner Alice owns the label data of the training sample

And characteristic data

And the data owner Bob owns the characteristic data of the training sample

。

In embodiments of the present description, the first and second member devices may comprise terminal devices or server devices. The server devices may include, but are not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The terminal devices may include, but are not limited to: any one of smart terminal devices such as a smart phone, a Personal Computer (PC), a notebook computer, a tablet computer, an electronic reader, a web tv, and a wearable device.

Further, the first member device 110 and the second member devices 120-1, 120-2 have a business model training means 111, a business model training means 121-1, and a business model training means 121-2, respectively. The business model training device 111, the business model training device 121-1, and the business model training device 121-2 may perform network communication via the network 130 to perform data interaction, whereby a cooperative process is performed to perform a model training process for a business model. The operation and structure of the business model training apparatus 111, the business model training apparatus 121-1, and the business model training apparatus 121-2 will be described in detail below with reference to the drawings.

In some embodiments, the network 130 may be any one or more of a wired network or a wireless network. Examples of network 130 may include, but are not limited to, a cable network, a fiber optic network, a telecommunications network, an intranet, the internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Public Switched Telephone Network (PSTN), a bluetooth network, a zigbee network (zigbee), Near Field Communication (NFC), an intra-device bus, an intra-device line, and the like, or any combination thereof.

FIG. 3 illustrates a flow diagram of a business model training method 300 for training a business model via a plurality of member devices in accordance with an embodiment of the present description. In the embodiment shown in fig. 3, the first member device 110 has first data, each of the second member devices 120 has second data, the first and second data constitute a training sample set for model training in a vertically sliced manner, and the first member device 110 has label data of the training sample. In addition, the first member device 110 and each second member device 120 respectively have a partial business model structure of the business model to be trained, and the partial business model structures of the first member device 110 and each second member device 120 form the business model to be trained in a vertical segmentation manner.

As shown in fig. 3, the first member device 110 performs operations 310 to 350 in a loop in cooperation with the respective second member devices 120 until a loop end condition is satisfied.

Specifically, during each cycle of training, at 310, the first member device 110 cooperates with each second member device 120 to train a current business model using a current training sample, and obtain a model prediction result of the current training sample according to the current business model. At the first training round, the current training sample may be a predetermined number (first predetermined number) of training samples randomly selected from a set of training samples, for example, 1 ten thousand training samples are randomly selected. In the subsequent cycle training, the current training sample is a training sample selected from unused training samples based on the model training result in the previous cycle training process.

In this specification, when performing current business model training, each member device uses a part of feature data of a current training sample of each member device to cooperatively train a current business model (i.e., obtain current model parameters of the business model). Here, the collaborative model training of each member device may adopt various collaborative model training schemes based on data privacy protection, which are applicable in the art, for example, a model training scheme based on Multi-Party security computing (MPC) and other applicable Multi-Party model training schemes based on privacy protection, such as a federal learning scheme.

After the model prediction results for the current training samples are obtained as described above, at 320, a predetermined number (a second predetermined number) of training samples with the largest prediction error are determined as first training samples from the model prediction results for each of the current training samples at first member device 110. Here, the second predetermined number may be pre-specified, for example, 100. The second predetermined number of values may be based on the scale of the training sample set, the training accuracy requirement of the service model, and/or the application scenario of the service model. Then, at 330, the first member device 110 sends the determined sample identification of the first training sample to each second member device 120, so that each second member device 120 can locally determine the corresponding training sample.

At 340, the first member device 110, in cooperation with each of the second member devices 120, selects a second training sample from the unused training samples of the set of training samples that is similar to the first training sample. In the present specification, the term "sample similarity" means that the sample similarity determined based on the sample characteristics exceeds a predetermined threshold.

In one example, the first member device 110, in cooperation with each of the second member devices 120, selects a second training sample from the set of unused training samples that is similar to the first training sample based on a non-secure computational manner. For example, the first member device 110 and each second member device 120 may share their respective feature data, thereby determining a sample similarity between each unused training sample and the first training sample, and selecting the second training sample.

In one example, first member device 110, in cooperation with each second member device 120, selects a second training sample from the unused training samples of the training sample set that is similar to the first training sample based on a multi-party security computation. For example, first member device 110 may select, in cooperation with respective second member devices 120, a second training sample similar to the first training sample from unused training samples of the set of training samples using secret sharing-based multi-party security computation. In other embodiments of the present description, first member device 110 may use other ways of multi-party security computation with respective second member devices 120 to select a second training sample from the unused training samples of the training sample set that is similar to the first training sample, e.g., a homomorphic encryption based multi-party security computation; multiparty security computing based on inadvertent transmissions; a obfuscated circuit-based multi-party security computation; or multi-party secure computing based on a trusted execution environment.

FIG. 4 illustrates a flow diagram of one example 400 of a training sample selection process for business model training in accordance with an embodiment of the present description. In the example shown in fig. 4, the second training sample is the training sample having the greatest sample similarity to the first training sample. Further, the example shown in fig. 4 is a training sample selection process performed for one first training sample. In the case where there are a plurality of first samples, the process shown in fig. 4 is performed once for each training sample.

As shown in fig. 4, at 410, the first member device 110, in cooperation with each of the second member devices 120, calculates a sample similarity of each unused training sample of the training sample set to the first training sample, the first and second member devices having respective similarity segmentation data for the sample similarity of each unused training sample. In one example, the sample similarity may include one of a sample similarity based on euclidean distance, a sample similarity based on manhattan distance, a sample similarity based on chebyshev distance, a sample similarity based on minkowski distance, a sample similarity based on hamming distance, and a sample similarity based on cosine similarity.

For example, in the case where the sample similarity is a sample similarity based on euclidean distance, assume that the first training sample is

The first member device 110 is member device A, the second member device 120 is member devices B and C, member device A has a first training sample

Part of the characteristic data

(i.e., first data), member device B has a first training sample

Part of the characteristic data

(i.e., second data)) And member device C has a first training sample

Part of the characteristic data

(i.e., the second data). For unused training samples in a set of training samples

Member device A has training samples

Part of the characteristic data

Member device B has training samples

Part of the characteristic data

And member device C has training samples

Part of the characteristic data

. Training sample

And

has a Euclidean distance of

Wherein, in the step (A),

as calculated by the member device a,

is calculated by member device B, an

Calculated by the member device C, thus resulting in a sample similarity based on euclidean distance. It is noted here that in the case where the second member device comprises one or more than two member devices, the euclidean distance between each unused training sample and the first training sample may be determined in a similar manner.

At 420, the first member device 110 cooperates with each of the second member devices 120-1, 120-2 to determine, according to the similarity degree sliced data of each unused training sample, a predetermined number of training samples having the greatest sample similarity with the first training sample from the unused training samples as the second training samples corresponding to the first training sample using a secret sharing based comparison protocol.

For example, for any two unused samples in the training sample set

And

assume a first data sample

With two untrained samples

And

respectively of Euclidean distances of

And

wherein

And

is owned by the member device a and,

and

owned by member device B, an

And

owned by member device C. In this case, a comparison protocol based on secret sharing may be used to compare euclidean distances

And

the size of (2). For untrained samples

And

in other words, the sample similarity of training samples having a small euclidean distance is large. In this manner, each unused training sample and the first training sample in the set of training samples can be determined

The magnitude relation between the similarity of the samples, thereby determining the similarity with the first sampleTraining sample

A predetermined number of training samples having the largest sample similarity as first training samples

A corresponding second training sample.

And circularly executing the process for each first training sample, and determining a second training sample corresponding to each first training sample, thereby obtaining a second training sample set corresponding to the determined first training sample set.

Optionally, in an example, the number of samples of the second training samples determined for each first training sample may be constrained, so that the number of samples of corresponding similar training samples (corresponding second training samples) of each first training sample in the second training sample set is equalized, so that the types of samples participating in the next cycle training are equalized, and thus the model training effect of the next cycle training is improved.

After the second training sample is obtained as described above, at 350, it is determined whether a loop over condition is satisfied. In one example, the loop over condition includes whether a predetermined number of loops has been reached. If the preset cycle number is reached, the process is ended, and the trained current business model is output. If the predetermined number of cycles has not been reached, returning to 310, the next cycle training process is performed using the second training sample as the current training sample for the next cycle process. In another example, the operations of 350 may be performed after 310. And after the preset cycle number is reached, ending the process, and outputting the trained current business model. If the predetermined number of cycles has not been reached, operations 320 through 340 continue and operations 310 return after the second training sample is obtained.

In another example, the end-of-loop condition may include determining whether the current predicted difference value is within a predetermined difference value range, such as determining whether the current predicted difference value is less than a predetermined threshold. The end-of-loop condition is satisfied if the current predicted difference is within a predetermined difference range, e.g., the current predicted difference is less than a predetermined threshold. In this case, the operation of 350 may be performed after 310. And after the loop ending condition is met, ending the process, and outputting the trained current business model. If the loop end condition is not satisfied, operations 320 through 340 continue to be performed, and operations 310 return to be performed after the second training sample is obtained.

As described above with reference to fig. 1 to 4, a business model training method and a business model training system for business model training via a plurality of member devices according to an embodiment of the present specification are described.

By using the business model training method, during each cycle training, a training sample with a large prediction error in the current training sample is determined according to the trained current business model, if the prediction error is large, the sample is not easy to learn by the model, and a sample similar to the training sample with the large prediction error is selected from unused samples in the training sample set as the current training sample of the next cycle, so that the business model intensively learns the sample in the next cycle of model training, thereby improving the model training effect and the model training efficiency.

In addition, by using the service model training method, when the service model training is carried out, a plurality of member devices are realized by adopting a service model training mode based on data privacy protection, and the data privacy safety of each member device can be ensured.

In addition, with the business model training method, by using a comparison protocol based on secret sharing to determine the second training sample from the unused training samples, the data privacy security at each member device can be further ensured.

In addition, by using the business model training method, when the second training sample is selected, the number of samples of the second training sample determined for each first training sample is constrained, so that the number of samples of each first training sample in the second training sample set corresponding to similar training samples (corresponding to the second training sample) is balanced, and the types of samples participating in the next cycle training are balanced, thereby improving the model training effect of the next cycle training.

FIG. 5 illustrates a block diagram of a business model training apparatus 500 for training a business model via a plurality of member devices in accordance with an embodiment of the present description. The business model training apparatus 500 is applied to the first member device 110. As shown in fig. 5, the traffic model training apparatus 500 includes a traffic model training unit 510, a traffic model prediction unit 520, a reference sample determination unit 530, a sample identification transmission unit 540, and a training sample selection unit 550.

In performing the business model training, the business model training unit 510, the business model prediction unit 520, the reference sample determination unit 530, the sample identification transmission unit 540, and the training sample selection unit 550 operate cyclically until a cycle end condition is satisfied. The loop-ending condition may include: a predetermined number of cycles is reached or the current predicted difference is within a predetermined difference range.

At each iteration, the business model training unit 510 is configured to train out a current business model using the current training samples in cooperation with the respective second member devices. Then, the business model prediction unit 520 is configured to cooperate with each second member device to obtain a model prediction result of the current training sample according to the current business model.

The reference sample determination unit 530 determines a first training sample, which is a predetermined number of training samples having the largest prediction error among the current training samples, according to the model prediction result of each current training sample. Subsequently, the sample identification transmission unit 540 transmits the sample identification of the first training sample to each of the second member devices.

The training sample selection unit 550 is configured to select, in cooperation with each of the second member devices, a second training sample similar to the first training sample from the unused training samples of the training sample set. When the loop end condition is not satisfied, the second training sample selected by the training sample selection unit 550 is used as the current training sample of the next loop process.

Fig. 6 shows a block diagram of an implementation example of the training sample selection unit 600 in fig. 5. As shown in fig. 6, the training sample selection unit 600 includes a sample similarity calculation module 610 and a training sample selection module 620.

The sample similarity calculation module 610 is configured to calculate sample similarities of respective unused training samples of the training sample set with a first training sample in cooperation with respective second member devices, the first and second member devices having respective similarity shard data of the sample similarity of each unused training sample. The operation of the sample similarity calculation module 610 may refer to the operation described above with reference to 410 of fig. 4.

The training sample selection module 620 is configured to cooperate with each second member device to determine, according to the similarity segmentation data of each unused training sample, a predetermined number of training samples having the greatest sample similarity with the first training sample from the unused training samples as the second training sample corresponding to the first training sample by using a secret sharing based comparison protocol. The operation of the training sample selection module 620 may refer to the operation described above with reference to 420 of fig. 4.

Fig. 7 shows a block diagram of a business model training apparatus 700 applied to a second member device for training a business model via a plurality of member devices according to an embodiment of the present description. As shown in fig. 7, the business model training apparatus 700 includes a business model training unit 710, a business model prediction unit 720, a sample identification receiving unit 730, and a training sample selection unit 740.

In the business model training, the business model training unit 710, the business model prediction unit 720, the sample identification receiving unit 730, and the training sample selection unit 740 operate in a loop until a loop end condition is satisfied. The loop-ending condition may include: a predetermined number of cycles is reached or the current predicted difference is within a predetermined difference range.

At each cycle, the business model training unit 710 is configured to train out a current business model using the current training samples in cooperation with the first member device and the other second member devices. Then, the business model prediction unit 720 cooperates with the first member device and other second member devices to obtain a model prediction result of the current training sample according to the current business model.

The sample identification receiving unit 730 is configured to receive a sample identification of a first training sample from a first member device.

The training sample selection unit 740 is configured to select, in cooperation with the first member device and the other second member devices, a second training sample similar to the first training sample from the unused training samples of the training sample set. When the loop end condition is not satisfied, the second training sample selected by the training sample selection unit 740 is used as the current training sample of the next loop process. It is noted that the training sample selection unit 740 may be implemented using the training sample selection unit shown in fig. 6.

As described above with reference to fig. 1 to 7, a business model training method and a business model training apparatus for training a business model via a plurality of member devices according to an embodiment of the present specification are described. The above data filtering device can be implemented by hardware, and can also be implemented by software, or a combination of hardware and software. In case of a software implementation, the individual units of the business model training apparatus shown in fig. 5 and 7 may be implemented as program modules in a computer program.

FIG. 8 illustrates an example schematic diagram of a computer-implemented business model-based training apparatus 800 at a first member device in accordance with an embodiment of the present description. As shown in FIG. 8, the business model training apparatus 800 may include at least one processor 810, a storage (e.g., non-volatile storage) 820, a memory 830, and a communication interface 840, and the at least one processor 810, the storage 820, the memory 830, and the communication interface 840 are coupled together via a bus 860. The at least one processor 810 executes at least one computer-readable program/instructions (i.e., elements described above as being implemented in software) stored or encoded in memory.

In one embodiment, a computer program is stored in the memory that, when executed, causes the at least one processor 810 to: the following processes are executed in a loop until a loop end condition is satisfied: the current training sample is used for training a current business model in cooperation with each second member device, and a model prediction result of the current training sample is obtained according to the current business model; determining a first training sample according to the model prediction result, and sending the sample identification of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample; and selecting, in cooperation with each second member device, a second training sample similar to the first training sample from the unused training samples of the training sample set, wherein the second training sample is used as a current training sample for a next cycle process when a cycle end condition is not satisfied.

It should be appreciated that the computer programs stored in the memory, when executed, cause the at least one processor 810 to perform the various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present specification.

Fig. 9 illustrates an example schematic diagram of a computer-implemented business model training apparatus 900 at a second member device in accordance with an embodiment of the present description. As shown in fig. 9, the business model training apparatus 900 may include at least one processor 910, a storage (e.g., a non-volatile storage) 920, a memory 930, and a communication interface 940, and the at least one processor 910, the storage 920, the memory 930, and the communication interface 940 are connected together via a bus 960. The at least one processor 910 executes at least one computer-readable program/instructions (i.e., elements described above as being implemented in software) stored or encoded in memory.

In one embodiment, a computer program is stored in the memory that, when executed, causes the at least one processor 910 to: the following processes are executed in a loop until a loop end condition is satisfied: the method comprises the steps that the current training sample is used for training a current business model in cooperation with first member equipment and other second member equipment, and a model prediction result of the current training sample is obtained according to the current business model; receiving sample identifications of first training samples from first member equipment, wherein the first training samples are a preset number of training samples with maximum prediction errors in current training samples determined at the first member equipment according to model prediction results; in conjunction with the first member device and the further second member devices, a second training sample is selected from the unused training samples of the training sample set, which second training sample is similar to the first training sample, wherein the second training sample is used as a current training sample for the next cycle process if the cycle end condition is not met.

It should be appreciated that the computer programs stored in the memory, when executed, cause the at least one processor 910 to perform the various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present description.

According to one embodiment, a program product, such as a computer-readable medium (e.g., a non-transitory computer-readable medium), is provided. The computer-readable medium may have a computer program (i.e., the elements described above as being implemented in software) that, when executed by a processor, causes the processor to perform various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present specification. Specifically, a system or apparatus may be provided which is provided with a readable storage medium on which software program code implementing the functions of any of the above embodiments is stored, and causes a computer or processor of the system or apparatus to read out and execute instructions stored in the readable storage medium.

In this case, the program code itself read from the readable medium can realize the functions of any of the above-described embodiments, and thus the machine-readable code and the readable storage medium storing the machine-readable code form part of the present invention.

Examples of the readable storage medium include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or from the cloud via a communications network.

According to one embodiment, a computer program product is provided that includes a computer program that, when executed by a processor, causes the processor to perform the various operations and functions described above in connection with fig. 1-7 in the various embodiments of the present specification.

It will be understood by those skilled in the art that various changes and modifications may be made in the above-disclosed embodiments without departing from the spirit of the invention. Accordingly, the scope of the invention should be determined from the following claims.

It should be noted that not all steps and units in the above flows and system structure diagrams are necessary, and some steps or units may be omitted according to actual needs. The execution order of the steps is not fixed, and can be determined as required. The apparatus structures described in the above embodiments may be physical structures or logical structures, that is, some units may be implemented by the same physical entity, or some units may be implemented by a plurality of physical entities, or some units may be implemented by some components in a plurality of independent devices.

In the above embodiments, the hardware units or modules may be implemented mechanically or electrically. For example, a hardware unit, module or processor may comprise permanently dedicated circuitry or logic (such as a dedicated processor, FPGA or ASIC) to perform the corresponding operations. The hardware units or processors may also include programmable logic or circuitry (e.g., a general purpose processor or other programmable processor) that may be temporarily configured by software to perform the corresponding operations. The specific implementation (mechanical, or dedicated permanent, or temporarily set) may be determined based on cost and time considerations.

The detailed description set forth above in connection with the appended drawings describes exemplary embodiments but does not represent all embodiments that may be practiced or fall within the scope of the claims. The term "exemplary" used throughout this specification means "serving as an example, instance, or illustration," and does not mean "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described technology. However, the techniques may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form in order to avoid obscuring the concepts of the described embodiments.

The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training a business model via a plurality of member devices, the plurality of member devices including a first member device having first data and at least one second member device each having second data, the first and second data comprising a set of training samples for model training in a vertically sliced manner, and the first member device having label data for the training samples, the method performed by the first member device, the method comprising:

the following processes are executed in a loop until a loop end condition is satisfied:

the method comprises the steps that the method is cooperated with each second member device, a current business model is trained by using a current training sample, and a model prediction result of the current training sample is obtained according to the current business model;

determining a first training sample according to the model prediction result, and sending a sample identifier of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample; and

selecting, in cooperation with each second member device, a second training sample similar to the first training sample from unused training samples of the set of training samples,

wherein the second training sample is used as a current training sample for a next cycle process when the cycle end condition is not satisfied.

2. The method of claim 1, wherein the second training sample is the training sample having the greatest sample similarity to the first training sample.

3. The method of claim 1, wherein the respective first training samples are equalized in number of samples of similar training samples in the second training samples.

4. The method of claim 2, wherein selecting, in conjunction with at least one second member device, a second training sample from the unused training samples of the set of training samples that is similar to the first training sample comprises:

selecting, in cooperation with at least one second member device, a second training sample from unused training samples of the set of training samples that is similar to the first training sample based on a multi-party security computation.

5. The method of claim 4, wherein selecting, in cooperation with at least one second member device, a second training sample from the unused training samples of the set of training samples that is similar to the first training sample based on a multi-party safety computation comprises:

for each of the first training samples, the training samples,

calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with each second member device, wherein the first member device and the second member device respectively have similarity slicing data of the sample similarity of each unused training sample; and

and in cooperation with each second member device, according to the similarity slicing data of each unused training sample, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples as second training samples corresponding to the first training sample by using a secret sharing-based comparison protocol.

6. The method of claim 2, wherein the sample similarity comprises one of:

sample similarity based on euclidean distance;

sample similarity based on manhattan distance;

sample similarity based on Chebyshev distance;

-sample similarity based on minkowski distance;

sample similarity based on hamming distance; and

sample similarity based on cosine similarity.

7. The method of any of claims 1 to 6, wherein the first data and second data comprise traffic data based on text data, image data and/or voice data.

8. A method for training a business model via a plurality of member devices, the plurality of member devices including a first member device having first data and at least one second member device each having second data, the first and second data comprising a set of training samples for model training in a vertically sliced manner, and the first member device having label data for the training samples, the method performed by the second member device, the method comprising:

the method comprises the steps that a current training sample is used for training a current business model in cooperation with first member equipment and other second member equipment, and a model prediction result of the current training sample is obtained according to the current business model;

receiving sample identifications of first training samples from a first member device, wherein the first training samples are a preset number of training samples with maximum prediction errors in the current training samples determined at the first member device according to the model prediction result;

selecting, in cooperation with a first member device and other second member devices, a second training sample similar to the first training sample from unused training samples of the set of training samples,

9. The method of claim 8, wherein the second training sample is the training sample having the greatest sample similarity to the first training sample.

10. The method of claim 8, wherein the respective first training samples are equalized in number of samples of similar training samples in the second training samples.

11. The method of claim 8, wherein selecting, in conjunction with a first member device and other second member devices, a second training sample from the unused training samples of the set of training samples that is similar to the first training sample comprises:

in cooperation with a first member device and other second member devices, a second training sample similar to the first training sample is selected from unused training samples of the training sample set based on a multi-party security computation.

12. The method of claim 11, wherein selecting, in conjunction with a first member device and other second member devices, a second training sample from unused training samples of the set of training samples that is similar to the first training sample based on a multi-party security computation comprises:

for each of the first training samples, the training samples,

calculating the sample similarity of each unused training sample of the training sample set and the first training sample in cooperation with a first member device and other second member devices, wherein the first member device and the second member devices respectively have similarity slicing data of the sample similarity of each unused training sample; and

and in cooperation with the first member device and other second member devices, according to the similarity slicing data of each unused training sample, determining a predetermined number of training samples with the maximum sample similarity with the first training sample from the unused training samples as second training samples corresponding to the first training sample by using a secret sharing based comparison protocol.

13. A method for training a business model via a plurality of member devices, the plurality of member devices including a first member device having first data and at least one second member device each having second data, the first and second data comprising a set of training samples for model training in a vertically sliced manner, and the first member device having label data for the training samples, the method performed jointly by the first and second member devices, the method comprising:

the method comprises the steps that first member equipment and each second member equipment cooperate, a current training sample is used for training a current business model, and a model prediction result of the current training sample is obtained according to the current business model;

determining a first training sample according to the model prediction result at a first member device, and sending a sample identifier of the first training sample to each second member device, wherein the first training sample is a preset number of training samples with the maximum prediction error in the current training sample;

the first member device and each second member device cooperate to select a second training sample similar to the first training sample from the unused training samples of the training sample set,

14. An apparatus for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the apparatus applied to the first member device, the apparatus comprising:

at least one processor for executing a program code for the at least one processor,

a memory coupled to the at least one processor, an

A computer program stored in the memory, the computer program being executable by the at least one processor to implement:

15. The apparatus of claim 14, wherein the second training sample is the training sample having the greatest sample similarity to the first training sample.

16. The apparatus of claim 14, wherein the at least one processor executes the computer program to implement:

in conjunction with each second member device, a second training sample similar to the first training sample is selected from unused training samples of the training sample set based on a multi-party security computation.

17. The apparatus of claim 16, wherein the at least one processor executes the computer program to implement:

for each of the first training samples, the training samples,

18. An apparatus for training a business model via a plurality of member devices, the plurality of member devices including a first member device and at least one second member device, the first member device having first data, each second member device having second data, the first and second data composing a training sample set for model training in a vertically sliced manner, and the first member device having label data of the training sample, the apparatus applied to the second member device, the apparatus comprising:

a memory coupled to the at least one processor, an

the method comprises the steps that a current training sample is used for training a current business model in cooperation with a first member device and other second member devices, and a model prediction result of the current training sample is obtained by using the current business model;

19. The apparatus of claim 18, wherein the second training sample is the training sample having the greatest sample similarity to the first training sample.

20. The apparatus of claim 18, wherein the at least one processor executes the computer program to implement:

21. The apparatus of claim 20, wherein the at least one processor executes the computer program to implement:

for each of the first training samples, the training samples,

22. A system for training a business model via a plurality of member devices, comprising:

a first member device comprising the apparatus of any one of claims 14 to 17; and

at least one second member device, each second member device comprising an apparatus according to any one of claims 18 to 21,

the first member device has first data, each second member device has second data, the first and second data form a training sample set for model training in a vertical segmentation mode, and the first member device has label data of the training sample.

23. A computer-readable storage medium storing a computer program for execution by a processor to implement the method of any one of claims 1 to 7.

24. A computer program product comprising a computer program for execution by a processor to implement the method of any one of claims 1 to 7.

25. A computer-readable storage medium storing a computer program for execution by a processor to implement the method of any one of claims 8 to 12.

26. A computer program product comprising a computer program for execution by a processor to implement the method of any one of claims 8 to 12.