CN110472745B - Information transmission method and device in federated learning - Google Patents

Information transmission method and device in federated learning Download PDF

Info

Publication number
CN110472745B
CN110472745B CN201910720601.2A CN201910720601A CN110472745B CN 110472745 B CN110472745 B CN 110472745B CN 201910720601 A CN201910720601 A CN 201910720601A CN 110472745 B CN110472745 B CN 110472745B
Authority
CN
China
Prior art keywords
result
participant
code
coordinator
rule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910720601.2A
Other languages
Chinese (zh)
Other versions
CN110472745A (en
Inventor
程勇
刘洋
陈天健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
WeBank Co Ltd
Original Assignee
WeBank Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by WeBank Co Ltd filed Critical WeBank Co Ltd
Priority to CN201910720601.2A priority Critical patent/CN110472745B/en
Publication of CN110472745A publication Critical patent/CN110472745A/en
Application granted granted Critical
Publication of CN110472745B publication Critical patent/CN110472745B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/46—Secure multiparty computation, e.g. millionaire problem

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Development Economics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Accounting & Taxation (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Economics (AREA)
  • Finance (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Technology Law (AREA)
  • General Business, Economics & Management (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The invention discloses an information transmission method and device in federated learning, wherein the method is suitable for federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant comprise the same sample object and different sample characteristics; the method comprises the following steps: the coordinator obtains a first result from the first participant and a second result from the second participant; the coordinator carries out operation on the first result and the second result according to a preset operation rule to obtain a third result after operation; the coordinator sends the third result to the first participant and the second participant. When the method is applied to financial technology (Fintech), the communication transmission time can be shortened on the basis of ensuring the communication safety, and the communication efficiency is obviously improved.

Description

Information transmission method and device in federated learning
Technical Field
The invention relates to the field of financial technology (Fintech) and computer software, in particular to an information transmission method and device in the Federation study.
Background
With the development of computer technology, more and more technologies (big data, distributed, Blockchain (Blockchain), artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to financial technology (Fintech). Currently, many financial strategies in the field of financial science and technology are adjusted depending on the result of data transmission in federal learning of a large amount of financial transaction data, and adjustment of the corresponding financial strategies is likely to affect profit and loss of financial institutions. Therefore, the efficiency of data transmission in federal learning is of great importance to a financial institution.
The two participants a and B are required to exchange intermediate results in the data transfer in federal learning. However, in the current data transmission scenario in federal learning, the participants a and B may not have a cooperative relationship or trust each other before, on one hand, the participants a and B may not establish a connection communication directly, and it takes a long time to create a new connection communication temporarily, and on the other hand, even if the connection communication is already established, the participants a and B need to negotiate a key to confirm the information of each other, which involves a complicated encryption and decryption algorithm, and the communication efficiency is low.
Disclosure of Invention
The embodiment of the application provides an information transmission method and device in federated learning, and solves the problem of low communication efficiency among participants in the prior art.
In a first aspect, an embodiment of the present application provides an information transmission method in federated learning, where the method is applied to federated learning including a first participant and a second participant, where training data sets of the first participant and the second participant include the same sample object and different sample features; the method comprises the following steps: the coordinator obtains a first result from the first participant and a second result from the second participant; wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training dataset of the first participant; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; the coordinator carries out operation on the first result and the second result according to a preset operation rule to obtain a third result after operation; and the coordinator sends the third result to the first participant and the second participant, so that the first participant solves the second result according to the preset operation rule, and the second participant solves the first result according to the preset operation rule.
According to the method, the coordinator obtains a third result after operation according to the first result and the second result and the preset operation rule, and then sends the third result to the first participant and the second participant, so that an encryption algorithm which is complex to the first result and the second result is avoided in the transmission process, the transmission time is reduced, the first participant and the second participant obtain respective required results through the third result and the first result or the second result, in addition, the first result or the second result is hidden through the third result, the required results can be obtained only through the first participant or the second participant which own the first result or the second result according to the preset operation rule, and therefore, the communication transmission time can be reduced on the basis of ensuring the communication safety, and the communication efficiency is remarkably improved.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the coordinator operates the first result and the second result according to a preset operation rule to obtain a third result after operation, and the method comprises the following steps: the coordinator correspondingly converts the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule; the coordinator performs the exclusive-or bit operation on the first code and the second code, and takes the result of the exclusive-or bit operation as the third result.
In the method, a coordinator correspondingly converts a first result and a second result into a first code and a second code which are easy to identify by a computer according to a binary bit conversion rule; and the first code and the second code are subjected to the XOR bit operation, and the XOR bit operation is the basic operation of a computer, so that the operation efficiency is higher, and the efficiency of obtaining a third result is improved.
In an alternative embodiment, the coordinator sending the third result to the first participant and the second participant, including: the coordinator sends the third result to the first participant and the second participant through broadcasting and/or multicasting.
In the method, the coordinator can select a broadcast and/or multicast mode, and can simultaneously send the third result to the first participant and the second participant, so that the flexibility of sending the third result is improved.
In a second aspect, an embodiment of the present application provides an information transmission method in federated learning, where the method is applied to federated learning including a first participant and a second participant, where training data sets of the first participant and the second participant include the same sample object and different sample features; the method comprises the following steps: a first participant uses data corresponding to the same sample object in the training data set of the first participant to calculate to obtain a first result; the first participant sends the first result to a coordinator; the first participant receiving a third result from the coordinator; the third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; and the first participant solves the second result according to the first result and the third result according to the preset operation rule and uses the second result to train a machine learning model.
In the above manner, the first participant trains using the data corresponding to the same sample object in the training data set of the first participant to obtain a first result, and then receives a third result from the coordinator; the third result is that the coordinator calculates the obtained result according to the first result and the second result according to the preset operation rule, so that the transmission time is reduced, the first participant can obtain the second result through the third result and the first result, the encryption algorithm which is complex to the second result is avoided in the transmission process, in addition, the second result is hidden through the third result, and the second result can be solved according to the third result only according to the preset operation rule and the first result, so that the communication transmission time can be reduced on the basis of ensuring the communication safety, and the communication efficiency is obviously improved.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the first participant solves the second result according to the first result and the third result according to the preset operation rule, and the method comprises the following steps: the first participant correspondingly converts the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule; the first participant performs the exclusive-or bit operation on the first code and the third code, and takes a result of the exclusive-or bit operation as the second result.
In the above manner, the first participant correspondingly converts the first result and the third result into a first code and a third code which are easy to be identified by the computer according to the binary bit conversion rule; and the XOR bit operation is carried out on the first code and the third code, and the XOR bit operation is the basic operation of a computer, so the operation efficiency is higher, and the efficiency of obtaining a second result is improved.
In a third aspect, the present application provides a data transmission apparatus in federated learning, which is adapted to federated learning including a first participant and a second participant, wherein training data sets of the first participant and the second participant include the same sample object and different sample features; the device comprises: an obtaining module for obtaining a first result from the first participant and a second result from the second participant; wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training dataset of the first participant; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; the processing module is used for calculating the first result and the second result according to a preset calculation rule to obtain a third result after calculation; and the third result is sent to the first participant and the second participant, so that the first participant solves the second result according to the preset operation rule and the second participant solves the first result according to the preset operation rule.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the processing module is specifically configured to: correspondingly converting the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule; and performing the XOR bit operation on the first code and the second code, and taking the result of the XOR bit operation as the third result.
In an optional implementation manner, the processing module is specifically configured to: sending the third result to the first participant and the second participant by broadcast and/or multicast.
For the advantages of the third aspect and the embodiments of the third aspect, reference may be made to the advantages of the first aspect and the embodiments of the first aspect, which are not described herein again.
In a fourth aspect, the present application provides a data transmission apparatus in federated learning, which is adapted to federated learning including a first participant and a second participant, wherein training data sets of the first participant and the second participant include the same sample object and different sample features; the device comprises: the training module is used for training data corresponding to the same sample object in the data set of the first participant to obtain a first result; the data transmission module is used for sending the first result to the coordinator; and for receiving a third result from the coordinator; the third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; and the processing module is used for solving the second result according to the first result and the third result according to the preset operation rule and training by using the second result.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the processing module is specifically configured to: correspondingly converting the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule; and performing the XOR bit operation on the first code and the third code, and taking the result of the XOR bit operation as the second result.
For the advantages of the embodiments of the fourth aspect and the fourth aspect, reference may be made to the advantages of the embodiments of the second aspect and the second aspect, which are not described herein again.
In a fifth aspect, embodiments of the present application provide a computer device, which includes a program or instructions, and when the program or instructions are executed, the computer device is configured to perform the method of each of the above first aspect and the first aspect or each of the above second aspect and the second aspect.
In a sixth aspect, embodiments of the present application provide a storage medium, which includes a program or instructions, and when the program or instructions are executed, the program or instructions are configured to perform the methods of the foregoing first aspect and the foregoing first aspect or the foregoing second aspect and the foregoing second aspect.
Drawings
Fig. 1 is a schematic diagram of an applicable framework of an information transmission method in federated learning according to an embodiment of the present application;
fig. 2 is a schematic flowchart illustrating steps of an information transmission method in federated learning according to an embodiment of the present application;
fig. 3 is a schematic flowchart illustrating steps of an information transmission method in federated learning according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data transmission apparatus in federated learning according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data transmission device in federated learning according to an embodiment of the present application.
Detailed Description
In order to better understand the technical solutions, the technical solutions will be described in detail below with reference to the drawings and the specific embodiments of the specification, and it should be understood that the specific features in the embodiments and examples of the present application are detailed descriptions of the technical solutions of the present application, but not limitations of the technical solutions of the present application, and the technical features in the embodiments and examples of the present application may be combined with each other without conflict.
In the operation process of a financial institution (a banking institution, an insurance institution or a security institution) in business (such as loan business, deposit business and the like of a bank), adjustment of a plurality of financial strategies in the financial science and technology field depends on the result of data transmission in federal study on a large amount of financial transaction data, and adjustment of the corresponding financial strategies probably influences profit and loss of the financial institution. Federal learning refers to a method of machine learning by federating different participants. The method comprises the following steps of: (a) each participant trains a machine learning model by using own data and sends model parameter update to a central coordinator; (b) the coordinator fuses (e.g., averages) the received model updates from the different participants and redistributes the fused model parameter updates to the individual participants. In the federal study, the participants do not need to expose own data to other participants or coordinators, so that the federal study can well protect the privacy of users and guarantee the data security.
For a financial institution, the efficiency of data transmission in federal learning is of great importance. In the prior art, the participants a and B may not have a cooperative relationship or trust each other before, on one hand, the participants a and B may not directly establish connection communication, and it takes a long time to temporarily create new connection communication, and on the other hand, even though the connection communication is already established, the participants a and B need to negotiate a secret key to mutually confirm information of each other, which involves a complicated encryption and decryption algorithm, and also results in low communication efficiency. This situation does not meet the requirements of financial institutions such as banks, and the efficient operation of various services of the financial institutions cannot be ensured.
To this end, an embodiment of the present application provides an information transmission method in federated learning, which is applicable to federated learning including a first participant and a second participant, where training data sets of the first participant and the second participant include the same sample object and different sample features. As shown in fig. 1, it is a schematic diagram of an applicable structure of the information transmission method in federal learning. For example, there are two participants belonging to the same region: a first participant and a second participant, wherein the first participant is a bank and the second participant is an e-commerce platform. The first participant and the second participant have more users in the same region, but the first participant and the second participant have different services and different recorded user data characteristics. In particular, the user data characteristics recorded by the first participant and the second participant may be complementary. In such a scenario, the architecture illustrated in fig. 1 may be used to help a first participant and a second participant build a joint machine learning model. To assist the first participant and the second participant in joint modeling, coordinator participation is required. It is generally assumed that both the first participant and the second participant trust the coordinator, but there is not necessarily a mutual trust relationship between the first participant and the second participant, and even a direct communication connection cannot be established between the first participant and the second participant. The method provided by the embodiment of the application can avoid the situation that the first participant and the second participant need to directly exchange intermediate results in federal learning, and is suitable for a scene that communication connection cannot be directly established between the participants or a scene that the participants do not trust or can not negotiate a secret key.
The following describes in detail an information transmission method in federated learning provided in an embodiment of the present application with reference to fig. 2 and fig. 3. Wherein, the step flowchart shown in fig. 2 is a step flowchart of coordinator-side data transmission; fig. 3 is a flowchart of steps of data transmission at the participant side (only the first participant is taken as an example, and steps performed by the second participant may refer to steps performed by the first participant, which are not described herein again).
Step 201: the coordinator obtains a first result from the first participant and a second result from the second participant.
Wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training data set of the first participant; the second result is a result of the second participant performing the present calculation using data corresponding to the same sample object in the training data set of the second participant. It should be noted that, in one possible scenario, the first result is a result that is calculated locally by the first participant using data corresponding to the same sample object in the training data set of the first participant.
Step 202: and the coordinator carries out operation on the first result and the second result according to a preset operation rule to obtain a third result after operation.
Step 203: the coordinator sends the third result to the first participant and the second participant.
Step 203 enables the first participant to solve the second result according to the preset operation rule and the second participant to solve the first result according to the preset operation rule.
Step 301: the first participant uses the data corresponding to the same sample object in the training data set of the first participant to calculate to obtain a first result.
Step 302: the first participant sends the first result to a coordinator;
step 303: the first participant receives a third result from the coordinator.
The third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant training using data corresponding to the same sample object in the training data set of the second participant;
step 304: and the first participant solves the second result according to the first result and the third result according to the preset operation rule and uses the second result to train a machine learning model.
Before step 201, the first participant performs step 301 to obtain a first result. The first participant then proceeds to step 302 to send the first result to the coordinator. It should be noted that the second participant can also follow the steps of the first participant to obtain a second result.
So that the coordinator then performs step 201, the coordinator receives the first result and the second result.
Step 202, in an optional implementation manner, the preset operation rule includes, but is not limited to, a binary bit conversion rule and an exclusive or operation; step 202 may specifically be:
the coordinator correspondingly converts the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule; the coordinator performs the exclusive-or bit operation on the first code and the second code, and takes the result of the exclusive-or bit operation as the third result.
In a first example, the first code and the second code are both binary bit streams. The first code is 0010, the second code is 0101, then the third code is 0111 according to an exclusive-or bit operation.
It should be noted that the preset operation rule in step 202 is not limited to the binary bit conversion rule and the exclusive or operation, and there may be various options, which are specifically described by taking an addition and subtraction method and an exclusive or operation as examples:
for example, when the predetermined operation rule is addition and subtraction, the first result is X1, the second result is X2, and the third result is Y ═ X1+ X2.
In a third example, when the predetermined operation rule is an exclusive-or operation, for example, the first code is 0010, the second code is 0101, and the third code is 1000 according to the exclusive-or operation.
An alternative implementation of step 203 is as follows:
the coordinator sends the third result to the first participant and the second participant through broadcasting and/or multicasting. Therefore, the communication transmission times can be reduced, and the communication efficiency is obviously improved.
After step 203, the first participant performs step 303, thereby receiving a third result. Accordingly, the second participant can also receive a third result.
Next, the first participant performs step 304. An alternative implementation of step 304 is: the preset operation rule includes but is not limited to a binary bit conversion rule and an exclusive-or bit operation; the first participant correspondingly converts the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule; the first participant performs the exclusive-or bit operation on the first code and the third code, and takes a result of the exclusive-or bit operation as the second result.
Alternative embodiments of step 304 are described below in terms of corresponding examples of three specific examples of step 202.
For the first example, the first code is 0010 and the third code is 0111, then the second code is 0101 based on an exclusive-or bit operation.
For the second example, the first code is X1, the third code is Y, and the second code is Y-X1 ═ X2 according to addition and subtraction.
For the third example, the first encoding is 0010, the third encoding is 1000, and the second encoding is 0101 according to an exclusive-nor operation.
The following describes, by way of example, a first result and a second result in an information transmission method in federated learning provided in an embodiment of the present application. The first participant is participant a; the second participant is participant B.
Together, participants a and B jointly constructed a linear regression model. Participant A owns the data
Figure BDA0002157051970000101
Wherein D isARepresents the training data set of participant a. Participant B owns the data and tags
Figure BDA0002157051970000102
Wherein D isBTraining data set, y, representing participant BiIndicating the ith group of data
Figure BDA0002157051970000103
The corresponding label, i.e.
Figure BDA0002157051970000104
And function f (-) is just the function that participants a and B need to learn jointly. In the general case of the above-mentioned,
Figure BDA0002157051970000105
and
Figure BDA0002157051970000106
are all multi-dimensional vectors, and yiIs a scalar (e.g., a scalar taking the value 0 or 1, indicating yes or no).
In a linear regression model, a loss function (also called a cost function) may be defined as
Figure BDA0002157051970000107
Wherein, wAAnd wBAre respectively corresponding to
Figure BDA0002157051970000108
And
Figure BDA0002157051970000109
the machine learning model parameters of (2) are also generally multidimensional vectors. The process of machine learning is to find the optimal parameter wAAnd wBThe loss function L is made as small as possible (minimizing the loss function).
Definition of
Figure BDA00021570519700001010
And
Figure BDA00021570519700001011
when y isiWhen scalar, the loss function L may be further decomposed into L ═ LA+LB+LABWherein
Figure BDA00021570519700001012
In the training process of the machine learning model, the parameter w is usually solved by using a gradient descent (gradient) method, or a deformation based on a gradient descent algorithmAAnd wB. Here, the loss function L and the parameter w are usedAAnd wBOf the gradient of (c). Loss function L versus parameter wAAnd wBCan be decomposed into LA，LBAnd LABFor parameter wAAnd wBThe sum of the gradients of (a). As can be seen from the above description, LAOnly with the parameter wAIn connection with this, the present invention is,and L isBOnly with the parameter wBIt is related.
Further define the
Figure BDA0002157051970000111
By GARepresenting the loss function L for a parameter wAGradient of (1) is
Figure BDA0002157051970000112
Similarly with GBRepresenting the loss function L for a parameter wBGradient of (1) is
Figure BDA0002157051970000113
Participant A required the use of a gradient GATo calculate and update the parameter wAWhereas participant B needs to use a gradient GBTo calculate and update the parameter wB. However, diAnd data
Figure BDA0002157051970000114
And
Figure BDA0002157051970000115
and a parameter wAAnd wBAre all relevant. Therefore, participant a cannot independently calculate the gradient GAAccordingly, participant B is also unable to independently calculate gradient GB. For the linear regression model, during one round of model parameter update for federal learning, participant a needs to calculate the first result
Figure BDA0002157051970000116
And will be sent to participant B through the coordinator; participant B needs to calculate the second result
Figure BDA0002157051970000117
And will send participant a through the coordinator.
For general machine learning models, use
Figure BDA0002157051970000118
Expressed in federal learningIn the k-th round of model parameter updating process, participant a needs a first result to be sent to participant B. Accordingly, use
Figure BDA0002157051970000119
Indicating that during the k-th round of model parameter update for federal learning, participant B needs a second result to be sent to participant a. Said intermediate result
Figure BDA00021570519700001110
And
Figure BDA00021570519700001111
not only parameter information and gradient information but also possibly loss function information may be included. Because participants a and B may not have a partnership or mutual trust before, it may not be possible to establish a communication connection and transfer intermediate results directly between a and B. Even if a connection can be established between a and B, additional communication overhead such as establishing trust and key agreement between a and B may be involved. Therefore, the information transmission method in federated learning provided by the embodiment of the application is suitable for a scenario in which communication connection cannot be directly established between participants, or a scenario in which the participants are not trusted or cannot negotiate a secret key.
The following steps can refer to the descriptions of step 201 to step 203 and step 301 to step 304.
It should be noted that the communication between the participants and the coordinator and the communication between the participants and the participants may be encrypted, for example, using homomorphic encryption, in view of data security.
Fig. 4 is a schematic structural diagram of a data transmission device in federated learning according to an embodiment of the present application.
The application provides a data transmission device in federated learning, which is suitable for federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant comprise the same sample object and different sample characteristics; the device comprises: an obtaining module 401, configured to obtain a first result from the first participant and a second result from the second participant; wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training dataset of the first participant; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; a processing module 402, configured to perform an operation on the first result and the second result according to a preset operation rule, and obtain a third result after the operation; and the third result is sent to the first participant and the second participant, so that the first participant solves the second result according to the preset operation rule and the second participant solves the first result according to the preset operation rule.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the processing module 402 is specifically configured to: correspondingly converting the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule; and performing the XOR bit operation on the first code and the second code, and taking the result of the XOR bit operation as the third result.
In an optional implementation manner, the processing module 402 is specifically configured to: sending the third result to the first participant and the second participant by broadcast and/or multicast.
Fig. 5 is a schematic structural diagram of a data transmission device in federated learning according to an embodiment of the present application.
The application provides a data transmission device in federated learning, which is suitable for federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant comprise the same sample object and different sample characteristics; the device comprises: a training module 501, configured to perform training using data corresponding to the same sample object in the data set of the first participant to obtain a first result; a data transmission module 502, configured to send the first result to the coordinator; and for receiving a third result from the coordinator; the third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant; the processing module 503 is configured to solve the second result according to the first result and the third result according to the preset operation rule, and perform training using the second result.
In an alternative embodiment, the preset operation rules include, but are not limited to, binary bit conversion rules and xor bit operations; the processing module 503 is specifically configured to: correspondingly converting the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule; and performing the XOR bit operation on the first code and the third code, and taking the result of the XOR bit operation as the second result.
The embodiment of the present application provides a computer device, which includes a program or an instruction, and when the program or the instruction is executed, the program or the instruction is used to execute the information transmission method and any optional method in federated learning provided by the embodiment of the present application.
The embodiment of the present application provides a storage medium, which includes a program or an instruction, and when the program or the instruction is executed, the program or the instruction is used to execute an information transmission method and any optional method in federated learning provided by the embodiment of the present application.
Finally, it should be noted that: as will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.

Claims (12)

1. The information transmission method in the federated learning is characterized by being applicable to the federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant contain the same sample object and different sample characteristics; the method comprises the following steps:
the coordinator obtains a first result from the first participant and a second result from the second participant; wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training dataset of the first participant; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant;
the coordinator carries out operation on the first result and the second result according to a preset operation rule to obtain a third result after operation; and the coordinator sends the third result to the first participant and the second participant, so that the first participant solves the second result according to the preset operation rule, and the second participant solves the first result according to the preset operation rule.
2. The method of claim 1, wherein the predetermined operation rules comprise binary bit conversion rules and exclusive-or bit operations; the coordinator operates the first result and the second result according to a preset operation rule to obtain a third result after operation, and the method comprises the following steps:
the coordinator correspondingly converts the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule;
the coordinator performs the exclusive-or bit operation on the first code and the second code, and takes the result of the exclusive-or bit operation as the third result.
3. The method of claim 1 or 2, wherein the coordinator sending the third result to the first participant and the second participant, comprising:
the coordinator sends the third result to the first participant and the second participant through broadcasting and/or multicasting.
4. The information transmission method in the federated learning is characterized by being applicable to the federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant contain the same sample object and different sample characteristics; the method comprises the following steps:
a first participant uses data corresponding to the same sample object in the training data set of the first participant to calculate to obtain a first result;
the first participant sends the first result to a coordinator;
the first participant receiving a third result from the coordinator; the third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant;
and the first participant solves the second result according to the first result and the third result according to the preset operation rule and uses the second result to train a machine learning model.
5. The method of claim 4, wherein the predetermined operation rules comprise binary bit conversion rules and XOR bit operations; the first participant solves the second result according to the first result and the third result according to the preset operation rule, and the method comprises the following steps:
the first participant correspondingly converts the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule;
the first participant performs the exclusive-or bit operation on the first code and the third code, and takes a result of the exclusive-or bit operation as the second result.
6. The data transmission device in the federated learning is characterized by being suitable for the federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant contain the same sample object and different sample characteristics; the device comprises:
an obtaining module for obtaining a first result from the first participant and a second result from the second participant; wherein the first result is a result of the first participant calculating using data corresponding to the same sample object in the training dataset of the first participant; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant;
the processing module is used for calculating the first result and the second result according to a preset calculation rule to obtain a third result after calculation; and the third result is sent to the first participant and the second participant, so that the first participant solves the second result according to the preset operation rule and the second participant solves the first result according to the preset operation rule.
7. The apparatus of claim 6, wherein the predetermined operation rules comprise binary bit translation rules and XOR bit operations; the processing module is specifically configured to:
correspondingly converting the first result and the second result into a first code and a second code respectively according to the binary bit conversion rule;
and performing the XOR bit operation on the first code and the second code, and taking the result of the XOR bit operation as the third result.
8. The apparatus of claim 6 or 7, wherein the processing module is specifically configured to:
sending the third result to the first participant and the second participant by broadcast and/or multicast.
9. The data transmission device in the federated learning is characterized by being suitable for the federated learning comprising a first participant and a second participant, wherein training data sets of the first participant and the second participant contain the same sample object and different sample characteristics; the device comprises:
the training module is used for training data corresponding to the same sample object in the data set of the first participant to obtain a first result;
the data transmission module is used for sending the first result to the coordinator; and for receiving a third result from the coordinator; the third result is a result obtained by the coordinator according to a preset operation rule and according to the first result and the second result; the second result is a result of the second participant calculating using data corresponding to the same sample object in the training dataset of the second participant;
and the processing module is used for solving the second result according to the first result and the third result according to the preset operation rule and training by using the second result.
10. The apparatus of claim 9, wherein the predetermined operation rules comprise binary bit translation rules and exclusive-or bit operations; the processing module is specifically configured to:
correspondingly converting the first result and the third result into a first code and a third code respectively according to the binary bit conversion rule;
and performing the XOR bit operation on the first code and the third code, and taking the result of the XOR bit operation as the second result.
11. A computer device comprising a program or instructions which, when executed, perform the method of any of claims 1 to 3 or 4 to 5.
12. A storage medium comprising a program or instructions which, when executed, perform the method of any one of claims 1 to 3 or 4 to 5.
CN201910720601.2A 2019-08-06 2019-08-06 Information transmission method and device in federated learning Active CN110472745B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910720601.2A CN110472745B (en) 2019-08-06 2019-08-06 Information transmission method and device in federated learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910720601.2A CN110472745B (en) 2019-08-06 2019-08-06 Information transmission method and device in federated learning

Publications (2)

Publication Number Publication Date
CN110472745A CN110472745A (en) 2019-11-19
CN110472745B true CN110472745B (en) 2021-04-27

Family

ID=68511327

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910720601.2A Active CN110472745B (en) 2019-08-06 2019-08-06 Information transmission method and device in federated learning

Country Status (1)

Country Link
CN (1) CN110472745B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11966818B2 (en) 2019-02-21 2024-04-23 Hewlett Packard Enterprise Development Lp System and method for self-healing in decentralized model building for machine learning using blockchain
CN113268776B (en) * 2019-12-09 2023-03-07 支付宝(杭州)信息技术有限公司 Model joint training method and device based on block chain
CN111212110B (en) * 2019-12-13 2022-06-03 清华大学深圳国际研究生院 Block chain-based federal learning system and method
US11748835B2 (en) 2020-01-27 2023-09-05 Hewlett Packard Enterprise Development Lp Systems and methods for monetizing data in decentralized model building for machine learning using a blockchain
US11218293B2 (en) 2020-01-27 2022-01-04 Hewlett Packard Enterprise Development Lp Secure parameter merging using homomorphic encryption for swarm learning
US11775667B2 (en) 2020-11-04 2023-10-03 Hewlett Packard Enterprise Development Lp Virtualizing secure storage of a baseboard management controller to a host computing device
CN112217706B (en) * 2020-12-02 2021-03-19 腾讯科技（深圳）有限公司 Data processing method, device, equipment and storage medium
CN113240127A (en) * 2021-04-07 2021-08-10 睿蜂群(北京)科技有限公司 Federal learning-based training method and device, electronic equipment and storage medium
CN113592097B (en) * 2021-07-23 2024-02-06 京东科技控股股份有限公司 Training method and device of federal model and electronic equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109165515A (en) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 Model parameter acquisition methods, system and readable storage medium storing program for executing based on federation's study
CN109492420A (en) * 2018-12-28 2019-03-19 深圳前海微众银行股份有限公司 Model parameter training method, terminal, system and medium based on federation's study

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180089587A1 (en) * 2016-09-26 2018-03-29 Google Inc. Systems and Methods for Communication Efficient Distributed Mean Estimation
CN109167695B (en) * 2018-10-26 2021-12-28 深圳前海微众银行股份有限公司 Federal learning-based alliance network construction method and device and readable storage medium
CN109886417B (en) * 2019-03-01 2024-05-03 深圳前海微众银行股份有限公司 Model parameter training method, device, equipment and medium based on federal learning
CN110008696A (en) * 2019-03-29 2019-07-12 武汉大学 A kind of user data Rebuilding Attack method towards the study of depth federation

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109165515A (en) * 2018-08-10 2019-01-08 深圳前海微众银行股份有限公司 Model parameter acquisition methods, system and readable storage medium storing program for executing based on federation's study
CN109492420A (en) * 2018-12-28 2019-03-19 深圳前海微众银行股份有限公司 Model parameter training method, terminal, system and medium based on federation's study

Also Published As

Publication number Publication date
CN110472745A (en) 2019-11-19

Similar Documents

Publication Publication Date Title
CN110472745B (en) Information transmission method and device in federated learning
CN112183730B (en) Neural network model training method based on shared learning
CN113159327B (en) Model training method and device based on federal learning system and electronic equipment
CN112733967B (en) Model training method, device, equipment and storage medium for federal learning
WO2022068575A1 (en) Calculation method for vertical federated learning, apparatus, device, and medium
CN111860829A (en) Method and device for training federal learning model
CN111368319A (en) Block chain-based data security access method in federated learning environment
CN112288094B (en) Federal network representation learning method and system
CN112199709A (en) Multi-party based privacy data joint training model method and device
CN111104968B (en) Safety SVM training method based on block chain
CN111428887A (en) Model training control method, device and system based on multiple computing nodes
CN113689003A (en) Safe mixed federal learning framework and method for removing third party
CN113609781B (en) Method, system, equipment and medium for optimizing automobile production die based on federal learning
CN113542228A (en) Data transmission method and device based on federal learning and readable storage medium
CN114595835A (en) Model training method and device based on federal learning, equipment and storage medium
US20230076260A1 (en) Systems and methods for converting data from int-64 to boolean for computations
CN113051239A (en) Data sharing method, use method of model applying data sharing method and related equipment
CN116468132A (en) System, method and device for processing user data
CN117171779B (en) Data processing device based on intersection protection
CN114757361A (en) Multi-mode intermodal transportation data sharing method and system based on federal learning
CN116743376A (en) Multiparty secret sharing data privacy comparison method based on efficient ciphertext confusion technology
CN113537516B (en) Training method, device, equipment and medium for distributed machine learning model
CN114723068A (en) Federal model training method and device
CN115361196A (en) Service interaction method based on block chain network
CN114723012A (en) Computing method and device based on distributed training system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant