CN116306959A

CN116306959A - Federal learning method and device, storage medium and electronic equipment

Info

Publication number: CN116306959A
Application number: CN202211338706.XA
Authority: CN
Inventors: 吴若凡
Original assignee: Alipay Hangzhou Information Technology Co Ltd
Current assignee: Alipay Hangzhou Information Technology Co Ltd
Priority date: 2022-10-28
Filing date: 2022-10-28
Publication date: 2023-06-23

Abstract

The specification provides a federal learning method, a federal learning device, a federal learning storage medium and electronic equipment. In the federal learning method provided in the present specification, determining a target node common to the second party, and determining a corresponding annotation; inputting data corresponding to the target node into a network model of the graph to be trained, and extracting first features of the target node through a feature extraction layer; encrypting the first characteristic by adopting a public key of the second participant to obtain a first encrypted characteristic; receiving a second encryption feature transmitted by a second participant; fusing the first encryption feature and the second encryption feature to obtain a fused encryption feature; inputting the fusion encryption characteristic into a subsequent processing layer of a network model of the graph to be trained to obtain an encryption output result output by the model; the encrypted output result is sent to a second participant, so that the second participant adopts a private key to decrypt the encrypted output result, and a decryption result is obtained; and receiving a decryption result sent by the second participant, and training the model by adopting the decryption result.

Description

Federal learning method and device, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a federal learning method, a federal learning device, a storage medium, and an electronic device.

Background

At present, in order to expand training samples adopted by a training model, the model is often trained by adopting a federal learning mode. The main purpose of federal learning is to combine training samples of two parties to promote the effect of training the model, in order to protect privacy, the original data of two parties are not revealed, and the training is usually performed by adopting a longitudinal federal learning mode.

However, the current longitudinal federal learning method still has hidden danger of leaking data, so how to ensure the safety of the original data in the longitudinal federal learning process is a problem to be solved urgently.

Disclosure of Invention

The present specification provides a federal learning method and federal learning apparatus to partially solve the above-mentioned problems existing in the prior art.

The technical scheme adopted in the specification is as follows:

the present specification provides a method of federal learning, the method being applied to a first party, comprising:

the first party determines a user node shared by the second party from the graph data stored by the first party as a target node, and determines the label of the target node according to a historical service execution result corresponding to the target node;

Inputting data corresponding to the target node into a feature extraction layer of a network model of a graph to be trained, and obtaining a first feature output by the feature extraction layer;

encrypting the first characteristic according to the public key of the second party through a homomorphic encryption algorithm, determining a first encryption characteristic, and receiving a second encryption characteristic sent by the second party, wherein the second encryption characteristic is obtained by encrypting the second characteristic by the second party through the public key, and the second characteristic is determined by the second party according to a target node in the graph data stored by the second party;

determining a fusion feature according to the first encryption feature and the second encryption feature;

inputting the fusion characteristics into a subsequent processing layer of the to-be-trained graph network model to obtain an encryption output result output by the to-be-trained graph network model;

the encryption output result is sent to the second participant, so that the second participant adopts a private key corresponding to the public key to decrypt the encryption output result;

and receiving a decryption result returned by the second participant, and training the network model of the graph to be trained according to the decryption result and the label.

Optionally, before sending the encrypted output result to the second party, the method further comprises:

encrypting the preset disturbance characteristics by adopting the public key to obtain encrypted disturbance characteristics;

scrambling the encryption output result by adopting the encryption disturbance characteristic;

training the network model of the graph to be trained according to the decryption result and the label, and specifically comprises the following steps:

according to the disturbance characteristics, eliminating the added disturbance in the decryption result;

and training the network model of the graph to be trained by adopting the decryption result after interference elimination and the labeling.

Optionally, the second feature is obtained by the second participant adjusting a dimension of a second intermediate feature to a specified dimension by using a second parameter, where the second intermediate feature is determined by the second participant according to a target node in the graph data stored by the second participant;

before encrypting the first feature according to the public key of the second party, the method further comprises:

and adjusting the dimension of the first feature to be a specified dimension by adopting a first parameter.

Optionally, training the network model of the graph to be trained according to the decryption result and the label specifically includes:

And adjusting the parameter to be trained and the first parameter by taking the minimum difference between the decryption result and the annotation as an optimization target.

The present specification provides a method of federal learning, the method being applied to a second party, comprising:

the second party determines a user node shared with the first party from the self-stored graph data as a target node;

extracting a second characteristic of data corresponding to the target node;

encrypting the second characteristics according to the public key of the second party through a homomorphic encryption algorithm to determine second encrypted characteristics;

the second encryption feature is sent to the first participant, and a fusion feature is determined according to a first encryption feature and the second encryption feature, wherein the first encryption feature is obtained by encrypting the first feature by the first participant through the public key, and the first feature is determined by the first participant by inputting data corresponding to a target node in graph data stored by the first participant into a feature extraction layer of a graph network model to be trained;

receiving an encryption output result sent by the first participant, wherein the encryption output result is determined by the first participant by inputting the fusion characteristic into a subsequent processing layer of the network model of the graph to be trained;

Decrypting the encrypted output result by adopting the private key of the second participant;

and returning the decryption result to the first participant, so that the first participant trains the network model of the graph to be trained according to the decryption result and labels, wherein the labels are determined by the first participant according to the historical service execution result of the first participant corresponding to the target node.

Optionally, the encryption output result is obtained by scrambling an intermediate result output by the to-be-trained graph network model of the first participant by using an encryption disturbance feature, and the intermediate result is determined by the first participant inputting the fusion feature into a subsequent processing layer of the to-be-trained graph network model.

Optionally, the first feature is obtained by the first participant adjusting a dimension of a first intermediate feature to be a specified dimension by adopting a first parameter, and the first intermediate feature is determined by the first participant inputting data corresponding to a target node in the graph data stored by the first participant into a feature extraction layer of a graph network model to be trained;

before encrypting the second feature according to the public key of the second party, the method further comprises:

And adopting a second parameter to adjust the dimension of the second feature to be a specified dimension.

Optionally, the decryption result is returned to the first participant, so that the first participant trains the to-be-trained graph network model by adopting the decryption result and the label, and the method specifically comprises the following steps:

and returning the decryption result to the first participant, so that the first participant takes the minimum difference between the decryption result and the label as an optimization target, and adjusts the parameters of the network model of the graph to be trained and the first parameter.

The present specification provides a federal learning apparatus comprising:

the first determining module is used for determining a user node shared by the first participant and the second participant from the graph data stored by the first participant, taking the user node as a target node, and determining the label of the target node according to a historical service execution result corresponding to the target node;

the first extraction module is used for inputting the data corresponding to the target node into a feature extraction layer of the network model of the graph to be trained to obtain a first feature output by the feature extraction layer;

the first encryption module is used for encrypting the first characteristic according to the public key of the second party through a homomorphic encryption algorithm, determining a first encryption characteristic, and receiving a second encryption characteristic sent by the second party, wherein the second encryption characteristic is obtained by encrypting the second characteristic by the second party through the public key, and the second characteristic is determined by the second party according to a target node in the graph data stored by the second party;

The fusion module is used for determining fusion characteristics according to the first encryption characteristics and the second encryption characteristics;

the output module is used for inputting the fusion characteristics into a subsequent processing layer of the to-be-trained graph network model to obtain an encryption output result output by the to-be-trained graph network model;

the sending module is used for sending the encryption output result to the second party, so that the second party adopts a private key corresponding to the public key to decrypt the encryption output result;

and the training module is used for receiving the decryption result returned by the second participant and training the network model of the graph to be trained according to the decryption result and the label.

The present specification provides a federal learning apparatus comprising:

the second determining module is used for determining a user node shared with the first participant from the graph data stored by the second participant, and taking the user node as a target node;

the second extraction module is used for extracting a second characteristic of the data corresponding to the target node;

the second encryption module is used for encrypting the second characteristics according to the public key of the second party through a homomorphic encryption algorithm to determine second encryption characteristics;

The feature sending module is used for sending the second encryption feature to the first participant, so that a fusion feature is determined according to the first encryption feature and the second encryption feature, wherein the first encryption feature is obtained by encrypting the first feature by the first participant through the public key, and the first feature is determined by the first participant inputting data corresponding to a target node in the self-stored graph data into a feature extraction layer of a graph network model to be trained;

the second receiving module is used for receiving an encryption output result sent by the first participant, wherein the encryption output result is determined by the first participant by inputting the fusion characteristic into a subsequent processing layer of the graph network model to be trained;

the decryption module is used for decrypting the encrypted output result by adopting the private key of the second participant;

and the return module is used for returning the decryption result to the first participant, so that the first participant trains the graph network model to be trained according to the decryption result and labels, and the labels are determined by the first participant according to the historical service execution result of the first participant corresponding to the target node.

The present specification provides a computer readable storage medium storing a computer program which when executed by a processor implements the method of federal learning described above.

The present specification provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of federal learning as described above when executing the program.

The above-mentioned at least one technical scheme that this specification adopted can reach following beneficial effect:

in the federal learning method provided in the present specification, determining a target node common to the second party, and determining a corresponding annotation; inputting data corresponding to the target node into a network model of the graph to be trained, and extracting first features of the target node through a feature extraction layer; encrypting the first characteristic by adopting a public key of the second participant to obtain a first encrypted characteristic; receiving a second encryption feature transmitted by a second participant; fusing the first encryption feature and the second encryption feature to obtain a fused encryption feature; inputting the fusion encryption characteristic into a subsequent processing layer of a network model of the graph to be trained to obtain an encryption output result output by the model; the encrypted output result is sent to a second participant, so that the second participant adopts a private key to decrypt the encrypted output result, and a decryption result is obtained; and receiving a decryption result sent by the second participant, and training the model by adopting the decryption result.

Drawings

The accompanying drawings, which are included to provide a further understanding of the specification, illustrate and explain the exemplary embodiments of the present specification and their description, are not intended to limit the specification unduly. In the drawings:

FIG. 1 is a schematic flow chart of a federal learning method provided in the present specification;

FIG. 2 is a detailed process diagram of a federal learning method provided herein;

FIG. 3 is a schematic flow chart of a federal learning method provided in the present specification;

FIG. 4 is a schematic illustration of a federal learning device provided herein;

FIG. 5 is a schematic illustration of a federal learning device provided herein;

fig. 6 is a schematic diagram of an electronic device corresponding to fig. 1 provided in the present specification.

Detailed Description

In general, longitudinal federal learning is a process that requires two parties to accomplish together. Wherein, two parties participating in federal learning can be divided into a training party and a feature provider. The traditional longitudinal federal learning main process is that a feature provider provides features of data for a training party according to original data of the own party, and the training party trains a model of the own party according to the features provided by the feature provider, so that joint training is completed under the condition that the data does not go out of a domain. The above only helps one party to train the model, and it is conceivable that the other party can be helped to complete the training of the model as long as the identities of the two parties participating in federal learning are exchanged.

For the purposes of making the objects, technical solutions and advantages of the present specification more apparent, the technical solutions of the present specification will be clearly and completely described below with reference to specific embodiments of the present specification and corresponding drawings. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present specification. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present disclosure.

The following describes in detail the technical solutions provided by the embodiments of the present specification with reference to the accompanying drawings.

Fig. 1 is a flow chart of a method for federal learning applied to a first participant provided in the present specification, including the steps of:

s100: the first party determines a user node shared by the second party from the graph data stored by the first party as a target node, and determines the label of the target node according to the historical service execution result corresponding to the target node.

In this specification, an execution body for implementing the federal learning method may refer to a designated device such as a server provided on a service platform, and for convenience of description, this specification uses only the server as an execution body as an example, and describes a federal learning method provided in this specification.

In this specification, both the first party and the second party are servers or other designated devices. The first party is the party who performs training, and the second party is the party who provides characteristic data. That is, in an embodiment of the federal learning method provided herein, features are provided by the second party to the first party, and the first party trains the model to be trained using the features provided by the second party.

In practical applications, there are many specific forms of samples and sample data. In the federal learning method provided in the present specification, sample data is stored in the form of graph data in the first party and the second party, in which case each sample may be represented by each node on the graph, and information of the sample may be represented by an edge on the graph. It should be noted that, besides the graph data, the training samples may also be stored in other data forms in the two parties participating in federal learning, which is not particularly limited in this specification.

A process step diagram of the federal learning method provided in the present specification may be as shown in fig. 2. When federal learning is performed, first, a user node shared by the first participant and the second participant, that is, a training sample capable of participating in training, needs to be determined as a target node. The function of federal learning is to train the model by additionally utilizing the characteristic data provided by the other party on the basis that the training party trains the model by utilizing own data. Thus, the training samples used to participate in the training should be target nodes that are common to both parties.

It should be noted that both parties have partially identical samples and do not represent that both parties have identical sample data. For example, assume a first participating party provides service a and service B, and a second participating party provides service C and service D. When the user is used as the node in the graph data, the user who uses the service provided by the first participant and the service provided by the second participant can be used as the user node shared by the first participant and the second participant, namely the target node. For the target node, the data owned by the first party is the data of the user when executing the service A and the service B, the data owned by the second party is the data of the user when executing the service C and the service D, and the sample data owned by the two parties are obviously different.

Meanwhile, in the embodiment of the present disclosure, since the first participant is a party for training the model, after determining the training sample, the first participant needs to determine the label corresponding to the training sample, where in general, the label may be determined according to the historical service execution result corresponding to the target node, and when training different models, the corresponding historical service may also be different. There are various ways to determine the label, for example, manually determining the label, and the present specification is not limited thereto.

S102: and inputting the data corresponding to the target node into a feature extraction layer of the network model of the graph to be trained, and obtaining a first feature output by the feature extraction layer.

After the target node is determined in step S100, data corresponding to the target node may be input into the network model of the graph to be trained in this step. After data corresponding to the target node is input, the data firstly enters a feature extraction layer in the graph network model, and the feature extraction layer extracts first features of the data. The to-be-trained graph network model is a model which is prestored in a first participant and needs to be trained.

S104: and encrypting the first characteristic according to the public key of the second party through a homomorphic encryption algorithm, determining a first encryption characteristic, and receiving a second encryption characteristic sent by the second party, wherein the second encryption characteristic is obtained by encrypting the second characteristic by the second party through the public key, and the second characteristic is determined by the second party according to a target node in the graph data stored by the second party.

In the federal learning method provided in the present specification, all the encryption employed is homomorphic encryption. In homomorphic encryption, homomorphic encrypted data is processed to obtain an output, and the output is decrypted, the result of which is identical to the output result obtained by processing unencrypted original data by the same method. For example, it is assumed that data a' can be obtained by performing a specific process on data a; in the case of homomorphic encryption, data a is encrypted to obtain [ [ a ] ], then the encrypted data [ [ a ] ] is subjected to the same specific processing to obtain data [ [ a ] ], at this time, the data [ [ a ] ] 'is decrypted, and the obtained data is the same as the data obtained by directly carrying out the specific processing on the data a, and is a'. Wherein the specific processing may be an operation consisting of addition and/or multiplication.

Those skilled in the art will appreciate that in data encryption techniques, there are means of asymmetric encryption. In asymmetric encryption, a public key is used for encryption and a private key is used for decryption. In the federal learning method provided in the present specification, both the first party and the second party have the public key of the second party, and data can be encrypted; and the private key of the second party is owned by the second party only, so that the data can be decrypted.

In this step, the first feature extracted in step S102 may be encrypted with the public key of the second party, so as to obtain a first encrypted feature, which is used in a subsequent step.

Meanwhile, because the feature data of the two parties are required to be combined for training in federal learning, in the step, a second encryption feature sent by the second parameter party can be received. The second encryption feature is obtained by the second party, similar to the first encryption feature, extracting the second feature of the target node and encrypting the second feature by using the public key. The public keys adopted by the first participant and the second participant in encrypting the data are the same, and are public keys corresponding to the private keys owned by the second participant, namely, only the private keys owned by the second participant can decrypt the data encrypted by the first participant and the second participant by the public keys.

In this case, since only the second party can decrypt the data after encrypting the data by using the public key, it is ensured that the first party cannot reverse attack the second encryption feature transmitted by the second party, thereby deducing the original data of the second party.

S106: and determining a fusion characteristic according to the first encryption characteristic and the second encryption characteristic.

In general, in order to make the model obtain a better effect, before the first encryption feature and the second encryption feature are input into a subsequent processing layer in the network model of the graph to be trained to continue to calculate, the second encryption feature is generally fused with the first encryption feature to obtain a fused encryption feature. The manner of fusing the first encryption feature and the second encryption feature may be various, for example, splicing, and the present disclosure is not limited thereto.

S108: and inputting the fusion characteristics into a subsequent processing layer of the to-be-trained graph network model to obtain an encryption output result output by the to-be-trained graph network model.

After determining the fusion encryption feature in step S106, the fusion encryption feature may be input to a subsequent processing layer of the network model of the graph to be trained in this step, and the fusion encryption feature is calculated, and a result obtained by the calculation is output. Because the first encryption feature and the second encryption feature are homomorphic encrypted secret state data, the fused encryption feature obtained after fusion is also encrypted data, and therefore the result output after calculation of the network model of the graph to be trained is also the secret state encrypted output result.

S110: and sending the encryption output result to the second party, so that the second party adopts a private key corresponding to the public key to decrypt the encryption output result.

Because the result output by the network model of the graph to be trained in step S108 is a secret-state encrypted output result, the result cannot be directly used for training, and only the second party has the corresponding private key and can decrypt the encrypted output result, the encrypted output result needs to be sent to the second party, so that the second party adopts the private key to decrypt the encrypted output result, and a decryption result is obtained.

S112: and receiving a decryption result returned by the second participant, and training the network model of the graph to be trained according to the decryption result and the label.

And after the second party finishes decrypting the encrypted output result to obtain a decryption result, returning the decryption result to the first party. Under homomorphic encryption, the decryption result is the same as the result obtained by directly inputting the unencrypted first feature and second feature into the network model of the drawing to be trained, so that the network model of the drawing to be trained can be trained by directly adopting the decryption result.

During training, the gradient of the feature can be calculated according to the decryption result, then the gradient of the feature is input into the network model to be trained, the gradient of the parameters of the network model is obtained, and the parameters in the network model to be trained are updated by adopting the obtained gradient of the parameters of the network model.

Specifically, the gradient of the feature may be determined according to the following formula:

where A represents a first party, S represents the total number of target nodes, i represents the ith target node,

gradient representing characteristics of the ith node, +.>

Representing the result of calculation of the network model of the graph to be trained on the basis of the second characteristics of the ith node,/th node>

Representing the first characteristic of the ith node, y _i Labeling, θ, representing the ith node _A Representing the first parameter.

When the federal learning method provided by the specification is adopted, the second party can encrypt the extracted second characteristic by adopting the public key and then send the encrypted output result to the first party, and after the first party obtains the encrypted output result through the graph network model to be trained, the encrypted output result is decrypted by adopting the corresponding private key and returned to the first party, the first party trains the model, so that the possibility that the first party can estimate the original data of the second party according to the characteristic data of the second party is effectively prevented, and the original data of the second party is protected.

Additionally, in the above embodiment, only the case that the first party may steal the original data of the second party and protect the second party is considered. In practical applications, however, the network model itself of the graph to be trained, which needs to be trained, usually needs to be protected in addition to the original data. And under the condition that the first participant transmits the encrypted output result output by the to-be-trained graph network model to the second participant and the second participant decrypts by adopting the private key to obtain the decryption result, the second participant can completely attack the to-be-trained graph network model of the first participant according to the decryption result.

Therefore, in order to be able to further protect the model of the first party, the encrypted output may be scrambled when the first party sends the encrypted output to the second party. Specifically, the public key can be adopted to encrypt the preset disturbance characteristics to obtain encrypted disturbance characteristics; and scrambling the encryption output result by adopting the encryption disturbance characteristic. Therefore, the encrypted output result obtained by the second party is additionally provided with the disturbance of the secret state, and when the second party decrypts the encrypted output result, the obtained decryption result is also the decryption result with the disturbance characteristic. Because the second participant does not know how the first participant scrambles, the second participant can not identify the disturbance from the scrambled decryption result, and the decryption result without the disturbance cannot be known, so that the model of the first participant cannot be attacked.

It is conceivable that in the above case, the decryption result returned by the second party to the first party is a result with disturbance, and the first party needs to cancel the disturbance before training the model by using the decryption result. Specifically, according to the disturbance characteristics, the added disturbance in the decryption result can be eliminated; and training the network model of the graph to be trained by adopting the decryption result after interference elimination and the labeling.

In practical applications, since the extraction networks adopted by the first and second parties in extracting the features of the target node are likely to be completely different, the dimensions of the extracted first and second features may also be different. In this case, in order to better fuse the first encryption feature and the second encryption feature, it may be considered to process the first feature and the second feature to the same dimension before performing encryption and fusion. Specifically, a first parameter may be adopted to adjust the dimension of the first feature to a specified dimension; and the second characteristic is obtained by adjusting the dimension of a second intermediate characteristic to a designated dimension by the second party by adopting a second parameter, wherein the second intermediate characteristic is determined by the second party according to a target node in the self-stored graph data.

Because homomorphic encryption is adopted and the same public key is adopted in encryption, after the first characteristic and the second characteristic are adjusted to the same appointed dimension, encryption is respectively carried out, and the obtained first encryption characteristic and second encryption characteristic are still in the same dimension, so that feature fusion is facilitated.

In the federal learning method provided in the present specification, although the first participant only trains the network model of the graph to be trained, it is preferable that if the above method of adjusting the first feature and the second feature to the same specified dimension is adopted, the first parameter for adjusting the first feature may be additionally updated during training. Specifically, the minimum difference between the decryption result and the label is used as an optimization target, and the parameters of the network model of the graph to be trained and the first parameters are adjusted. Wherein the gradient for adjusting the first parameter may be determined according to the following formula:

representing the gradient of the first parameter S _A Representing the sum of the results of the network model of the graph to be trained calculated on the basis of the second characteristics of the respective target node,/->

Representing the first characteristic of each node of the ith, < >>

Representing the first characteristic, y, of the ith node after the first parameter adjustment _i Labeling representing the ith node, +.>

And (3) representing. Fig. 3 is a flow chart of a method provided in the present specification for federal learning applied to a second party, comprising the steps of:

S200: the second party determines a user node shared with the first party from the graph data stored by the second party as a target node.

It should be noted that, the method for federal learning applied to the second participant provided in the present specification is a method corresponding to the method for federal learning applied to the first participant provided in the present specification, and the two methods are actually execution flows of the same method in different participants. Therefore, in order to make the description more concise, the steps that have been explained in detail in the method of federal learning applied to the first party provided in the present description will not be repeated in the method of federal learning applied to the second party described below.

Similar to step S100, in this step, as the second party providing the feature data, it is also necessary to first determine a user node common to the first party as a target node for assisting the training of the graph network model to be trained of the first party.

S202: and extracting a second characteristic of the data corresponding to the target node.

When the target node is determined, the second party extracts the second feature of the target node for use in a subsequent step.

S204: and encrypting the second characteristic according to the public key of the second party through a homomorphic encryption algorithm, and determining a second encryption characteristic.

In this step, the second party encrypts the second feature using the public key of the second party in a homomorphic encryption manner, resulting in a second encrypted feature.

S206: and sending the second encryption feature to the first participant, and determining a fusion feature according to the first encryption feature and the second encryption feature, wherein the first encryption feature is obtained by encrypting the first feature by the first participant by adopting the public key, and the first feature is determined by the first participant inputting data corresponding to a target node in the self-stored graph data into a feature extraction layer of the graph network model to be trained.

After the encryption of the second feature is completed, the obtained second encryption feature can be sent to the first party, so that the first party fuses the first encryption feature and the second encryption feature. The first encryption feature is obtained by extracting a first feature of the target node by a first party through a network of a to-be-trained graph and encrypting the first feature by adopting a public key of a second party, and the first party adopts the same public key as the public key adopted when the first party participates in Fang Jiami second feature.

S208: and receiving an encryption output result sent by the first participant, wherein the encryption output result is determined by the first participant by inputting the fusion characteristic into a subsequent processing layer of the graph network model to be trained.

When the first party fuses the first encryption feature and the second encryption feature to obtain a fused encryption feature, the fused encryption feature can be input into a subsequent processing layer of the network model of the graph to be trained, and the fused encryption feature is calculated through the subsequent processing layer to obtain an encryption output result. Since the first party does not own the private key for decryption itself, the encrypted output result can be sent to the second party.

S210: and decrypting the encrypted output result by adopting the private key of the second participant.

The second party decrypts the encrypted output result by using the private key to obtain a decrypted result. In the federal learning method provided in the present specification, both the first party and the second party have public keys that can encrypt data, but only the second party has private keys that can decrypt the encrypted data. Thus, only the second party can decrypt the encrypted output.

S212: and returning the decryption result to the first participant, so that the first participant trains the network model of the graph to be trained according to the decryption result and labels, wherein the labels are determined by the first participant according to the historical service execution result of the first participant corresponding to the target node.

In this step, the second party sends the decryption result back to the first party, so that the first party can train the network model of the graph to be trained, which needs to be trained, by using the received decryption result.

Corresponding to the additional embodiments presented in this description as applied to the federal learning method of the first party, a corresponding operation is also required in the second party.

Preferably, in order to protect the model of the first participant, the first participant may scramble the fused encryption feature and then send the scrambled fusion encryption feature to the second participant. Specifically, the encryption output result is obtained by scrambling an intermediate result output by the to-be-trained graph network model of the first participant by using an encryption disturbance feature, and the intermediate result is determined by the first participant inputting the fusion feature into a subsequent processing layer of the to-be-trained graph network model.

Preferably, before encryption, the first feature and the second feature can be adjusted to a specified dimension, so that the first feature and the second feature after the same encryption can be better fused in the same dimension. Specifically, the first characteristic is obtained by the first participant adopting a first parameter to adjust the dimension of a first intermediate characteristic to be a specified dimension, and the first intermediate characteristic is determined by the first participant inputting data corresponding to a target node in the self-stored graph data into a characteristic extraction layer of a graph network model to be trained; and adopting a second parameter to adjust the dimension of the second feature to be a specified dimension.

Preferably, the first participant is actually able to train not only the network model of the graph to be trained, but also a first extraction model that extracts the first features and a first parameter for adjusting the dimensions of the first features. Specifically, the decryption result may be returned to the first participant, so that the first participant uses the minimum difference between the decryption result and the label as an optimization target, and adjusts the parameters of the network model of the graph to be trained and the first parameter.

It should be noted that, in the embodiment provided in the present specification, only the second party provides the feature data to assist the first party in training, which is described as an example, it is conceivable that the training of the other party can be achieved by exchanging the identities of the first party and the second party in the method.

The foregoing is a method for implementing federal learning for one or more of the present specification, and the present specification further provides a corresponding federal learning device based on the same concept, as shown in fig. 4 and 5.

Fig. 4 is a schematic diagram of a federal learning device provided in the present specification, including:

a first determining module 300, configured to determine, from graph data stored by the first participant, a user node shared by the second participant as a target node, and determine a label of the target node according to a historical service execution result corresponding to the target node;

the first extraction module 302 is configured to input data corresponding to the target node into a feature extraction layer of a network model of a graph to be trained, so as to obtain a first feature output by the feature extraction layer;

a first encryption module 304, configured to encrypt the first feature according to a public key of the second party by using a homomorphic encryption algorithm, determine a first encryption feature, and receive a second encryption feature sent by the second party, where the second encryption feature is obtained by encrypting, by the second party, the second feature by using the public key, and the second feature is determined by the second party according to a target node in graph data stored by the second party;

A fusion module 306, configured to determine a fusion feature according to the first encryption feature and the second encryption feature;

the output module 308 is configured to input the fusion feature into a subsequent processing layer of the to-be-trained graph network model, to obtain an encrypted output result output by the to-be-trained graph network model;

a sending module 310, configured to send the encrypted output result to the second party, so that the second party decrypts the encrypted output result by using a private key corresponding to the public key;

and a training module 312, configured to receive a decryption result returned by the second party, and train the network model of the graph to be trained according to the decryption result and the label.

Optionally, the apparatus further includes a scrambling module 314, specifically configured to encrypt a preset disturbance feature by using the public key to obtain an encrypted disturbance feature; scrambling the encryption output result by adopting the encryption disturbance characteristic;

the training module 312 is specifically configured to eliminate the interference added in the decryption result according to the disturbance characteristic; and training the network model of the graph to be trained by adopting the decryption result after interference elimination and the labeling.

the apparatus further comprises a first adjustment module 316, in particular for adjusting the dimension of the first feature to a specified dimension using a first parameter.

Optionally, the training module 312 is specifically configured to adjust the parameter to be trained and the first parameter with a minimum difference between the decryption result and the label as an optimization target.

Fig. 5 is a schematic diagram of a federal learning device provided in the present specification, including:

a second determining module 400, configured to determine, from the graph data stored in the second participant, a user node that is common to the first participant, as a target node;

a second extracting module 402, configured to extract a second feature of data corresponding to the target node;

a second encryption module 404, configured to encrypt the second feature according to the public key of the second party by using a homomorphic encryption algorithm, to determine a second encrypted feature;

The feature sending module 406 is configured to send the second encryption feature to the first participant, so that a fusion feature is determined according to a first encryption feature and the second encryption feature, where the first encryption feature is obtained by encrypting, by the first participant, the first feature by using the public key, and the first feature is determined by the first participant inputting, into a feature extraction layer of a to-be-trained graph network model, data corresponding to a target node in graph data stored by the first participant;

a second receiving module 408, configured to receive an encrypted output result sent by the first participant, where the encrypted output result is determined by the first participant by inputting the fusion feature into a subsequent processing layer of the graph network model to be trained;

a decryption module 410, configured to decrypt the encrypted output result using a private key of the second party;

and a return module 412, configured to return the decryption result to the first participant, so that the first participant trains the graph network model to be trained according to the decryption result and a label, where the label is determined by the first participant according to the historical service execution result corresponding to the target node.

the apparatus further comprises a second adjustment module 414, in particular for adjusting the dimension of the second feature to a specified dimension using a second parameter.

Optionally, the returning module 412 is specifically configured to return the decryption result to the first participant, so that the first participant uses a difference between the decryption result and the label as an optimization target to adjust the parameters of the network model to be trained and the first parameter.

The present specification also provides a computer readable storage medium having stored thereon a computer program operable to perform a method of federal learning as provided in fig. 1 above.

The present specification also provides a schematic structural diagram of an electronic device corresponding to fig. 1 shown in fig. 6. At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile storage, as illustrated in fig. 6, although other hardware required by other services may be included. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs to implement the federal learning method described above with respect to fig. 1. Of course, other implementations, such as logic devices or combinations of hardware and software, are not excluded from the present description, that is, the execution subject of the following processing flows is not limited to each logic unit, but may be hardware or logic devices.

In the 90 s of the 20 th century, improvements to one technology could clearly be distinguished as improvements in hardware (e.g., improvements to circuit structures such as diodes, transistors, switches, etc.) or software (improvements to the process flow). However, with the development of technology, many improvements of the current method flows can be regarded as direct improvements of hardware circuit structures. Designers almost always obtain corresponding hardware circuit structures by programming improved method flows into hardware circuits. Therefore, an improvement of a method flow cannot be said to be realized by a hardware entity module. For example, a programmable logic device (Programmable Logic Device, PLD) (e.g., field programmable gate array (Field Programmable Gate Array, FPGA)) is an integrated circuit whose logic function is determined by the programming of the device by a user. A designer programs to "integrate" a digital system onto a PLD without requiring the chip manufacturer to design and fabricate application-specific integrated circuit chips. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, such programming is mostly implemented by using "logic compiler" software, which is similar to the software compiler used in program development and writing, and the original code before the compiling is also written in a specific programming language, which is called hardware description language (Hardware Description Language, HDL), but not just one of the hdds, but a plurality of kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), lava, lola, myHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog are currently most commonly used. It will also be apparent to those skilled in the art that a hardware circuit implementing the logic method flow can be readily obtained by merely slightly programming the method flow into an integrated circuit using several of the hardware description languages described above.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (e.g., software or firmware) executable by the (micro) processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable logic controllers, and embedded microcontrollers, examples of which include, but are not limited to, the following microcontrollers: ARC625D, atmel AT91SAM, microchip PIC18F26K20, and Silicone Labs C8051F320, the memory controller may also be implemented as part of the control logic of the memory. Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller may thus be regarded as a kind of hardware component, and means for performing various functions included therein may also be regarded as structures within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.

The system, apparatus, module or unit set forth in the above embodiments may be implemented in particular by a computer chip or entity, or by a product having a certain function. One typical implementation is a computer. In particular, the computer may be, for example, a personal computer, a laptop computer, a cellular telephone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being functionally divided into various units, respectively. Of course, the functions of each element may be implemented in one or more software and/or hardware elements when implemented in the present specification.

It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present description is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.

It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The description may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, as relevant to see a section of the description of method embodiments.

The foregoing is merely exemplary of the present disclosure and is not intended to limit the disclosure. Various modifications and alterations to this specification will become apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, or the like, which are within the spirit and principles of the present description, are intended to be included within the scope of the claims of the present description.

Claims

1. A method of federal learning, the method applied to a first party, comprising:

2. The method of claim 1, prior to transmitting the encrypted output to the second party, the method further comprising:

3. The method of claim 1, wherein the second characteristic is obtained by the second party adjusting a dimension of a second intermediate characteristic to a specified dimension using a second parameter, the second intermediate characteristic being determined by the second party according to a target node in the graph data stored by the second party;

4. The method of claim 3, training the network model of the graph to be trained according to the decryption result and the label, specifically comprising:

5. A method of federal learning, the method being applied to a second party, comprising:

extracting a second characteristic of data corresponding to the target node;

6. The method of claim 5, wherein the encrypted output result is obtained by the first party scrambling an intermediate result output by the first party's graph network model to be trained using an encrypted perturbation feature, the intermediate result being determined by the first party inputting the fusion feature into a subsequent processing layer of the graph network model to be trained.

7. The method of claim 5, wherein the first feature is obtained by the first participant adjusting a dimension of a first intermediate feature to a specified dimension by using a first parameter, and the first intermediate feature is determined by the first participant inputting data corresponding to a target node in the graph data stored in the first participant into a feature extraction layer of a graph network model to be trained;

8. The method of claim 7, wherein the step of returning the decryption result to the first party to enable the first party to train the graph network model to be trained by using the decryption result and the label specifically comprises:

9. A federal learning apparatus, comprising:

10. The device of claim 9, further comprising a scrambling module, specifically configured to encrypt a preset disturbance feature by using the public key to obtain an encrypted disturbance feature; scrambling the encryption output result by adopting the encryption disturbance characteristic;

the training module is specifically configured to eliminate interference added in the decryption result according to the disturbance characteristic; and training the network model of the graph to be trained by adopting the decryption result after interference elimination and the labeling.

11. The apparatus of claim 9, the second characteristic being obtained by the second party adjusting a dimension of a second intermediate characteristic to a specified dimension using a second parameter, the second intermediate characteristic being determined by the second party from a target node in self-stored graph data;

the device further comprises a first adjustment module, in particular for adjusting the dimension of the first feature to a specified dimension by adopting a first parameter.

12. The apparatus of claim 10, wherein the training module is specifically configured to adjust the parameter to be trained and the first parameter with a minimum difference between the decryption result and the label as an optimization target.

13. A federal learning apparatus, comprising:

14. The apparatus of claim 13, the encrypted output result is a result of the first party scrambling an intermediate result output by a graph network model to be trained of the first party using an encrypted perturbation feature, the intermediate result being determined by the first party inputting the fusion feature into a subsequent processing layer of the graph network model to be trained.

15. The apparatus of claim 13, wherein the first feature is obtained by the first participant adjusting a dimension of a first intermediate feature to a specified dimension by using a first parameter, and the first intermediate feature is determined by the first participant inputting data corresponding to a target node in the graph data stored in the first participant into a feature extraction layer of a graph network model to be trained;

the device further comprises a second adjustment module, in particular for adjusting the dimension of the second feature to a specified dimension by adopting a second parameter.

16. The apparatus of claim 15, wherein the return module is specifically configured to return the decryption result to the first party, so that the first party adjusts the parameters of the graph network model to be trained and the first parameter with a minimum difference between the decryption result and the label as an optimization target.

17. A computer readable storage medium storing a computer program which, when executed by a processor, implements the method of any of the preceding claims 1-8.

18. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the preceding claims 1-8 when the program is executed.