CN116667996A - Verifiable federal learning method based on mixed homomorphic encryption - Google Patents

Abstract

The invention discloses a verifiable federated learning method based on hybrid homomorphic encryption, which is characterized in that on the client-server infrastructure of centralized federated learning, the hybrid homomorphic encryption technology supporting SIMD operation is used to encrypt data, while ensuring data Simultaneously mobilize the advantages of simple symmetric encryption calculation and no ciphertext expansion while maintaining privacy and correct aggregation, and make up for the shortcomings of public key homomorphic encryption schemes such as complex calculation and serious ciphertext expansion; use Lagrangian interpolation method to construct verification codes, and support clients Check the aggregation results. Specifically include: system initialization, model training and data encryption, aggregation, verification of aggregation results, and model update. Compared with the existing technical solutions, the present invention not only ensures the confidentiality of user gradients, the correctness of aggregation, and the integrity of aggregation results in the federated learning aggregation process, but also reduces the number of public key encryption times, reduces the burden of calculation and communication, and has greater efficiency promote.

Description

A Verifiable Federated Learning Method Based on Hybrid Homomorphic Encryption

technical field

The invention relates to the field of homomorphic encryption of cryptography technology and the field of privacy protection of machine learning, in particular to a verifiable federated learning method based on hybrid homomorphic encryption.

Background technique

With the advent of the fourth industrial revolution, a new generation of artificial intelligence, big data and other technologies have brought opportunities for the intelligent transformation and development of traditional industries. As one of the main methods of big data analysis in recent years, machine learning covers a variety of intelligent algorithms such as deep neural network and regression algorithm, and has been successfully applied in many fields such as medical care, automatic driving, finance and industrial manufacturing. High-quality machine learning models usually require massive amounts of reliable data as training support, so the performance of AI-based services is largely affected by the quality of training data, and large-scale data collection is crucial. However, the limited and poor quality of data in most industries often leads to problems such as overfitting and unreliability of the trained machine learning models, which are not enough to support the realization of artificial intelligence technology. An intuitive solution is data sharing, which integrates and uses data from different industries and different companies in the same industry. In traditional centralized training, the server requires all participants to upload their local data to the cloud. The server then initializes the deep neural network on the cloud and trains it using training samples until optimal parameters are obtained. Finally, the cloud server will release the prediction service interface or return the optimal parameters to all participants. This centralized training method raises a series of data privacy and security issues: the user's local data may contain sensitive private information, for example, in the healthcare system, patients may not be willing to communicate with third-party service providers (such as cloud servers) share their medical data. In addition, there are often hard-to-break barriers between different data sources. In most industries, data exists in the form of silos. Therefore, it is difficult to directly integrate data scattered in various industries and regions, or it is costly and privacy-threatening.

In order to solve the problem of "data islands", the concept of federated learning was proposed: in the process of machine learning, each participant can conduct joint modeling with the help of other parties' data. All parties do not need to share data resources, that is, when the data does not come out of the local area, data joint training is carried out to establish a shared machine learning model. According to different communication architectures, federated learning can be divided into centralized federated learning and decentralized federated learning. Centralized federated learning is based on client-server architecture, while decentralized federated learning is based on end-to-end network architecture. Compared with decentralized federated learning, centralized federated learning is usually simple and effective, and its scalability and stability make it more widely used. In a centralized federated learning scheme, each participant trains a local model locally using private data. In each round of model update, the user uploads the update parameters (gradient) of the local model to the aggregation server. The aggregation server aggregates the update parameters of all users, and returns the aggregated value to each user for model update. However, it has been shown that an adversary can still obtain sensitive information indirectly based on shared gradients. Thus, the research branch of privacy-preserving federated learning was extended.

In order to protect the privacy of participants in federated learning, many strategies have been proposed, among which secure multi-party computation, differential privacy and homomorphic encryption technologies are widely used. Secure multi-party computation allows multiple parties to collaboratively compute a contracted function with private data in a way that each party knows nothing about its inputs and outputs. The principle of differential privacy is to ensure the privacy of each independent sample in the data set by injecting noise into the data set to blur sensitive information, making it impossible for third parties to distinguish individuals. Homomorphic encryption allows certain computations to be performed directly on ciphertexts without decrypting them first. Compared with multi-party secure computing and differential privacy methods, homomorphic encryption technology has stronger privacy protection and has little impact on the accuracy of the training model. However, because the commonly used homomorphic encryption technology belongs to the public key cryptosystem, it has a long key and requires complex mathematical calculations, which will lead to expensive calculation overhead and ciphertext expansion. Therefore, the development of federated learning systems based on homomorphic encryption techniques is limited by high computational and communication overhead bottlenecks. In the field of cryptography research, some scholars have recently proposed the concept of hybrid homomorphic encryption, which combines symmetric encryption and public key homomorphic encryption, and proposes a method that supports SIMD (Single Instruction Multiple Data, single instruction multiple data) operations. Feasible examples of pasta (a symmetric stream encryption algorithm cluster) symmetric cipher and BFV (a fully homomorphic encryption scheme based on the RLWE (Ring-Learning With Errors, fault-tolerant learning on the ring) problem) fusion of homomorphic ciphers (hereinafter referred to as for the "Pasta+BFV" scheme). The computationally efficient characteristics of symmetric cryptography are complementing homomorphic encryption, which provides a new idea for privacy-preserving federated learning schemes.

In addition, there are data integrity issues in federated learning. The aggregation server exists as a third party, which can easily cause a single point of failure problem. Without integrity guarantees, once a server is compromised, an adversary controlling the server can manipulate the global model. Malicious servers can reversely analyze the private data of the participants or destroy the user's local model by forging the aggregation results, resulting in wrong classification results.

To sum up, protecting user privacy and data integrity are two fundamental issues in the federated learning training process. In addition, in the federated learning scheme based on homomorphic encryption, how to reduce the number of public key encryptions, reduce the burden of calculation and communication, and balance security and efficiency while achieving strong privacy protection to protect data security is an open question. problem. Therefore, it is urgent and meaningful to design a safe and efficient verifiable federated learning method.

Contents of the invention

In order to solve the privacy security, data integrity and efficiency problems existing in the field of federated learning described in the background technology, the purpose of the present invention is to provide a verifiable federated learning method based on hybrid homomorphic encryption. This method is applicable to the centralized federated learning scenario under the client-server architecture, allowing all parties to rely on the aggregation server for data joint training when the source data does not go out of the local area, which solves the "data island" problem. Aiming at privacy, security and efficiency issues, this invention uses the "Pasta+BFV" hybrid homomorphic encryption technology that supports SIMD, and introduces the idea of "symmetric transient homomorphism" to transfer complex calculations from the client to the server to reduce the burden on users , and alleviate the communication pressure caused by ciphertext expansion; based on BFV homomorphic encryption, it can be correctly aggregated under encrypted conditions to protect the privacy of user source data; based on SIMD packaging, it can encrypt multiple data at one time and reduce the number of encryptions. For data integrity issues, the Lagrangian interpolation method is used to construct verification codes to support the client to verify the aggregation results.

The concrete technical scheme that realizes the object of the invention is:

A verifiable federated learning method based on hybrid homomorphic encryption, including entities: PKG (Public Key Generator), n clients, and aggregation servers; used in federated learning scenarios under client-server architecture, features Yes, the method includes the following specific steps:

Step A: Initialize

The n clients first negotiate according to business needs, and reach an agreement on the training model; the key generation agency PKG generates and initializes the global model, key and public parameters, and distributes them to the client and the aggregation server as required;

Step B: Model Training and Data Encryption

In each round of model update, each client uses local data to train the local model, and calculates the model update parameters of this round of iteration; then, preprocesses the update parameters, constructs the verification code and composes the plaintext; finally, the plaintext Pasta symmetric encryption, and the Pasta symmetric ciphertext is sent to the aggregation server;

Step C: Aggregation

After the aggregation server receives the Pasta symmetric ciphertext from all clients, it first converts the ciphertext to obtain the BFV homomorphic ciphertext; then aggregates all the BFV homomorphic ciphertext and sends the aggregation result to all clients;

Step D: Aggregation result verification and model update

After each client receives the aggregation result from the aggregation server, it performs homomorphic decryption; then, it relies on the Lagrangian interpolation method to verify the aggregation result, and if the verification is passed, the aggregation result is used to update the model; otherwise, the aggregation result is discarded; then , enter the next round of iterations until the model converges or reaches the maximum number of federation rounds.

Wherein, the step A specifically includes:

A1: Model initialization

Each client first reaches an agreement on the training model according to the training goal. According to this agreement, PKG generates an initial global model, learning rate, gradient quantization accuracy, mapping finite field, and the maximum number of federation rounds, and distributes it to all clients as the initial local model. ;

A2: Key initialization

PKG generates Pasta encryption scheme, BFV encryption scheme and security parameter λ; then generates a Pasta key for all clients and distributes it to each client; then, generates a set of public BFV keys, BFV keys include public key, private key key and calculation key, where the public key and calculation key are open to all participants, that is, n clients and the aggregation server, and the private key is shared with all clients, but kept secret to the aggregation server; then, use the BFV public key to encrypt in turn The pasta key of each client forms a user list and sends it to the aggregation server;

A3: Public parameter initialization

PKG generates a parameter sequence for Lagrangian interpolation and sends it to all clients.

Described step B specifically comprises:

B1: Model training

Each client uses the local private data set to train the local model, and calculates the loss function of the current round of training and the gradient used for updating;

B2: Data Preprocessing

In order to encrypt the gradient, the gradient is preprocessed to convert it into a form suitable for "PASTA+BFV" hybrid homomorphic encryption; specifically, firstly, the floating-point gradient is quantized and converted into an integer, and then the quantized gradient Mapped to a finite field to adapt to the encryption algorithm; finally, all the data to be encrypted are grouped according to the threshold parameter of the SIMD operation supported by Pasta;

B3: Construct verification code

For each preprocessed gradient packet, a verification code is constructed using Lagrangian interpolation;

B4: encryption

Construct an encrypted plaintext vector based on a preprocessed gradient group and its verification code; use the Pasta password to encrypt the plaintext vector, and send the Pasta symmetric ciphertext to the aggregation server.

Described step C specifically comprises:

C1: Ciphertext conversion

After the aggregation server receives the Pasta symmetric ciphertext from the client, it first retrieves the user's Pasta key encrypted by BFV from the user list received from PKG during the initialization process; then, according to the fully homomorphic BFV cipher , decrypt the symmetric ciphertext homomorphically to convert it into a BFV homomorphic ciphertext;

C2: aggregation

After converting all received Pasta symmetric ciphertexts into BFV homomorphic ciphertexts, the aggregation server aggregates the BFV homomorphic ciphertexts, and then sends the aggregation results to each client.

Described step D specifically comprises:

D1: Verification

After receiving the aggregation result from the aggregation server, the client decrypts it and divides the aggregated gradient grouping and aggregation verification code; then uses the Lagrangian interpolation method to construct the verification code for the aggregated gradient grouping, and verifies the verification code Whether it is equal to the aggregation verification code; if they are equal, the verification is successful; otherwise, the verification fails;

D2: Update the model

If the verification is successful, the user processes the aggregated gradient, that is, the inverse operation in the data preprocessing of the model training and data encryption process; then uses the restored aggregated gradient to update the model; if the verification fails, the aggregated value is discarded;

D3: Iterate into the next round of training, that is, repeat steps B to D until the model converges or reaches the maximum number of federation rounds.

Compared with the existing privacy protection federated learning method, the beneficial effects of the present invention are:

(1) The present invention applies the concept of hybrid homomorphic encryption to the federated learning scenario, mobilizes the advantages of simple symmetric encryption calculation and no ciphertext expansion, and makes up for the shortcomings of public key homomorphic encryption schemes that are complex in calculation and serious ciphertext expansion. In the existing federated learning method based on homomorphic encryption, the client usually directly uses the public key homomorphic encryption scheme to encrypt the gradient, and then transmits the homomorphic ciphertext to the aggregation server to calculate the aggregation result. However, the ciphertext produced by a public-key encryption scheme is usually much longer than the plaintext, and its inflation factor depends on the security parameters of the encryption scheme. In order to guarantee the security of the cryptographic scheme, this security parameter needs to be large enough. It has been shown that implementing this type of federated learning method results in more than 150 times more data transfer than without encryption. In addition, the implementation of the public key homomorphic encryption scheme needs to perform complex cryptographic operations (such as modular multiplication and exponentiation), which will bring great computational pressure to some clients with limited computing power. In the present invention, the client uses a symmetric encryption algorithm to encrypt the gradient, and then transmits the symmetric ciphertext to the aggregation server to calculate the aggregation result. Note that the symmetric encryption scheme is simple to calculate, and the plain and ciphertext lengths are equal, that is, the ciphertext expansion factor is 1. This shifts the computing burden from the client with limited computing power to the aggregation server, reducing the computing overhead of the client and alleviating the communication pressure caused by the ciphertext expansion problem of public key encryption.

(2) The hybrid homomorphic encryption scheme used in the present invention is the "Pasta+BFV" scheme, which supports the SIMD idea. The BFV homomorphic encryption scheme supports polynomial packaging, that is, encoding a plaintext vector into a polynomial, and converting the encryption of the vector into polynomial encryption. Furthermore, homomorphic operations on ciphertexts are equivalent to element-to-element operations on vectors. In order to continue to use the packaging advantages of BFV in the hybrid homomorphic encryption scheme, the present invention selects the Pasta symmetric encryption scheme that supports the SIMD idea. A Pasta encryption scheme with a SIMD operation threshold of 1 can be transformed into a BFV homomorphic ciphertext at one time after encrypting plaintext. Compared with only encrypting one plaintext at a time and performing ciphertext conversion one by one, the number of encryption and decryption on the client side and the number of ciphertext conversions on the aggregation server side are reduced to Improved computational efficiency.

(3) The present invention uses the Lagrangian interpolation method to realize the verification of the aggregation result. Not only data integrity is guaranteed, but the calculation is simpler and more efficient than using homomorphic hashing and the general method based on linear pairing.

Description of drawings

Fig. 1 is a structure diagram of the present invention;

Fig. 2 is a flowchart of the present invention;

Fig. 3 is a schematic diagram of the gradient grouping method in the present invention.

Detailed ways

The present invention will be further described in detail in conjunction with the following specific embodiments and accompanying drawings. The process, conditions, experimental methods, etc. for implementing the present invention are general knowledge and common knowledge in the art except for the content specifically mentioned below, and the present invention has no special limitation content. It should be noted that like numerals and letters denote similar items in the drawings, therefore, once an item is defined in one drawing, it does not require further definition and explanation in subsequent drawings.

Explanation of terms:

(1) Symmetric Key Encryption (SKE): According to whether the keys used for encryption and decryption are the same, the cryptosystem is divided into symmetric cryptosystem and asymmetric cryptosystem, and asymmetric cryptosystem is also called public key cryptosystem. Symmetric encryption is a cryptographic system that uses the same key for encryption and decryption. Symmetric ciphers use a key and an encryption algorithm to convert plaintext into ciphertext. Using the same key and decryption algorithm, the plaintext can be recovered from the ciphertext.

A symmetric encryption scheme SKE consists of three probabilistic polynomial time algorithms:

SKE = (SKE.keyGen, SKE.Enc.SKE.Dec).

Among them, SKE.keyGen is the key generation algorithm, SKE.Enc is the encryption algorithm, and SKe.Dec is the decryption algorithm.

(2) Homomorphic Encryption (HE): Homomorphic encryption allows certain computations to be performed directly on ciphertexts without decrypting them first. Commonly used homomorphic encryption schemes all belong to public key cryptography. Note that the encrypted ciphertexts of the given plaintext m ₁ and m ₂ are respectively Then the homomorphism of homomorphic encryption is expressed as:

Additive homomorphism:

Multiplicative homomorphism:

According to the types and times of operations supported by homomorphic encryption, it can be divided into partial homomorphic encryption, partial homomorphic encryption and full homomorphic encryption. Partially homomorphic encryption only supports homomorphism for addition or multiplication operations. Some homomorphic encryption supports a limited number of addition and multiplication operations; fully homomorphic encryption supports the homomorphism of any calculation on the ciphertext, and does not limit the number of calculations. The BFV encryption scheme adopted in the present invention is a fully homomorphic encryption scheme.

A public key homomorphic encryption scheme HE consists of four probabilistic polynomial time algorithms:

HE=(HE.keyGen, HE.Enc, HE.Dec, HE.Eval).

Among them, HE.keyGen is a key generation algorithm, which outputs a set of public key homomorphic keys, including public key, private key and calculation key. HE.Enc and HE.Dec are encryption algorithm and decryption algorithm respectively. HE.Eval is a homomorphic evaluation algorithm. Input the calculation key, target calculation function and ciphertext. After decryption, the output result of the algorithm is equal to the value obtained by directly using the target calculation function to operate on the original plaintext.

(3) Hybrid Homomorphic Encryption (HHE): Considering that the commonly used homomorphic encryption technology belongs to the public key cryptosystem, it has a long key and requires complex mathematical calculations, which will lead to expensive calculation overhead and ciphertext expansion problem. Some researchers have proposed the concept of hybrid homomorphic encryption. The main idea is: use a symmetric encryption scheme with a ciphertext expansion factor of 1 to encrypt data instead of a homomorphic encryption with a large expansion factor, then use a homomorphic encryption to encrypt the key of the symmetric encryption scheme, and combine it with the symmetric ciphertext sent to the cloud service provider. Next, the cloud service provider first executes the symmetric decryption circuit homomorphically to convert the symmetric ciphertext into homomorphic ciphertext, and then continues to perform required computing operations. The definition of hybrid homomorphic encryption will be derived from a public key homomorphic encryption scheme and a symmetric encryption scheme as follows:

Based on a public key homomorphic encryption scheme HE and a symmetric encryption scheme SKE, a hybrid homomorphic encryption scheme HHE can be constructed, which consists of five probabilistic polynomial time algorithms:

HHE=(HHE.keyHen, HHE.Enc, HHE.Decomp, HHE.Eval, HHE.Dec).

The key generation algorithm HHE.keyGen calls HE.keyGen and SKE.keyGen; the encryption algorithm HHE.Enc calls SKE.Enc to encrypt the plaintext, and calls HE.Enc to encrypt the symmetric key; the ciphertext conversion algorithm HHE.Decomp Call HE.Rval, execute the decryption circuit of the symmetric cipher homomorphically, and convert the symmetric ciphertext into the homomorphic ciphertext; the homomorphic evaluation algorithm HHW.Eval and the decryption algorithm HHE.Dec directly call HE.Eval and HE.Dec respectively.

A finite field, also known as a Galois field, is a field containing only a finite number of elements, and F _q represents a finite field containing q elements. Homomorphic encryption schemes applied in federated learning scenarios generally take the finite field F _q as the input plaintext space, and many popular homomorphic encryption algorithms (such as BFV) support SIMD operations: encode a plaintext vector into a polynomial, and The encryption of the vector is transformed into polynomial encryption, so the homomorphic operation on the ciphertext is equivalent to the operation between elements on the vector. In order to maintain this feature in the hybrid homomorphic encryption scheme, many scholars have conducted research on symmetric encryption schemes that meet the corresponding requirements, and have obtained some results, such as Pasta. Therefore, in order to rely on SIMD to obtain further efficiency improvement, the present invention selects the Pasta symmetric encryption scheme and the BFV homomorphic encryption scheme to construct a hybrid homomorphic encryption scheme, wherein Pasta is A family of symmetric stream encryption algorithms on is a dimensional vector space on the finite field F _q , and q is generally selected as a large prime number satisfying 2 ⁽¹⁶⁾ <q<2 ⁽⁶⁰⁾ , which is the threshold for SIMD operation, that is, batch conversion. According to the given q and security level requirements, suitable can be calculated. In the "Pasta+BFV" encryption scheme constituted in this way, the encrypted plaintext over a finite field F _q can be transformed into a BFV homomorphic ciphertext at one time.

It should be noted that the above statement about the composition of the hybrid homomorphic encryption scheme algorithm is only to illustrate the construction method and execution process of the hybrid homomorphic encryption, and in the above definition, the hybrid homomorphic encryption algorithm is accompanied by each time the plaintext is encrypted. The operation of the homomorphic encryption symmetric cipher, and these operations can be simplified and divided in practical applications. Therefore, the following description of the embodiments of the present invention does not directly use the formal definition form of the hybrid homomorphic encryption scheme, but based on the idea of hybrid homomorphic encryption, the terms of basic homomorphic encryption and symmetric encryption are used for description.

(4) Lagrange interpolation method: Lagrange interpolation method is a polynomial interpolation method. Given +1 points with different coordinates, Lagrangian interpolation can give an order polynomial function that passes through exactly these +1 points. The calculation idea is to first calculate the node basis function on the given node, and then do the linear combination of the basis functions to obtain an interpolation polynomial whose combination coefficient is the value of the node function.

The specific calculation process is described as follows:

Given +1 different interpolation points x _i , (i=0,1,...,n.), and corresponding values f(xi ₎ . First compute the interpolation basis functions:

Obviously, l _i (x) is also a polynomial of order n, and satisfies

Then do a linear combination of the basis functions:

Thus, an n-order polynomial L _n (x) is obtained, which obviously satisfies L _n (n _i )=f(xi ₎ , that is, a Lagrangian interpolation polynomial passing through n+1 given interpolation points.

Example

Referring to Fig. 1, the present invention adopts a centralized federated learning structure based on a client-server architecture, including three types of entities: a key generation organization PKG, n clients, and an aggregation server. The key generation organization is responsible for parameter initialization and key distribution, and will no longer participate in the subsequent process after completing the initialization task. Each client P _i , (+∈N, N={1,2,…,n}) has a private data set D _i ={<x _j ,y _j >|j=1,2,…, T}, where x _j is the input, y _j is the label, and T = |D _i | denotes the size of the dataset. Using this data set, the client trains a local model f(x,M) locally, where x is the input and M is the model parameter. The goal of training is to obtain the model parameters that minimize the loss function, that is, to achieve model convergence. Therefore, in each round of model update, P _i selects a random subset of D _i to train the model locally, and then calculates the gradient W _i of the loss function. In order to speed up model convergence and make up for the lack of local data, each client does not directly use the local gradient W _i to update, but obtains the aggregate value of the local gradient and other client gradients, that is, the global gradient to update. Therefore, each client encrypts the local gradient and adds a verification code to obtain the ciphertext _ci and uploads it, and requests the aggregation server to aggregate the local gradients of all clients. After the aggregation server receives the ciphertext from all clients, it aggregates the ciphertext to obtain C and sends it back to each client. After each client receives the aggregated ciphertext, it unpacks the ciphertext to obtain the aggregated verification code and the global gradient W _globPl . According to the verification result, the client decides whether to use W _globPl to update the model, and then enters the next round of iteration until the model converges or reaches the agreed maximum number of federation rounds.

Referring to Fig. 2, the present invention proposes a verifiable federated learning method based on hybrid homomorphic encryption, including the following steps:

Step A: Initialize

N clients negotiate according to business requirements and reach an agreement on the training model; the key generation agency PKG generates and initializes the global model, keys and public parameters, and distributes them to clients and aggregation servers as required;

Step B: Model Training and Data Encryption

In each round of model update, each client uses local data to train the local model, and calculates the model update parameters of this round of iteration; then, preprocesses the update parameters, constructs the verification code and composes the plaintext; finally, the plaintext Pasta symmetric encryption, and the Pasta symmetric ciphertext is sent to the aggregation server;

Step C: Aggregation

After the aggregation server receives the Pasta symmetric ciphertext from all clients, it first converts the ciphertext to obtain the BFV homomorphic ciphertext; then aggregates all the BFV homomorphic ciphertext and sends the aggregation result to all clients;

Step D: Aggregation result verification and model update

After each client receives the aggregation result from the aggregation server, it performs homomorphic decryption; then, it relies on the Lagrangian interpolation method to verify the aggregation result, and if the verification is passed, the aggregation result is used to update the model; otherwise, the aggregation result is discarded; then , to enter the next round of iterations, that is, repeat steps B to D until the model converges or reaches the maximum number of federation rounds.

Described step A specifically comprises:

Step A1: Model initialization. All clients reach an agreement on the training model according to the training goal, and PKG generates an initial global model d(x,M), learning rate η, gradient quantization accuracy l _w , mapping finite field F _q and the model's maximum federated model according to this agreement The number of rounds r _max is distributed to all clients as the initial model, that is, the local model used in the first round of model update.

Step A2: Key initialization. PKG calculates the SIMD operation threshold t according to the finite field F _q and security level requirements, and generates Pasta encryption scheme, BFV encryption scheme and security parameters on the above. Then, for each client P _i generate a Pasta key _ki that is kept secret from other clients P _j , (j∈N, j≠i) and the aggregation server. Next, generate public BFV keys (pk, sk, evk), where the public key pk and calculation key evk are open to all participants (n clients and aggregation servers), and the private key sk is shared with all clients. It is kept secret from the aggregation server. Next, use the BFV public key pk to encrypt the pasta key of each client in turn, that is, for the client P _i , i∈N, calculate /> form user list sent to the aggregation server.

Step A3: Public parameter initialization. PKG generates a parameter sequence {a ₁ ,a ₂ ,…,a _t } for Lagrangian interpolation and sends it to all clients. where t is the SIMD operation threshold of the Pasta encryption scheme.

Step B of the present invention specifically includes:

Step B1: Model training. In each round of model update, client P _i randomly selects a subset of the local private dataset Train the local model and calculate the loss function of the current round of training /> and for updating gradients /> where, /> is the gradient operator. remember Where n _gn =|W _i | is the length of W _i .

Step B2: Data preprocessing. To encrypt the gradients, the gradients are preprocessed to transform them into a form suitable for "PASTA+BFV" hybrid homomorphic encryption. Specifically, first, the floating-point vector gradient W _i is quantized and converted into an integer vector, and then the quantized gradient is mapped to the finite field F _q to obtain The specific calculation process is: for each component of the gradient vector /> calculation /> get Among them, (·) _round and ψ(·) are quantization function and finite field mapping function respectively:

Represents the largest integer less than or equal to x.

Finally, referring to Figure 3, all the data to be encrypted are grouped according to the threshold parameter of the SIMD operation supported by Pasta: consecutive -1 components form a group, and when the number of the last group of components is less than -1, it is filled with zeros, so grouping total where /> Represents the largest integer greater than or equal to x. Record the jth group as /> Then />

Step B3: Construct the verification code. For each preprocessed gradient group Use the Lagrange interpolation method to construct a verification code, the specific process is: according to -1 interpolation point /> Find a Lagrangian polynomial of order t-2 /> Then substitute in to get the verification code />

Step B4: Encryption. for each group and its corresponding verification code /> A plaintext vector can be constructed /> Use the Pasta cipher to encrypt the plaintext vector to obtain the symmetric ciphertext And send all Pasta symmetric ciphertexts to the aggregation server.

Step C of the present invention specifically includes:

Step C1: Ciphertext conversion. The aggregation server receives the pasta symmetric ciphertext from the client Afterwards, for each client P _i from the user list from PKG received in step A2 /> retrieve the pasta key encrypted by the user via BFV /> Then according to the fully homomorphic property of the BFV cipher, the symmetric ciphertext is decrypted homomorphically to convert it into a BFV homomorphic ciphertext />

Step C2: Polymerization. After converting all received Pasta symmetric ciphertexts into BFV ciphertexts, the aggregation server aggregates the BFV ciphertexts where h is the aggregation function. Then the aggregation result (C ^(j) ) _HE ,j∈N _group is sent to each client.

Step D of the present invention specifically includes:

Step D1: Verification. After the client receives the aggregation result from the aggregation server, it decrypts m ^(j) = BFV.Dec((R ^(j) ) _HE ), j∈N _group , according to m ^(j) = G ^(j) | |v ^(j) obtains the aggregated gradient group G ^(j) and the aggregated verification code v ^(j) . Then group the aggregated gradients Use the Lagrange interpolation method to construct the check code: that is, for each j∈N _group , according to t-1 interpolation points Construct the Lagrange polynomial (L _t-2 (x)) ^(j) , and then substitute a _t to get the verification code (L _t-2 (a _t )) ^(j) . Then, verify whether the verification code (L _t-2 ( _at ) ^(j) is equal to the aggregation verification code v ^(j) . If they are equal, the verification is successful; otherwise, the verification fails.

Step D2: Update the model. If the verification is successful, the aggregated gradient can be obtained according to all aggregated groups Relying on the additive homomorphism of BFV, it can be proved that /> That is, the correctness of the aggregation process. Therefore, the client restores the aggregated gradient, that is, for each component /> Perform the inverse of the data preprocessing step B2 /> get restored gradients /> Each client then updates the model with the recovered aggregated gradients If validation fails, the aggregate value is discarded.

Step D3: Enter the next round of training iteratively, that is, repeat steps B1 to D3 until the model converges or reaches the maximum number of federation rounds r _max .

In the embodiment, the client P _* +∈N, N={1,2,...,n}) is taken as an example to describe the client-related operations in the process of the present invention. It should be noted that P _i is not limited to a certain specific client, while representing all clients will operate equally.

The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still carry out the foregoing embodiments The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (5)

1. A verifiable federal learning method based on hybrid homomorphic encryption, comprising the entities: a key generation mechanism PKG, a personal client and an aggregation server; the federation learning scenario for use in a client-server architecture, the method comprising the specific steps of:

step A: initialization of

The individual clients negotiate according to the service requirements, and the agreement is achieved by the training model; the PKG generates an initialized global model, a secret key and public parameters and distributes the initialized global model, the secret key and the public parameters to the client and the aggregation server according to requirements;

and (B) step (B): model training and data encryption

In each round of model updating, each client trains a local model by using local data, and calculates model updating parameters of the round of iteration; then, preprocessing the updated parameters, constructing verification codes and forming a plaintext; finally, encrypting the plaintext by using a symmetrical encryption scheme Pasta to obtain a Pasta symmetrical ciphertext, and transmitting the ciphertext to the aggregation server;

step C: polymerization

After receiving the Pasta symmetric ciphertext from all clients, the aggregation server firstly uses the full homomorphic encryption scheme BFV to carry out ciphertext conversion to obtain BFV homomorphic ciphertext; then, all BFV homomorphic ciphertexts are aggregated, and an aggregation result is sent to all clients;

step D: aggregation result verification and model update

After receiving the aggregation result from the aggregation server, each client performs homomorphic decryption; then, checking the aggregation result by means of a Lagrange interpolation method, if the verification is passed, updating a model by using the aggregation result, otherwise, discarding the aggregation result; the next iteration is then entered until the model converges or the maximum federal number of rounds is reached.

2. The verifiable federal learning method based on hybrid homomorphic encryption according to claim 1, wherein said step a specifically comprises:

a1: model initialization

The individual clients firstly reach a contract for a training model according to a training target, and PKG generates an initialization global model, a learning rate, gradient quantization precision, a mapping finite field and a maximum federal round number according to the contract and distributes the initialization global model, the learning rate, the gradient quantization precision, the mapping finite field and the maximum federal round number to all the clients as initial local models;

a2: key initialization

PKG generates a Pasta encryption scheme, a BFV encryption scheme and security parameters; then generating a Pasta key for all clients and distributing the Pasta key to each client; then, generating a group of public BFV keys, wherein the BFV keys comprise public keys, private keys and calculation keys, the public keys and the calculation keys are disclosed to all the participants, namely, individual clients and an aggregation server, the private keys are shared to all the clients, and the private keys are kept secret from the aggregation server; then, the Pasta keys of all clients are encrypted by using BFV public keys in sequence to form a user list and then sent to the aggregation server;

a3: public parameter initialization

The PKG generates a sequence of parameters for lagrangian interpolation and sends it to all clients.

3. The verifiable federal learning method based on hybrid homomorphic encryption according to claim 1, wherein said step B specifically comprises:

b1: model training

In each round of model updating, each client trains a local model by using a local private data set, and calculates a loss function of the round of training and a gradient for updating;

b2: data preprocessing

To encrypt the gradient, the gradient is preprocessed to convert it into a form suitable for "pasta+bfv" mixed homomorphic encryption; firstly, carrying out quantization conversion on floating point number gradients to form integers, and then mapping the quantized gradients onto a finite field to adapt to an encryption algorithm; finally, grouping all data to be encrypted according to threshold parameters of batch conversion operation supported by Pasta;

b3: construction verification code

For each preprocessed gradient group, constructing a verification code by adopting a Lagrange interpolation method;

b4: encryption

Constructing a once-encrypted plaintext vector according to a preprocessed gradient packet and a verification code thereof; the plaintext vector is encrypted using a Pasta cipher and the Pasta symmetric ciphertext is sent to the aggregation server.

4. The verifiable federal learning method based on hybrid homomorphic encryption according to claim 1, wherein said step C specifically comprises:

c1: ciphertext conversion

After receiving the Pasta symmetric ciphertext from the client, the aggregation server firstly retrieves the Pasta key encrypted by the user through BFV from a user list from PKG received in the initialization process; then, according to the full homomorphism of the BFV password, homomorphically decrypting the symmetric ciphertext to convert the symmetric ciphertext into a BFV homomorphism ciphertext;

c2: polymerization

After all the received Pasta symmetric ciphertexts are converted into BFV homomorphic ciphertexts, the aggregation server aggregates the BFV homomorphic ciphertexts, and then the aggregation result is sent to each client.

5. The verifiable federal learning method based on hybrid homomorphic encryption according to claim 1, wherein said step D comprises:

d1: verification

After receiving the aggregation result from the aggregation server, the client decrypts the aggregation result to divide an aggregated gradient packet and an aggregation verification code; then constructing a check code for the aggregated gradient group by using a Lagrangian interpolation method, and verifying whether the check code is equal to the aggregated check code or not; if the two types of data are equal, the verification is successful; otherwise, the verification fails;

d2: updating a model

If the verification is successful, the user processes the aggregated gradient, namely, the inverse operation in the data preprocessing of the model training and data encryption process; updating the model using the restored aggregation gradient; discarding the aggregate value if the verification fails;

d3: and (3) iteratively entering the next training round, namely repeatedly executing the steps B to D until the model converges or the maximum federal round number is reached.