CN115310121B - Real-time reinforced federal learning data privacy security method based on MePC-F model in Internet of vehicles - Google Patents

Abstract

The invention discloses a real-time reinforced federal learning data privacy security method based on a MePC-F model in the internet of vehicles, which comprises the following steps: building multiple edge servers E _i And a cloud server CS; edge server E _i Downloading initial type A gradients from cloud server CS

And decrypted into

Random initialization of type B gradients

Carrying out local model training; edge server E _i By a decoding function from

To obtain partial gradient information to be preserved

And the remaining gradient information is used

Is homomorphically encrypted as

Then broadcast and send to all other edge servers E through MePC algorithm _j (ii) a The class A gradient information after all the edge servers are updated and shared is respectively

All edge servers will

Uploading the global parameters to a cloud server CS, and aggregating the global parameters by the cloud server CS through a PreFLa algorithm; the above steps are repeated until a termination condition is reached. The invention prevents data leakage between terminals, realizes data privacy safety protection, and reduces communication overhead while preventing original data leakage.

Description

Real-time enhanced federated learning data privacy security based on MePC-F model in Internet of Vehicles Method

Technical Field

The present invention relates to the technical field of real-time safety behavior analysis for collaborative processing by networked vehicle users, and in particular to a real-time enhanced federated learning data privacy security method based on a MePC-F model in a network of vehicles.

Background Art

With the development of the Internet of Vehicles supporting various real-time communications and services, the amount of data generated by interconnected devices such as vehicle-mounted units is unprecedentedly huge. Faced with the large amount of heterogeneous data of vehicle users and the differences in device computing capabilities, federated learning provides an effective solution to meet the data security protection requirements during real-time training of network models. It allows different edge devices to collaboratively train machine learning models without exposing the original data.

The massive data of edge computing is closely related to the personal privacy of users. For example, the user's trajectory, credit card, bill and other data are closely related to the privacy and security of users. If data leakage occurs, it will bring major security risks to users. Federated learning can protect data to a certain extent, but there is still the risk of information leakage, which mainly includes four types: 1) member leakage, 2) unexpected feature leakage, 3) class representing the original data leakage, and 4) original data leakage. The last type of data leakage is the most unacceptable for privacy-sensitive participants.

In order to protect the data privacy of mobile users and solve the above-mentioned original data leakage problem, researchers have conducted a lot of research on data security protection based on cryptography: differential privacy, homomorphic encryption, and multi-party secure computing. Differential privacy usually uses three noise addition mechanisms: Laplace mechanism, Gaussian mechanism, and exponential mechanism. By adding noise to interfere with contextual information, the privacy of data is protected, but if the noise increases too much, the performance of model training will be affected. Additive and multiplicative homomorphic encryption are commonly used in homomorphic encryption: studies have shown that when using Paillier additive homomorphic encryption, the noise will double, while when using El Gamal multiplicative homomorphic encryption, the noise will increase quadratically. In order to increase the availability of data and overcome the noise problem, researchers introduced bootstrapping, which reduces noise by setting a threshold for encryption and decryption, allowing the scheme to calculate an unlimited number of operations. Batch processing, parallel homomorphic computing, or deletion of compression can also be performed to solve the noise problem. Secure multi-party computing refers to the problem of multiple participants securely calculating an agreed function without a trusted third party. The main purpose is to ensure that the private inputs of all parties are independent during the calculation process and no local data is leaked during the calculation. Research has shown that using secure multi-party computing can solve the gradient leakage problem in federated learning, and that only information exchange on the first hidden layer is needed to ensure accuracy while protecting data security. However, the information exchange process is P2P, so there will be a problem of high communication overhead.

Most cryptography-based data security protection research is a centralized solution. In order to solve the time overhead problem while protecting data security: Federated learning allows edge devices to collaboratively train machine learning models without exposing raw data. Federated learning usually adopts a parameter server architecture, in which the client synchronizes local model training by the parameter server. It is usually implemented using a synchronous method, that is, the central server sends the global model to multiple clients synchronously, and multiple clients train the model based on local data and then synchronously return the updated model to the central server. This may become slow due to falling behind. Due to limited computing power and battery time, availability and completion time vary from device to device, so global synchronization is very difficult, especially in federated learning scenarios. A new joint optimization asynchronous algorithm has been proposed to solve the regularization local problem to ensure convergence, so that multiple devices and servers can collaboratively and efficiently train models without leaking privacy.

Although there are many studies on data security, most of them are limited to solving the original data security problem. How to meet the privacy and availability of mobile users' big data in the complex Internet of Vehicles space, and design an effective federated learning algorithm to reduce communication overhead while preventing data recovery due to gradient leakage are still open issues.

First, in federated learning, data is stored in local nodes, which can reduce the risk of original data leakage during data transmission. However, if only gradient information is transmitted, there is still a possibility that the original data can be recovered. Data interaction in secure multi-party computing allows multiple parties to have data, reducing the possibility of sample information recovery after gradient information is leaked. However, the existing way of user interaction information in secure multi-party computing is that all users send it to other users, which is simply to use unicast, which will bring high time overhead. Therefore, when dealing with the data security and real-time requirements of vehicle users, it is important to find a suitable solution to reduce the risk of data being attacked and recovered, and reduce transmission delay. Secondly, due to the differences in data and devices of different edge servers, it is also necessary to improve the training accuracy of the entire model in a targeted manner during the training process. Using the typical federated average synchronization method for global parameter aggregation will cause falling behind and become slow. While balancing the computational communication time overhead, it is also important to train multiple models in a personalized way to ensure global accuracy. However, most federated learning algorithms based on data security rely on synchronous aggregation algorithms, which will bring high latency and are challenging to meet the real-time requirements of the Internet of Vehicles. Therefore, a federated learning algorithm based on reinforcement learning is necessary to reduce latency, improve accuracy and ensure data security.

Summary of the invention

The technical problem to be solved by the present invention is to provide a real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles in response to the defects in the prior art.

The technical solution adopted by the present invention to solve the technical problem is:

The present invention provides a real-time enhanced federated learning data privacy security method based on the MePC-F model in an Internet of Vehicles, the method comprising the following steps:

S1. Build multiple edge servers E _i and a cloud server CS; obtain vehicle data D = {D ₁ , D ₂ , ..., D _i }, and the edge server E _i obtains the corresponding vehicle data D _i ;

并解密为

随机初始化B型梯度

边缘服务器E_i根据其车辆数据D_i的最小化损失函数来计算本地网络模型训练中的梯度，边缘服务器E_i完成T轮本地训练完后的梯度信息记为

S2. In the kth round of federated tasks, the edge server E _i downloads the initial A-type gradient from the cloud server CS

and decrypted to

Randomly initialize B-type gradient

The edge server E _i calculates the gradient in the local network model training according to the minimized loss function of its vehicle data D _i . The gradient information of the edge server E _i after completing T rounds of local training is recorded as

从

中获取需要保留的部分梯度信息

并将剩余的梯度信息

经同态加密为

再通过MePC算法广播发送给其它所有的边缘服务器E_j；边缘服务器E_i根据解码函数

获取来自其它边缘服务器E_j的对应部分梯度信息

所有边缘服务器更新共享后的A类梯度信息分别为

i∈[1，n]，n为边缘服务器的总数；S3, edge server E _i through decoding function

from

Get some gradient information that needs to be retained

And the remaining gradient information

After homomorphic encryption,

Then, the MePC algorithm is used to broadcast the message to all other edge servers _Ej ; the edge server _Ei receives the message according to the decoding function

Get the corresponding partial gradient information from other edge servers _Ej

The updated and shared A-type gradient information of all edge servers are

i∈[1,n], n is the total number of edge servers;

上传到云服务器CS，云服务器CS通过PreFLa算法聚合全局参数，PreFLa算法通过强化学习获得最大化回报来选择边缘服务器E_i的最优参数权重比a_i，k，全局梯度参数

根据a_i，k进行聚合；参数的上传和下载过程是并行的，所有参数都经过HE加密；S4, all edge servers will

Uploaded to the cloud server CS, the cloud server CS aggregates global parameters through the PreFLa algorithm, and the PreFLa algorithm selects the optimal parameter weight ratio a _{i, k} of the edge server E _i and the global gradient parameter by maximizing the return through reinforcement learning

Aggregate according to a _{i, k} ; the parameter upload and download process is parallel, and all parameters are HE encrypted;

S5. Repeat steps S2-S4 until the termination condition is reached. The cloud server CS calculates the final global gradient parameters and sends them to each edge server. The edge server extracts features from multiple vehicle data, calculates the accuracy and optimal loss function of the MePC-F model, obtains the trained MePC-F model, completes the entire training process, and outputs it to the corresponding service of the Internet of Vehicles in real time.

Furthermore, in step S2 of the present invention, the specific method of training the local network model is:

A deep neural network (DNN) model is used. The DNN performs end-to-end feature learning and classifier training by taking different vehicle data as raw input, and uses stochastic gradient descent as a subroutine to minimize the loss value in each local training.

并解密为A型梯度

随机初始化B型梯度

其中，k∈[1，K]，K表示联邦任务的总轮数；若为第一轮联邦任务，CS随机初始化

在本地训练之前，E_i通过使用同态加密对

解密为

并记为

E _i downloads the base layer parameters from the cloud server CS in the kth round of communication, i.e., the initial A-type gradient before decryption

and decrypted into a type A gradient

Randomly initialize B-type gradient

Among them, k∈[1, K], K represents the total number of rounds of the federated task; if it is the first round of the federated task, CS is randomly initialized

Before local training, E _i is encrypted using homomorphic encryption

Decrypted to

And record it as

The loss function of the local model is set as follows:

L( _wi )=l( _wi )+λ( _wi,t - _wi,t+1 ) ²

Among them, l() represents the loss of the network, the second term is the L2 regularization term, and λ is the regularization coefficient; _wi represents the total weight information in the local model, _wi,t is the weight information of the local model at time t, and _wi,t+1 is the weight information of the local model at time t+1;

E _i initializes G _k and replaces the model's weight parameters w _i , and continues local model training by minimizing the loss function as follows:

w _i = w _i −ηG _k

和

的总表示，这里的

随机初始化；Where η is the learning rate and _Gk is

and

The total representation here is

Random initialization;

After the edge server E _i reaches T rounds of local training, the accuracy of each local model acc _i,k and

Furthermore, in step S3 of the present invention, the specific method of the MePC algorithm is:

其中，

表示第k轮联邦任务中第n个边缘服务器的A类加密数据，

表示第k轮联邦任务中第i个边缘服务器的A类加密数据，

表示第k轮联邦任务中第i个边缘服务器广播发给其他边缘服务器的A类加密数据，

即为

去除了自己保留的那一份后的加密数据；In the kth round of federated tasks, all edge servers use MePC to exchange base layer gradients.

in,

represents the encrypted data of type A of the nth edge server in the kth round of federated tasks,

represents the encrypted data of type A of the i-th edge server in the k-th round of federated tasks,

It indicates the encrypted data of type A broadcasted by the i-th edge server to other edge servers in the k-th round of federated tasks.

That is

The encrypted data after removing the copy you keep;

梯度即为

并保持同一轮联邦的随机比例χ相同，再将

加密为

在不同轮的联邦任务中，随机比例χ是变化的，χ∈[1，1/n]；

剩下的梯度通过同态加密为

被均分为n-1份

的值划分为：In order to avoid the risk of data being cracked, a random ratio of χ is taken in each network.

The gradient is

Keep the random proportion χ of the federation in the same round the same, and then

Encrypted as

In different rounds of federated tasks, the random ratio χ varies, χ∈[1, 1/n];

The remaining gradients are homomorphically encrypted as

Divided equally into n-1 parts

The values are divided into:

被保留在E_i中，其它部分和随机参数χ将会以密文的形式广播发送给其它E_j；通过这种方式，即使部分传输内容被攻击，最初的数据

也不会泄露；only

is retained in E _i , and the other parts and random parameters χ will be broadcasted to other E _j in the form of ciphertext; in this way, even if part of the transmission content is attacked, the original data

It will not be leaked;

The gradient information shared to other E _j is

它在本地执行数据验证。When Ei receives a data packet sent by other servers

It performs data validation locally.

Furthermore, in step S3 of the present invention, the specific method of performing data verification locally is:

In the kth round of federated tasks, the corresponding "multiplication" method is used for verification, and each edge server designs two decoding functions by itself, as follows:

的长度，L’是

的长度；解码函数的下标k，表示第k轮联邦任务中的解码函数；Among them, _L0 is

The length, L' is

The length of the decoding function; the subscript k of the decoding function represents the decoding function in the k-th round of federated tasks;

L ₀ = x·L

的长度，

与

的长度相等；Where L is

Length,

and

are of equal length;

满足所有边缘服务器的解码函数对同一个数据包执行“并”操作得到全0，并执行“交”操作得到全1，即：Require

The decoding functions that satisfy all edge servers perform a "and" operation on the same data packet to obtain all 0s, and perform a "cross" operation to obtain all 1s, that is:

First, initialize the decoding function as follows:

与其它服务器中相应的解码函数相乘；由于

中0的二进制位被乘得0，所以E_i保证只得到它自己的部分数据包；当

中的二进制位为1时，得到对应位置的梯度信息的密文，如下：Packet

Multiply with the corresponding decoding function in other servers; since

The 0 bits in are multiplied by 0, so E _i is guaranteed to get only its own part of the data packet; when

When the binary bit in is 1, the ciphertext of the gradient information at the corresponding position is obtained as follows:

即：E _i adds all the data packet arrays obtained from other edge servers E _j to the corresponding positions, obtains all the ciphertext data, and updates to the final

Right now:

的二进制向左循环移动m个单位，以保证

共享的动态性，并将它们平均划分到E₁，E₂，…，E_n中，且每个部分的数据信息不重复。Each time a secure multi-party computation is performed, as k increases, the decoding function in each E _i is replaced by

The binary is circularly shifted to the left by m units to ensure

The dynamics of sharing are evenly divided into E ₁ , E ₂ , …, E _n , and the data information of each part is not repeated.

Furthermore, in step S4 of the present invention, the specific method of the PreFla algorithm is:

PreFLa uses reinforcement learning RL to adapt to select the optimal parameter weight ratio a _i,k to aggregate global parameters

和

上传到CS；在聚合阶段，由于每个ES的分布不平衡和数据异构性，其模型参数用于聚合对该阶段的收敛速度具有至关重要的影响；因此，有必要考虑k轮联邦聚合中参与者E_i的参数权重比a_i，k；In the uplink communication phase, each edge server not only trains the local model, but also uploads the local parameters to the cloud server CS for joint aggregation; after executing the MePC algorithm in the kth round of federation, E _i transmits the parameters to the cloud server CS through the TLS/SSL secure channel.

and

Upload to CS; In the aggregation stage, due to the imbalanced distribution and data heterogeneity of each ES, its model parameters used for aggregation have a crucial impact on the convergence speed of this stage; therefore, it is necessary to consider the parameter weight ratio a _i,k of participant E _i in the k-round federated aggregation;

强化学习包括：状态、动作、奖励函数以及反馈。Use DQN-based reinforcement learning to predict parameter weight ratios and store information through Q functions to prevent spatial multidimensional disasters; in order to better personalize the model and reduce the waiting time for uploading weights in MePC-F, use DQN to select the optimal parameter weight ratio a _{i, k} and aggregate to update the global parameters in CS

Reinforcement learning includes: state, action, reward function, and feedback.

Furthermore, in step S4 of the present invention, the specific methods of state, action, reward function and feedback are:

其中，

是精度差，表示为：Status: Status of round k

in,

is the precision difference, expressed as:

Action: The parameter weight ratio _ai,k represents the action of the federated task in round k. To avoid falling into the local optimal solution, the ε-greedy algorithm is used to optimize the action selection process and obtain _ai,k :

where P is a set of weight permutations, rand is a random number, rand ∈ [0, 1], Q( _{si, k} , ai _{, k} ) refers to the cumulative discounted return of the agent when taking action ai _{, k} in state si _{, k} ; once the DQN is trained to approximate Q(si _{, k} , ai _{, k} ) during testing, the DQN agent will calculate {Q(si _{, k} , ai _{, k} )|ai _{, k} ∈ [P]} for all actions in the kth round; each action value represents the maximum expected return that the agent can obtain by choosing a specific action ai _{, k in state si,} k;

Reward: Set the reward observed at the end of the kth federation round to:

是一个正数，确保r_k随着测试准确度Δacc_i，k呈指数增长；第一项激励代理选择能够实现更高测试精度的设备；

用来控制随着Δacc_i，k增长r_k的变化；当Δacc_i，k小于0时，有r_k∈(-1，0)；in,

is a positive number, ensuring that r _k grows exponentially with the test accuracy Δacc _i,k ; the first term motivates the agent to select equipment that can achieve higher test accuracy;

It is used to control the change of r _k as Δacc _i,k increases; when Δacc _i,k is less than 0, r _k ∈(-1,0);

The DQN agent is trained to maximize the expectation of the cumulative discounted reward as shown below:

Among them, γ∈(0,1], represents a factor that discounts future rewards;

After obtaining r _k , the cloud server CS saves the multidimensional quadruple B _k = (s _{i, k} , a _i, k , r _k , s _{i, k + 1} ) for each round of federated tasks; the optimal action-value function Q(s _{i, k} , a _{i, k} ) is the cheat sheet sought by the RL agent and is defined as the maximum expectation of the cumulative discounted return starting from s _{i, k} :

Q(s _i,k ,a _i,k )=E(r _i,k +γmax Q(s _i,k+1 ,a _i,k )|s _i,k ,a _i,k )

Function approximation techniques are applied to learn a parameterized value function Q(s _{i, k} , a _{i, k} ; w _k ) to approximate the optimal value function Q(s _{i, k} , a _{i, k} ); r _k +γmax Q(s _{i, k+1} , a _{i, k} ) is the goal of learning Q(s _{i, k} , a _{i, k} ; w _k ); DNN is used to represent the function approximator; the RL learning problem becomes minimizing the MSE loss between the target and the approximator, defined as:

l( _wk )=(ri _,k +γmax Q(si _,k+1 ,ai _,k ; _wk )-Q(si _,k ,ai _,k ; _wk )) ²

CS updates the global parameter _wk as:

Where η≥0 is the step size;

更新为：After the cloud server CS obtains the best learning model, it obtains the k-th round weight ratio sequence a _{i, k} and sets the global parameter

Updated to:

并开始接下来的T轮本地训练。All edge servers update global parameters

And start the next T rounds of local training.

Furthermore, the HE encryption method in the method of the present invention is specifically as follows:

The encryption schemes for the weight matrix and the bias vector follow the same idea. The additive homomorphic encryption of a real number a is represented by a ^E . In additive homomorphic encryption, for any two numbers a and b, a ^E + b ^E = (a+b) ^E . The method to convert any real number r into an encoded rational fixed point v is:

中的每个编码实数r可以表示为有理定理的H位数，由一个符号位、z位整数位和d位小数位组成；因此，每个可以编码的有理数都由其H＝1+z+d位定义；执行编码以允许乘法运算，这需要运算模数为H+2d以避免比较；Think gradient

Each coded real number r in can be represented as an H-bit rational theorem, consisting of a sign bit, z integer bits, and d fraction bits; therefore, each coded rational number is defined by its H = 1 + z + d bits; the encoding is performed to allow multiplication operations, which require operations modulo H + 2d to avoid comparisons;

Decode is defined as:

Multiplication of these encoded numbers requires the removal of the factor 1/2d; when using Paillier addition encryption, the encoded multiplication can be calculated exactly, but only one homomorphic multiplication is guaranteed; for simplicity, it is handled during decoding;

The largest encryptable integer is V-1, so the largest encryptable real number must take this into account. Therefore, the integer z and the decimal d are selected as follows:

V≥2 ^H+2d ≥2 ^1+z+3d .

Furthermore, the optimal loss function in step S5 of the present invention is

Where L( _wi ) represents the loss of the _Ei network.

The beneficial effects produced by the present invention are:

(1) A federated learning model with multi-party broadcast secure computing (MePC-F) is proposed. This model combines the MePC algorithm and the PreFla algorithm to solve the problems of data security and communication overhead in federated learning training in the Internet of Vehicles. The hybrid advantages of homomorphic encryption and secure multi-party computing are considered to prevent data leakage between ends, and reduce the degree of restoration of original data after the data is attacked, so as to maximize the privacy and security protection of data.

(2) A secure broadcast multi-party computation (MePC) is proposed. For secure multi-party computation, only sharing the gradient information of the first layer can greatly reduce the risk of data recovery and reduce the amount of communication. In the sharing process, the broadcast method is adopted. The edge server model takes its own part through the decoding function, which can reduce the time complexity from O(n ² ) to O(n), preventing the leakage of original data while reducing the communication overhead.

(3) A federated learning algorithm PreFla based on weight ratio is proposed. PreFla is used to find the optimal gradient weight ratio to aggregate global parameters, and the accuracy difference of each edge server is used to design the reward function, so that the action selection with the largest overall return is the weight ratio of each round of federation. An L2 regularization term is added to the loss function to promote edge server collaboration and reduce the latency and performance issues caused by data heterogeneity. This can better generalize the global model and accelerate convergence.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

FIG1 is a MePC-F model according to an embodiment of the present invention;

FIG2 is a flow chart of a MePC-F model according to an embodiment of the present invention;

FIG3 is a MePC algorithm according to an embodiment of the present invention;

FIG4 is a DLG result of four methods on MNIST in an embodiment of the present invention when the first hidden layer is not hidden; (a) FL; (b) MePC-F; (c) PeMPC; (d) Gaussian; (e) Laplacian;

FIG5 is the performance of DLG on MNIST according to an embodiment of the present invention when the gradient of the first hidden layer is replaced by four methods (Gaussian distribution, Laplace distribution, PEMPC, and MePC-F);

FIG6 is an average accuracy and loss of No-IID MNIST data according to an embodiment of the present invention;

FIG. 7 shows the average accuracy and loss of No-IID CAFIR-10 data according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

The parameters involved in the embodiments of the present invention are described as follows:

Table 1 Parameter Description

Among them, E _i represents the current edge server, E _j represents the edge server other than the current edge server, and _Es represents all edge servers.

The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles of the embodiment of the present invention includes the following steps:

S1. Build multiple edge servers E _i and a cloud server CS; obtain vehicle data D = {D ₁ , D ₂ , ..., D _i }, and the edge server E _i obtains the corresponding vehicle data D _i ;

并解密为

随机初始化B型梯度

边缘服务器E_i根据其车辆数据D_i的最小化损失函数来计算本地网络模型训练中的梯度，边缘服务器E_i完成T轮本地训练完后的梯度信息记为

S2. In the kth round of federated tasks, the edge server E _i downloads the initial A-type gradient from the cloud server CS

and decrypted to

Randomly initialize B-type gradient

The edge server E _i calculates the gradient in the local network model training according to the minimized loss function of its vehicle data D _i . The gradient information of the edge server E _i after completing T rounds of local training is recorded as

从

中获取需要保留的部分梯度信息

并将剩余的梯度信息

经同态加密为

再通过MePC算法广播发送给其它所有的边缘服务器E_j；边缘服务器E_i根据解码函数

获取来自其它边缘服务器E_j的对应部分梯度信息

所有边缘服务器更新共享后的A类梯度信息分别为

i∈[1，n]，n为边缘服务器的总数；S3, edge server E _i through decoding function

from

Get some gradient information that needs to be retained

And the remaining gradient information

After homomorphic encryption,

Then, the MePC algorithm is used to broadcast the message to all other edge servers _Ej ; the edge server _Ei receives the message according to the decoding function

Get the corresponding partial gradient information from other edge servers _Ej

The updated and shared A-type gradient information of all edge servers are

i∈[1,n], n is the total number of edge servers;

上传到云服务器CS，云服务器CS通过PreFLa算法聚合全局参数，PreFLa算法通过强化学习获得最大化回报来选择边缘服务器E_i的最优参数权重比a_i，k，全局参数

根据a_i，k进行聚合；参数的上传和下载过程是并行的，所有参数都经过HE加密；S4, all edge servers will

Uploaded to the cloud server CS, the cloud server CS aggregates the global parameters through the PreFLa algorithm, and the PreFLa algorithm selects the optimal parameter weight ratio a _{i, k} of the edge server E _i by maximizing the return through reinforcement learning, and the global parameters

Aggregate according to a _{i, k} ; the parameter upload and download process is parallel, and all parameters are HE encrypted;

S5. Repeat steps S2-S4 until the termination condition is reached and the entire training process is completed. The termination condition can be the maximum number of training cycles, the convergence of the loss function, or other user-defined conditions. Finally, the optimal loss function can be obtained according to the following formula (1).

Where L( _wi ) represents the loss of the _Ei network.

The specific method of local training is:

In the local model stage, a deep neural network (DNN) is used to learn the cloud model and ES model. DNN performs end-to-end feature learning and classifier training by taking different user data as raw input. Stochastic gradient descent is used as a subroutine in the proposed algorithm to minimize the loss value in each local training.

并随机初始化

其中，K表示联邦任务的总轮数。若为第一轮联邦任务，CS随机初始化

在本地训练之前，E_i需要通过使用同态加密(公式(4))对

解密为

并记为

In the downlink communication phase, E _i downloads the base layer parameters from CS in the kth (k∈[1, K]) round of communication.

And randomly initialize

Where K represents the total number of rounds of the federated task. If it is the first round of the federated task, CS is randomly initialized.

Before local training, E _i needs to be encrypted using homomorphic encryption (Formula (4)).

Decrypted to

And record it as

In order to better reflect the model personalization, the loss function of the local model is set as follows:

L( _wi )=l( _wi )+λ( _wi,t - _wi,t+1 ) ² (16)

Where l() represents the loss of the network, such as the cross entropy loss of the classification task. The second term is the L2 regularization term, which can both retain its own personalization ability and improve the efficiency of cooperation with other participants. λ is the regularization coefficient.

E _i initializes G _k and replaces the model's weight parameter w _i , and continues local model training as follows

w _i = w _i - ηG _k (17)

和

的总表示。这里的

随机初始化。Where η is the learning rate and _Gk is

and

Here is the total representation of

Random initialization.

和

端与端之间直接共享用户信息是禁止的，边缘服务器中的数据在通信前需要进行加密，防止数据在通信前被攻击。这个过程使用HE来避免信息泄露。以下将展示使用实数进行加法HE的过程。权重矩阵和偏置向量的加密方案遵循相同的思想，实数a的加法同态加密表示为a^E。在加法同态加密中，对于任意两个数a和b，有a^E+b^E＝(a+b)^E。将任何实数r转换为编码的有理数不动点v的方法是：After E _i reaches T rounds of local training, the accuracy of each local model acc _i,k ,

and

Direct sharing of user information between ends is prohibited. Data in the edge server needs to be encrypted before communication to prevent data from being attacked before communication. This process uses HE to avoid information leakage. The following will show the process of additive HE using real numbers. The encryption schemes for weight matrices and bias vectors follow the same idea. The additive homomorphic encryption of a real number a is represented as a ^E . In additive homomorphic encryption, for any two numbers a and b, a ^E +b ^E ＝(a+b) ^E . The method to convert any real number r into an encoded rational number fixed point v is:

中的每个编码实数r可以表示为有理定理的H位数，由一个符号位、z位整数位和d位小数位组成。因此，每个可以编码的有理数都由其H＝1+z+d位定义。执行编码以允许乘法运算，这需要运算模数为H+2d以避免比较。Think gradient

Each coded real number r in can be represented as an H-bit rational theorem, consisting of a sign bit, z integer bits, and d fraction bits. Therefore, each coded rational number is defined by its H = 1 + z + d bits. The encoding is performed to allow multiplication operations, which require operations modulo H + 2d to avoid comparisons.

Decode is defined as:

Multiplication of these encoded numbers requires the removal of the factor 1/2d. When encrypting with Paillier addition, the encoded multiplication can be calculated exactly, but only one homomorphic multiplication is guaranteed. For simplicity, it is handled during decoding.

If only one code multiplication occurs, then it is correct. Since the largest encryptable integer is V-1, the largest encryptable real number must take this into account. Therefore, the integer z and the decimal d must be chosen as follows:

V≥2 ^H+2d ≥2 ^1+z+3d (5)

和aCC_i，k分别表示为

和

Encrypted

and aCC _i,k are represented as

and

The specific method of the MePC algorithm is shown in Figure 3.

为了避免数据被破解的风险，在每个网络中取随机比例χ的

梯度即为

并保持同一轮联邦的随机比例χ相同。在不同轮的联邦任务中，随机比例χ(χ∈[1，1/n])是变化的，

剩下的梯度被均分为n-1份

如图3所示，

的值划分为：In the kth round of federated tasks, MePC is used to exchange the base layer gradients

In order to avoid the risk of data being cracked, a random ratio of χ is taken in each network.

The gradient is

And keep the random ratio χ of the same round of federation the same. In different rounds of federation tasks, the random ratio χ (χ∈[1, 1/n]) is changed.

The remaining gradient is divided equally into n-1 parts

As shown in Figure 3,

The values are divided into:

被保留在E_i中，其他部分和随机参数χ将会以密文的形式广播发送给其他ESs。通过这种方式，即使部分传输内容被攻击，最初的数据

也不会泄露。具体来说，如果攻击者想要获取数据

就必须获取

的所有部分。但是，

和χ在参与者E_i和接收者E_j之间通信时都是通过同态加密保持密文形式。only

is retained in E _i , and the other parts and random parameters χ will be broadcasted to other ESs in the form of ciphertext. In this way, even if part of the transmission content is attacked, the original data

Specifically, if an attacker wants to obtain data

You must obtain

However,

When communicating between participant E _i and receiver E _j, both χ and χ are kept in ciphertext form through homomorphic encryption.

The gradient information shared to other ESs is

它在本地执行数据验证。具体来说，它使用相应的“乘法”方法进行验证。每个边缘服务器自己设计两个解码功能，如下：When E _i receives a data packet sent by other servers

It performs data verification locally. Specifically, it uses the corresponding "multiplication" method for verification. Each edge server designs two decoding functions by itself, as follows:

的长度，L’是

的长度。Among them, _L0 is

length, L' is

Length.

L ₀ = χ·L (9)

的长度，

与

的长度相等。Where L is

Length,

and

are of equal length.

满足所有ES的解码函数对同一个数据包执行“并”操作得到全0，并执行“交”操作得到全1，即Require

The decoding function that satisfies all ESs performs a "and" operation on the same data packet to obtain all 0s, and performs a "cross" operation to obtain all 1s, that is,

First, initialize the decoding function as follows,

It should be noted that, at the time of initialization, the data packets sent by different E _i in the same federated task also use the same function for data decoding.

与其他服务器中相应的解码函数相乘。由于

中0的二进制位被乘得0，所以E_i可以保证只得到它自己的部分数据包。当

中的二进制位为1时，可以得到对应位置的梯度信息的密文，如下：Packet

Multiply by the corresponding decoding function in the other servers.

The 0 bits in are multiplied to 0, so E _i can ensure that it only gets its own part of the data packet.

When the binary bit in is 1, the ciphertext of the gradient information at the corresponding position can be obtained as follows:

即E _i adds all the data packet arrays obtained from other ES to the corresponding positions, obtains all the ciphertext data, and updates to the last

Right now

的二进制向左循环移动m个单位，以保证

共享的动态性，并可以将它们平均划分到E₁，E₂，…，E_n中，且每个部分的数据信息不重复。Each time a secure multi-party computation is performed, the decoding function in each E _i is replaced by

The binary is circularly shifted to the left by m units to ensure

The shared dynamics can be evenly divided into E ₁ , E ₂ , …, E _n , and the data information in each part is not repeated.

The specific method of the PreFla algorithm is:

个性化层的参数

随机生成并使用本地数据进行微调。为了满足实时性要求，实现ES的个性化需求，PreFLa采用强化学习(RL)进行适配来选择最优参数权重比a_i，k聚合全局参数

The data distribution in the Internet of Vehicles is relatively scattered, and the data is unbalanced and heterogeneous, making it difficult to meet real-time requirements while improving personalized service needs. In order to prevent user privacy leakage during communication between different edge servers, HE is used to encrypt parameters during the communication process. In order to better achieve personalized training for different user data, the first layer is set as the base layer and trained in a collaborative manner using the existing federated learning method, while the other layers are trained locally as personalized layers, thereby being able to capture personal information of different ES devices. In this way, after the joint training process, the globally shared base layer can be transferred to ES to build its own personalized deep learning model and use its unique personalized layer. Only download the base layer parameters from CS

Parameters of the personalization layer

Randomly generate and fine-tune using local data. In order to meet the real-time requirements and realize the personalized needs of ES, PreFLa uses reinforcement learning (RL) to adapt to select the optimal parameter weight ratio a _i,k aggregate global parameters

和

上传到BS。在聚合阶段，由于每个ES的分布不平衡和数据异构性，其模型参数用于聚合对该阶段的收敛速度具有至关重要的影响。因此，有必要考虑k轮联邦聚合中参与者E_i的参数权重比a_i，k。In the uplink communication phase, each ES not only trains the local model, but also uploads the local parameters to the CS for joint aggregation. After executing the MePC algorithm in the kth round of federation, E _i transmits the parameters to the CS through the TLS/SSL secure channel.

and

Upload to BS. In the aggregation stage, due to the imbalanced distribution and data heterogeneity of each ES, its model parameters used for aggregation have a crucial impact on the convergence speed of this stage. Therefore, it is necessary to consider the parameter weight ratio a _i,k of participant E _i in the k-round federated aggregation.

强化学习中核心内容有：状态、动作、奖励函数以及反馈，定义如下：In this invention, DQN-based reinforcement learning is used to predict parameter weight ratios, and information is stored through Q functions instead of table storage in Q-Learning to prevent spatial multidimensional disasters. In order to better realize model personalization and reduce the waiting time for uploading weights in MePC-F, DQN is used to select the optimal parameter weight ratio a _{i, k} to aggregate and update the global parameters in CS.

The core contents of reinforcement learning are: state, action, reward function and feedback, which are defined as follows:

其中，

是精度差，表示为：Status: Status of round k

in,

is the precision difference, expressed as:

Action: The parameter weight ratio _ai,k represents the action of the federated task in round k. To avoid falling into the local optimal solution, the ε-greedy algorithm is used to optimize the action selection process, and ai _,k can be obtained:

Where P is a set of weight permutations, rand is a random number (rand ∈ [0, 1]), and Q( _{si, k} , ai _{, k} ) refers to the cumulative discounted return of the agent when taking action ai _{, k} in state si _, k. Once DQN is trained to approximate Q(si _{, k} , ai _{, k} ) during testing, the DQN agent computes {Q(si _{, k} , ai _{, k} )|ai _{, k} ∈ [P]} for all actions in the kth round. Each action value represents the maximum expected return that the agent can obtain by choosing a specific action _{ai, k in state si,} k.

Reward: Set the reward observed at the end of the kth federation round to

是一个正数，确保r_k随着测试准确度Δacc_i，k呈指数增长。第一项激励代理选择能够实现更高测试精度的设备。

用来控制随着Δacc_i，k增长r_k的变化。一般来说，随着机器学习训练的进行，模型准确率会以较慢的速度增加。但在联邦合作任务中，由于数据分布不平衡和异质性，模型精度可能会降低。因此，随着FL进入后期阶段，使用指数项来放大边界准确度的增加。第二项-1用来鼓励智能体提高模型准确性，因为当Δacc_i，k小于0时，有r_k∈(-1，0)。in,

is a positive number that ensures that r _k grows exponentially with the test accuracy Δacc _i,k . The first term motivates the agent to select devices that can achieve higher test accuracy.

It is used to control the change of r _k as Δacc _i,k grows. In general, as machine learning training progresses, model accuracy increases at a slower rate. However, in federated cooperative tasks, model accuracy may decrease due to imbalanced data distribution and heterogeneity. Therefore, as FL enters the later stages, an exponential term is used to amplify the increase in boundary accuracy. The second term -1 is used to encourage the agent to improve model accuracy because when Δacc _i,k is less than 0, there is r _k ∈(-1, 0).

The DQN agent is trained to maximize the expectation of the cumulative discounted reward as shown below

where γ∈(0,1] is a factor that discounts future rewards.

After obtaining r _k , CS saves a multidimensional quadruple B _k = (s _{i , k} , a _{i , k} , r _k , s _{i , k + 1} ) for each round of federated tasks. The optimal action-value function Q (s _{i , k} , a _{i , k} ) is the cheat sheet sought by the RL agent and is defined as the maximum expectation of the cumulative discounted return starting from s _{i , k} :

Q(s _i,k ,a _i,k )=E(r _i,k +γmax Q(s _i,k+1 ,a _i,k )|s _i,k ,a _i,k ) (22)

Then, function approximation techniques can be applied to learn a parameterized value function Q(s _{i, k} , a _{i, k} ; w _k ) to approximate the optimal value function Q(s _{i, k} , a _{i, k} ). The first step r _k +γmax Q(s _{i, k+1} , a _{i, k} ) is the target of Q(s _{i, k} , a _{i, k} ; w _k ) learning. Typically, DNN is used to represent the function approximator. The RL learning problem becomes minimizing the MSE loss between the target and the approximator, defined as:

l( _wk )=(ri _,k +γmax Q(si _,k+1 ,ai _,k ; _wk )-Q(si _,k ,ai _,k ; _wk )) ² (23 )

CS updates the global parameter _wk as:

where η ≥ 0 is the step size.

更新为：CS repeats the above steps to obtain the best learning model. CS can obtain the k-th round weight ratio sequence a _{i, k} and set the global parameter

Updated to:

并开始接下来的T轮本地训练。All ES update global parameters

And start the next T rounds of local training.

Experimental test example:

取3。各参数取值见表2。In order to verify the effectiveness of the proposed mechanism, experimental results and analysis are given. Consider a system with 1 cloud server and 10 edge servers respectively. The experimental learning rate α is 0.01 and the discount factor γ is 0.9. Positive integer

The value is 3. See Table 2 for the values of each parameter.

Table 2 Parameter settings

The effectiveness of the proposed model is verified on two datasets: MNIST and CIFAR-10. The performance of the proposed federated learning model MePC-F is evaluated based on the reconstructed images, average precision, and average loss of DLG. First, the performance of five schemes for defending against DLG attacks is presented, and then the performance of the proposed federated learning model MePC-F is compared with centralized and PeMPC. All results in the following scenarios are the average of 1000 independent experiments.

1) Performance of defending against DLG attacks

This section evaluates the effectiveness of MePC-F and compares it with FL, PeMPC, and DP algorithms (Gaussian distributed noise and Laplace distributed noise) on DLG reconstructed images. The common gradient of the network is calculated for a single image on the MNIST dataset, and the results of different schemes are shown in Figure 4. Since the study [17] showed that the gradient of the hidden first layer can reduce the reconstruction of the data, four methods are used to replace the gradient of the first layer (weights and bias terms): the proposed MePC-F, PeMPC, Gaussian distributed (μ = 0, σ = 1) noise, and Laplace distributed (μ = 0, σ = 1) noise to see the behavior of DLG. After completing the gradient of the hidden first layer

After quantization, DLG uses these gradients to recover the image creating a common shared gradient.

As can be seen from Figure 4, when there is no method to hide the gradient of the first layer (FL in Figure 4(a)), the DLG process can accurately reconstruct the training data. When the gradient of the first layer is protected by the method MePC-F proposed in the present invention, information leakage can be effectively prevented in Figure 4(b). When the number of iterations reaches 500, DLG still cannot construct the image. From Figure 4(c), it can be seen that the results are similar to those in Figure 4(b), and PeMPC can also defend against the DLG attack in Figure 4. From Figure 4(d), it can be seen that by adding Gaussian noise to the first layer, the reconstructed image is partially displayed from the 15th round, and the basic outline of the original image has been constructed by the 20th round. As the number of iterations increases to 500, the image can be clearly restored. The Laplace noise in Figure 4(e) also has similar phenomena to the driven Gaussian noise.

As can be seen from Figure 5, if the malicious server receives all the gradients of the hidden layers as plain text, the reconstruction process is able to obtain the lowest gradient loss and the MSE of the image (the green line in Figure 5). As the number of rounds increases, PeMPC and MePC-F do not converge to zero, and the MSE of the image reaches 10 ⁷ . Adding Laplace and Gaussian noise to the original gradient converges to 10 ^-5 , and Figure 4 also proves that the data can be reconstructed when 20 rounds are reached. The larger the MSE of the image, the less likely it is that the image is reconstructed.

Based on the above experimental results, it is verified that adding Laplace and Gaussian noise to the original gradient can prevent partial gradient leakage in the early stage, but as the number of rounds increases, the original data will still be restored due to deep leakage. However, PeMPC and MePC-F are effective methods to prevent DLG attacks from reconstructing the original data regardless of the number of training rounds.

2) Performance comparison of average accuracy and average loss

In this section, the effectiveness of MePC-F is evaluated and compared with Centralized and PeMPC in terms of mean average accuracy and mean loss on MNIST and CIFAR-10 datasets.

Figure 6(a) shows the number of rounds required for the model to achieve 98% accuracy on the MNIST dataset. The average accuracy of the three methods increases with the increase in training rounds. To achieve the target accuracy on the MNIST data, the centralized method requires 25 rounds, PeMPC requires 140 rounds, and MePC-F requires 40 rounds. The number of training rounds required for MePC-F is 71.2% lower than that of PeMPC. The reason is that the proposed reinforced federated learning algorithm PreFLa can find better aggregate parameter weights a _i,k through interaction with the environment, which can better cope with No-IID data, accelerate model convergence, and achieve the target accuracy. The centralized method is trained on a combination of all data, so its accuracy will be higher than that of the federated learning algorithm. However, it can be seen from the figure that the convergence of PeMPC can almost reach the accuracy of the centralized method.

From Figure 6(b), we can see that the average losses of the three schemes decrease as the number of training rounds increases. For the centralized scheme, the average loss decreases from 0.233 to 0.052. The average loss of PeMPC decreases from 0.35 to 0.084. At the same time, the average loss of MePC-F proposed in the present invention is reduced to 0.06, which is lower than 28.6% of PeMPC. When the number of training rounds reaches 100, the proposed MePC-F can almost reach the loss value of the centralized scheme.

From Figure 7(a), we can see the number of rounds required for the model to reach the target accuracy of 50% on CAFIR-10. Similar results can be seen as in Figure 6(a). The average accuracy of the three models increases until the target value is reached. For the centralized model, the average accuracy increases from 0.42 to 0.5 in 23 rounds. The average accuracy of PeMPC increases from 0.372 to 0.5 in 89 rounds. Meanwhile, the average accuracy of the proposed MePC-F reaches the target accuracy at 41 rounds, which is 53.9% lower than PeMPC. Figure 7(a) shows that MePC-F uses better weights a _i,k than PeMPC to update the global model, which leads to faster convergence.

As can be seen from Figure 7(b), the average loss of the three schemes is decreasing until they reach a stable value. Centralized, MePC-F, and PeMPC reach the minimum loss in turn, and the time efficiency of the proposed MePC-F is better under PeMPC.

Table 3 The highest accuracy of the three schemes within 100 rounds

MNIST CIFAR-10 centralized 98.4% 51.4% MePC-F 98.2% 51.1% PcP 97.6% 49.2%

Table 3 gives the accuracy of the three schemes within 100 rounds. For MNIST data, the average accuracy of the proposed MePC-F is 98.2%, which is 0.6% higher than PeMPC. The accuracy of PeMPC can almost reach the accuracy of centralized training. For CAFIR-10 data, the average accuracy of MePC-F in 100 rounds is as high as 0.511, which is 1.9% higher than PeMPC. It shows that MePC-F can better aggregate global parameters than PeMPC through the optimal weight update a _i,k , thereby achieving higher accuracy and closer to the centralized accuracy.

It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims (8)

1. A real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles, characterized in that the method comprises the following steps: S1. Build multiple edge servers E _i and a cloud server CS; obtain vehicle data D = {D ₁ , D ₂ , ..., D _i }, and the edge server E _i obtains the corresponding vehicle data D _i ; S2. In the kth round of federated tasks, the edge server E _i downloads the initial A-type gradient from the cloud server CS

and decrypted to

Randomly initialize B-type gradient

The edge server E _i calculates the gradient in the local network model training according to the minimized loss function of its vehicle data D _i . The gradient information of the edge server E _i after completing T rounds of local training is recorded as

S3, edge server E _i through decoding function

from

Get some gradient information that needs to be retained

And the remaining gradient information

After homomorphic encryption,

Then, the MePC algorithm is used to broadcast the message to all other edge servers _Ej ; the edge server _Ei receives the message according to the decoding function

Get the corresponding partial gradient information from other edge servers _Ej

The updated and shared A-type gradient information of all edge servers are

i∈[1,n], n is the total number of edge servers; S4, all edge servers will

Uploaded to the cloud server CS, the cloud server CS aggregates global parameters through the PreFLa algorithm, and the PreFLa algorithm selects the optimal parameter weight ratio a _{i, k} of the edge server E _i and the global gradient parameter by maximizing the return through reinforcement learning

Aggregate according to a _{i, k} ; the parameter upload and download process is parallel, and all parameters are HE encrypted; S5. Repeat steps S2-S4 until the termination condition is reached. The cloud server CS calculates the final global gradient parameters and sends them to each edge server. The edge server extracts features from multiple vehicle data, calculates the accuracy and optimal loss function of the MePC-F model, obtains the trained MePC-F model, completes the entire training process, and outputs it to the corresponding service of the Internet of Vehicles in real time. 2. According to the method for real-time enhanced federated learning data privacy security based on the MePC-F model in the Internet of Vehicles according to claim 1, it is characterized in that in the step S2, the specific method of training the local network model is: A deep neural network (DNN) model is used. The DNN performs end-to-end feature learning and classifier training by taking different vehicle data as raw input, and uses stochastic gradient descent as a subroutine to minimize the loss value in each local training. E _i downloads the base layer parameters from the cloud server CS in the kth round of communication, i.e., the initial A-type gradient before decryption

and decrypted into a type A gradient

Randomly initialize B-type gradient

Among them, k∈[1, K], K represents the total number of rounds of the federated task; if it is the first round of the federated task, CS is randomly initialized

Before local training, E _i is encrypted using homomorphic encryption

Decrypted to

And record it as

The loss function of the local model is set as follows: L( _wi )=l( _wi )+λ( _wi,t - _wi,t+1 ) ² Among them, l() represents the loss of the network, the second term is the L2 regularization term, and λ is the regularization coefficient; _wi represents the total weight information in the local model, wi _,t is the weight information of the local model at time t, and _wi,t+1 is the weight information of the local model at time t+1; E _i updates G _k and replaces the model's weight parameters w _i , and continues local model training by minimizing the loss function as follows: w _i = w _i −ηG _k Where η is the learning rate and _Gk is

and

The total representation here is

Random initialization; After the edge server E _i reaches T rounds of local training, the accuracy of each local model aCC _i,k and

3. According to claim 1, the real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles is characterized in that in step S3, the specific method of the MePC algorithm is: In the kth round of federated tasks, all edge servers use MePC to exchange base layer gradients.

in,

represents the encrypted data of type A of the nth edge server in the kth round of federated tasks,

represents the encrypted data of type A of the i-th edge server in the k-th round of federated tasks,

It indicates the encrypted data of type A broadcasted by the i-th edge server to other edge servers in the k-th round of federated tasks.

That is

The encrypted data after removing the copy you keep; In order to avoid the risk of data being cracked, a random ratio of χ is taken in each network.

The gradient is

Keep the random proportion χ of the federation in the same round the same, and then

Encrypted as

In different rounds of federated tasks, the random ratio χ varies, χ∈[1, 1/n];

The remaining gradients are homomorphically encrypted as

Divided equally into n-1 parts

The values are divided into:

only

is retained in E _i , and the other parts and random parameters χ will be broadcasted to other E _j in the form of ciphertext; in this way, even if part of the transmission content is attacked, the original data

It will not be leaked; The gradient information shared to other E _j is

When E _i receives a data packet sent by other servers

It performs data validation locally. 4. The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles according to claim 3 is characterized in that in the step S3, the specific method of performing data verification locally is: In the kth round of federated tasks, the corresponding "multiplication" method is used for verification, and each edge server designs two decoding functions by itself, as follows:

Among them, _L0 is

The length, L' is

The length of the decoding function; the subscript k of the decoding function represents the decoding function in the k-th round of federated tasks; L ₀ = x·L

Where L is

Length,

and

are of equal length; Require

The decoding functions that satisfy all edge servers perform a "and" operation on the same data packet to obtain all 0s, and perform a "cross" operation to obtain all 1s, that is:

First, initialize the decoding function as follows:

Packet

Multiply with the corresponding decoding function in other servers; since

The 0 bits in are multiplied by 0, so E _i is guaranteed to get only its own part of the data packet; when

When the binary bit in is 1, the ciphertext of the gradient information at the corresponding position is obtained as follows:

E _i adds all the data packet arrays obtained from other edge servers E _j to the corresponding positions, obtains all the ciphertext data, and updates to the final

Right now:

Each time a secure multi-party computation is performed, as k increases, the decoding function in each E _i is replaced by

The binary is circularly shifted to the left by m units to ensure

The dynamics of sharing are evenly divided into E ₁ , E ₂ , …, E _n , and the data information of each part is not repeated. 5. The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles according to claim 1 is characterized in that in the step S4, the specific method of the PreFla algorithm is: PreFLa uses reinforcement learning RL to adapt to select the optimal parameter weight ratio a _i,k to aggregate global parameters

In the uplink communication phase, each edge server not only trains the local model, but also uploads the local parameters to the cloud server CS for joint aggregation; after executing the MePC algorithm in the kth round of federation, E _i transmits the parameters to the cloud server CS through the TLS/SSL secure channel.

and

Upload to CS; In the aggregation stage, due to the imbalanced distribution and data heterogeneity of each ES, its model parameters used for aggregation have a crucial impact on the convergence speed of this stage; therefore, it is necessary to consider the parameter weight ratio a _i,k of participant E _i in the k-round federated aggregation; Use DQN-based reinforcement learning to predict parameter weight ratios and store information through Q functions to prevent spatial multidimensional disasters; in order to better personalize the model and reduce the waiting time for uploading weights in MePC-F, use DQN to select the optimal parameter weight ratio a _{i, k} and aggregate to update the global parameters in CS

Reinforcement learning includes: state, action, reward function, and feedback. 6. The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles according to claim 5 is characterized in that in step S4, the specific methods of state, action, reward function and feedback are: Status: Status of round k

in,

is the precision difference, expressed as:

Action: The parameter weight ratio _ai,k represents the action of the federated task in round k. To avoid falling into the local optimal solution, the ε-greedy algorithm is used to optimize the action selection process and obtain _ai,k :

where P is a set of weight permutations, rand is a random number, rand ∈ [0, 1], Q( _{si, k} , ai _{, k} ) refers to the cumulative discounted return of the agent when taking action ai _{, k} in state si _{, k} ; once the DQN is trained to approximate Q(si _{, k} , ai _{, k} ) during testing, the DQN agent will calculate {Q(si _{, k} , ai _{, k} )|ai _{, k} ∈ [P]} for all actions in the kth round; each action value represents the maximum expected return that the agent can obtain by choosing a specific action ai _{, k in state si,} k; Reward: Set the reward observed at the end of the kth federation round to:

in,

is a positive number that ensures that r _k grows exponentially with the training accuracy Δacc _i,k ; the first term incentivizes the agent to choose a device that can achieve higher test accuracy;

It is used to control the change of r _k as Δacc _i,k increases; when Δacc _i,k is less than 0, r _k ∈(-1,0); The DQN agent is trained to maximize the expectation of the cumulative discounted reward as shown below:

Among them, γ∈(0,1], represents a factor that discounts future rewards; After obtaining r _k , the cloud server CS saves the multidimensional quadruple B _k = (s _{i, k} , a _i, k , r _k , s _{i, k + 1} ) for each round of federated tasks; the optimal action-value function Q(s _{i, k} , a _{i, k} ) is the cheat sheet sought by the RL agent and is defined as the maximum expectation of the cumulative discounted return starting from s _{i, k} : Q(s _i,k ,a _i,k )=E(r _i,k +γmax Q(s _i,k+1 ,a _i,k )|s _i,k ,a _i,k ) Function approximation techniques are applied to learn a parameterized value function Q(s _{i, k} , a _{i, k} ; w _k ) to approximate the optimal value function Q(s _{i, k} , a _{i, k} ); r _k +γmax Q(s _{i, k+1} , a _{i, k} ) is the goal of learning Q(s _{i, k} , a _{i, k} ; w _k ); DNN is used to represent the function approximator; the RL learning problem becomes minimizing the MSE loss between the target and the approximator, defined as: l( _wk )=(ri _,k +γmax Q(Si _,k+1 ,ai _,k ; _wk )-Q(si _,k ,ai _,k ; _wk )) ² CS updates the global parameter _wk as:

Where η≥0 is the step size; After the cloud server CS obtains the best learning model, it obtains the k-th round weight ratio sequence a _{i, k} and sets the global parameter

Updated to:

All edge servers update global parameters

And start the next T rounds of local training. 7. The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles according to claim 1 is characterized in that the HE encryption method in the method is specifically: The encryption schemes for the weight matrix and the bias vector follow the same idea. The additive homomorphic encryption of a real number a is represented by a ^E . In additive homomorphic encryption, for any two numbers a and b, a ^E + b ^E = (a+b) ^E . The method to convert any real number r into an encoded rational fixed point v is:

Think gradient

Each coded real number r in can be represented as an H-bit rational theorem, consisting of a sign bit, z integer bits, and d fraction bits; therefore, each coded rational number is defined by its H = 1 + z + d bits; the encoding is performed to allow multiplication operations, which require operations modulo H + 2d to avoid comparisons; Decode is defined as:

Multiplication of these encoded numbers requires the removal of the factor 1/2d; when using Paillier addition encryption, the encoded multiplication can be calculated exactly, but only one homomorphic multiplication is guaranteed; for simplicity, it is handled during decoding; The largest encryptable integer is V-1, so the largest encryptable real number must take this into account. Therefore, the integer z and the decimal d are selected as follows: V≥2 ^H+2d ≥2 ^1+z+3d . 8. The real-time enhanced federated learning data privacy security method based on the MePC-F model in the Internet of Vehicles according to claim 1 is characterized in that the optimal loss function in step S5 is

Where L( _wi ) represents the loss of the _Ei network.

Images