CN116227631A - Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal - Google Patents
Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal Download PDFInfo
- Publication number
- CN116227631A CN116227631A CN202211565002.6A CN202211565002A CN116227631A CN 116227631 A CN116227631 A CN 116227631A CN 202211565002 A CN202211565002 A CN 202211565002A CN 116227631 A CN116227631 A CN 116227631A
- Authority
- CN
- China
- Prior art keywords
- model
- data
- client
- training
- federal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
- G06N20/20—Ensemble learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
- G06F21/6245—Protecting personal data, e.g. for financial or medical purposes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Bioethics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention belongs to the field of artificial intelligence, and discloses a federal learning method and a federal learning system for classification and prediction of connection data of a vehicle networking terminal, wherein the federal learning method comprises the following steps: constructing a federal learning model for classification prediction of the connection data of the vehicle networking terminal; performing iterative training on the model by the client based on the federal differential privacy algorithm, processing the gradient by adopting cutting operation and noise adding, and uploading the updated gradient to a server; and updating and aggregating the new global model by the server, finally transmitting the new global model to the client in a model broadcasting mode, and repeating the local updating calculation operation until training is finished, so that the connection data of the Internet of vehicles terminal is classified and predicted. The invention achieves the effect of sharing a large amount of data while protecting privacy in a scattered and cooperative way. According to the invention, the data is shared under the condition of guaranteeing personal privacy information, and the model is trained cooperatively, so that the calculation efficiency and accuracy are improved, and a new thought is provided for future big data sharing and cooperation.
Description
Technical Field
The invention belongs to the field of artificial intelligence, and particularly relates to a federal learning method and a federal learning system for classification and prediction of connection data of a vehicle networking terminal.
Background
The internet of vehicles, namely the technology of the mobile internet of things of the automobile, refers to the fact that electronic tags loaded on vehicles extract and effectively utilize attribute information, static and dynamic information of all vehicles on an information network platform through the identification technology of wireless radio frequency and the like, and effectively supervise and provide comprehensive services for the running states of all vehicles according to different functional requirements, and the technology is widely applied to the fields of distance protection, real-time navigation and the like, and improves the efficiency of traffic running by tracking the positions and communication of the vehicles. The realization of the function of the internet of vehicles is free from a large amount of data brought by users and vehicles, but the internet of vehicles is huge in scale, the wireless channel is open and lacks confidentiality, the vehicle track is easy to track, lawbreakers can steal the data privacy of the users in a mode of intercepting information broadcast by the users, predicting the vehicle track and the like, so that the privacy safety problem of the users gradually becomes a main factor for limiting the participation of the vehicles and the users in data analysis. The present technology aims to better implement vehicle network termination functions by safely analyzing and processing data. Machine learning aims to extract useful information from data, while privacy is protected by hidden information, both of which need to be kept balanced when mining sensitive data. The federal learning is used as a shared machine learning algorithm, and is mainly characterized in that data of a data provider are kept locally and are not transmitted, so that leakage of data privacy is inhibited from the source, user privacy is protected, learning effect is better, and meanwhile, the privacy safety of a calculation process is protected by utilizing differential privacy in federal learning.
With the rapid development of the internet of vehicles technology, calculating the communication performance and the privacy protection degree is increasingly important for the internet of vehicles terminal. Federal learning is used as a machine learning method with privacy protection capability, can safely share data and improve model accuracy, and is beneficial to solving two major problems of data island and data privacy. According to the invention, a federal learning model (FLIoVT) for classification and prediction of the connection data of the Internet of vehicles terminal is constructed aiming at the client connection data of the Internet of vehicles terminal, and the connection normal terminal and the connection abnormal terminal are experimentally classified in the connection data of the Internet of vehicles terminal for 15 days, so that the normal use condition of the client is dynamically monitored.
Through the above analysis, the problems and defects existing in the prior art are as follows:
(1) The internet of vehicles is huge in scale, the wireless channels are open and lack of confidentiality, the vehicle tracks are easy to track, and lawbreakers can steal the data privacy of the same user by intercepting the information broadcast by the user, predicting the vehicle tracks and the like, so that the privacy security problem of the user gradually becomes a main factor for limiting the participation of the vehicle and the user in data analysis.
(2) Machine learning aims to extract useful information from data, while privacy is protected by hidden information, both of which need to be kept balanced when mining sensitive data.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a federal learning method and a federal learning system for classification and prediction of connection data of a vehicle networking terminal.
The invention is realized in such a way that the federal learning method for classification and prediction of the connection data of the vehicle networking terminal comprises the following steps:
constructing a federal learning model (FLIoVT) for classification prediction of the connection data of the vehicle networking terminal; performing iterative training on the model by utilizing a federal differential privacy algorithm on the client, processing the gradient by adopting cutting operation and noise adding, and uploading the updated gradient to a server; and updating and aggregating a new global model on the server, sending the new global model to the client in a model broadcasting mode, and repeating the local updating calculation operation until training is finished, so that the connection data of the Internet of vehicles terminal is subjected to classified prediction.
Further, the FLIoVT takes an MLP network and an LR network as model trunks, and comprises training of a client federal differential privacy algorithm and aggregation of a server federal differential privacy algorithm;
the architecture of the MLP network is as follows: constructing a feedforward neural network by using a Sequential class derived from a torch.nn.module class, passing through a Linear layer, then passing through a random inactivation Dropout layer with a probability of 0.2, adopting a ReLU activation function, and finally passing through the Linear layer for classifying final output;
the LR network has the structure: firstly, a feedforward neural network is built by a Linear layer, then a Sigmoid activation function is used, and then a Sequential class derived from a torch.nn.module class is used for building, and finally, a probability value between [0,1] is output by the Sigmoid activation function after the Linear layer.
Further, the specific process of iterative training by the federal differential privacy algorithm is as follows:
the user end k and the global model parameter theta 0 The local model learning rate eta, the loss function l, the training sample size X of each round and the local model theta are input into the model; in each iteration t=1, 2, …, T, the local data is split into |x| pieces of data X p For each batch b ε X p A gradient descent is performed and the gradient is set down,
and clipping parameters, wherein the clipping gradient is clip (theta-theta 0 ) The client computes a gradient update, θ=θ 0 +clip(θ-θ 0 ) Finally, gaussian noise is added, the local model is updated, and the local model is output as delta k ＝θ-θ 0 。
Further, the processing procedure of the server is as follows:
step one, performing local calculation, wherein the client i performs local calculation according to a local database D i And accepted server-side global model
As local parameter, i.e. +.>
t represents the current iteration value, and local model training is carried out by carrying out gradient descent strategy to obtain +.>
Step two, model disturbance is carried out, noise n conforming to Gaussian distribution is randomly added to each client, and the model disturbance is used
Disturbing the local model;
step three, model aggregation is carried out, and the server side utilizes the federal average algorithm to aggregate the received data from the client side
Obtaining new global model parameters->
Further, the average value of the model parameters is calculated to update the model of the server side;
step four, model broadcasting is carried out, and the server side broadcasts new model parameters to each client side;
and fifthly, carrying out local model updating, updating model parameters by each client, and carrying out local calculation again. The federal learning iteration is randomly selected from the client, the parameters of the trainable model are downloaded from the server, the current global model is transmitted to the client, the local model of the client is updated according to local data, local training is carried out, the local model is returned, new model parameters are uploaded to the server, and meanwhile the server is required to aggregate the updating of a plurality of clients, so that the model is further improved.
Further, the specific process of model aggregation is as follows:
input training sample data { x } 1 ,x 2 ,……,x N Loss function
Gradient clipping boundary value C, gaussian noise standard deviation sigma and initialization model parameter theta 0 Weight corresponding to client k>
Weight set w= Σfor client k ω k ；
For each round of iterations t=1, 2, …, T, the set of clients K participating in the training is randomly chosen with probability q t For each user K e K t Local training is performed:
obtaining the parameters of an aggregation client:
cut delta t The values are:
calculating the standard deviation of Gaussian noise:
using a gaussian distribution N (0,I sigma 2 ) Generating noise data, adding the noise data and then updating global model parameters: θ t ＝θ t-1 +Δ t +N(0,Iσ 2 ) And broadcast the global model to clients.
Another object of the present invention is to provide a federal learning system for classification prediction of connection data of a vehicle networking terminal, the federal learning system for classification prediction of connection data of a vehicle networking terminal comprising:
the model construction module is used for constructing a federal learning model for classification prediction of the connection data of the vehicle networking terminal;
the training module is used for training the federal differential privacy algorithm;
an aggregation module, configured to aggregate updates of a client, update global model parameters, and broadcast to the client, and another object of the present invention is to provide a computer device, where the computer device includes a memory and a processor, where the memory stores a computer program, and where the computer program when executed by the processor, causes the processor to execute the steps of the federal learning method for internet of vehicles terminal connection data classification prediction.
It is a further object of the present invention to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the step of federal learning for internet of vehicles terminal connection data classification prediction.
The invention further aims to provide an information data processing terminal which is used for realizing the federal learning system for classification and prediction of the connection data of the internet of vehicles terminal.
In combination with the technical scheme and the technical problems to be solved, the technical scheme to be protected has the following advantages and positive effects:
first, aiming at the technical problems in the prior art and the difficulty in solving the problems, the technical problems solved by the technical proposal of the invention are analyzed in detail and deeply by tightly combining the technical proposal to be protected, the results and data in the research and development process, and the like, and some technical effects brought after the problems are solved have creative technical effects. The specific description is as follows:
(1) The invention simulates the non-IID data distribution in the actual application scene, and uses the inclined class division data set as four clients.
(2) The invention builds a model using a client-server architecture in horizontal federal learning.
(3) The invention uses a local differential privacy method based on Gaussian noise to protect gradient information, thereby protecting the privacy of the federal learning calculation process.
Secondly, the technical scheme is regarded as a whole or from the perspective of products, and the technical scheme to be protected has the following technical effects and advantages:
the invention provides a federal learning method for classifying and predicting connection data of a vehicle networking terminal, which is characterized in that a normal connection terminal and an abnormal connection terminal are experimentally classified in connection data of a vehicle networking terminal for 15 days, so that normal use conditions of clients are dynamically monitored, safe shared data can be analyzed in a vehicle networking application scene, and a new direction is provided for research of federal learning of privacy protection.
Thirdly, as inventive supplementary evidence of the claims of the present invention, the following important aspects are also presented:
(1) The expected benefits and commercial values after the technical scheme of the invention is converted are as follows:
under the application scene of the internet of vehicles, the invention classifies the normal connection terminals and the abnormal connection terminals from the connection data of the 300 trolley internet of vehicles terminals, and achieves the effect of sharing a large amount of data while protecting privacy in a scattered and cooperative mode. The method can be popularized to actual scenes such as hospitals and banks for protecting data privacy, and can share data and cooperate with a training model under the condition of guaranteeing personal privacy information, so that the calculation efficiency and accuracy are improved, a new thought is provided for future large data sharing and cooperation, and the method has huge commercial value.
(2) The technical scheme of the invention fills the technical blank in the domestic and foreign industries:
under the application scene of the internet of vehicles, the invention classifies the normal connection terminals and the abnormal connection terminals from the connection data of the 300 trolley internet of vehicles terminals, achieves the effect of sharing a large amount of data while protecting privacy in a scattered and cooperative mode, and fills the technical blank.
(3) Whether the technical scheme of the invention solves the technical problems that people want to solve all the time but fail to obtain success all the time is solved:
with the rapid development of the internet of vehicles technology, calculating the communication performance and the privacy protection degree is increasingly important for the internet of vehicles terminal. Aiming at the client connection data of the internet of vehicles terminal, the invention provides the federal learning method for classifying and predicting the internet of vehicles terminal connection data, and the federal learning machine learning method with privacy protection capability is used, so that the data can be safely shared, the accuracy of a model is improved, and the privacy of a user is effectively protected.
(4) The technical scheme of the invention overcomes the technical bias:
the internet of vehicles is huge in scale, wireless channels are open and lack of confidentiality, vehicle tracks are easy to track, lawbreakers can steal data privacy of the same user by intercepting information broadcast by the user, predicting the vehicle tracks and the like, privacy safety problems of the user gradually become main factors for limiting the participation of the vehicle and the user in data analysis.
Drawings
FIG. 1 is a schematic diagram of a model framework of FLIoVT provided by an embodiment of the invention;
FIG. 2 is a connection record data set provided by an embodiment of the present invention;
FIG. 3 is a schematic diagram of an accuracy fit curve provided by an embodiment of the present invention;
FIG. 4 is a graph showing model accuracy and calculation time of a conventional deep learning method according to an embodiment of the present invention, (a) a graph showing model accuracy and (b) a graph showing calculation time;
FIG. 5 is a graph showing the effect of the magnitude of the C value on the model accuracy and the calculation time, (a) the model accuracy, and (b) the calculation time;
FIG. 6 is a graph of model accuracy and calculation time for training of unbalanced data distribution, (a) model accuracy versus graph, (b) calculation time versus graph, provided by an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
In order to fully understand how the invention may be embodied by those skilled in the art, this section is an illustrative embodiment in which the claims are presented for purposes of illustration.
The federal learning method for classification prediction of the connection data of the internet of vehicles terminal provided by the embodiment of the invention comprises the following steps:
constructing a federal learning model (FLIoVT) for classification prediction of the connection data of the vehicle networking terminal, as shown in FIG. 1; performing iterative training on the model by the client based on the federal differential privacy algorithm, processing the gradient by adopting cutting operation and noise adding, and uploading the updated gradient to a server; and updating and aggregating the new global model by the server, finally transmitting the new global model to the client in a model broadcasting mode, and repeating the local updating calculation operation until training is finished, so that the connection data of the Internet of vehicles terminal is classified and predicted.
The invention aims to construct a federal learning classification prediction model for balancing privacy protection and performance, and uses connection records within 15 days of a 300-trolley networking terminal as a data set to carry out experiments, simulate non-IID data distribution in an actual application scene, divide the data set into four clients by using inclined classes, and construct FLIoVT by using a client-server architecture in transverse federal learning due to the characteristics of more sample coincidence feature coincidence and less source coincidence, wherein each round of federal learning model training comprises: the client trains a local model issued by the server, updates the training model according to the local private data set, uses the local differential privacy based on Gaussian noise to cut and noise the gradient, uploads the updated gradient to the server after finishing, updates and aggregates a new global model, and finally broadcasts the model to the client to repeatedly perform local updating calculation until the federal learning training round is finished. The FLIoVT uses MLP and LR networks as model backbones, and adopts a momentum gradient algorithm as a training optimizer. The method has the advantages that the safe shared data is analyzed in the application scene of the Internet of vehicles, and a new direction is provided for research of federal study of privacy protection.
Three important parts in the FLIoVT framework are described in detail, including architecture of training network (MLP and LR), training of client-side federal differential privacy algorithm and aggregation of server-side federal differential privacy algorithm. The architecture of the training network is as follows:
and establishing a deep learning network (MLP and LR) to classify connection data of the 300 trolley network terminals for 15 days into a normal connection terminal and an abnormal connection terminal, taking connection data of each terminal as input, and outputting classification accuracy of two types of connection conditions. MLP and LR networks for text data classification are implemented using Pytorch as model backbones. Two network architectures are output as follows.
MLP(
(model):Sequential(
(0):Linear(in_features＝3,out_features＝200,bias＝True)
(1):Dropout(p＝0.2,inplace＝False)
(2):ReLU()
(3):Linear(in_features＝200,out_features＝2,bias＝True)
)
)
The feedforward neural network is built by using a Sequential class derived from a torch.nn.module class, linear is a Linear layer, represents a Linear connection part of one layer of the neural network, prevents over fitting by randomly inactivating the Dropout layer with a probability of 0.2, improves the network training process by introducing a regularization method, and is a common activation function which can enable the neural network to have stronger nonlinear expression capability and finally classifies final output by passing through the Linear layer. The ReLU is more suitable for the MLP network than the sigmoid activation function, has higher convergence rate, does not have the problem of gradient disappearance, and has simple calculation and high efficiency.
LR(
(linear):Linear(in_features＝3,out_features＝2,bias＝True)
(sigmoid):Sigmoid()
(model):Sequential(
(0):Linear(in_features＝3,out_features＝2,bias＝True)
(1):Sigmoid()
)
)
Firstly, a Linear layer is passed, then a sigmoid activation function commonly used in logistic regression is passed, two classification functions are realized, then a Sequential class derived from a torch.nn.module class is used for building a feedforward neural network, the Linear layer is passed, and finally, a probability value between (0 and 1) is outputted through the sigmoid activation function.
The specific process of training the client based on the federal differential privacy algorithm is as follows:
model training is performed at the client, each joint client has a fixed data set and computing power to perform momentum gradient descent, the algorithm 1 is used to process clients with all the same network architecture and loss functions, each local model is initialized by a global model from the server, the iteration number of performing momentum gradient descent is the same as the training round number, after each step of local iteration update, parameters are cut, the client calculates gradient update, and an update model is generated and shared with the aggregation server. But local data is private to each client and not shared. The client-based federal differential privacy algorithm is shown in algorithm 1.
The pseudo code of the federal differential privacy algorithm based on the client of the algorithm 1 is as follows:
input: user side k and global model parameter theta 0 Local model learning rate eta, loss function l, training sample size X of each round and local model theta
For each round of iteration t=1, 2, …, tdo
Splitting local data into |X| parts of data X
For each batch b.epsilon.X.do
Gradient descent is performed:
parameter cutting: theta+. 0 +clip(θ-θ 0 )
Adding Gaussian noise
End
End
Output: local model update: delta k ＝θ-θ 0 。
Further, the specific process of the server processing is as follows:
s101, constructing a training network, establishing a deep learning network (MLP and LR), classifying connection normal terminals and connection abnormal terminals from connection data of 300 trolley network terminals for 15 days, taking connection data of each terminal as input, and outputting classification accuracy of two types of connection conditions;
s102, training a federal differential privacy algorithm, training a local model issued by a server by a client, updating the training model according to a local private data set, processing gradients by using cutting operation and noise adding by using the local differential privacy, and uploading the updated gradients to the server;
s103, a server aggregation end federation differential privacy algorithm receives and aggregates the updates from all participating clients in federation learning at the server end, builds a new model with update parameters, and sends the new model to the clients in a model broadcasting mode;
s104, updating model parameters of each client, retraining, downloading parameters of the trainable model from the server side by the client with randomly selected federal learning iteration, transmitting the current global model into the client, updating the local model of the client according to local data, carrying out local training, returning to the local model, uploading new model parameters to the server side, and simultaneously requesting the server side to aggregate the updating of a plurality of clients to further improve the model.
Further, the specific aggregation process of the server is as follows: and the server side with the global model manages the overall progress of model training and distributes the original model to all participating clients. The new model with updated parameters is built using algorithm 2 to receive and aggregate updates from all participating clients in each federal study. The federal differential privacy algorithm based on the server side is shown as algorithm 2.
The pseudo code of the federal differential privacy algorithm based on the server side of the algorithm 2 is as follows:
shear boundary value C, gaussian noise standard deviation sigma and initialization model parameter theta 0 Weight corresponding to client k
Let w= Σ k ω k
For each round of iteration t=1, 2, …, tdo
Randomly selecting client set K participating in training with probability q t
For each user K e K t do
Performing local training:
End
aggregation client parameters:
cut delta t Value:
calculating the standard deviation of Gaussian noise:
using a gaussian distribution N (0,I sigma 2 ) Generating noise data
Noise data is added in the global model aggregation operation, and global model parameters are updated: θ t ←θ t-1 +Δ t +N(0,Iσ 2 )
Broadcasting global models to clients
End
The federal learning system for classification prediction of connection data of a vehicle networking terminal provided by the embodiment of the invention specifically comprises the following components:
the model construction module is used for constructing a federal learning model for classification prediction of the connection data of the vehicle networking terminal;
the training module is used for training the federal differential privacy algorithm;
the aggregation module is used for aggregating the update of the client, updating the global model parameters and broadcasting the global model parameters to the client
In order to prove the inventive and technical value of the technical solution of the present invention, this section is an application example on specific products or related technologies of the claim technical solution.
The FLIoVT provided by the invention classifies the normal connection terminal and the abnormal connection terminal from the connection data of the 300 trolley network terminals for 15 days. In this decentralized and collaborative framework, the effect of private data sharing can be achieved while protecting privacy. The FLIoVT architecture is a client-server architecture, also known as a master-slave architecture, in which a federal averaging algorithm is used to solve the federal optimization problem. The federal averaging algorithm is divided into two types: the gradient average is that the participant sends the gradient information to the server, the server aggregates the received gradient information, and then sends the aggregated gradient information to the participant; the model average is that the participator calculates the model parameter locally and sends the parameter information to the server, the server aggregates the received model parameter and then sends the aggregated model parameter to the participator. The invention adopts a safe federal average algorithm on the basis of gradient average, which means that a privacy protection technology is added on the basis of the federal average algorithm, and a local differential privacy protection method based on Gaussian noise is used.
Firstly, data preparation is carried out, wherein the data preparation comprises loading of a data set and dividing of the data set;
the experimental data used in the invention is connection record of 300 trolley networking terminals for 15 days, only a connection record part and terminal numbers are reserved for protecting the privacy of clients to which the terminals belong, a named two-class data set is 101.Csv (connection record data set), 300 observation values are shared, the first 15 columns are used as independent variables (input x), the extraction characteristics are used for digitizing the daily connection record, the connectionless record is marked as 0, the connection record is marked as 1, the last column is used as a class value (output y), the class value is < = 8 (abnormal connection) and >8 (normal connection), and the data set of the intercepted part is shown in figure 2.
The CSV data set class is a CSV data set loading class, is processed into a data format suitable for a model, randomly divides the training set and the testing set into 240 pieces of data with the proportion of 8:2, the testing set into 60 pieces of data, then splits the training data set into 4 clients, distributes 60 pieces of data for IID data to each client, and according to the same distribution, all the clients have the training data with the same quantity and the same class proportion; for non-IID data distribution, the data set is divided using a sloped class distribution such that each client obtains a different proportion of data from each class, but the total number of data for the four clients is the same; for unbalanced data distribution, the entire training sample is distributed over 4 clients, each having a different number and class ratio of training data at random. The IID data distribution belongs to ideal distribution and has no practical meaning, and when IID data is used, the actual and centralized learning is equivalent, so that the invention performs model training on non-IID data distribution in order to be suitable for practical application scenes without considering the situation.
The experimental environment of the examples of the present invention is shown in table 1:
table 1 experimental environment
The model parameters of the embodiment of the invention are set as follows:
when the dependent variables are of different types, the MLP and LR networks divide 300 terminals into corresponding types according to the input 15-day connection data, and in order to obtain the optimal training result, proper optimizers and training step sizes need to be selected on model setting so that the value of the loss function is minimum. And (3) a Momentum gradient descent method is selected for the MLP and LR networks to update the optimized network weight, and Momentum is set to 0.9. In the selection of the loss function, the MLP network adopts a cross entropy loss function, nn. Cross EntopyLoss is the integration of nn. LogSoftmax and nn. NLLLoss, and for log-softmax output, the loss is calculated by using negative log likelihood loss nn. NLLLoss, and parameter updating is carried out by back propagation, so that the loss is minimized; the LR network selects a log loss function, takes-log for the likelihood function to convert the maximized likelihood function into a minimized loss function so as to solve the parameter value. Setting related model parameters: the output size is 2, the number of clients is 4, the learning rate is 0.01, the batch size is 128, the training round is 50, and in consideration of calculation cost, the local update round number of each client is equal to 1 in federal learning, because the consumed privacy budget can be large due to the fact that more update rounds are set, the gradient clipping boundary value C is 0.5, and the Gaussian noise standard deviation sigma is 0.5.
Customer normal usage is dynamically monitored by FLIoVT. To evaluate model performance, model accuracy of the test data was reported after each round of federal learning, and the validity and feasibility of the method was further verified.
It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.
The embodiment of the invention has a great advantage in the research and development or use process, and has the following description in combination with data, charts and the like of the test process.
The FLIoVT is used for a training experiment of the connection data of the vehicle networking terminal, and is respectively called FL-MLP and FL-LR when an MLP and LR network are used as a model backbone. Training the FL-MLP model and the FL-LR model with data privacy protection capability to obtain accuracy and calculation time. In order to further verify the effectiveness and the practical feasibility of the model, three groups of comparison experiments are set, firstly, the traditional deep learning method is compared, and the model has low accuracy rate while protecting the data privacy; comparing the client cooperation level to prove that the number of clients in each round of cooperative training has positive influence on the model accuracy; and finally, comparing unbalanced data distribution to prove that the performance of the model under the scattered data in the actual scene is effective. The obtained FLIoVT has privacy protection capability, and meanwhile, still keeps higher model accuracy and calculation time which is not higher than that of the traditional deep learning method, shows forward significance of a collaborative training mode, and is suitable for practical application scenes.
(1) Training results of FLIoVT
The accuracy of the FL-MLP model of all the clients participating in the collaboration training is 0.8113, the calculation time is 0.916s, the accuracy of the FL-LR model is 0.783, the calculation time is 0.562s, and the fitted curve is shown in FIG. 3. As can be seen from fig. 3, the model accuracy trained on the non-IID data distribution is low and grows slowly in the early stage, because non-uniform sampling results in a large amount of data from one class per client and little data from another class, with increasing number of rounds, the accuracy stabilizes to higher levels.
(2) Model accuracy and calculation time compared with those of the traditional deep learning method
When MLP and LR networks are also used as model trunks on the traditional deep learning method, they are called DL-MLP and DL-LR, respectively. Compared with the model accuracy of the traditional deep learning training, whether the FLIoVT reduces the performance of the model while protecting the data privacy is verified, and only if the FLIoVT and the FLIoVT are balanced, the performance of the model is better. The accuracy of the trained DL-MLP model is 0.8302, the calculation time is 3.05s, the accuracy of the trained DL-LR model is 0.8019, and the calculation time is 2.11s. Model accuracy and calculation time of FL-MLP and FL-LR models versus conventional deep learning methods are shown in FIG. 4.
As can be seen from fig. 4, the convergence speed of the conventional deep learning method at the initial training stage is faster than that of the federal learning method, the federal learning method converges faster with increasing rounds, the accuracy is gradually stabilized, the conventional deep learning method has no privacy protection capability, while the FLIoVT keeps not lower model accuracy than the conventional deep learning method while preventing the leakage of the shared data, and the calculation time is also less than that of the conventional deep learning method.
(3) Comparing the influence of the client cooperation level on the model accuracy and the calculation time
The client cooperation level is also called a C value, the parallel quantity of multiple clients is controlled, and in non-IID data distribution, the influence of the C value on the model performance is compared and studied. On the premise of setting the number of the clients to be 4, when C=1, selecting all the four involved clients to perform collaborative training in each round; when c=0.75, three clients are selected for collaborative training in each round, when c=0.5, two clients are selected for collaborative training in each round, and when c=0.25, only one client is selected for training in each round, and clients are not learned in parallel but are learned sequentially. To demonstrate the impact of collaboration levels between clients on model performance, experiments were performed with c=1, c=0.75, c=0.5, and c=0.25 with data non-IID distribution between clients, with trained FL-MLP model accuracies of 0.8113, 0.7981, 0.7736, and 0.7547, calculation times of 0.365s, 0.551s, 0.726s, and 0.916s, respectively, trained FL-LR model accuracies of 0.783, 0.7633, 0.74, and 0.717, and calculation times of 0.233, 0.353, 0.469, and 0.562, respectively. The effect of comparing the magnitude of the C value on model accuracy and computation time is shown in fig. 5.
As can be seen from fig. 5, after training 50 rounds, when c=1, four clients perform cooperative training in each round, and the FL-MLP and FL-LR models with non-IID data distribution training have maximum accuracy and calculation time; when c=0.75, three clients perform cooperative training in each round, and the FL-MLP and FL-LR model accuracy and calculation time of non-IID data distribution training are smaller than those of c=1; when c=0.5, two clients perform cooperative training in each round, and the FL-MLP and FL-LR model accuracy and calculation time of non-IID data distribution training are smaller than when c=0.75; when c=0.25, only one client trains in each round, and the FL-MLP and FL-LR models for non-IID data distribution training have minimal accuracy and computation time. The highest model accuracy of the non-IID data distribution training when c=1 is due to the fact that as the number of clients participating in training is reduced, the model quality obtained by the clients through collaborative training is poorer, so that the accuracy obtained by training after updating the model sent by the clients by each round of server is lower, and the calculation time is more important because communication, calculation cost and time length are increased due to collaboration among all the clients participating in training, so that the number of the clients participating in training is reasonably set. Thus, for non-IID data distribution, increasing the number of collaboration clients in each round can improve accuracy, but can prolong calculation time, and the influence of collaboration level among clients on model accuracy is positive.
(4) Model accuracy and computation time for comparison of unbalanced data distribution training
The unbalanced data distribution can more represent the data distribution under the actual application scenes such as the internet of vehicles, in order to verify the actual feasibility of the FLIoVT, experiments are carried out under the condition that the data distribution among clients is unbalanced, and when C=1, the accuracy of FL-MLP and FL-LR models trained by the unbalanced data distribution is 0.76 and 0.7264, and the calculation time is 1.53 and 1.001. Model accuracy and computation time for training against unbalanced data distribution are shown in fig. 6.
As can be seen from fig. 6, the model accuracy of the training of the non-IID data distribution is higher than that of the unbalanced data distribution because the data amount is different between the four clients of the unbalanced data distribution, but the model accuracy of the training of the non-IID data distribution is close to that of the model after several rounds of training, and is higher than that of the model trained when no cooperation is performed between the clients. The model computation time for unbalanced data distribution training is longer than that of non-IID data distribution, but still less than that of the traditional depth method, which means that the influence of data volume unbalance between clients on the model performance is small.
Under the application scene of the internet of vehicles, the FLIoVT provided by the invention classifies the normal connection terminal and the abnormal connection terminal from the connection data of the 300 trolley internet of vehicles terminals, and achieves the effect of sharing a large amount of data while protecting privacy in a scattered and cooperative mode. The local differential privacy method based on Gaussian noise is used for protecting gradient information, so that privacy of a federal learning calculation process is protected, an MLP and LR network is used as a model backbone, a momentum gradient algorithm is adopted on selection of a training optimizer, FL-MLP model accuracy trained while protecting data privacy is 0.8113, calculation time is 0.916, FL-LR model accuracy trained is 0.783, and calculation time is 0.562. In a comparison experiment for verifying the performance of a model, the traditional deep learning method is compared, and the accuracy rate of the FLIoVT is not lower than that of the traditional deep learning method without privacy protection capability while protecting the data privacy, so that the time is less; further, the influence of the client cooperation level on the model performance is compared and tested, and the influence of the cooperation level between the clients on the model accuracy is proved to be positive, but the calculation time is increased along with the increase of the clients participating in the cooperation training; finally, the model is trained on unbalanced data distribution, the accuracy is higher than the model accuracy trained when no collaboration is performed between clients along with the increase of training round number, the calculation time is still smaller than that of the traditional depth method, and the performance of the model under scattered data is effective.
The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.
Claims (10)
1. The federal learning method for classification and prediction of the connection data of the internet of vehicles is characterized by comprising the following steps of:
constructing a federal learning model for classification prediction of the connection data of the vehicle networking terminal; performing iterative training on the model by the client based on the federal differential privacy algorithm, processing the gradient by adopting cutting operation and noise adding, and uploading the updated gradient to a server; and updating and aggregating the new global model by the server, finally transmitting the new global model to the client in a model broadcasting mode, and repeating the local updating calculation operation until training is finished, so that the connection data of the Internet of vehicles terminal is classified and predicted.
2. The federal learning method for classification prediction of connection data of a vehicle networking terminal according to claim 1, wherein the FLIoVT uses an MLP network and an LR network as model trunks, and further comprises training of a client federal differential privacy algorithm and aggregation of a server-side federal differential privacy algorithm;
the architecture of the MLP network is as follows: the feedforward neural network is built by using a Sequential class derived from a torch.nn.module class, and the feedforward neural network is used for classifying the final output by passing through a Linear layer, then passing through a random inactivation Dropout layer with the probability of 0.2, then adopting a ReLU activation function and finally passing through the Linear layer.
3. The federal learning method for classification prediction of connection data of a vehicle networking terminal according to claim 2, wherein the LR network has a structure of: firstly, a feedforward neural network is built by a Linear layer, then a Sigmoid activation function is used, and then a Sequential class derived from a torch.nn.module class is used for building, and finally, a probability value between [0,1] is output by the Sigmoid activation function after the Linear layer.
4. The federal learning method for classification prediction of connection data of a vehicle networking terminal according to claim 2, wherein the training of the client federal differential privacy algorithm comprises the following specific procedures:
client k and global model parameters theta 0 The local model learning rate eta, the loss function l, the training sample size X of each round and the local model theta are input into the model; in each iteration t=1, 2, …, T, the local data is split into |x| pieces of data X p For each batch b ε X p A gradient descent is performed and the gradient is set down,
and clipping parameters, wherein the clipping gradient is clip (theta-theta 0 ) The client computes a gradient update, θ=θ 0 +clip(θ-θ 0 ) Finally, gaussian noise is added, and the local model update parameter delta is updated k ＝θ-θ 0 。
5. The federal learning method for classification prediction of connection data of a vehicle networking terminal according to claim 2, wherein the processing procedure of the server side is:
step one, performing local calculation, wherein the client i performs local calculation according to a local database D i And accepted server-side global model
As local parameter, i.e. +.>
t represents the current iteration value, and the local model training is carried out by adopting a gradient descent strategy to obtain
Step two, model disturbance is carried out, noise n conforming to Gaussian distribution is added to each client, and the model disturbance is used
Disturbing the local model;
step three, model aggregation is carried out, and the server side utilizes the federal average algorithm to aggregate the received data from the client side
Obtaining new global model parameters->
Further, the average value of the model parameters is calculated to update the model of the server side;
step four, model broadcasting is carried out, and the server side broadcasts new model parameters to each client side;
and fifthly, carrying out local model updating.
6. The federal learning method for classification prediction of connection data of a vehicle networking terminal according to claim 5, wherein the specific process of model aggregation is:
input training sample data { x } 1 ,x 2 ,……,x N Loss function
Gradient clipping boundary value C, gaussian noise standard deviation sigma and initialization model parameter theta 0 Weight corresponding to client k>
Weight set w= Σfor client k ω k ；
For each round of iterations t=1, 2, …, T, the set of clients K participating in the training is randomly chosen with probability q t For each userk∈K t Executing model parameters obtained by local training:
and then executing the aggregation operation to obtain the model parameters of the aggregation client:
cut delta t The values are:
the standard deviation of the gaussian noise is calculated as:
wherein z is the mean value of noise, and S is the normalized value;
using a gaussian distribution N (0,I sigma 2 ) Generating noise data, adding the noise data and then updating global model parameters: θ t ＝θ t-1 +Δ t +N(0,Iσ 2 ) And broadcast the global model to clients.
7. A federal learning system for classification prediction of internet of vehicles terminal connection data according to any one of claims 1 to 6, wherein the federal learning system for classification prediction of internet of vehicles terminal connection data comprises:
the model construction module is used for constructing a federal learning model for classification prediction of the connection data of the vehicle networking terminal;
the training module is used for training the federal differential privacy algorithm;
and the aggregation module is used for aggregating the update of the client, updating the global model parameters and broadcasting the global model parameters to the client.
8. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the federal learning method for internet of vehicles terminal connection data classification prediction as claimed in any one of claims 1-6.
9. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the federal learning method for internet of vehicles terminal connection data classification prediction as in any one of claims 1-6.
10. An information data processing terminal, wherein the information data processing terminal is used for realizing the federal learning system for classification prediction of connection data of a vehicle networking terminal according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211565002.6A CN116227631A (en) | 2022-12-07 | 2022-12-07 | Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211565002.6A CN116227631A (en) | 2022-12-07 | 2022-12-07 | Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116227631A true CN116227631A (en) | 2023-06-06 |
Family
ID=86579321
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211565002.6A Pending CN116227631A (en) | 2022-12-07 | 2022-12-07 | Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116227631A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116611115A (en) * | 2023-07-20 | 2023-08-18 | 数据空间研究院 | Medical data diagnosis model, method, system and memory based on federal learning |
| CN117094382A (en) * | 2023-10-19 | 2023-11-21 | 曲阜师范大学 | Personalized federal learning method, device and medium with privacy protection |
| CN118151540A (en) * | 2024-05-09 | 2024-06-07 | 深圳市大数据研究院 | Model training method based on Gaussian distribution, vehicle control method and device |
| CN118278506A (en) * | 2024-05-31 | 2024-07-02 | 南方电网能源发展研究院有限责任公司 | Load prediction model establishment method, device, equipment and storage medium based on multi-gradient fair descent federal learning algorithm |
| CN118468988A (en) * | 2024-07-09 | 2024-08-09 | 浙江大学 | Terminal data leakage event prediction method and system based on horizontal federal learning |
-
2022
- 2022-12-07 CN CN202211565002.6A patent/CN116227631A/en active Pending
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116611115A (en) * | 2023-07-20 | 2023-08-18 | 数据空间研究院 | Medical data diagnosis model, method, system and memory based on federal learning |
| CN117094382A (en) * | 2023-10-19 | 2023-11-21 | 曲阜师范大学 | Personalized federal learning method, device and medium with privacy protection |
| CN117094382B (en) * | 2023-10-19 | 2024-01-26 | 曲阜师范大学 | Personalized federal learning method, device and medium with privacy protection |
| CN118151540A (en) * | 2024-05-09 | 2024-06-07 | 深圳市大数据研究院 | Model training method based on Gaussian distribution, vehicle control method and device |
| CN118151540B (en) * | 2024-05-09 | 2024-08-09 | 深圳市大数据研究院 | Model training method based on Gaussian distribution, vehicle control method and device |
| CN118278506A (en) * | 2024-05-31 | 2024-07-02 | 南方电网能源发展研究院有限责任公司 | Load prediction model establishment method, device, equipment and storage medium based on multi-gradient fair descent federal learning algorithm |
| CN118278506B (en) * | 2024-05-31 | 2024-09-06 | 南方电网能源发展研究院有限责任公司 | Load prediction model establishment method, device, equipment and storage medium based on multi-gradient fair descent federal learning algorithm |
| CN118468988A (en) * | 2024-07-09 | 2024-08-09 | 浙江大学 | Terminal data leakage event prediction method and system based on horizontal federal learning |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116227631A (en) | Federal learning method and system for classification prediction of connection data of Internet of vehicles terminal | |
| JP2023505973A (en) | federated mixed model | |
| Tonellotto et al. | Neural network quantization in federated learning at the edge | |
| Nouman et al. | Malicious node detection using machine learning and distributed data storage using blockchain in WSNs | |
| CN110798314B (en) | Quantum key distribution parameter optimization method based on random forest algorithm | |
| CN112734032A (en) | Optimization method for horizontal federal learning | |
| CN112749749A (en) | Classification method and device based on classification decision tree model and electronic equipment | |
| CN115130536A (en) | Training method of feature extraction model, data processing method, device and equipment | |
| CN113691594B (en) | Method for solving data imbalance problem in federal learning based on second derivative | |
| CN115481441A (en) | Difference privacy protection method and device for federal learning | |
| CN112989186A (en) | Information recommendation model training method and device, electronic equipment and storage medium | |
| CN116629376A (en) | Federal learning aggregation method and system based on no data distillation | |
| Chen et al. | Distributed community detection over blockchain networks based on structural entropy | |
| CN115309647A (en) | Federal learning-based software defect prediction privacy protection method | |
| Zhou et al. | Intelligent analysis system for signal processing tasks based on LSTM recurrent neural network algorithm | |
| CN116227632A (en) | Federation learning method and device for heterogeneous scenes of client and heterogeneous scenes of data | |
| Al-Maslamani et al. | Toward secure federated learning for iot using drl-enabled reputation mechanism | |
| Yi et al. | pFedLHNs: Personalized federated learning via local hypernetworks | |
| Zhang et al. | Cross-fcl: Toward a cross-edge federated continual learning framework in mobile edge computing systems | |
| Yang et al. | Adaptive safety evaluation for connected and automated vehicles with sparse control variates | |
| Shaikh et al. | Bayesian optimization with stacked sparse autoencoder based cryptocurrency price prediction model | |
| Buelens et al. | Predictive inference for non-probability samples: a simulation study | |
| CN117033997A (en) | Data segmentation method, device, electronic equipment and medium | |
| Yang et al. | Horizontal federated learning | |
| Liu et al. | Optimization of Federated Learning Communications with Heterogeneous Quantization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |