CN117135597A - Intelligent network-connected automobile low-delay data sharing method based on distributed learning - Google Patents

Intelligent network-connected automobile low-delay data sharing method based on distributed learning Download PDF

Info

Publication number
CN117135597A
CN117135597A CN202311088483.0A CN202311088483A CN117135597A CN 117135597 A CN117135597 A CN 117135597A CN 202311088483 A CN202311088483 A CN 202311088483A CN 117135597 A CN117135597 A CN 117135597A
Authority
CN
China
Prior art keywords
model
data
global
student
intelligent network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311088483.0A
Other languages
Chinese (zh)
Inventor
黄晓舸
肖亚莉
李春磊
杨帆行
陈前斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202311088483.0A priority Critical patent/CN117135597A/en
Publication of CN117135597A publication Critical patent/CN117135597A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W28/00—Network traffic management; Network resource management
    • H04W28/02—Traffic management, e.g. flow control or congestion control
    • H04W28/08—Load balancing or load distribution
    • H04W28/09—Management thereof
    • H04W28/0925—Management thereof using policies
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Traffic Control Systems (AREA)

Abstract

The application relates to an intelligent network-connected automobile low-time-delay data sharing method based on distributed learning, and belongs to the technical field of mobile communication. The method comprises the steps of obtaining a global model issued by a roadside unit; the global model is obtained by polymerizing a student model and global aggregation weights; collecting vehicle data, and locally training a student model and a teacher model corresponding to the global model; uploading the student model to a roadside unit, and determining global aggregation weights based on the teacher model; if the deviation degree of the global model and the student model exceeds a preset threshold, uploading part of vehicle data to a data buffer area of the roadside unit; obtaining shared data issued by a roadside unit, and locally correcting a student model and a teacher model corresponding to the global model; the shared data is determined by the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters. According to the technical scheme provided by the embodiment of the application, the precision of the model can be obviously improved.

Description

Intelligent network-connected automobile low-delay data sharing method based on distributed learning
Technical Field
The application belongs to the technical field of mobile communication, and relates to an intelligent network-connected automobile low-time-delay data sharing method based on distributed learning.
Background
An intelligent networking car (Intelligent Connected Vehicle, ICV) is a car that enables high connectivity and information exchange between vehicles, between vehicles and infrastructure, between vehicles and the internet by various means of technology. The vehicles and Road Side Units (RSUs) are mutually associated, so that information exchange among the vehicles, vehicle-Road coordination and intelligent traffic management are realized. Federal learning (Federated Learning, FL) has important application potential in ICV fields, where ICV data relates to private content such as location and travel track. And the FL aggregates the local knowledge into a global model through the sharing model, so that the privacy of the original data is ensured. Meanwhile, due to the influences of driving habits, environments and the like, the data collected by the ICV in the driving process are characterized by Non-independent identical distribution (Non-Independent and Identically Distributed, non-IID), so that differences exist in the local training process, the convergence speed of the model is reduced, and the accuracy is reduced.
Knowledge distillation (Knowledge Distillation, KD) is a widely used model compression method. It transfers large, complex model knowledge into small, simplified models, thereby reducing the resource requirements for model computation, transmission and storage. KD-based mutual distillation (Mutual Knowledge Distillation, MKD) methods involve complex teacher models and simplified student models that enable more complete knowledge sharing among models through two-way learning. In the FL process, student models are typically used to upload to the RSU to aggregate global models, learning knowledge from multiple ICVs. Meanwhile, the teacher model is stored locally for further KD and student model training.
Common FL is classified into synchronous FL and asynchronous FL, and the adoption of synchronous FL leads to problems of inefficiency in training and imbalance in resource utilization due to ICV computational power and constraints of communication resources. In the Non-IID data scene, the asynchronous FL is adopted, so that the model diverges in the training process and cannot converge. Furthermore, asynchronous FL requires frequent parameter transmission and synchronization between devices or nodes, which can result in significant communication power consumption and bandwidth overhead.
Due to the influences of driving habits, environments and the like, data acquired by the ICV in the driving process show Non-IID characteristics, so that model differences of different ICV training are large, and accuracy is reduced.
Disclosure of Invention
Based on the problems, the application designs an intelligent network-connected automobile low-time-delay data sharing method based on distributed learning, so that model accuracy is improved, and communication resource expenditure and training time are reduced.
The embodiment of the application provides an intelligent network-connected automobile low-delay data sharing method based on distributed learning, which is used for solving the problems of high federal learning communication overhead and low Non-IID data training model accuracy in the Internet of vehicles.
In order to solve the technical problems, the implementation process of the application is as follows:
in a first aspect of the present application, the present application provides a low-latency data sharing method for an intelligent network-connected vehicle based on distributed learning, which is applied to one or more intelligent network-connected vehicles, where the intelligent network-connected vehicles are communicable devices in the internet of vehicles and can communicate with roadside units in the internet of vehicles, and the method includes:
acquiring a global model issued by a roadside unit; the global model is obtained by polymerizing a student model and global aggregation weights;
collecting vehicle data, and locally training a student model and a teacher model corresponding to the global model;
uploading a locally trained student model and a global aggregation weight determined based on a teacher model to a roadside unit;
if the deviation degree of the global model and the locally trained student model exceeds a preset threshold value, uploading part of vehicle data to a data buffer area of the roadside unit;
obtaining shared data issued by a roadside unit, and locally correcting a student model and a teacher model corresponding to the global model; the shared data is determined by the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters.
In a second aspect of the present application, the present application further provides a low-latency data sharing method for an intelligent network-connected vehicle based on distributed learning, which is applied to one or more roadside units, where the roadside units are communicable devices in the internet of vehicles and can communicate with the intelligent network-connected vehicle in the internet of vehicles, and the method includes:
acquiring student models uploaded by a plurality of intelligent network-connected automobiles and determining global aggregation weights based on teacher models;
m uploaded student models with the nearest arrival time and corresponding global aggregation weights are selected, and the M student models are aggregated into an overall model according to the global aggregation weights;
issuing global models to a plurality of intelligent network-connected automobiles;
if the deviation degree of the global model and the locally trained student model exceeds a preset threshold, receiving partial vehicle data uploaded by the intelligent network-connected automobile;
integrating according to the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters to generate shared data;
and issuing shared data to a plurality of intelligent network-connected automobiles.
In the technical scheme provided by the embodiment of the application, the optimal number of learning models is calculated through the time delay sum of the arrival time of the minimum chemical raw model in the expected iteration times, so that the student models with communication energy consumption exceeding the constraint are removed, the convergence time delay of the global model is minimum, and the aim of low time delay is fulfilled; determining vehicle liveness through the latest training times of the intelligent network-connected automobile participating in global aggregation, and determining the self-adaptive learning rate of the student model through the vehicle liveness so as to adapt to the influence of Non-IID data on convergence speed and improve the precision of the global model in a Non-IID data scene; the new and old data of different ICVs are fused by incremental learning, the change rate of important parameters is minimized, the small change amplitude of the important parameters is ensured, the proportion of the new and old data to the training is balanced, the student model can continuously learn, and the degree of forgetting the data is reduced.
Drawings
Preferred embodiments of the present application will be described in detail below with reference to the attached drawings;
FIG. 1 is a diagram of an intelligent networked automobile data sharing architecture based on distributed learning according to an exemplary embodiment of the present application;
FIG. 2 is a flowchart of a low-latency data sharing method for an intelligent network-connected vehicle based on distributed learning according to an exemplary embodiment of the present application;
FIG. 3 is a flowchart of a method for sharing low-latency data of an intelligent network-connected vehicle based on distributed learning according to another exemplary embodiment of the present application;
FIG. 4 is a schematic diagram of vehicle selection based on semi-asynchronous federal learning in accordance with an exemplary embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship; in the formula, the character "/" indicates that the front and rear associated objects are a "division" relationship. The term "plurality" as used herein refers to two or more, if not specified.
In order to clearly describe the technical solution of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", etc. are used to distinguish the same item or similar items having substantially the same function or effect, and those skilled in the art will understand that the terms "first", "second", etc. do not limit the number and execution order.
In embodiments of the application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g." in an embodiment should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion. In the embodiments of the present application, unless otherwise indicated, the meaning of "plurality" means two or more.
The scheme provided by the embodiment of the application relates to a car networking federal learning technology in a mobile communication technology, and the combination of federal learning and the car networking technology can be realized in the existing car networking federal learning technology, but due to the influences of driving habits, environment and the like, data acquired by ICV in the driving process can show Non-IID characteristics, so that the model difference of different ICV training is larger, and the accuracy is reduced. In order to reduce the influence of Non-IID data on model accuracy, partial data is shared among ICVs through incremental learning, so that the model is prevented from forgetting previously learned knowledge, and generalization capability and adaptability of the model are improved.
Based on this, the embodiment of the application provides an intelligent network-connected automobile low-time-delay data sharing method based on distributed learning, which can be applied to a communication scene of an intelligent network-connected automobile data sharing architecture diagram based on distributed learning, as shown in fig. 1, in which the network considers the internet-of-vehicles scene of interaction of N ICVs and 1 RSU, and the scene comprises dynamic MKD and incremental learning. The student model is distilled through the MKD, so that the student model obtains richer knowledge, and the consumption of computing resources is reduced; because the ICV collects Non-IID data, the accuracy of the student model can be affected, when the student model of the ICV has large deviation, partial data are shared among the ICVs through incremental learning, and the accuracy and generalization capability of the model are improved. The following will describe the present communication scenario in detail.
In the embodiment of the present application, the adaptive mutual distillation may be replaced by other algorithms, such as KD algorithm, off-line distillation, on-line distillation, etc., as long as training based on "teacher-student-network" can be achieved.
Fig. 2 is a flowchart of a method for intelligent network-connected vehicle security data collaboration according to an exemplary embodiment of the present application, as shown in fig. 2, applied to one or more intelligent network-connected vehicles, where the intelligent network-connected vehicles are communicable devices in the internet of vehicles and can communicate with roadside units in the internet of vehicles, and the method includes:
101. acquiring a global model issued by a roadside unit; the global model is obtained by polymerizing a student model and global aggregation weights;
in the embodiment of the application, the intelligent network-connected automobile can obtain a corresponding global model from the roadside units communicated with the intelligent network-connected automobile, and the global model can be an initialized model or a model obtained by aggregation of a student model and global aggregation weight in the training process.
102. Collecting vehicle data, and locally training a student model and a teacher model corresponding to the global model;
in the embodiment of the application, in the local training stage, a mutual distillation technology is adopted, and the intensity of each distillation is dynamically adjusted so as to strengthen the knowledge absorption and generalization capability of a student model; by initializing the student model and the teacher model, different network layers of the student model are distilled. The CE (Cross-end) loss and KL (Kullback-Leibler) divergence of the two are calculated, the distillation intensity is adjusted by dynamic distillation loss, and the teacher model and the student model are updated based on the distillation loss, respectively.
In the embodiment of the application, the MKD method learns each other through the student model and the teacher model, and dynamically adjusts the distillation intensity each time to strengthen the knowledge absorbing and generalization capability of the student model, wherein the ICV only uploads the student model to the RSU, thereby realizing model sharing and collaborative training among the ICVs and reducing the communication overhead and the model convergence time. Meanwhile, the teacher model is stored locally for further KD student model training. The federal learning method based on MKD mainly comprises the following steps:
(1) Distillation student model
The CNN is used for initializing a teacher model and a student model, the teacher model adopts a more complex network structure compared with the student model, the student model comprises an input layer, a convolution layer, a full connection layer, an output layer and the like, different student model structures are selected, and model compression is carried out on different network layers, so that the size and calculation cost of the student model are reduced. Will v i Is denoted as y by a sample soft label of student model and teacher model t,i And y s,i 。
(2) Loss calculation
When the model accuracy is lower, the distillation loss value is large, in order to balance the knowledge transfer between the teacher model and the student model, the performance of the student model is improved, and the distillation strength can be adjusted according to the quality of the soft label through dynamic distillation loss. By y t,i And y s,i And calculating CE losses and KL divergences of the student model and the teacher model respectively, wherein the CE losses of the student model and the teacher model are defined as:
KL divergence of student model and teacher model is defined as:
dynamic distillation loss is defined as:
(3) Updating student models
Gradient of student model and teacher model respectively pass throughAnd->Calculate and perform local updates as follows:
the student model is updated by meta learning, common characteristics can be extracted from a plurality of training data, and generalization capability of the student model is enhanced. First, gradient is calculated according to dynamic distillation loss, and back propagation is performed, and then 2 SGD updates are performed to obtain 2 student models, further expressed as:
and->Is from v i Is a random extraction of data. Averaging 2 student models, namely, carrying out 2 times of random gradient descent (Stochastic Gradient Descent, SGD) on the student models, improving generalization capability of a global model, and obtaining a reliable student model, namely:
in the updating process of the student model, in the preferred embodiment of the application, the influence of Non-IID data on the convergence speed can be reduced by adaptively adjusting the learning rate. Learning rate is subject to ICV liveness A i Influence, v i The learning rate of (2) can be expressed as:
wherein eta i Representing student model v i Is used for the self-adaptive learning rate of the system; η represents the learning rate of the global model; a is that i Representing a locally trained student model v i The vehicle liveness of the intelligent network-connected automobile; gamma is the adjustment coefficient.
In a preferred embodiment of the application, ICV liveness is calculated based on the number of times of participation in the aggregation. The local training student model v i The method for calculating the vehicle activity of the intelligent network-connected automobile comprises the steps of determining the vehicle activity of the intelligent network-connected automobile according to the latest training times of the intelligent network-connected automobile participating in global aggregation; if the latest training times of the intelligent network-connected automobile participating in global aggregation is lower than a preset threshold value, forcing the intelligent network-connected automobile to acquire a global model issued by a roadside unit, uploading a locally trained student model to the roadside unit at a specified time,and a global aggregate weight determined based on the teacher model; and resetting the vehicle activity of the intelligent network-connected vehicle.
For example, if an ICV participates in the current round of polymerization, the ICV activity is increased by 1, i.e., A i ＝A i +1. The ICV whose continuous three rounds of liveness are unchanged is labeled as an inactive ICV, the latest global model will be issued for local model training on the next round of RSUs received, and the inactive ICV needs to upload its local model at a given time.
According to the embodiment of the application, the learning rate is adaptively adjusted when the teacher model and the student model are updated according to the liveness of each ICV, so that the influence of Non-IID data on the convergence speed can be adapted.
103. Uploading a locally trained student model and a global aggregation weight determined based on a teacher model to a roadside unit;
in the embodiment of the application, a teacher model evaluation method is also provided, which evaluates the quality of the teacher model by calculating the CE loss value of the teacher model, namelyThen expressed as weights in the global model aggregate by exponential transformation, model v i The global aggregate weights of (a) are expressed as:
in the embodiment of the application, the CE loss value of the teacher model is calculated, the quality of the teacher model is estimated by using the CE loss value, namely, the smaller the CE loss value, the more important the knowledge taught by the teacher model is, the higher the weight is given to the teacher model, and the weight of the student model in global aggregation is calculated based on the CE loss value.
104. If the deviation degree of the global model and the locally trained student model exceeds a preset threshold value, uploading part of vehicle data to a data buffer area of the roadside unit;
in the scene of the Internet of vehicles, due to the influences of driving habits, environments and the like, the data acquired by the ICVs in the driving process show Non-IID characteristics, so that the deviation of the student models is obvious, in order to improve the model precision, the deviation of the student model and the global model of each ICV is firstly measured through cosine similarity, and whether the ICV triggers data sharing or not is judged according to the deviation. And when the data sharing mechanism is triggered, each ICV needs to upload part of original data to the RSU data buffer area, and then the change rate of important parameters is minimized based on the Fisher information matrix to obtain the training proportion of new and old data of the ICV, and the student model is corrected according to the training proportion. After the data buffer is saturated, stale data will be discarded.
In the embodiment of the application, in order to reduce the influence of Non-IID data on the accuracy of a global model, a model correction method based on incremental learning is provided. Under the Non-IID data scene, the student model has obvious deviation, so that the model precision is reduced, and therefore, each ICV shares part of local data, and the model deviation caused by the Non-IID data is reduced. In addition, old knowledge learned before can be gradually lost in the training process, new and old data of different ICVs are fused by adopting incremental learning, so that the model can continuously learn, and the degree of forgetting the data is reduced.
For the deviation degree, the Non-IID data can cause the student models of different ICVs to generate obvious deviation, and the model precision is affected. The cosine similarity is used to measure the deviation between the models and is calculated as follows:
wherein CS is k+1 Is the kth round global modelAnd v i Local model uploaded at k+1 roundsCorrelation between them. Based on this, the degree of deviation between the local model and the global model can be quantified, and when the ICV student model deviation exceeds a set threshold, the ICV will triggerAnd a data sharing mechanism, otherwise, directly performing global aggregation.
If the data sharing mechanism is triggered, the M ICVs participating in the model aggregation will upload part of the local dataTo the RSU data buffer. To protect user data privacy, the Laplace noise is added to meet the differential privacy requirement before local data is uploaded, and a shared data set D is synthesized in a data buffer area s 。
105. Obtaining shared data issued by a roadside unit, and locally correcting a student model and a teacher model corresponding to the global model; the shared data is determined by the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters.
In an embodiment of the application, the ICV triggering the sharing mechanism will be from the shared data set D s Random extraction of partial data D s ' and ICV raw local data D M And correcting the student model by fusion until the deviation of the student models of all ICVs is lower than a threshold value, and then performing global aggregation. When the RSU data buffer reaches saturation, stale data will be discarded. After the data sharing mechanism is triggered, the newly uploaded data is reserved in the RSU data buffer area. However, as the data in the data buffer increases, early data will be forgotten. In the training of the kth wheel, defining the forgetting degree of the data according to the importance of the model parameters as follows:
wherein L (w) represents the total change rate of the model parameters, H is the current training wheel number, n is the number of the model parameters, and f h Training the proportion of old data for the h round, f H Training the proportion of new data for the current H-th wheel, F h,j Fisher information matrix for the jth model parameter of the h-th training round,representing the H-th trainingJth parameter, w, of the training local model of the kth-1 time h,j The jth parameter representing the global model of the h-th round of training.
Based on the analysis, the shared data proportion of the previous H-1 times in the old data is f 1 ,f 2 ,…,f H-1 Will sum f in the H-th new upload data H Partial co-training, correction v i Is a student model of (a).
Therefore, the present embodiment finds the vehicle data of the corresponding important model parameter and the corresponding proportion as shared data by minimizing the total change rate of the model parameters determined by the new and old proportions of the uploaded partial vehicle data and the change rate of the important model parameter, and at the same time, the shared data can be used to train the learning model, and the shared data is stored in the data buffer of the roadside unit, so that the vehicle data of the data buffer can also be updated.
In the embodiment of the application, a training representative traverses a batch of data to perform a training; one round of training represents traversing all batches of data and performing multiple exercises.
Fig. 3 is a flowchart of an intelligent network-connected vehicle security data collaboration method according to an exemplary embodiment of the present application, as shown in fig. 3, applied to one or more roadside units, where the roadside units are communicable devices in the internet of vehicles and can communicate with intelligent network-connected vehicles in the internet of vehicles, and the method includes:
201. acquiring student models uploaded by a plurality of intelligent network-connected automobiles and determining global aggregation weights based on teacher models;
in the embodiment of the application, the roadside units receive student models and corresponding global aggregation weights from a plurality of intelligent network-connected automobiles, so that partial learning models in the student models can be globally aggregated in a subsequent process.
202. M uploaded student models with the nearest arrival time and corresponding global aggregation weights are selected, and the M student models are aggregated into an overall model according to the global aggregation weights;
in the embodiment of the application, a vehicle selection scheme based on semi-asynchronous federal learning is adopted, and model training time and model uploading time are calculated based on ICV calculation and communication capacity, so that model preparation time is obtained. The order of ICV arrival at RSU is ordered according to the model preparation time of ICV, so that M student models with the earlier arrival time are selected for aggregation.
As shown in fig. 4, the semi-asynchronous federal learning-based vehicle selection process is mainly divided into the following steps:
(1) Model preparation process
In this embodiment, the internet of vehicles system is composed of a plurality of ICVs and RSUs, where N ICVs, v= { V are located in a coverage area of one RSU 1 ,v 2 ,…,v N Each vehicle v i Training a student model v i For convenience of description, the vehicle v will be herein i And model v i Equivalently, v i The model preparation time is defined as the sum of the model training time, the model uploading time and the model aggregation time. The model uploading time and the model training time depend on the computing capacity and communication resources of the ICV, and the model issuing time and the model aggregation time are negligible compared with the model uploading time and the model training time, and based on the model uploading time and the model training time are respectively defined as:
wherein θ is an iteration number factor;the number of iterations required to achieve the desired model accuracy, ε representing the desired accuracy; c (C) i V is a training model i The number of CPU cycles required; d (D) i Is v i Training data sample number; f (f) i V is a training model i Is a CPU frequency of (2); b is the transmission bandwidth allocated to each ICV; p (P) i Is v i Is used for the transmission power of the (a); g i Is v i Channel gain to RSU; n (N) 0 Is the noise power.
Based on the above analysis, v i Model preparation time q of (2) i E Q, including model training time and model upload time, expressed as:
(2) Selectively polymerizing ICV
Let r be i Representing the current turn v i Is prepared according to the ICV model preparation time q= { Q 1 ,q 2 ,…,q M Model arrival time of M ICVs can be predicted to be R= { R 1 ,r 2 ,…,r M }. R is arranged in ascending order to obtain R (M) ＝{r (1) ,r (2) ,…,r (M) }，R (M) Is the model arrival order of the ICV. Let the semi-asynchronous FL aggregation time at the kth training be t k ＝r (M) . For all ICVs, model arrival times are less than or equal to t k V of (2) i Select the k-th aggregate ICV and model arrival time r i Reset to q i The remaining unselected ICV arrival times are set to r i -t k . At the beginning of each round of training, the model arrival times of the ICV are first arranged in ascending order, and the aggregate ICV model is selected according to the method described above until the training number reaches a given iteration number K.
As shown in FIG. 4, there are 5 ICVs in the system, v 1 ，v 2 ，v 3 ，v 4 ，v 5 Its corresponding model preparation time satisfies q 1 ，q 2 ，q 3 ，q 4 ，q 5 . At the beginning of each round of training, the model arrival time is initialized to the model preparation time, i.e., r 1 ＝q 1 ，r 2 ＝q 2 ，r 3 ＝q 3 ，r 4 ＝q 4 ，r 5 ＝q 5 Then for r 1 ，r 2 ，r 3 ，r 4 ，r 5 Arranged in ascending order to obtain R (5) ＝{r (1) ，r (2) ，r (3) ，r (4) ，r (5) }，r (1) <r (2) <r (3) <r (4) <r (5) . Assuming that m=2 at this time, i.e. the model of the 2 uploads with the nearest arrival time is selected, due to r (1) And r (2) Is the smallest two, thus v 1 And v 2 Global aggregation involving training 1 and completion time t of first training 1 ＝r (2) 。
Similarly, at the beginning of the fourth training, v takes part in the training 2 And v 4 Model arrival time reinitialization of r 2 And r 4 While the arrival time of the rest ICV model is r respectively 1 ＝q 1 -t 3 ，r 3 ＝q 3 -t 3 ，r 5 ＝q 5 Because v5 is an inactive ICV, the latest global model will be issued for local model training on the next round of RSUs received, and an inactive ICV needs to upload its model at a given time. Then for r again 1 ，r 2 ，r 3 ，r 4 ，r 5 In ascending order, it is apparent that r 1 <r 5 <r 3 <r 2 <r 4 Thus v 1 And v 5 Global aggregation to participate in the fourth training. According to the method, the arrival model is adopted, the time delay sum of the arrival time of the minimum chemical model in the expected iteration times is calculated, and the optimal number of learning models is calculated, so that the student models with communication energy consumption exceeding the constraint are removed, the convergence time delay of the global model is minimum, and the purpose of low time delay is achieved.
(3) Selecting an optimal number of aggregated ICVs
When the training times reach K, the communication energy consumption of each ICV is calculated, and M values, which are more than the constraint, of the communication energy consumption are removed. And calculating the corresponding total training time delay T of the rest M, and selecting M with the minimum total training time delay T as the optimal aggregate ICV number.
The numerical value determination mode of M comprises the steps of calculating expected iteration times required by model precision according to the iteration times factor; determining the value of M according to the minimum time delay sum of the arrival time of the student model in the expected iteration times, wherein the minimum time delay sum model of the arrival time of the student model in the expected iteration times is expressed as:
wherein T represents the sum of time delays of arrival times of the student model within the expected iteration number; k represents the expected iteration number, θ represents the iteration number factor, ε represents the expected accuracy;representing student model v during the kth iteration i Time of arrival of->Representing student model v during the kth iteration i Is set up for a time period of preparation of (2); />Representing student model v i Model training time,/->Representing student model v i Model upload time of (2); />Representing a maximum model training time; />Representing the maximum model upload time.
According to the method and the device, the optimal learning model number M is calculated through the time delay sum of the minimum chemical raw model within the expected iteration times, so that the student models with communication energy consumption exceeding the constraint are removed, the convergence time delay of the global model is minimum, and the purpose of low time delay is achieved.
203. Issuing global models to a plurality of intelligent network-connected automobiles;
in this embodiment, the aggregated global model is further issued to a plurality of intelligent network-connected vehicles for local training.
204. If the deviation degree of the global model and the locally trained student model exceeds a preset threshold, receiving partial vehicle data uploaded by the intelligent network-connected automobile;
in this embodiment, if the intelligent network-connected vehicle compares the issued global model with the locally trained student model and finds that the deviation degree exceeds the preset threshold, a part of vehicle data is actively uploaded to the roadside unit, so as to prepare for subsequent data sharing and incremental learning.
205. Integrating according to the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters to generate shared data;
in the embodiment of the application, new data is continuously added into the data buffer area during the running process of the ICV, and old data is discarded as the data of the data buffer area is increased to be saturated.
In the embodiment of the application, in order to protect the privacy of user data, laplacian noise is added to meet the differential privacy requirement before local data is uploaded, and a shared data set D is synthesized in a data buffer area s 。
206. And issuing shared data to a plurality of intelligent network-connected automobiles.
In this embodiment, since the shared data set already meets the differential privacy requirement, the problem of privacy disclosure will not be caused when the roadside unit issues the shared data to the intelligent network-connected vehicle.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
It can be understood that the intelligent network steam-connection low-delay data sharing method based on distributed learning can be applied to different execution subjects, and the corresponding technical features can be mutually cited, so that the application is not repeated.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, etc.) to perform the method of secure data collaboration according to the embodiments of the present application.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, etc.
Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application, the scope of which is defined in the appended claims and their equivalents.

Claims (8)

1. The utility model provides a low time delay data sharing method of intelligent network allies oneself with car based on distributed study which characterized in that is applied to one or more intelligent network allies oneself with the car, but intelligent network allies oneself with the car and is the communication equipment in the car networking to can communicate with the roadside unit in the car networking, said method includes:
acquiring a global model issued by a roadside unit; the global model is obtained by polymerizing a student model and global aggregation weights;
collecting vehicle data, and locally training a student model and a teacher model corresponding to the global model;
uploading a locally trained student model and a global aggregation weight determined based on a teacher model to a roadside unit;
if the deviation degree of the global model and the locally trained student model exceeds a preset threshold value, uploading part of vehicle data to a data buffer area of the roadside unit;
obtaining shared data issued by a roadside unit, and locally correcting a student model and a teacher model corresponding to the global model; the shared data is determined by the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters.
2. The intelligent internet-connected vehicle low-latency data sharing method based on distributed learning according to claim 1, wherein the locally training the student model and the teacher model corresponding to the global model includes training with an adaptive learning rate based on vehicle activity, the adaptive learning rate based on vehicle activity is expressed as:
wherein eta i Representing student model v i Is used for the self-adaptive learning rate of the system; η represents the learning rate of the global model; a is that i Representing a locally trained student model v i The vehicle liveness of the intelligent network-connected automobile; gamma is the adjustment coefficient.
3. The intelligent network-connected automobile low-time-delay data sharing method based on distributed learning according to claim 2, wherein the local training student model v is characterized in that i The method for calculating the vehicle activity of the intelligent network-connected automobile comprises the steps of determining the vehicle activity of the intelligent network-connected automobile according to the latest training times of the intelligent network-connected automobile participating in global aggregation; if the latest training times of the intelligent network-connected automobile participating in global aggregation is lower than a preset threshold, forcing the intelligent network-connected automobile to acquire a global model issued by a roadside unit, uploading a locally trained student model to the roadside unit at a specified time, and determining global aggregation weights based on a teacher model; and resetting the vehicle activity of the intelligent network-connected vehicle.
4. The intelligent network-connected vehicle low-latency data sharing method based on distributed learning according to claim 1, wherein the determining of the shared data from the new-old ratio of the uploaded partial vehicle data and the change rate of the important model parameters includes obtaining the corresponding important model parameters and the vehicle data of the corresponding ratio as the shared data by minimizing the total change rate of the model parameters determined from the new-old ratio of the uploaded partial vehicle data and the change rate of the important model parameters, and updating the vehicle data of the data buffer.
5. The intelligent network-connected automobile low-latency data sharing method based on distributed learning according to claim 4, wherein the minimum model of the forgetting degree of the data to be shared is expressed as:
wherein L (w) represents the total change rate of the model parameters, H is the current training wheel number, n is the number of the model parameters, and f h Training the proportion of old data for the h round, f H Training the proportion of new data for the current H-th wheel, F h,j Fisher information matrix for the jth model parameter of the h-th training round,the j-th parameter, w, representing the local model of the kth-1 training in the H-th round of training h,j The jth parameter representing the global model of the h-th round of training.
6. The utility model provides an intelligent network allies oneself with car low time delay data sharing method based on distributed study which characterized in that is applied to in one or more roadside units, but roadside unit is communication equipment in the car networking to but with intelligent network allies oneself with car communication in the car networking, the method includes:
acquiring student models uploaded by a plurality of intelligent network-connected automobiles and determining global aggregation weights based on teacher models;
m uploaded student models with the nearest arrival time and corresponding global aggregation weights are selected, and the M student models are aggregated into an overall model according to the global aggregation weights;
issuing global models to a plurality of intelligent network-connected automobiles;
if the deviation degree of the global model and the locally trained student model exceeds a preset threshold, receiving partial vehicle data uploaded by the intelligent network-connected automobile;
integrating according to the new-old proportion of the uploaded partial vehicle data and the change rate of the important model parameters to generate shared data;
and issuing shared data to a plurality of intelligent network-connected automobiles.
7. The intelligent network-connected automobile low-time-delay data sharing method based on distributed learning as claimed in claim 6, wherein the numerical determination mode of M comprises calculating expected iteration times required by model precision according to iteration times factors; and determining the value of M according to the minimum time delay sum of the arrival time of the student model in the expected iteration times.
8. The intelligent network-connected automobile low-delay data sharing method based on distributed learning according to claim 7, wherein the minimum delay of the arrival time of the student model in the expected iteration number and the model are expressed as:
s.t.
C1:
C2:
C3:
C4:
C5:
C6:
wherein T represents the sum of time delays of arrival times of the student model within the expected iteration number; k represents the expected iteration number, θ represents the iteration number factor, ε represents the expected accuracy;representing student model v during the kth iteration i Time of arrival of->Representing student model v during the kth iteration i Is set up for a time period of preparation of (2); />Representing student model v i Is used for the model training time of the model (a),representing student model v i Model upload time of (2); />Representing a maximum model training time; />Representing the maximum model upload time. />
CN202311088483.0A 2023-08-28 2023-08-28 Intelligent network-connected automobile low-delay data sharing method based on distributed learning Pending CN117135597A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311088483.0A CN117135597A (en) 2023-08-28 2023-08-28 Intelligent network-connected automobile low-delay data sharing method based on distributed learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311088483.0A CN117135597A (en) 2023-08-28 2023-08-28 Intelligent network-connected automobile low-delay data sharing method based on distributed learning

Publications (1)

Publication Number Publication Date
CN117135597A true CN117135597A (en) 2023-11-28

Family

ID=88859437

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311088483.0A Pending CN117135597A (en) 2023-08-28 2023-08-28 Intelligent network-connected automobile low-delay data sharing method based on distributed learning

Country Status (1)

Country Link
CN (1) CN117135597A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220036194A1 (en) * 2021-10-18 2022-02-03 Intel Corporation Deep neural network optimization system for machine learning model scaling
CN114091667A (en) * 2021-11-22 2022-02-25 北京理工大学 Federal mutual learning model training method oriented to non-independent same distribution data
CN114912532A (en) * 2022-05-20 2022-08-16 电子科技大学 Multi-source heterogeneous sensing data fusion method for automatic driving automobile
CN115238906A (en) * 2022-08-01 2022-10-25 重庆邮电大学 Asynchronous federal learning method based on block chain in Internet of vehicles
US20230135179A1 (en) * 2021-10-21 2023-05-04 Meta Platforms, Inc. Systems and Methods for Implementing Smart Assistant Systems

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220036194A1 (en) * 2021-10-18 2022-02-03 Intel Corporation Deep neural network optimization system for machine learning model scaling
US20230135179A1 (en) * 2021-10-21 2023-05-04 Meta Platforms, Inc. Systems and Methods for Implementing Smart Assistant Systems
CN114091667A (en) * 2021-11-22 2022-02-25 北京理工大学 Federal mutual learning model training method oriented to non-independent same distribution data
CN114912532A (en) * 2022-05-20 2022-08-16 电子科技大学 Multi-source heterogeneous sensing data fusion method for automatic driving automobile
CN115238906A (en) * 2022-08-01 2022-10-25 重庆邮电大学 Asynchronous federal learning method based on block chain in Internet of vehicles

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
RAHUL MISHRA: "A Network Resource Aware Federated Learning Approach using Knowledge Distillation", 《IEEE INFOCOM 2021 - IEEE CONFERENCE ON COMPUTER COMMUNICATIONS WORKSHOPS (INFOCOM WKSHPS》, 19 July 2021 (2021-07-19) *
周亚如: "车联网中基于深度分布式学习的场景识别算法研究", 《中国优秀硕士学位论文全文数据库》, 15 February 2023 (2023-02-15) *
李国法;陈耀昱;吕辰;陶达;曹东璞;成波;: "智能汽车决策中的驾驶行为语义解析关键技术", 汽车安全与节能学报, no. 04, 15 December 2019 (2019-12-15) *
黄晓舸: "基于区块链的雾网络中的任务卸载优化方案研究", 《电子与信息学报》, 31 July 2022 (2022-07-31) *

Similar Documents

Publication Publication Date Title
CN114827198B (en) Multi-layer center asynchronous federal learning method applied to Internet of vehicles
CN113435472A (en) Vehicle-mounted computing power network user demand prediction method, system, device and medium
CN105302858B (en) A kind of the cross-node enquiring and optimizing method and system of distributed data base system
CN114553661A (en) Mobile user equipment clustering training method for wireless federal learning
CN113518007B (en) Multi-internet-of-things equipment heterogeneous model efficient mutual learning method based on federal learning
CN112637883A (en) Federal learning method with robustness to wireless environment change in power Internet of things
CN114386570A (en) Heterogeneous federated learning training method based on multi-branch neural network model
CN110110903A (en) A kind of distribution vehicle paths planning method based on neural evolution
Li et al. FEEL: Federated end-to-end learning with non-IID data for vehicular ad hoc networks
CN113887748B (en) Online federal learning task allocation method and device, and federal learning method and system
CN117195019A (en) VANET-oriented lightweight federal learning framework optimization method
CN115966083A (en) Distributed traffic flow prediction method and system based on time-space decoupling
CN115022322A (en) Edge cloud cooperation task unloading method based on crowd evolution in Internet of vehicles
Lv et al. Edge computing task offloading for environmental perception of autonomous vehicles in 6G networks
CN109064750B (en) Urban road network traffic estimation method and system
Chen et al. Traffic signal optimization control method based on adaptive weighted averaged double deep Q network
CN117151208B (en) Asynchronous federal learning parameter updating method based on self-adaptive learning rate, electronic equipment and storage medium
CN117135597A (en) Intelligent network-connected automobile low-delay data sharing method based on distributed learning
CN107465571A (en) Tactical network simulated training background service flow generation method based on statistical property
CN113705902B (en) Traffic hub passenger transport organization evaluation method based on crowd management
CN115118591B (en) Cluster federation learning method based on alliance game
CN113516163B (en) Vehicle classification model compression method, device and storage medium based on network pruning
CN114401192B (en) Multi-SDN controller cooperative training method
CN113313343B (en) Instant vehicle ride matching method based on dynamic time slicing and heat migration
CN114860396B (en) Instant perception even execution task allocation method based on reinforcement learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination