CN115103313A - Intelligent road task cooperative processing method and system based on position prediction - Google Patents
Intelligent road task cooperative processing method and system based on position prediction Download PDFInfo
- Publication number
- CN115103313A CN115103313A CN202210551476.9A CN202210551476A CN115103313A CN 115103313 A CN115103313 A CN 115103313A CN 202210551476 A CN202210551476 A CN 202210551476A CN 115103313 A CN115103313 A CN 115103313A
- Authority
- CN
- China
- Prior art keywords
- task
- vehicle
- tasks
- intelligent road
- time delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000002787 reinforcement Effects 0.000 claims abstract description 28
- 238000009826 distribution Methods 0.000 claims abstract description 9
- 238000004364 calculation method Methods 0.000 claims description 49
- 238000012545 processing Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 17
- 238000003860 storage Methods 0.000 claims description 8
- 238000003062 neural network model Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 abstract description 31
- 230000033001 locomotion Effects 0.000 abstract description 22
- XNWFRZJHXBZDAG-UHFFFAOYSA-N 2-METHOXYETHANOL Chemical compound COCCO XNWFRZJHXBZDAG-UHFFFAOYSA-N 0.000 abstract 1
- 230000009471 action Effects 0.000 description 15
- 238000004422 calculation algorithm Methods 0.000 description 15
- 238000004891 communication Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 12
- 238000005457 optimization Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000007704 transition Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000002829 reductive effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 239000000654 additive Substances 0.000 description 1
- 230000000996 additive effect Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003205 fragrance Substances 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000005303 weighing Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/029—Location-based management or tracking services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5066—Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W24/00—Supervisory, monitoring or testing arrangements
- H04W24/02—Arrangements for optimising operational condition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a position prediction-based intelligent road task cooperative processing method and system, wherein the method comprises the following steps: acquiring state information and task information of a vehicle; predicting a vehicle position based on the state information; cooperatively distributing the task to a plurality of mobile edge computing servers based on a reinforcement learning model with the goal of minimizing the overall time delay of the task, wherein the mobile edge computing servers are adjacent to the predicted vehicle position. By predicting the vehicle track, the vehicle tasks are transversely and cooperatively distributed on the MECS, and the transmission delay is saved by matching with the vehicle motion, so that the distribution of the whole vehicle tasks is optimized.
Description
Technical Field
The invention relates to the technical field of intelligent roads, in particular to a method and a system for intelligent road task cooperative processing based on position prediction.
Background
The intelligent highway is a highway system which can realize cooperative management and control and innovation service by means of a new-generation information communication technology, taking comprehensive accurate perception intelligent decision of a human-vehicle road environment as a core and through human-vehicle road interconnection and cooperation. The intelligent road relies on basic facilities such as a communication system, a monitoring system and the like, and implements automatic safety detection, release of relevant road environment information and real-time automatic operation on vehicles, thereby providing safer, more economic and more comfortable basic services for realizing intelligent road transportation. Intelligent highways enable emerging vehicle services, such as road condition reminders, including reminders for accident-prone sections of bridges, tunnels, sharp curves, and the like; intelligent road monitoring, namely collecting information such as weather, environment, traffic road conditions and the like by using sensing equipment at the road side, and providing latest road information for a driver; the vehicle auxiliary driving comprises basic applications of collision early warning, emergency braking early warning, lane change assistance, vehicle out-of-control early warning and the like. Obviously, the realization of the service is established on the basis of rapid and accurate processing and calculation of mass data, and the intelligent highway information communication system is required to meet the corresponding business requirements of low time delay, high reliability and the like.
Mobile Edge Computing (MEC) is highly compatible with intelligent road scenes due to its Edge processing and mobility-supporting features. With the development of the 5G technology, the MEC has become the first choice of the intelligent highway as a key technology for providing computing power on the edge side, which can fully exert the network advantages of 5G, such as high bandwidth, low time delay, large connection and the like. The MEC technology greatly slows down the self calculation pressure of the vehicle, reduces the service processing time delay and improves the overall efficiency.
However, at present, the coordination of vehicle tasks is based on longitudinal coordination performed by allocating tasks to a roadside unit (RSU), an MEC, and a cloud server, and adverse effects caused by transmission delay due to high-speed movement of a vehicle cannot be considered.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides an intelligent road task cooperative processing method and system based on position prediction.
The invention provides an intelligent road task cooperative processing method based on position prediction, which comprises the following steps:
acquiring state information and task information of a vehicle;
predicting a vehicle position based on the state information;
cooperatively distributing the tasks to a plurality of mobile edge computing servers based on a reinforcement learning model with the aim of minimizing the overall time delay of the tasks, wherein the mobile edge computing servers are close to the predicted vehicle position.
According to the intelligent road task cooperative processing method based on the position prediction, which is provided by the invention, the goal of minimizing the whole time delay of the task based on the reinforcement learning model comprises the following steps:
acquiring a plurality of tasks in a time slot;
and taking the minimization of the average integral time delay of a plurality of tasks in the time slot as the target of the reinforced learning model.
According to the intelligent road task cooperative processing method based on the position prediction, the objective of the reinforcement learning model by minimizing the average overall time delay of a plurality of tasks in the time slot comprises the following steps:
computing network formed by all mobile edge computing servers distributed for a plurality of tasks in the time slot;
and taking the minimization of the average overall time delay of a plurality of tasks in the time slot as the target of the reinforcement learning model, and simultaneously taking the load balance of the computing network as the target.
According to the intelligent road task cooperative processing method based on the position prediction, the task is cooperatively distributed to a plurality of mobile edge computing servers, and the method comprises the following steps:
when the task is cooperatively distributed, the requirement that the time delay of the vehicle running to the current mobile edge calculation server range is greater than or equal to the whole time delay of the task and less than the maximum time delay acceptable by the task is met.
According to the intelligent road task cooperative processing method based on the position prediction provided by the invention, the vehicle position is predicted based on the state information, and the method comprises the following steps:
through the GRU neural network model, a prediction is made of the vehicle location based on the vehicle's historical location, time, and speed.
According to the intelligent road task cooperative processing method based on the position prediction, provided by the invention, the reinforcement learning model is a DDQN model.
The invention also provides an intelligent road task cooperative processing system based on position prediction, which comprises:
the system comprises an acquisition module, a task information acquisition module and a task information acquisition module, wherein the acquisition module acquires state information and task information of a vehicle;
a prediction module that predicts a vehicle location based on the state information;
a coordination module that coordinates the task to a plurality of mobile edge computing servers based on a reinforcement learning model, with the goal of minimizing the overall time delay of the task, wherein the mobile edge computing servers are proximate to the predicted vehicle location.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of the intelligent road task co-processing method based on the position prediction.
The present invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method for cooperative processing of intelligent road task based on location prediction according to any of the above embodiments.
The invention also provides a computer program product, which comprises a computer program, wherein the computer program is used for realizing the steps of the intelligent road task cooperative processing method based on the position prediction when being executed by a processor.
According to the intelligent road task cooperative processing method and system based on the position prediction, the vehicle tasks are transversely and cooperatively distributed on the MECS through predicting the vehicle track, transmission delay is saved by matching with vehicle motion, and distribution of the whole vehicle tasks is optimized.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings required for the embodiments or the prior art descriptions will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a diagram of a GRU structure employed in the present embodiment;
FIG. 2 is a schematic diagram of the trajectory prediction model of the embodiment using a GRU neural network model;
fig. 3 is a schematic diagram comparing the system delay indicators mainly concerned in the present embodiment;
FIG. 4 is a diagram comparing the variation of the calculation delay of different algorithms with the increase of the number of the collaborative MECS;
FIG. 5 is a schematic flow chart of a method for intelligent road task cooperative processing based on location prediction according to the present invention;
FIG. 6 is a schematic diagram of a location prediction-based intelligent road task cooperative processing system according to the present invention;
fig. 7 is a schematic physical structure diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
The intelligent road task cooperative processing method based on location prediction provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings through specific embodiments and application scenarios thereof.
Example 1
The present embodiment aims to research a Location Prediction-based Edge task Collaboration (CETLP) mechanism. Aiming at the requirements of massive tasks and high-speed movement of vehicles on low time delay of task processing in an intelligent highway scene, the embodiment sufficiently combines the characteristics of vehicle motion tracks, and provides a position prediction model based on GRU. The model fully utilizes the resources of the edge server to realize the cooperative calculation of the tasks in the vehicle motion, greatly reduces the return transmission delay of the tasks, provides an edge task cooperative algorithm based on deep reinforcement learning, optimally distributes the tasks according to the load condition and the vehicle motion condition of the edge server, and provides stable and sustainable service for the vehicle.
The embodiment sufficiently combines the characteristic of high-speed movement of vehicles in the same direction in a smart road scene, and provides a cloud edge-end network architecture facing the smart road, which is composed of an edge layer and a terminal layer, wherein the edge layer is composed of a cloud end, an MECS (namely, a plurality of mobile edge computing servers) and an RSU (remote server unit).
1) The cloud end is composed of cloud servers with strong computing power, and meanwhile, the cloud end is used as a center covering the whole network and can acquire the use conditions and task information of various resources in the network, learn and make decisions on the task cooperation strategy, and issue instructions to the edge MECS.
2) The edge layer is composed of an MECS and an RSU, and the MECS and the RSU are in one-to-one correspondence, and the RSU is deployed on the two sides of a road and used as an access device for wireless communication of the vehicle V2I to provide wireless access service for the vehicle. Each RSU covers a range, so each vehicle can only access the RSU device responsible for the range. The MECS is used as a main calculation unit to perform collaborative calculation on the tasks of vehicle unloading, and the calculation results are integrated and then returned to the terminal.
3) The terminal layer consists of high-speed moving vehicles. The vehicle driving direction under the intelligent road scene is mostly straight line driving, a calculation task is generated in the driving process, and task data are transmitted and unloaded to an edge layer in a wireless communication mode for calculation processing.
This embodiment uses M ═ M 1 ,…,m i ,…,m M }、R＝{r 1 ,…,r i ,…,r M Denotes the MECS set and RSU set in the network, respectively. The task generated by the vehicle in a unit time slot is denoted as T k ＝{d k,max ,a k ,s k V, where V represents a set of vehicle unloading task categories, d k,max Represents the minimum requirement of the current task on the time delay, a k Number of CPU cycles required to complete the current task, s k The amount of data offloaded for the current task. The number of different kinds of tasks that a vehicle unloads per unit time is independent of each other and subject to a poisson distribution. The proportion of different types of tasks in all vehicle unloading tasks on the road is lambda k K is equal to V, and
the cooperative computing mode of the edge layer for the vehicle unloading task is discussed in combination with the vehicle motion characteristics, so that the cloud does not serve as a computing server in the embodiment. The RSU is used as an access device for transmitting vehicle tasks and simultaneously takes charge of a small amount of calculation tasks, and the RSU cannot be in direct communication with each other. The MECS, as an important research object of the present embodiment, performs wired communication with each other, and can reduce mutual interference between information and improve transmission efficiency compared with wireless communication. m is a unit of i The task computing is internally responsible for a set U composed of computing units i ＝{u i1 ,…,u ij ,…,u iN And N represents the number of computing units in the server. The computing power of the MECS computing unit, i.e. the CPU frequency, is denoted P M 。
Vehicle { c 1 ,c 2 At m 1 Within the coverage area, the vehicle passes through r 1 Offloading tasks to m 1 The above. Considering a state where the vehicle travels at a high speed on an intelligent road, if the task completion calculation requires a long time, the vehicle may pass through a plurality of MECS when the task calculation is completed. At this time, the task calculation result needs to be transmitted back, and a back transmission path is transmitted from the MECS responsible for calculation to the MECS within the range to which the vehicle currently belongs. This method of returning results does not take into account the vehicle motion characteristics.
In order to reduce the time delay caused by backhaul, the present embodiment provides a prediction mode-based edge layer task cooperation model. Based on accurate prediction of vehicle motion speed, task transmission and calculation time delay, the vehicle can reach a communication interval of the RSU and return a calculation result when task calculation of unloading the vehicle to the front MECS is completed.
The model coordinates partial tasks to m in front of the running of the vehicle in advance 2 And m 3 The above. { c' 1 ,c′ 2 And { c ″ } 1 ,c″ 2 Mean that the vehicle travels to m, respectively 2 And m 3 Status within coverage. When the vehicle reaches the forward MECS (here m 2 And m 3 ) Within the corresponding RSU range, the calculation result can be returned quickly, and the task calculation time and the calculation result return time are greatly reduced. Different types of tasks have different calculation times and different sensitivity degrees to time delay, and how to efficiently cooperate the tasks is the key point of the embodiment in order to better utilize the calculation resources of the edge layer.
The present embodiment employs a Gate Recycling Unit (GRU) to predict the vehicle motion trajectory.
In order to solve the problem of complicated calculation, GRU is proposed to be easier to calculate, fig. 1 is a diagram of the structure of the GRU adopted in this embodiment, and as shown in fig. 1, a gated loop unit uses reset gating r t And updating gating z t To determine the output of gated loop units that can hold useful information in long sequences. Reset gating determines how to put new input information x t And history memory h t-1 In combination, the expression is as follows:
r t ＝σ(W r ·[h t-1 ,x t ]) (1)
update gating defines the amount of information that the previous memory holds until the current time:
z t ＝σ(W z ·[h t-1 ,x t ]) (2)
current data content h' t Input data x including the current time t And past history information stored by using reset gating, wherein the expression is as follows:
update memory phase, GRU using a gate control z t Forget and select memory at the same time, when z t Closer to 1, more current data is represented as remembered; conversely, the more current data is forgotten. Therefore, the current update to the memory means forgetting part of the history information, and keeping part of the current information, and the expression is as follows:
the motion trail data of the vehicle under the intelligent highway scene is a time-sequence data set, and the motion information of the vehicle is represented as mov x ＝(t x ,pos x ,v x ) It is formed by time t x Position pos x And vehicle speed v x And (4) forming. The vehicle motion trajectory can thus be expressed as:
T＝(mov 1 ,...,mov x ,...) (5)
by combining the time-series characteristics of the vehicle trajectory data, fig. 2 is a schematic diagram of the embodiment in which a GRU neural network model is used as a trajectory prediction model, and as shown in fig. 2, the future motion trajectory of the vehicle can be predicted by training data such as the vehicle historical trajectory, time, speed, and the like.
Because the vehicle approaches a plurality of mobile edge computing servers, the model for vehicle trajectory prediction is also trained in the cloud. The vehicle can carry out preprocessing on data before uploading the driving data, the preprocessing comprises deleting abnormal data, removing duplicate data and the like, and the preprocessed data are synchronized to the cloud in real time and used for training a motion trail model. According to data uploaded by the vehicles in real time, in the subsequent task cooperation, the co-directional cooperation of the tasks is predicted according to the positions of the vehicles.
The delay model constructed in this embodiment is as follows:
(1) transmission time delay
The present embodiment contemplates two transmission paths including wireless transmission from the vehicle to the RSU, wired transmission from the RSU to the MECS, and between the MECS. The wireless transmission, considering the communication signal-to-noise ratio, is expressed as:
wherein p is i,j Representing the transmission power between devices, h i,j Representing the antenna gain, σ 2 Representing an additive white gaussian noise power. The wireless bandwidth is denoted B, and according to the fragrance formula, the wireless transmission rate can be expressed as:
the wireless transmission makes an expected bandwidth allocation decision, and the influence of bandwidth jitter on task delay is relieved by allocating enough bandwidth for different services, so that the service stability is improved. The wired transmission rate is denoted by v wired The delay in the calculation is collectively denoted by v. T denotes the size of the task transmitted on the path, dis denotes the physical distance between the two devices, and C denotes the propagation speed. The end-to-end time delay of task data in the network comprises transmission time delay and propagation time delayThe following is disclosed:
then the vehicle unloads task T k The transmission delay for obtaining the calculation result can be expressed as follows:
wherein n is i Indicates the number of MECS passed by the vehicle after the task calculation is finished, n i And 0 represents that the task uploading and the result returning are completed within the range of the same MECS.
Representing the transmission delay of the task path MECS.
It can be seen that the forward transmission of too many tasks will bring about a large transmission delay, and therefore, balancing the task quantity ratio between different MECS is a key issue of this embodiment.
(2) Calculating time delay
The MECS allocates tasks to the idle computing units for computation, and the computation delay of a single computing unit computing a single task is denoted as c k /P M 。
Queuing delay is discussed as part of the computation delay, since the computational resources of the MECS are limited, when all the computing units have tasks to execute, the arrival of a new task creates a queuing situation. Q t Representing the amount of tasks waiting to be calculated in the current queue, MECS, Q t-1 Representing the amount of tasks in the queue of the previous time slot, the queue length is represented as:
Q t ＝max[Q t-1 +∑T new -∑T end ,0] (10)
T new indicating a new set of tasks, T, offloaded to the MECS at the start of the current time slot end And indicating the task set which is executed after the current time slot is finished. Thus, when task T k To MThe queuing time required for ECS is:
wherein S is ij Is allocated to the computing unit u in the MECS ij And at task T k Set of previously computed tasks, T ij Then it means that T is preceded k Is assigned to u ij After the calculation of the calculating unit is finished, the head task of the queue is always preferentially distributed. Thus, the computational latency of the MECS to the task is:
(3) integral time delay
The overall task time delay represents the time from the task unloading of the vehicle to the task processing result acquisition, and comprises task uploading time delay, calculation time delay, transmission time delay and return time delay. Then task T i The overall delay of (a) is as follows:
unloading of vehicles to m per unit time i Satisfies T i ＝∑T k K ∈ V. Part of the tasks are calculated under the current MECS, and the rest of the tasks cooperate with Z-1 MECS in front respectively. Set I i ＝{i 1 ,…,i x ,…,i Z Denotes the occupation ratio of the task data volume of multiple MECS cooperative computing in the task set, wherein
Whereby the vehicle is unloaded to m per unit time i Overall delay of task setCan be expressed as:
wherein,
representing the latency of the part of the task coordinated to the x-th MECS ahead.
From the vehicle trajectory prediction model, the time for the vehicle to reach the x-th MECS in front can be obtained, and is expressed as
At the edge layer to task T k And at the same time of coordination, the requirement that the time delay of the vehicle running to the current MECS range is more than or equal to the task calculation time delay and is less than the maximum time delay acceptable by the task coordinated to the current MECS is met:
the present embodiment builds a load model for the edge layer.
Under the condition of meeting the optimal time delay, the single-point load is likely to be larger, so that the load balance degree of the MECS is added as one of the task cooperative consideration conditions, and the sustainable processing of the network to the task is ensured. By counting the number N of active computing units in the MECS working And total number of calculation units N total To describe the server load state:
with L mean Representing the average state of the load of the edge layer network, the load balancing state L of the edge layer network balance The following were used:
according to the vehicle movement characteristics, the edge task collaborative model based on prediction provided by the embodiment considers factors such as task calculation delay and edge layer load balance, and provides continuous and efficient service for users. The latency considers the average latency of all tasks offloaded to the edge layer at the same time slot to be expressed as follows:
ω 1 and ω 2 The method is characterized in that the method comprises the following steps of respectively weighing and normalizing coefficients of time delay and network load balance, and the collaborative optimization goal is as follows:
min Y＝ω 1 D mean +ω 2 L balance (20)
on the basis of the model, the embodiment provides an edge task cooperation algorithm based on position prediction, and when a vehicle unloads a new task into a network, the network state is learned and optimal distribution is performed, so that the overall low time delay of the task and the load balance of the network are ensured. The network parameters are as follows:
(1) system state
At the beginning of each timeslot, the vehicle will offload new tasks to the edge layer. State S in the network at this time t The vehicle-mounted intelligent management system comprises task information uploaded by a vehicle, task information in MECS calculation and queue information.
(2) Task collaboration
The MECS transmits the task header data unloaded from the vehicle to the edge layer to the cloud, and the cloud determines the forward cooperation proportion of the task according to the edge device information, namely the action A executed in the current state t . When the number of coordinated MECS is Z and the number of current offload tasks is n, action A t Of spatial order n Z-1 From this, forward synergy is seenA larger number of MECS results in a larger motion space. The task cooperation vector is I i ＝{i 1 ,…,i x ,…,i Z Represents the proportion of tasks coordinated to the forward MECS. The final choice of action will be determined by the size of the reward and the selection process will be described in the algorithm flow.
(3) Probability of state transition
In the markov decision process, the state transition probability represents the probability that the current network is transferred from the current state to another state, and the next state is only related to the previous state, and the state transition probability of this embodiment is represented as P (S) t+1 |S t ,A t ) In which S is t+1 Indicating the next state of the network.
(4) Reward function
In taking action A t The system will then receive the reward R t For a random process, the instant prize does not represent the maximum expected long-term cumulative prize. In the DRL (Deep Reinforcement Learning) method, these actions affect not only the instant reward, but also the next case and hence all subsequent rewards. In order to obtain many rewards, reinforcement learning must be inclined to take actions that have been tried in the past that are considered to be effective in generating the reward. The change in the optimization model value is used to represent the reward, namely:
R t ＝Y t-1 -Y t (21)
wherein, Y t-1 、Y t Y in (1) is a co-optimization target in the formula (21), Y t-1 I.e. the co-optimization target value of the last state, Y t I.e. the co-optimization target value of the current state.
The vehicle unloads the task set to the MECS in the same time slot, and at the moment, whether part of tasks are coordinated to the front MECS is judged through an algorithm, and the tasks in the task set are independent and not separable, so that the separating action can be regarded as limited selection. For the finite motion space, the present embodiment solves the problem using the DDQN algorithm. Compared with the DQN algorithm, the two steps of action selection and target value calculation are split by using two Q networks, the problem of over-estimation is eliminated to a certain extent, and the algorithm can converge to a better result.
The current edge side cooperative network state is S t Including vehicle location information, task calculation queuing conditions, and server load conditions. Outputting the action value function Q (S) in the current state through the Q network t ,A t ). The action A with the maximum value is selected from the output Q values of the epsilon-greedy network, namely the MECS coordinates the task at the moment t to other computing equipment in the network, so that the reward R corresponding to the current state can be obtained t And judges whether the current state is the termination state is _ end. { S t ,A t ,R t ,S t+1 The is _ end set is stored as experience into an experience playback set D, and the user updates Q network parameters. In order to prevent excessive dependence between the calculation of the target Q value and the action selection, the Q value is calculated by constructing a set of target networks of the same structure. The iterative values of the network parameters are only specific to the current Q network, and the target network only needs to periodically copy the network parameters, so that the model is prevented from being overestimated.
The algorithm finds a task cooperation mode which can minimize time delay and has the best load balancing degree in the current Q network as a selected action, namely:
wherein S is t+1 Is the new state of the network, omega t Is all parameters of the network at the current time Q, and a is the execution action that the network can choose. By action selected in the current Q network
Utilizing target network parameters in a target network
Calculating a target Q value Y t Namely:
from the above equation and the network termination state, the target Q value is expressed as follows:
updating the neural network parameters in an experience playback mode, randomly selecting m pieces of historical experience data from an experience pool, and comparing the difference with the current Q value in a mean square error mode to ensure that an algorithm model is continuously converged, wherein a loss function is as follows:
fig. 3 is a schematic diagram comparing the system delay indexes mainly concerned in the embodiment, and as shown in fig. 3, the CETLP of the embodiment is compared with A3C and DQN, and as time goes by, the reinforcement learning algorithm model gradually converges, and the calculation delay of the task of offloading the vehicle to the edge side network gradually decreases. Compared with a comparative algorithm, the algorithm of the embodiment can realize lower system time delay, because the algorithm of the embodiment fully considers the moving characteristics of the vehicle while performing coordination on the tasks, and reduces the time delay caused by unnecessary data transmission.
Fig. 4 is a schematic diagram comparing changes of different algorithm calculation delays with the increase of the number of the cooperative MECS, and as shown in fig. 4, it is found that the system calculation delay is significantly reduced by the appropriate number of the MECS, and gradually becomes gentle when the number is 3. The larger number of synergies means longer cooperative distance and transmission delay, and blind increase of the number of synergies does not necessarily bring higher benefit, but increases the computational complexity of the system.
In addition, in the implementation, the cooperative proportion of the tasks with different sizes is gradually increased along with the increase of the number of the tasks, but the cooperative proportion of the tasks with larger calculation amount and data amount is higher overall. This is because the calculation and transmission time delay of the task with large calculation amount and data amount is relatively longer, and the number of RSUs that the vehicle passes through is increased, so the cooperative distribution system applied in the technical solution of the embodiment tends to cooperate such task.
Example 2
Fig. 5 is a schematic flow chart of a method for cooperatively processing an intelligent road task based on location prediction according to the present invention, and as shown in fig. 5, the method for cooperatively processing an intelligent road task based on location prediction according to the present invention includes:
s100, acquiring state information and task information of a vehicle;
s200, predicting the position of the vehicle based on the state information;
and S300, based on the reinforcement learning model, cooperatively distributing the tasks to a plurality of mobile edge computing servers by taking the overall time delay of the tasks as a target, wherein the mobile edge computing servers are close to the predicted vehicle positions.
Optionally, the implementation subject of the method is a cloud server, and the reinforcement learning model is erected in the cloud server to perform cooperative distribution on the whole task.
Optionally, status information and task information of the vehicle; are transmitted to the MECS through the RSU, and the calculation result of the MECS is transmitted to the vehicle by the corresponding RSU.
Optionally, the mobile edge computing server located in front of the vehicle, after completing the assignment task, passes the task result to the vehicle when the vehicle comes within its corresponding RSU range. Thus, a large amount of transmission delay can be saved.
According to the method, the vehicle tasks are transversely and cooperatively distributed on the MECS by predicting the vehicle track, and the transmission delay is saved by matching with the vehicle motion, so that the distribution of the whole vehicle tasks is optimized.
Further, on the basis of the foregoing embodiment, in another embodiment, the present embodiment provides a method for intelligent road task co-processing based on location prediction, which is based on a reinforcement learning model and aims to minimize the overall time delay of a task, and includes:
acquiring a plurality of tasks in a time slot;
the minimization of the average overall time delay of a plurality of tasks in a time slot is taken as the target of the reinforced learning model.
Optionally, the overall delay includes a calculation delay and a transmission delay, where the transmission delay includes all non-task calculation-induced delays such as vehicle task uploading, task result returning to the vehicle, task data transmission during cooperative allocation, and the like.
The embodiment discloses that the overall delay of the tasks is minimized, the tasks are not limited to a single task, but a plurality of tasks are considered comprehensively as a whole, so that the average overall delay of the whole is minimized, and correspondingly, in a reinforcement learning model, not only the instant direct reward but also the subsequent attenuated reward are considered.
Further, based on the foregoing embodiment, in another embodiment, the present embodiment provides an intelligent road task cooperative processing method based on location prediction, with regard to minimization of an average overall time delay of a plurality of tasks within a time slot, as a target of a reinforcement learning model, including:
a computing network formed by all mobile edge computing servers distributed to a plurality of tasks in the time slot;
the minimization of the average overall time delay of a plurality of tasks in a time slot is taken as the target of a reinforced learning model, and meanwhile, the load balance of a computing network is taken as the target.
Optionally, a certain number of mobile edge computing servers on the highway side are divided into one computing network according to the geographical location.
The embodiment discloses that the load balance of the computing network is considered while the vehicle task is optimized, the equipment problem caused by load concentration is prevented, and the stability of the whole computing network is improved.
Further, on the basis of the foregoing embodiment, in another embodiment, the present embodiment provides a method for intelligent road task cooperative processing based on location prediction, cooperatively distributing tasks to a plurality of mobile edge computing servers, including:
when the task is cooperatively distributed, the requirement that the time delay of the vehicle running to the current mobile edge calculation server range is greater than or equal to the whole time delay of the task and less than the maximum time delay acceptable by the task is met.
With the additional constraint, when the vehicle reaches the front MECS, the task result can be immediately obtained.
The time delay limiting condition of the embodiment reduces the time complexity and the calculated amount of the reinforcement learning model on one hand, and ensures the timeliness of the completion of the vehicle task on the other hand.
Further, on the basis of the foregoing embodiment, in another embodiment, the present embodiment provides a method for intelligent road task cooperative processing based on location prediction, which predicts a vehicle location based on status information, and includes:
and predicting the position of the vehicle based on the state information through a GRU neural network model.
Alternatively, the status information is historical position, time, and speed of the vehicle.
The embodiment realizes accurate prediction of the vehicle position through the GRU neural network model.
Further, on the basis of the foregoing embodiment, in another embodiment, the present embodiment provides a method for intelligent road task cooperative processing based on location prediction, and the reinforcement learning model is a DDQN model.
Compared with the DQN model, the DDQN model divides two steps of action selection and target value calculation, and eliminates the problem of over-estimation to a certain extent.
In the embodiment, the optimized calculation of the task cooperation is realized through the DDQN model, and a more excellent task cooperation effect is obtained.
The intelligent road task cooperative processing system based on the position prediction provided by the invention is described below, and the intelligent road task cooperative processing system based on the position prediction described below and the intelligent road task cooperative processing method based on the position prediction described above can be referred to correspondingly.
Fig. 6 is a schematic structural diagram of an intelligent road task cooperative processing system based on location prediction according to the present invention, as shown in fig. 6, the present invention further provides an intelligent road task cooperative processing system based on location prediction, the system includes:
the acquisition module acquires the state information and the task information of the vehicle;
a prediction module that predicts a vehicle position based on the state information;
and the cooperation module is used for cooperatively distributing the tasks to a plurality of mobile edge computing servers based on the reinforcement learning model and aiming at minimizing the overall time delay of the tasks, wherein the mobile edge computing servers are close to the predicted vehicle positions.
According to the method, the vehicle tasks are transversely and cooperatively distributed on the MECS by predicting the vehicle track, and the transmission delay is saved by matching with the vehicle motion, so that the distribution of the whole vehicle tasks is optimized.
Fig. 7 is a schematic physical structure diagram of an electronic device provided in the present invention, and as shown in fig. 7, the electronic device may include: a processor (processor)810, a communication Interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication Interface 820 and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to perform a method for intelligent road task co-processing based on location prediction, the method comprising:
acquiring state information and task information of a vehicle;
predicting a vehicle position based on the state information;
cooperatively distributing the task to a plurality of mobile edge computing servers based on a reinforcement learning model with the goal of minimizing the overall time delay of the task, wherein the mobile edge computing servers are adjacent to the predicted vehicle position.
In addition, the logic instructions in the memory 830 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the method for cooperative processing of intelligent road tasks based on location prediction provided by the above methods, the method including:
acquiring state information and task information of a vehicle;
predicting a vehicle position based on the state information;
cooperatively distributing the tasks to a plurality of mobile edge computing servers based on a reinforcement learning model with the aim of minimizing the overall time delay of the tasks, wherein the mobile edge computing servers are close to the predicted vehicle position.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the location prediction-based intelligent road task cooperative processing method provided above, the method comprising:
acquiring state information and task information of a vehicle;
predicting a vehicle position based on the state information;
cooperatively distributing the tasks to a plurality of mobile edge computing servers based on a reinforcement learning model with the aim of minimizing the overall time delay of the tasks, wherein the mobile edge computing servers are close to the predicted vehicle position.
The above-described embodiments of the apparatus are merely illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, i.e. may be located in one place, or may also be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A position prediction-based intelligent road task cooperative processing method is characterized by comprising the following steps:
acquiring state information and task information of a vehicle;
predicting a vehicle position based on the state information;
cooperatively distributing the tasks to a plurality of mobile edge computing servers based on a reinforcement learning model with the aim of minimizing the overall time delay of the tasks, wherein the mobile edge computing servers are close to the predicted vehicle position.
2. The intelligent road task cooperative processing method based on location prediction as claimed in claim 1, wherein the objective of minimizing the overall time delay of the task based on the reinforcement learning model comprises:
acquiring a plurality of tasks in a time slot;
and taking the minimization of the average integral time delay of a plurality of tasks in the time slot as the target of the reinforced learning model.
3. The intelligent road task cooperative processing method based on location prediction according to claim 2, wherein the minimizing of the average overall time delay of the tasks in the time slot as the target of the reinforcement learning model comprises:
a computing network formed by all mobile edge computing servers distributed for a plurality of tasks in the time slot;
and taking the minimization of the average overall time delay of a plurality of tasks in the time slot as the target of the reinforcement learning model, and simultaneously taking the load balance of the computing network as the target.
4. The intelligent road task cooperative processing method based on location prediction as claimed in claim 1, wherein the cooperative distribution of the tasks to a plurality of mobile edge computing servers comprises:
when the task is cooperatively distributed, the requirement that the time delay of the vehicle running to the current mobile edge calculation server range is greater than or equal to the whole time delay of the task and less than the maximum time delay acceptable by the task is met.
5. The intelligent road task cooperative processing method based on position prediction as claimed in claim 1, wherein predicting the vehicle position based on the state information comprises:
through the GRU neural network model, a prediction is made of the vehicle location based on the vehicle's historical location, time, and speed.
6. The intelligent road task cooperative processing method based on location prediction as claimed in any one of claims 1 to 5, wherein the reinforcement learning model is a DDQN model.
7. An intelligent road task co-processing system based on position prediction, the system comprising:
the acquisition module acquires state information and task information of a vehicle;
a prediction module that predicts a vehicle location based on the state information;
a coordination module that, based on a reinforcement learning model, cooperatively allocates the task to a plurality of mobile edge computing servers with a goal of minimizing an overall time delay of the task, wherein the mobile edge computing servers are close to the predicted vehicle position.
8. An electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the intelligent road task cooperative processing method based on location prediction according to any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium, having stored thereon a computer program, wherein the computer program, when being executed by a processor, implements the steps of the method for intelligent road task cooperative processing based on location prediction according to any one of claims 1 to 6.
10. A computer program product comprising a computer program, wherein the computer program when executed by a processor implements the steps of the location prediction based intelligent road task co-processing method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210551476.9A CN115103313A (en) | 2022-05-18 | 2022-05-18 | Intelligent road task cooperative processing method and system based on position prediction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210551476.9A CN115103313A (en) | 2022-05-18 | 2022-05-18 | Intelligent road task cooperative processing method and system based on position prediction |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115103313A true CN115103313A (en) | 2022-09-23 |
Family
ID=83289414
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210551476.9A Pending CN115103313A (en) | 2022-05-18 | 2022-05-18 | Intelligent road task cooperative processing method and system based on position prediction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115103313A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116208669A (en) * | 2023-04-28 | 2023-06-02 | 湖南大学 | Intelligent lamp pole-based vehicle-mounted heterogeneous network collaborative task unloading method and system |
| CN117575113A (en) * | 2024-01-17 | 2024-02-20 | 南方电网数字电网研究院股份有限公司 | Edge collaborative task processing method, device and equipment based on Markov chain |
-
2022
- 2022-05-18 CN CN202210551476.9A patent/CN115103313A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116208669A (en) * | 2023-04-28 | 2023-06-02 | 湖南大学 | Intelligent lamp pole-based vehicle-mounted heterogeneous network collaborative task unloading method and system |
| CN116208669B (en) * | 2023-04-28 | 2023-06-30 | 湖南大学 | Intelligent lamp pole-based vehicle-mounted heterogeneous network collaborative task unloading method and system |
| CN117575113A (en) * | 2024-01-17 | 2024-02-20 | 南方电网数字电网研究院股份有限公司 | Edge collaborative task processing method, device and equipment based on Markov chain |
| CN117575113B (en) * | 2024-01-17 | 2024-05-03 | 南方电网数字电网研究院股份有限公司 | Edge collaborative task processing method, device and equipment based on Markov chain |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xu et al. | Service offloading with deep Q-network for digital twinning-empowered internet of vehicles in edge computing | |
| CN110213796B (en) | Intelligent resource allocation method in Internet of vehicles | |
| CN115103313A (en) | Intelligent road task cooperative processing method and system based on position prediction | |
| CN111311959B (en) | Multi-interface cooperative control method and device, electronic equipment and storage medium | |
| CN111835827A (en) | Internet of things edge computing task unloading method and system | |
| CN111831427B (en) | Distributed inter-vehicle task unloading method based on mobile edge calculation | |
| CN112700642B (en) | Method for improving traffic passing efficiency by using intelligent internet vehicle | |
| CN113709249B (en) | Safe balanced unloading method and system for driving assisting service | |
| CN113316116B (en) | Vehicle calculation task unloading method | |
| CN112929849B (en) | Reliable vehicle-mounted edge calculation unloading method based on reinforcement learning | |
| CN117062025B (en) | Energy-saving combined computing unloading and resource allocation method for Internet of vehicles | |
| CN114189869A (en) | Unmanned vehicle collaborative path planning and resource allocation method based on edge calculation | |
| CN114074680B (en) | Vehicle channel change behavior decision method and system based on deep reinforcement learning | |
| CN115052262A (en) | Potential game-based vehicle networking computing unloading and power optimization method | |
| CN115134242A (en) | Vehicle-mounted computing task unloading method based on deep reinforcement learning strategy | |
| CN116204315A (en) | Track-based dynamic task unloading method for vehicle resource pool in Internet of vehicles | |
| CN112750298B (en) | Truck formation dynamic resource allocation method based on SMDP and DRL | |
| CN115022322A (en) | Edge cloud cooperation task unloading method based on crowd evolution in Internet of vehicles | |
| CN114168328A (en) | Mobile edge node calculation task scheduling method and system based on federal learning | |
| CN113724507A (en) | Traffic control and vehicle induction cooperation method and system based on deep reinforcement learning | |
| CN115208892B (en) | Vehicle-road collaborative online task scheduling method and system based on dynamic resource demand | |
| CN116896561A (en) | Calculation unloading and resource allocation method for vehicle-road cooperative system | |
| CN114138466A (en) | Task cooperative processing method and device for intelligent highway and storage medium | |
| Wu et al. | Density-aware federated imitation learning for connected and automated vehicles with unsignalized intersection | |
| Shi et al. | A computation offloading method with distributed double deep Q‐network for connected vehicle platooning with vehicle‐to‐infrastructure communications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |