CN113114721B - Software defined Internet of vehicles service migration method based on MEC - Google Patents

Software defined Internet of vehicles service migration method based on MEC Download PDF

Info

Publication number
CN113114721B
CN113114721B CN202110274917.0A CN202110274917A CN113114721B CN 113114721 B CN113114721 B CN 113114721B CN 202110274917 A CN202110274917 A CN 202110274917A CN 113114721 B CN113114721 B CN 113114721B
Authority
CN
China
Prior art keywords
migration
task
vehicle
path
vehicles
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110274917.0A
Other languages
Chinese (zh)
Other versions
CN113114721A (en
Inventor
刘开健
蔡磊
张海波
张益峰
罗伊娜
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202110274917.0A priority Critical patent/CN113114721B/en
Publication of CN113114721A publication Critical patent/CN113114721A/en
Application granted granted Critical
Publication of CN113114721B publication Critical patent/CN113114721B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L45/00—Routing or path finding of packets in data switching networks
    • H04L45/12—Shortest path evaluation
    • H04L45/122—Shortest path evaluation by minimising distances, e.g. by selecting a route with minimum of number of hops
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Mobile Radio Communication Systems (AREA)
  • Navigation (AREA)

Abstract

The invention belongs to the technical field of wireless short-distance communication of Internet of vehicles, and particularly relates to a software defined Internet of vehicles service migration method based on MEC, which comprises the following steps: determining a time delay function of a vehicle user according to an optimization target of the service migration of the Internet of vehicles; determining the optimal path of the vehicle task by adopting a Dijkstra routing algorithm, optimizing a vehicle task migration delay function by adopting a Q-learning algorithm to obtain the optimal value of the vehicle user delay function, and migrating the task by adopting a server with the optimal value of the delay function; the method for determining the service migration of the Internet of vehicles by combining the SDN, the virtual machine and the Internet of vehicles realizes the optimal migration mode of the service migration of the Internet of vehicles.

Description

Software defined Internet of vehicles service migration method based on MEC
Technical Field
The invention belongs to the technical field of wireless short-distance communication of Internet of vehicles, and particularly relates to a software defined Internet of vehicles service migration method based on MEC.
Background
The internet of vehicles has gained wide attention and research at home and abroad as the most potential development and application of the internet of things theory in the intelligent traffic system. Over the past decade, the automotive industry has developed rapid in hardware and software technology, a large number of services and applications, and various advanced communication technologies, to make driving safer and more comfortable. The automotive industry has invested a large amount of resources to achieve vehicle automation, and automotive autopilot technology has been rapidly developed. These autonomous smart cars are running a large number of compute-intensive applications that require cloud assistance for data processing and storage. However, the computing power of the car terminal is limited. Mobile Edge Computing (MEC) is considered a promising solution to push cloud services towards the edge of the wireless access network and to provide cloud-based computing offloading in close proximity to the mobile vehicle terminals.
The concept of Software Defined Networking (SDN) shows great potential in facilitating data scheduling, improving resource utilization of vehicle networks, and enhancing service management. The core idea of SDN is the separation of the control plane and the data plane. An SDN controller with global network knowledge formulates data broadcasting and forwarding rules for network nodes such as Road Side Units (RSUs), Base Stations (BSs) and vehicles through a control plane. As a result, the SDN controller may define the behavior of individual vehicles, RSUs, and BSs by making scheduling decisions based on the global view.
Resource virtualization is one of the key support technologies for cloud resource allocation and management. It allows tasks of various tenants to be performed simultaneously on a shared hardware platform in the form of Virtual Machines (VMs). To facilitate load balancing, fault management, server maintenance, etc., traditional cloud data centers advocate live VM migration. However, Vehicle Cloud Computing (VCC) based on roadside cloud (RSC) is largely different from conventional cloud computing platforms in that cloud resources are highly dispersed and users are highly dynamic. This makes VM migration in RSC-based VCC more challenging and user service experience poor.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a software defined internet of vehicles service migration method based on MEC, which comprises the following steps: the vehicle user inputs the generated calculation task to the Internet of vehicles in the driving process; the Internet of vehicles performs task migration according to the calculation task; selecting a task migration path by adopting a Dijkstra routing algorithm to obtain an optimal path; determining an optimization target of the vehicle user according to the optimal path, and calculating a time delay function of the vehicle user according to the optimization target; optimizing the time delay function by adopting a Q-learning algorithm to obtain an optimal time delay function; calculating a transfer reward function according to the optimal time delay function, and when the reward function is maximum, taking the task transfer path as an optimal transfer scheme; wherein MEC represents the calculation of moving edges, Dijkstra routing algorithm represents Dijkstra routing algorithm, and Q-learning represents the reinforcement learning algorithm.
Preferably, the calculation task generated by the vehicle user comprises three parts of local execution, original roadside unit execution and target roadside unit execution; when a vehicle user moves, tasks are transferred from an original roadside unit to a target roadside unit, so that a virtual machine VM is transferred; and selecting an optimal path from a plurality of migration paths from the original roadside unit to the target roadside unit for migration during the migration of the virtual machine, and optimizing the migration time of the virtual machine under the optimal path.
Preferably, the process of selecting the task migration path by using the Dijkstra routing algorithm comprises the following steps:
step 1: acquiring a city distribution model graph G (S, E, W), and determining the positions of a starting roadside unit A and a target roadside unit B in the graph;
step 2: calculating a weight matrix D corresponding to different nodes in the graph according to the urban distribution model graph, and generating an initial node V ═ ones (1, n);
and step 3: calculating the distances from the initial node to all paths in the next node according to the weight matrix D, and selecting the path with the shortest distance as the optimal sub-path between the two nodes;
and 4, step 4: taking the end point of the optimal sub-path as the starting point of a new path, calculating the distances of all paths in the next node, and selecting the path with the shortest distance as the optimal sub-path of the two nodes;
and 5: repeating the step 4 until the task is migrated to the target roadside unit;
step 6: and according to all the optimal sub-paths, reverse searching paths from the target roadside unit to the starting roadside unit by adopting a backtracking method to obtain a communication link I, the number of servers of the paths and the shortest migration path.
Preferably, the optimization objective of the car networking service migration comprises: time delay function t for vehicle to transmit partial data to roadside unitTLocal calculation of the time function t for the vehiclelThe original MEC server edge calculates the delay function tioEdge computation of the delay function t by the destination MEC ServeridMigration time T of virtual machinemigAnd virtual machine downtime Tdown。
Preferably, the time delay function of the vehicle user determined according to the optimization objective of the internet of vehicles service migration is as follows: the optimization target of the service migration of the Internet of vehicles is weighted and summed to obtain a time delay function of a vehicle user, and the expression is as follows:
Figure BDA0002976224880000031
preferably, the specific steps of optimizing the delay function by using the Q-learning algorithm are as follows:
step 1: abstracting an objective function according to the task migration state and the optimization target of the vehicle user, and acquiring a state set S (R, L) and a behavior set A (lambda, N) of the vehicle user through the objective function;
and 2, step: constructing a Q matrix according to the state set and the behavior set of the vehicle user, and initializing the Q matrix; the Q matrix of the Q-learning algorithm is a matrix taking a state as a row and an action as a column, and is initialized at first, and all values are set to be 0;
and step 3: the method comprises the following steps that a vehicle user randomly selects an initial state S in a state set, and a greedy strategy is adopted according to the initial state to obtain corresponding behaviors;
and 4, step 4: updating the state of the selected behavior by adopting a Q value formula, namely updating rewards obtained by different behaviors in a Q matrix, wherein the corresponding values are the rewards obtained by adopting different behaviors in different states;
and 5: taking the updated state as the state of the current task to update the state of the next task; continuously repeating the steps 3, 4 and 5 until the Q matrix is converged;
step 6: when the Q matrix is converged, obtaining the optimal reward r of each state through the Q matrix;
and 7: adding all the optimal rewards to obtain the final reward TM(ii) a When T isMAnd when the task is maximum, the migration path is the optimal migration scheme.
Further, the Q value calculation formula is as follows:
Q(s,a)←Q(s,a)+α[r(s,a)+γmaxQ(s′,a′)-Q(s,a)]
further, the final prize TMThe expression of (a) is:
Figure BDA0002976224880000041
the method for determining the service migration of the Internet of vehicles by combining the SDN, the virtual machine and the Internet of vehicles realizes the optimal migration mode of the service migration of the Internet of vehicles; according to the invention, the optimal migration path is found in the urban base station distribution through the Dijkstra algorithm to reduce resource waste and time consumption, and then the migration is optimized through the Q learning algorithm, so that the user experience is improved, the time complexity is reduced, the total time delay of the user is reduced, and better experience is provided for the user.
Drawings
FIG. 1 is a diagram of an architecture model of the software defined vehicle networking based on MEC in the vehicle networking of the present invention;
FIG. 2 is a flowchart of an embodiment of resource allocation based on MEC cache service in the Internet of vehicles according to the present invention;
FIG. 3 is a flowchart illustrating an implementation of the service migration of the virtual machine based on the MEC software definition in the Internet of vehicles according to the present invention;
FIG. 4 is a comparison graph of total time delay calculated for vehicle user tasks before and after migration in accordance with the present invention;
FIG. 5 is a comparison of average delay for different data amounts according to the present invention;
FIG. 6 is a graph comparing migration times of the virtual machine migration algorithm based on path selection and Q learning and the methods such as MDP, AMEChe and OHBAC under different migration data;
FIG. 7 is a comparison of the migration failure rates of the migration algorithm of the present invention with AMEChe, MDP, and OHCAC strategies;
FIG. 8 is a graph comparing the percentage of VM migration performed by the migration algorithm based on path finding and Q learning and the strategies AEEChe, MDP, OHBAC, etc. in accordance with the present invention;
FIG. 9 is a diagram illustrating Q-learning state process transition according to the present invention.
Detailed Description
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive effort based on the embodiments of the present invention, are within the scope of the present invention.
A software defined Internet of vehicles service migration method based on MEC comprises the following steps: determining a time delay function of a vehicle user according to an optimization target of the service migration of the Internet of vehicles; the SDN, the virtual machines and the Internet of vehicles are combined, firstly, the SDN can solve the problem of network redirection of virtual machine migration, the Internet of vehicles inevitably faces the problem of virtual machine migration, the SDN improves the expandability of the Internet of vehicles, and the virtual machine migration routing can be better managed. Using SDN technology, the network control unit is separated from the network forwarding unit by a standardized protocol, so all network functions are integrated into one unified network control plane, i.e. SDN controller. By means of a centralized SDN controller, network states may be monitored and collected in real time, thereby making SDN enabled network devices transparent and controllable. The combination of the SDN, the virtual machine and the Internet of vehicles can provide a complete structural framework, the resource allocation and virtual machine migration problems of the Internet of vehicles are better managed, an optimal migration path is found in urban base station distribution through a Dijkstra algorithm so as to reduce resource waste and time consumption, and then the migration is optimized through a Q learning algorithm, so that the user experience is improved.
As shown in FIG. 1, during the driving process of the vehicle, a user can generate a calculation task; due to the limited computing capacity of the vehicle, tasks are sent to a nearby base station, an MEC server equipped with the base station assists in the computing tasks, and during the period that the MEC server serves the vehicle, the vehicle is switched among different roadside units along with the driving of the vehicle, in this case, customized VMs can be transmitted across the roadside units for the continuity of the service. This process is referred to as VM migration. As shown in FIG. 1, when a vehicle V requests service in the communication range of the RSU A, the RSU A can be customized for the vehicle V, VM-A is generated, a task possibly leaves the service range of the RSU A without being completed due to the mobility of the vehicle, and when the vehicle reaches the RSU B, two schemes are provided, wherein one scheme is that the VM-A is moved to the RSU B, and the other scheme is that the VM-A is still left in the RSU A, and the vehicle and the RSU A communicate to obtain service. Obviously, the communication distance of the VM migration path is long, so that finding a proper VM migration strategy is particularly important, and model building is performed for finding an optimal service migration path.
An MEC-based software defined internet of vehicles service migration method, as shown in fig. 2, includes: the vehicle user inputs the generated calculation task into the Internet of vehicles in the driving process; the Internet of vehicles performs task migration according to the calculation task; selecting a task migration path by adopting a Dijkstra routing algorithm to obtain an optimal path; determining an optimization target of the vehicle user according to the optimal path, and calculating a time delay function of the vehicle user according to the optimization target; optimizing the time delay function by adopting a Q-learning algorithm to obtain an optimal time delay function; calculating a transfer reward function according to the optimal time delay function, and when the reward function is maximum, taking the task transfer path as an optimal transfer scheme; wherein MEC represents the calculation of moving edges, Dijkstra routing algorithm represents Dijkstra routing algorithm, and Q-learning represents the reinforcement learning algorithm.
Optimization goals for internet of vehicles service migration include: time delay function t for vehicle to transmit partial data to roadside unitTVehicle local calculation of the time function tlThe original MEC server edge calculates the delay function tioEdge computation of the delay function t by the destination MEC ServeridMigration time T of virtual machinemigTime of virtual machine down Tdown。
The processing procedure of the car networking system comprises the following steps: a multi-cell multi-user software-defined Internet of vehicles network, a cell deployment base station and a roadside unit, wherein an MEC server is provided. The user has a task during the movement. The SDN controller has a global view and can be used for intensively scheduling vehicle and base station information.
As shown in fig. 1, there are M roadside units that deploy M MEC servers, defined as service nodes, M ═ 1,2 …, M](ii) a Vehicles may offload tasks to roadside units and base stations through a cellular network. A number of task request vehicles, denoted V, are deployed, subject to a Poisson distributioni＝{v1,v2,...vA}. For spectral reuse, interference exists between multiple roadside units, assuming that they share one spectral resource. Dividing a bandwidth W into P channels; vehicle users are connected to a base station using Orthogonal Frequency Division Multiple Access (OFDMA), i.e., orthogonal between users within the coverage of a base station.
Let the city distribution model graph be G (S, E, W), where S denotes the set of RSUs in the city, E denotes the communication links between these roadside units, and W denotes the distance weighting factor. Each computational task may be described as Γi(Di,Ci)，DiRepresenting a computational task ΓiInput data size of CiRepresenting completion of task ΓiThe number of CPU cycles required. When the task calculation migration is carried out, the task is divided into three parts to be executed, including vehicle local execution, original roadside unit execution and target roadside unit execution. In the process of executing the task, the task is limited in two computing nodes to be executed; if the tasks are calculated at a plurality of computing nodes, the load of IO ports of a communication link is increased, and when the number of task vehicles is increased, the blockage is caused, so that the efficiency of calculating and transferring the whole tasks is reduced. Thus limiting the tasks to the start node and the destination node for computation, which may improve overall system efficiency.
For a vehicle single data calculation task, a calculation task gamma is definediInput data size DiThe sum of the sizes of the task data of the local vehicle, the original roadside unit and the target roadside unit is represented by the following expression:
Figure BDA0002976224880000071
wherein, epsilon, phi,
Figure BDA0002976224880000072
The distribution ratio coefficients are respectively.
Defining completion task ΓiRequired number of CPU cycles CiFor local partial tasks, original pathThe sum of the number of CPU cycles required by the tasks of the edge unit and the target roadside unit is expressed as follows:
Ci＝Cil+Cio+Cid＝αCi+βCi+χCi
wherein, alpha, beta and chi are distribution ratio coefficients respectively.
RVThe up-transmission rate, R, which can be realized when the vehicle and the nearby roadside unit carry out task unloadingVThe expression of (a) is:
Figure BDA0002976224880000073
where, w is the bandwidth of the channel,
Figure BDA0002976224880000074
p, h are the transmission power and channel gain of the vehicle at channel n, respectively.
Figure BDA0002976224880000075
σ2The system is characterized by comprising a vehicle terminal, a base station and a roadside unit, wherein the vehicle terminal is Gaussian white noise of the vehicle terminal, I is interference between a vehicle user and the base station and the roadside unit, and d is a distance between the vehicle user and the roadside unit when a vehicle uploads a task.
Uploading a portion of data for a single vehicle mission to a roadside unit at time tTThe expression is:
Figure BDA0002976224880000076
the time expression for transmitting the total uploading part data of A task request vehicles which comprise M MEC servers and obey Poisson distribution to the roadside unit is as follows:
Figure BDA0002976224880000077
computing a time function t locally for a vehiclelThe calculation process of (2) includes: local divisionThe CPU period number required by the vehicle is divided by the computing capacity of the vehicle, and the expression of the vehicle task edge computing time delay function is as follows:
Figure BDA0002976224880000081
the MEC edge calculation time can be divided into two parts, namely the edge calculation time t corresponding to the original MEC serverioEdge computation time t corresponding to destination MEC serveridThe corresponding calculation process comprises: the number of CPU cycles required for computing a task is divided by the computing power of the MEC server, and the expression of the MEC edge computation delay function is:
Figure BDA0002976224880000082
Figure BDA0002976224880000083
the overall edge computation time can be expressed as:
Figure BDA0002976224880000084
for virtual machine migration time TmigThe expression is:
Figure BDA0002976224880000085
in the formula (I), the compound is shown in the specification,
Figure BDA0002976224880000086
r is the dirty page rate during migration, L is the network bandwidth of the data center where the virtual machine is located, and N is the number of iterations.
For virtual machine downtime TdownThe expression is:
Figure BDA0002976224880000087
in the formula, tresIs the restart time of the virtual machine at the destination physical machine.
The total time consumption of virtual machine migration is TMThe expression is:
Figure BDA0002976224880000088
because the virtual machine migration is to find an algorithmically determined link Y in all migration links EiMigration is performed, so the corresponding expression is:
Figure BDA0002976224880000091
the service quality is guaranteed, and meanwhile the total time delay of the whole system, especially the migration time delay of the virtual machine is reduced. The virtual machine migration latency is optimized by Dijkstra algorithm, Q-learning algorithm to minimize the total system latency for the vehicle user. Weighting the optimization target of the service migration of the Internet of vehicles to obtain a time delay function of a vehicle user, wherein the expression is as follows:
Figure BDA0002976224880000092
the limiting conditions include:
Figure BDA0002976224880000093
α+β+χ＝1,α+β+χ＝1、C2:0<Fio,Fid<FB、C3:Yi∈E,∑Yi＝1、
Figure BDA0002976224880000094
C5:0<λ<1、C6:0<N<NMAX。
where C1 is the CPU cycles required to enter data and complete a task in the task modelThe satisfaction of the weighting factor of the period number. C2 indicates that the computing resources allocated to the task by the MEC server are less than the total computing resources F of the MEC serverB. C3 shows that virtual machine migration must select a link YiMigration is performed. C4 shows that the amount of data transferred during virtual machine migration is less than the capacity A of the linkLINK. And the value range of the model parameter lambda of the migration time of the C5 virtual machine. C6 indicates the range of iterations, where NMAXIndicating the maximum number of iterations. w is the bandwidth, p, h are the transmission power and channel gain of the vehicle on channel n [22]。σ2Is Gaussian white noise of an automobile terminal, I is interference between an automobile user and a base station and a roadside unit, D is a distance between the automobile user and the roadside unit when a vehicle uploads a task, and D is a distance between the automobile user and the roadside unit when the vehicle uploads a taskiRepresenting a computational task ΓiInput data size of CiRepresenting completion of task ΓiThe number of CPU cycles required.
As shown in fig. 3, in order to reduce the computational complexity of the user service delay function, the algorithm mainly adopts a Dijkstra-based shortest migration path selection algorithm and a Q-learning algorithm to optimize the virtual machine service migration time.
Step 1: inputting a city distribution model diagram G (S, E, W), the position of a starting roadside unit A and the position of a target roadside unit B;
step 2: inputting the relevant parameters obtained in the step 1 into a Dijkstra-based shortest migration path selection algorithm, wherein the algorithm firstly generates a weight matrix D ═ W (A,: for different nodes) and generates an initial node V ═ ones (1, n); and V (A) < 0 >, recording the last node of each node, starting from the starting point, finding the next point with the shortest distance, not repeating the original track every time, setting V to judge whether the node is accessed, after the previous operation is completed, using the node with the shortest path found above as the starting point, if the node passes through, shortening the path length from the starting point to each node, then updating and recording the previous node, and finally searching the searching path from the tail part to the front part by using a backtracking method to obtain I and the number E of servers on the path, thereby obtaining the shortest path.
And step 3: inputting the obtained shortest path into a Q-learning algorithm, wherein the migration time of the virtual machine on different nodes is dynamically changed and is a non-convex optimization problem, so that the migration time of the virtual machine is abstracted into a migration function, and then the Q-learning algorithm is used for optimizing to obtain the shortest time; the Q value is expressed as:
Q(s,a)←Q(s,a)+α[r(s,a)+γmaxQ(s′,a′)-Q(s,a)]
where S represents a state set. A denotes an action set, α is a learning rate, and has a value between 0 and 1. r (S, a) represents the return that would result from performing action a in state S, and γ represents a discount factor that determines how close and far the time has affected the return.
Establishing a target optimization function according to a Q value formula; the expression of the objective optimization function is:
Figure BDA0002976224880000101
Figure BDA0002976224880000102
wherein, λ represents, N represents, E represents, Y representsiIs represented byidIs represented by L, tresAnd (4) showing.
Since the Q value formula is a non-convex optimization problem under given constraints, the formula for f (λ, N) is calculated as:
Figure BDA0002976224880000103
the second order partial derivative is calculated for f (λ, N), namely:
Figure BDA0002976224880000111
Figure BDA0002976224880000112
Figure BDA0002976224880000113
Figure BDA0002976224880000114
the expression of the Hessian matrix obtained according to the formula is as follows:
Figure BDA0002976224880000115
at 0<λ<1，0<N<NMAXAnd N is an integer, under the condition that,
Figure BDA0002976224880000116
then there is
Figure BDA0002976224880000117
The values of the other subforms are less than 0, so the objective function formula is a non-convex optimization problem. Namely, the Q value formula is also a non-convex optimization problem, and therefore, the Q-learning algorithm is adopted to optimize the problem.
And 4, step 4: a set of states is established as S (R, L) and a set of behaviors as A (lambda, N). Because it is the reward maximization sought, the reward is set to:
Figure BDA0002976224880000118
the reward maximization, namely the time minimization, is realized so as to achieve the aim of optimization;
and 5: and when the shortest path obtained by the Dijkstra algorithm is optimized to the shortest time by using the Q-learning algorithm, namely the Q-learning algorithm is converged, the obtained final service is migrated to be the optimal virtual machine service migration method.
The shortest migration path selection algorithm based on Dijkstra is as follows:
Figure BDA0002976224880000119
Figure BDA0002976224880000121
the shortest migration path optimization algorithm based on Q-learning is as follows:
Figure BDA0002976224880000122
Figure BDA0002976224880000131
when there are E states, S ═ S1,S2,...,SES can be converted every time the action set A is selected. The process of obtaining a reward r, Q-learning status is transformed as shown in fig. 9.
Dijkstra's shortest migration path selection method constantly compares the distance of next node through fixed starting point roadside unit and purpose roadside unit, selects the next node that the distance is the shortest, and constantly repeats the above process and obtains the shortest path from starting point roadside unit to purpose roadside unit, after obtaining the shortest path, reuse Q-learning method to optimize the migration delay of each node, because Q-learning method pursues the reward maximization, can be the time minimization for Q-learning method goal through the setting of reward. The final virtual machine migration method obtained through the two algorithms is the optimal migration method.
As shown in fig. 4, the time delay after the transition is significantly reduced, because if the vehicle still performs communication transmission with the original MEC server after driving out of the service range of the original roadside unit MEC server, the communication distance between the vehicle and the corresponding virtual machine is long, and the number of hops is increased, which takes a long time and also causes waste of communication resources. However, if the virtual machine migration mode is adopted, because the virtual machine migration occurs between the MEC servers, and the communication with the vehicle is finally completed at the target server, the time consumption is less, and the communication resources are also saved.
As shown in fig. 5, the larger the amount of data, the higher the average delay. It is also worth noting that as vehicles increase, the average time delay increases first and then decreases slowly, because the increase of vehicles within a certain service range of a base station or a roadside unit may aggravate the service delay, however, when the vehicles increase to a certain extent, the equivalent uniform distribution of the vehicle positions of users may be indirectly caused, and the communication distance between users and the base station is relatively decreased, thereby causing the time delay to decrease. This document compares with MDP (markov chain), AMEChe (energy saving migration) and OHBAC (load balancing algorithm), which have the advantage that a good compromise between user quality of service and migration cost can be achieved, the AMEChe algorithm optimizes energy consumption during migration, and the OHBAC algorithm reduces energy consumption, VM migration number and shutdown host number using an automatic deployment mechanism. They ignore the effect of migration time.
As shown in fig. 6, it can be seen that the overall migration time consumption is less for the methods herein compared to other methods. The reason is that Q learning is a dynamic and continuous learning process, and can select an optimal migration parameter for migration, and in addition, the Dijkstra algorithm is used to select the shortest path for task migration.
As shown in fig. 7, the algorithm herein has a smaller migration failure rate because it reduces the number of executed migrations compared to other methods, thereby avoiding some unnecessary migrations. This is because the shortest migration path is found by the path selection algorithm after the positions of the original roadside unit MEC server and the destination roadside unit MEC server are known, thereby reducing the number of times of migration. The time for virtual machine migration is reduced accordingly, which also reduces service delay and optimizes user experience.
As shown in fig. 8, it can be observed that the number of VM migrations is less for the algorithm herein compared to other methods. This is because the algorithm herein is more intelligent and can be continuously learned by the Q-learning algorithm to provide efficient migration decisions. AMEChe, MDP and OBAC do not take into account time factors, resulting in some unnecessary migration.
The above-mentioned embodiments, which further illustrate the objects, technical solutions and advantages of the present invention, should be understood that the above-mentioned embodiments are only preferred embodiments of the present invention, and should not be construed as limiting the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (7)

1. A software defined Internet of vehicles service migration method based on MEC is characterized by comprising the following steps: the vehicle user inputs the generated calculation task into the Internet of vehicles in the driving process; the Internet of vehicles performs task migration according to the calculation task; selecting a task migration path by adopting a Dijkstra routing algorithm to obtain an optimal path; determining an optimization target of the vehicle user according to the optimal path, and calculating a time delay function of the vehicle user according to the optimization target; optimizing the time delay function by adopting a Q-learning algorithm to obtain an optimal time delay function; calculating a transfer reward function according to the optimal time delay function, and when the reward function is maximum, taking the task transfer path as an optimal transfer scheme; the MEC represents the calculation of a moving edge, the Dijkstra routing algorithm represents the Dijkstra routing algorithm, and the Q-learning represents the reinforcement learning algorithm;
determining a time delay function of a vehicle user according to an optimization target of the service migration of the Internet of vehicles as follows:
Figure DEST_PATH_IMAGE002
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE004
for the total mission data size of the vehicle,
Figure DEST_PATH_IMAGE006
the number of CPU cycles required to complete the vehicle's mission,
Figure DEST_PATH_IMAGE008
in order to distribute the proportionality coefficient,
Figure DEST_PATH_IMAGE010
is the bandwidth, p, h are the vehicle on-channel respectively
Figure DEST_PATH_IMAGE012
The transmission power and the channel gain of the mobile station,
Figure DEST_PATH_IMAGE014
is the white gaussian noise of the car terminal,
Figure DEST_PATH_IMAGE016
is interference of car users with base stations and roadside units,
Figure DEST_PATH_IMAGE018
is the distance from the roadside unit when the vehicle uploads a task,
Figure DEST_PATH_IMAGE020
which represents the computing power of the vehicle itself,
Figure DEST_PATH_IMAGE022
and
Figure DEST_PATH_IMAGE024
the computing resources allocated to the task for the original MEC server and the destination MEC server,
Figure DEST_PATH_IMAGE026
is the link selected for the migration of the virtual machine,
Figure DEST_PATH_IMAGE028
，
Figure DEST_PATH_IMAGE030
for the dirty page rate during migration purposes,
Figure DEST_PATH_IMAGE032
for the network bandwidth of the data center where the virtual machine is located,
Figure DEST_PATH_IMAGE034
in order to be able to perform the number of iterations,
Figure DEST_PATH_IMAGE036
is the restart time of the virtual machine at the destination physical machine.
2. The MEC-based software-defined internet of vehicles service migration method of claim 1, wherein the vehicle user generated calculation task comprises three parts of local execution, original roadside unit execution and target roadside unit execution; when a vehicle user moves, tasks are transferred from an original roadside unit to a target roadside unit, so that a virtual machine VM is transferred; and selecting an optimal path from a plurality of migration paths from the original roadside unit to the target roadside unit for migration during the migration of the virtual machine, and optimizing the migration time of the virtual machine under the optimal path.
3. The MEC-based software-defined internet of vehicles service migration method of claim 1, wherein the process of selecting the task migration path by using Dijkstra routing algorithm comprises:
step 1: obtaining city distribution model diagram
Figure DEST_PATH_IMAGE038
Determining the positions of a starting point road side unit A and a destination road side unit B in the graph;
step 2: calculating a weight matrix D corresponding to different nodes in the graph according to the urban distribution model graph, and generating an initial node V = ones (1,
Figure DEST_PATH_IMAGE040
)；
and step 3: calculating the distances from the initial node to all paths in the next node according to the weight matrix D, and selecting the path with the shortest distance as the optimal sub-path between the two nodes;
and 4, step 4: taking the end point of the optimal sub-path as the starting point of a new path, calculating the distances of all paths in the next node, and selecting the path with the shortest distance as the optimal sub-path of the two nodes;
and 5: repeating the step 4 until the task is migrated to the target roadside unit;
step 6: and (4) according to all the optimal sub-paths, a back trace method is adopted to search the paths from the target roadside unit to the starting roadside unit in a reverse mode, and a communication link I, the number of servers of the paths and the shortest migration path are obtained.
4. The MEC-based software defined Internet of vehicles service migration method of claim 1, wherein the optimization objective of Internet of vehicles service migration comprises: time delay function for vehicle to transmit partial data to roadside unit
Figure DEST_PATH_IMAGE042
Vehicle local computation time function, original MEC server edge computation time delay function
Figure DEST_PATH_IMAGE044
Destination MEC server edge computation delay function
Figure DEST_PATH_IMAGE046
Migration time of virtual machine
Figure DEST_PATH_IMAGE048
And virtual machine downtime
Figure DEST_PATH_IMAGE050
。
5. The MEC-based software-defined Internet of vehicles service migration method of claim 1, wherein the specific steps of optimizing the delay function by using Q-learning algorithm are as follows:
step 1: abstracting an objective function according to the task migration state and the optimization target of the vehicle user, and acquiring a state set of the vehicle user through the objective function
Figure DEST_PATH_IMAGE052
And action set
Figure DEST_PATH_IMAGE054
；
Step 2: constructing a Q matrix according to the state set and the behavior set of the vehicle user, and initializing the Q matrix; the Q matrix of the Q-learning algorithm is a matrix taking a state as a row and an action as a column, and is initialized at first, and all values are set to be 0;
and step 3: the method comprises the following steps that a vehicle user randomly selects an initial state S in a state set, and a greedy strategy is adopted according to the initial state to obtain corresponding behaviors;
and 4, step 4: updating the state of the selected behavior by adopting a Q value formula, namely updating rewards obtained by different behaviors in a Q matrix, wherein the corresponding values are the rewards obtained by adopting different behaviors in different states;
and 5: taking the updated state as the state of the current task to update the state of the next task; continuously repeating the steps 3, 4 and 5 until the Q matrix is converged;
step 6: when the Q matrix is converged, the optimal reward of each state is obtained through the Q matrixr；
And 7: adding all the optimal rewards to obtain the final reward
Figure DEST_PATH_IMAGE056
(ii) a When in use
Figure 540508DEST_PATH_IMAGE056
And when the task is maximum, the migration path is the optimal migration scheme.
6. The MEC-based software-defined internet of vehicles service migration method of claim 5, wherein the Q value calculation formula is as follows:
Figure DEST_PATH_IMAGE058
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE060
a set of states is represented that is,
Figure DEST_PATH_IMAGE062
a set of actions is represented that is,
Figure DEST_PATH_IMAGE064
it is indicated that the learning rate is,
Figure DEST_PATH_IMAGE066
is shown in a state
Figure 706916DEST_PATH_IMAGE060
Lower execution action
Figure DEST_PATH_IMAGE068
The obtained return is returned to the user,
Figure DEST_PATH_IMAGE070
representing a discount factor.
7. The MEC-based software-defined internet of vehicles service migration method of claim 5, wherein the final reward is
Figure 142446DEST_PATH_IMAGE056
The expression of (a) is:
Figure DEST_PATH_IMAGE072
wherein the content of the first and second substances,
Figure DEST_PATH_IMAGE074
representing a virtual machine migration time model parameter,
Figure DEST_PATH_IMAGE076
representing the number of iterations, E representing the communication links between all roadside units within the system,
Figure 134060DEST_PATH_IMAGE026
representing the link selected for the migration of the virtual machine,
Figure DEST_PATH_IMAGE078
the size of the task data of the target roadside unit is shown, L represents the network bandwidth of the data center where the virtual machine is positioned,
Figure DEST_PATH_IMAGE080
representing the restart time of the virtual machine at the destination physical machine.
CN202110274917.0A 2021-03-15 2021-03-15 Software defined Internet of vehicles service migration method based on MEC Active CN113114721B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110274917.0A CN113114721B (en) 2021-03-15 2021-03-15 Software defined Internet of vehicles service migration method based on MEC

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110274917.0A CN113114721B (en) 2021-03-15 2021-03-15 Software defined Internet of vehicles service migration method based on MEC

Publications (2)

Publication Number Publication Date
CN113114721A CN113114721A (en) 2021-07-13
CN113114721B true CN113114721B (en) 2022-05-03

Family

ID=76711418

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110274917.0A Active CN113114721B (en) 2021-03-15 2021-03-15 Software defined Internet of vehicles service migration method based on MEC

Country Status (1)

Country Link
CN (1) CN113114721B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114089919A (en) * 2021-11-25 2022-02-25 中国联合网络通信集团有限公司 Heterogeneous MEC data collaboration method, collaboration service center and system
CN114118300B (en) * 2022-01-21 2022-05-20 苏州浪潮智能科技有限公司 Service migration model training method and Internet of vehicles service migration method and system
CN115002155B (en) * 2022-05-25 2023-02-21 安徽江淮汽车集团股份有限公司 Time delay optimization-based road side unit service migration method

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111885155A (en) * 2020-07-22 2020-11-03 大连理工大学 Vehicle-mounted task collaborative migration method for vehicle networking resource fusion

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111885155A (en) * 2020-07-22 2020-11-03 大连理工大学 Vehicle-mounted task collaborative migration method for vehicle networking resource fusion

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
面向5G网络的边缘计算迁移策略研究与仿真;赵丹;《中国优秀硕士学位论文》;20181130;全文 *

Also Published As

Publication number Publication date
CN113114721A (en) 2021-07-13

Similar Documents

Publication Publication Date Title
CN113114721B (en) Software defined Internet of vehicles service migration method based on MEC
Hu et al. Twin-timescale artificial intelligence aided mobility-aware edge caching and computing in vehicular networks
CN108777852B (en) Internet of vehicles content edge unloading method and mobile resource distribution system
CN109391681B (en) MEC-based V2X mobility prediction and content caching offloading scheme
Cui et al. A novel offloading scheduling method for mobile application in mobile edge computing
CN110557732B (en) Vehicle edge computing network task unloading load balancing system and balancing method
CN111010684B (en) Internet of vehicles resource allocation method based on MEC cache service
CN113543074B (en) Joint computing migration and resource allocation method based on vehicle-road cloud cooperation
CN111586696A (en) Resource allocation and unloading decision method based on multi-agent architecture reinforcement learning
CN111683381A (en) End-to-end network slice resource allocation algorithm based on deep reinforcement learning
CN112685186A (en) Method and device for unloading computing tasks, electronic equipment and storage medium
CN114116047A (en) V2I unloading method for vehicle-mounted computation-intensive application based on reinforcement learning
Dhiman et al. SHANN: an IoT and machine-learning-assisted edge cross-layered routing protocol using spotted hyena optimizer
Zhang et al. An approach of multi-objective computing task offloading scheduling based NSGS for IOV in 5G
Ren et al. Vehicular network edge intelligent management: A deep deterministic policy gradient approach for service offloading decision
Lin et al. Deep reinforcement learning-based task scheduling and resource allocation for NOMA-MEC in Industrial Internet of Things
Li et al. Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN
CN116261119A (en) Intelligent collaborative task calculation and on-demand resource allocation method in vehicle-mounted environment
Li et al. DQN-enabled content caching and quantum ant colony-based computation offloading in MEC
Suzuki et al. Multi-agent deep reinforcement learning for cooperative offloading in cloud-edge computing
Yu et al. UAV-assisted cooperative offloading energy efficiency system for mobile edge computing
Shinde et al. A markov decision process solution for energy-saving network selection and computation offloading in vehicular networks
Su et al. Towards fast and energy-efficient offloading for vehicular edge computing
Gupta et al. Priority based V2V data offloading scheme for FiWi based vehicular network using reinforcement learning
KR102497226B1 (en) Offloading method using autonomous vehicular ad hoc network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant