CN113542376A - Task unloading method based on energy consumption and time delay weighting - Google Patents

Task unloading method based on energy consumption and time delay weighting Download PDF

Info

Publication number
CN113542376A
CN113542376A CN202110749538.2A CN202110749538A CN113542376A CN 113542376 A CN113542376 A CN 113542376A CN 202110749538 A CN202110749538 A CN 202110749538A CN 113542376 A CN113542376 A CN 113542376A
Authority
CN
China
Prior art keywords
task
vehicle
unloading
mec
time delay
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110749538.2A
Other languages
Chinese (zh)
Inventor
曾伟良
周镖华
黄永慧
孙为军
杨超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangdong University of Technology
Original Assignee
Guangdong University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangdong University of Technology filed Critical Guangdong University of Technology
Priority to CN202110749538.2A priority Critical patent/CN113542376A/en
Publication of CN113542376A publication Critical patent/CN113542376A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computing Systems (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention discloses a task unloading method based on energy consumption and time delay weighting, which comprises the following steps: s1, constructing a system model according to communication between the task vehicle and the cooperative vehicle and communication between the task vehicle and an MEC server configured on a roadside unit arranged at the edge of the intersection road to obtain an objective function; s2, judging whether the time for the task vehicle to solve the task of the task vehicle exceeds the maximum tolerance time delay, if so, performing local calculation, otherwise, entering the step S3; s3, converting the task allocation process decision problem into a Markov decision process according to the constructed system model and the obtained objective function; s4, solving the strategy selection problem in the Markov decision process by applying the DQN algorithm, thereby obtaining the optimal unloading node. The method effectively reduces the calculation time delay of the task vehicle, relieves the calculation pressure of the task vehicle and the MEC server, and can solve the problem that a heuristic algorithm falls into a local optimal solution.

Description

Task unloading method based on energy consumption and time delay weighting
Technical Field
The invention relates to the technical field of Internet of vehicles, in particular to a task unloading method based on energy consumption and time delay weighting.
Background
With the development of technologies such as internet of things, artificial intelligence and virtual reality, the computation-intensive business with high energy consumption is continuously increased, and the conflict between the computation-intensive application and the mobile computing system with limited resources brings unprecedented challenges to the development of future mobile business. To address this challenge, Mobile Cloud Computing (MCC) is often used to offload Computing tasks on Mobile terminals to resource-rich remote clouds. However, the conventional MCC method has disadvantages of long delay and low reliability caused by data transmission through a wide area network.
In recent years, Mobile Edge Computing (MEC), which can provide cloud Computing capability in the vicinity of Mobile users, has been proposed as one of the key technologies of 5G. Offloading the user's computing tasks to a nearby MEC server, i.e., mobile edge computing offloading, is considered a promising solution to address the above challenges. Edge calculation can achieve lower latency and higher reliability compared to the conventional MCC scheme, and has been a research hotspot.
With the proliferation of vehicles on roads, a large amount of computing resources are required to meet the computing demands of users, and the proliferation of vehicles is accompanied by a lot of idle computing resources available. For vehicles, the main optimization problem for task offloading of vehicles using edge calculation is: (1) and the task calculation time delay is reduced. I.e., how to reasonably allocate the proportion of resources to minimize the time from the task generated by the vehicle to the feedback that the vehicle received the task. (2) The energy consumption is reduced. I.e., the vehicle is tasked off-loading, the energy consumed is minimal.
Currently existing vehicle task offloading strategies are:
1. in the research on the minimum time delay multi-edge node unloading balancing strategy, data flow in a Network is monitored through a Software Defined Network (SDN), data in a hot spot area are regulated and controlled to be unloaded to peripheral nodes in a multi-hop mode to execute a computing task, the purpose of reducing heat in the hot spot area and reducing time delay of the executed task is achieved, and meanwhile, an edge node unloading algorithm based on a quantum particle swarm algorithm and an edge node load balancing algorithm based on a heuristic algorithm are provided to solve the problem.
2. Vehicle networking cooperative service caching and calculation unloading based on multi-agent reinforcement learning carries out service caching and task scheduling facing vehicle application by researching cooperative cooperation of multi-path side units (RSUs), and models the service caching and task scheduling as a mixed integer nonlinear programming problem, aiming at minimizing service delay of a vehicle networking system. In order to reduce the difficulty of solving an optimization problem, firstly, a double-layer multi-RSU cooperative cache framework is provided to decouple the problem, a multi-agent meta reinforcement learning method is adopted at the outer layer, and a long-term and short-term memory network is adopted as a meta-agent to balance local decisions and accelerate the learning process while each local agent performs decision learning, so that an optimal RSU cache strategy is obtained; and the inner layer adopts a Lagrange multiplier method to solve the optimal collaborative unloading strategy, so that task allocation among RSUs is realized.
3. A genetic algorithm-based multi-site collaborative computing unloading algorithm GAMCCO is provided. The algorithm abstracts an application program into a task dependency relationship graph model, analyzes the dependency relationship among tasks, models the problem of multi-site collaborative computing unloading into a cost model, and searches an unloading scheme with the minimum cost by using a genetic algorithm.
However, the above solution has the following disadvantages:
1) the scheme does not consider energy consumption; 2) the adopted long and short term memory network needs a large amount of data training and the training process is time-consuming; 3) by applying the genetic algorithm, the heuristic algorithm is easy to fall into the local optimal solution, so that the global optimal solution is neglected.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a task unloading method based on energy consumption and time delay weighting.
In order to achieve the purpose, the technical scheme provided by the invention is as follows:
a task unloading method based on energy consumption and time delay weighting releases computing resources of cooperative vehicles with idle computing resources in a road and provides computing services for task vehicles, and comprises the following steps:
s1, constructing a system model according to communication between the task vehicle and the cooperative vehicle and communication between the task vehicle and an MEC server configured on a roadside unit arranged at the edge of the intersection road to obtain an objective function;
s2, judging whether the time for the task vehicle to solve the task of the task vehicle exceeds the maximum tolerance time delay, if so, performing local calculation, otherwise, entering the step S3;
s3, converting the task allocation process decision problem into a Markov decision process according to the system model constructed in the step S1 and the obtained objective function;
and S4, according to the Markov decision process obtained by conversion in the step S3, in the process, solving the problem of strategy selection in the Markov decision process by using a DQN algorithm, so as to obtain an optimal unloading node, namely the node for providing calculation service for the task vehicle.
Further, the step S1 includes:
s1-1, carrying out scene modeling analysis, including dividing unloading modes into a vehicle-to-vehicle V2V unloading mode and an MEC unloading mode;
s1-2, building a mathematical model for the system model.
Further, the step S1-2 specifically includes:
V2V unload mode:
in the unloading mode of V2V, assuming that each task vehicle can find the optimal cooperative vehicle and becomes an unloading pair with it, according to shannon's formula, the transmission rate between the task vehicle and the cooperative vehicle is:
Figure BDA0003144027830000031
in the formula (1), ω isV2VIs bandwidth, PiIn order to transmit the power, the power is transmitted,
Figure BDA0003144027830000032
for channel gain, N0Is noise;
in the unloading mode of V2V, the time delay comprises the time of data uploading and the time of data calculation in the cooperative vehicle, and the return time is ignored because the processed data of the task is small;
time delay of transmission process
Figure BDA0003144027830000033
And energy consumption
Figure BDA0003144027830000034
Respectively as follows:
Figure BDA0003144027830000041
Figure BDA0003144027830000042
in the formula (2), DiIs the size of the data of the task,
Figure BDA0003144027830000043
Figure BDA0003144027830000044
a set of task vehicles representing the inability to resolve their own tasks within a maximum tolerated delay;
collaborative vehicle computation latency
Figure BDA0003144027830000045
And energy consumption
Figure BDA0003144027830000046
Comprises the following steps:
Figure BDA0003144027830000047
Figure BDA0003144027830000048
in the formula (3), BiRepresenting the number of CPU cycles required to calculate one bit size data,
Figure BDA0003144027830000049
fjfor the computational capability of cooperating vehicles, PjIn order to cooperate with the power consumption of the vehicle,
Figure BDA00031440278300000410
mu is an energy parameter;
so the total delay in the unloading mode of V2V
Figure BDA00031440278300000411
And total energy consumption
Figure BDA00031440278300000412
Comprises the following steps:
Figure BDA00031440278300000413
Figure BDA00031440278300000414
considering the overhead of latency and energy together, the total cost function in the V2V offload mode is:
Figure BDA00031440278300000415
in the formula (5), α is a weight factor of time delay and energy consumption, and the closer α is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
MEC uninstallation mode:
when the MEC unloading mode is selected by the task vehicle, the task vehicle is communicated with the roadside unit through the cellular network and is communicated with an MEC server deployed on the roadside unit;
according to the shannon formula, the transmission rate between the mission vehicle and the MEC server is:
Figure BDA00031440278300000416
in the formula (6), ωMECIs the bandwidth;
in the MEC unloading mode, the time delay and the energy consumption are formed by two parts of task uploading to an MEC server and MEC server calculation;
time delay of task uploading part
Figure BDA0003144027830000051
And energy consumption
Figure BDA0003144027830000052
Respectively, as follows:
Figure BDA0003144027830000053
Figure BDA0003144027830000054
the time delay and the energy consumption calculated by the MEC server are respectively expressed as follows:
Figure BDA0003144027830000055
in the formula (8), fMECFor the computing power of the MEC server:
Figure BDA0003144027830000056
so the total delay in MEC offload mode
Figure BDA0003144027830000057
And energy consumption
Figure BDA0003144027830000058
Expressed as:
Figure BDA0003144027830000059
Figure BDA00031440278300000510
the time delay and energy overhead are comprehensively considered, so the total cost function in the MEC offload mode is:
Figure BDA00031440278300000511
in the formula (11), α is a weight factor of time delay and energy consumption; the closer alpha is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
then the cost function for the entire task is:
Figure BDA00031440278300000512
in equation (12), β ═ {0, 1} is an unloading decision parameter, and if the task vehicle selects the V2V unloading mode to unload, β ═ 0, and otherwise, β ═ 1;
after the cost function, the objective function becomes as follows:
Figure BDA00031440278300000513
Figure BDA00031440278300000514
in equation (13), C1 represents a constraint in the sense that neither V2V nor MEC offload mode computations can exceed the maximum tolerable latency of a task.
Further, in step S3, the converting the task allocation process decision problem into the markov decision process specifically includes:
when a task vehicle carries out task unloading, tasks of the task vehicle are continuously unloaded to an MEC server or a cooperative vehicle, the state of a system is composed of the data size of the current task, the task complexity, the maximum tolerance time delay and the communication state between the vehicle and an unloading point, and the state space of the system is expressed as follows:
St＝{Ti，V2V1，V2V2，...，V2Vj，MEC1，MEC2，...，MECm} (14)
in formula (14), Ti: for task type, V2V1，V2V2，...，V2VjFor the vehicle to communicate in the V2V unloaded mode, MEC1，MEC2，...，MECmPerforming a credit communication state in an MEC unloading mode for the vehicle;
and the motion space is composed of selectable unloading points, and the motion space is expressed as:
At＝{V1，V2，，...，Vj，M1，M2，...，Mm} (15)
in the formula (15), V1，V2，，...，VjFor cooperative vehicles, M1，M2，...，MmIs an MEC server;
the task vehicle selects one of the optimal unloading nodes according to a strategy, namely from AtSelecting an action to perform the current task;
the reward function is set as
Figure BDA0003144027830000061
s is the current state, a is the action, the reward function is set as the negative total cost function, in order to make the reward of the whole process maximum, namely the smaller the total cost function;
the unloading strategy pi represents that the vehicle is in the state space StThe basis for taking action is as follows; the merits of a policy are judged by a policy-based merit function, which is divided into: a state cost function and a behavior cost function;
wherein the content of the first and second substances,
the state cost function:
Figure BDA0003144027830000062
a behavioral cost function: qπ(s，a)＝E(Gt|St＝s，At＝a)；
Through the Be | | man equation, the state cost function is converted to:
Figure BDA0003144027830000063
in formula (16), s is the current state, r(s) is the reward function in the current state, γ is the discount factor γ ∈ [0, 1], s ' is the next state, P (s ' | s) is the transition probability of the current state transitioning to the next state, and V (s ') is the value of the next state;
through the Be | man equation, it can Be seen that the value of the current state is equal to the reward to reach the current state plus the value of the next state.
Further, in step S4, the method for solving the problem of policy selection in the markov decision process by using the DQN algorithm specifically includes:
let the behavioral cost function be updated according to the following formula:
Figure BDA0003144027830000071
in the formula (17), R is a reward function,
Figure BDA0003144027830000072
in order to learn the rate of speed,
Figure BDA0003144027830000073
q (s, a) is the current behavioral merit function, Q*(s, a) is the updated behavioral cost function;
the optimal strategy can obtain the optimal behavior cost function through the formula (17), and the strategy is to select the action which can obtain the maximum behavior cost function value in the next state as the optimal action;
in the DQN algorithm, a neural network is used for fitting a Q value, and in each time slot, current condition information is input to a trained model, so that the Q value and corresponding actions, namely corresponding unloading nodes, can be obtained; and searching a new unloading node by using an epsilon-greedy search algorithm according to an epsilon probability, and selecting the unloading node according to a current Q value updating formula according to a 1-epsilon probability.
Compared with the prior art, the principle and the advantages of the scheme are as follows:
1) according to the scheme, vehicles (cooperative vehicles) with idle computing resources on the road are considered, the functions of the vehicles are fully played, the idle computing resources are released, the computing time delay of the task vehicle can be effectively reduced, and the computing pressure of the task vehicle and the MEC server is relieved.
2) According to the scheme, the DQN algorithm is applied in the process of obtaining the unloading node, and as the state of the vehicle is high-dimensional, the common q-table is limited by the dimension, and the DQN algorithm can not be limited by the dimension. Due to the high latitude, the application of the deep neural network can avoid falling into the local optimal solution, so that the problem that the heuristic algorithm falls into the local optimal solution is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the services required for the embodiments or the technical solutions in the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a task offloading method based on energy consumption and time delay weighting according to the present invention;
fig. 2 is a schematic diagram of a system model involved in the present invention.
Detailed Description
The invention will be further illustrated with reference to specific examples:
according to the task unloading method based on energy consumption and time delay weighting, the computing resources of the cooperative vehicles with free computing resources in the road are released, and computing services are provided for the task vehicles.
As shown in fig. 1, the method specifically comprises the following steps:
s1, constructing a system model according to communication between the task vehicle and the cooperative vehicle and communication between the task vehicle and an MEC server configured on a roadside unit arranged at the edge of the intersection road to obtain an objective function;
in this step, include:
s1-1, carrying out scene modeling analysis:
as shown in fig. 2, the considered scene is an intersection, a Road Side Unit (RSU) is deployed on the Road, and an MEC server is configured on the RSU and is marked as M ═ M1，M2，...，Mm}. Vehicles with unused computing resources on the road, i.e., cooperating vehicles, denoted V ═ V, can provide computing services1，V2，，...，VjAnd N task vehicles are available. This embodiment discretizes time, divides time into time segments, and defines a time set as t ═ 1, 2, 3.
It is assumed that the state information of the vehicles does not change within each time block and the task type of each task vehicle is Ti＝(Di，Bi，τi) Where Di represents the data size of the task in units of bit, BiRepresents the number of CPU cycles required to calculate one bit size data, and has the unit of cycle/bit, tauiRepresents the maximum tolerated delay of the task, and has the unit of S.
The task vehicles are divided into two groups,
wherein one group is DiBi/fi≤τiWherein f isiFor the computing capacity of the current task vehicle, the computing tasks of the current task vehicle in the group can be solved in the vehicle, and the maximum tolerable delay is not exceeded, that is, the task unloading is not required, and the embodiment records the task vehicles as a set L ═ 1, 2.
The other group isiBi/fi＞τiThat is to say, the calculation task of the current task vehicle cannot be solved within the maximum tolerance time delay, and the task needs to be unloaded, and this embodiment records this type of vehicle as a set
Figure BDA0003144027830000091
In the collection
Figure BDA0003144027830000092
The task in the interior has two unloading modes, namely a Vehicle-to-Vehicle (V2V) mode and an MEC unloading mode, and defines an unloading decision parameter β ═ 0, 1}, where β ═ 0 if the task Vehicle selects the V2V mode for unloading, and β ═ 1 otherwise.
S1-2, establishing a mathematical model for the system model, specifically comprising:
V2V unload mode:
in the unloading mode of V2V, assuming that each task vehicle can find the optimal cooperative vehicle and becomes an unloading pair with it, according to shannon's formula, the transmission rate between the task vehicle and the cooperative vehicle is:
Figure BDA0003144027830000093
in the formula (1), ω isV2VIs bandwidth, PiIn order to transmit the power, the power is transmitted,
Figure BDA0003144027830000094
for channel gain, N0Is noise;
in the unloading mode of V2V, the time delay comprises the time of data uploading and the time of data calculation in the cooperative vehicle, and the return time is ignored because the processed data of the task is small;
time delay of transmission process
Figure BDA0003144027830000095
And energy consumption
Figure BDA0003144027830000096
Respectively as follows:
Figure BDA0003144027830000097
Figure BDA0003144027830000098
in the formula (2), DiIs the size of the data of the task,
Figure BDA0003144027830000099
Figure BDA00031440278300000910
a set of task vehicles representing the inability to resolve their own tasks within a maximum tolerated delay;
collaborative vehicle computation latency
Figure BDA0003144027830000101
And energy consumption
Figure BDA0003144027830000102
Comprises the following steps:
Figure BDA0003144027830000103
Figure BDA0003144027830000104
in the formula (3), BiRepresenting a calculation of bit number of CPU cycles required for size data,
Figure BDA0003144027830000105
fjfor the computational capability of cooperating vehicles, PjFor coordinating the power consumption of the vehicles, Pj＝μ(fj)2μ is an energy parameter;
so the total delay in the unloading mode of V2V
Figure BDA0003144027830000106
And total energy consumption
Figure BDA0003144027830000107
Comprises the following steps:
Figure BDA0003144027830000108
Figure BDA0003144027830000109
considering the overhead of latency and energy together, the total cost function in the V2V offload mode is:
Figure BDA00031440278300001010
in the formula (5), α is a weight factor of time delay and energy consumption, and the closer α is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
MEC uninstallation mode:
when the MEC unloading mode is selected by the task vehicle, the task vehicle is communicated with the roadside unit through the cellular network and is communicated with an MEC server deployed on the roadside unit;
according to the shannon formula, the transmission rate between the mission vehicle and the MEC server is:
Figure BDA00031440278300001011
in the formula (6), ωMECIs the bandwidth;
in the MEC unloading mode, the time delay and the energy consumption are formed by two parts of task uploading to an MEC server and MEC server calculation;
time delay of task uploading part
Figure BDA00031440278300001012
And energy consumption
Figure BDA00031440278300001013
Respectively, as follows:
Figure BDA0003144027830000111
Figure BDA0003144027830000112
the time delay and the energy consumption calculated by the MEC server are respectively expressed as follows:
Figure BDA0003144027830000113
in the formula (8), fMECFor the computing power of the MEC server:
Figure BDA0003144027830000114
so the total delay in MEC offload mode
Figure BDA0003144027830000115
And energy consumption
Figure BDA0003144027830000116
Expressed as:
Figure BDA0003144027830000117
Figure BDA0003144027830000118
the time delay and energy overhead are comprehensively considered, so the total cost function in the MEC offload mode is:
Figure BDA0003144027830000119
in the formula (11), α is a weight factor of time delay and energy consumption; the closer alpha is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
then the cost function for the entire task is:
Figure BDA00031440278300001110
in equation (12), β ═ {0, 1} is an unloading decision parameter, and if the task vehicle selects the V2V unloading mode to unload, β ═ 0, and otherwise, β ═ 1;
after the cost function, the objective function becomes as follows:
Figure BDA00031440278300001111
Figure BDA00031440278300001112
in equation (13), C1 represents a constraint in the sense that neither V2V nor MEC offload mode computations can exceed the maximum tolerable latency of a task.
S2, judging whether the time for the task vehicle to solve the task of the task vehicle exceeds the maximum tolerance time delay, if so, performing local calculation, otherwise, entering the step S3;
the steps are equivalentPutting task vehicles capable of completing tasks in the set L and putting task vehicles incapable of completing tasks within the maximum tolerance time delay in the set L
Figure BDA0003144027830000123
In, then pair sets
Figure BDA0003144027830000124
The task vehicle in (1) pulls in steps S3 and S4 for further processing.
S3, converting the task allocation process decision problem into a Markov decision process according to the system model constructed in the step S1 and the obtained objective function, which specifically comprises the following steps:
when a task vehicle carries out task unloading, tasks of the task vehicle are continuously unloaded to an MEC server or a cooperative vehicle, the state of a system is composed of the data size of the current task, the task complexity, the maximum tolerance time delay and the communication state between the vehicle and an unloading point, and the state space of the system is expressed as follows:
St＝{Ti’V2V1，V2V2，...，V2Vj，MEC1，MEC2，...，MECm} (14)
in formula (14), TiFor task type, V2V1，V2V2，...，V2VjFor the vehicle to communicate in the V2V unloaded mode, MEC1，MEC2，...，MECmPerforming a credit communication state in an MEC unloading mode for the vehicle;
and the motion space is composed of selectable unloading points, and the motion space is expressed as:
At＝{V1，V2，，...，Vj，M1，M2，...，Mm} (15)
in the formula (15), V1，V2，，...，VjFor cooperative vehicles, M1，M2，...，MmIs an MEC server;
the task vehicle selects one of the optimal unloading vehicles according to a strategyCarrier node, i.e. slave AtSelecting an action to perform the current task;
the reward function is set as
Figure BDA0003144027830000121
s is the current state, a is the action, the reward function is set as the negative total cost function, in order to make the reward of the whole process maximum, namely the smaller the total cost function;
the unloading strategy pi represents that the vehicle is in the state space StThe basis for taking action is as follows; the merits of a policy are judged by a policy-based merit function, which is divided into: a state cost function and a behavior cost function;
wherein the content of the first and second substances,
the state cost function:
Figure BDA0003144027830000122
(meaning the desired reward that all actions can achieve in state s)
A behavioral cost function: qπ(s，a)＝E(Gt|St＝s，AtA) (meaning the reward that can be obtained by taking action a in state s)
Through the Be | | man equation, the state cost function is converted to:
Figure BDA0003144027830000131
in formula (16), s is the current state, r(s) is the reward function in the current state, γ is the discount factor γ ∈ [0, 1], s ' is the next state, P (s ' | s) is the transition probability of the current state transitioning to the next state, and V (s ') is the value of the next state;
through the Be | man equation, it can Be seen that the value of the current state is equal to the reward to reach the current state plus the value of the next state.
And S4, according to the Markov decision process obtained by conversion in the step S3, in the process, solving the problem of strategy selection in the Markov decision process by using a DQN algorithm, so as to obtain an optimal unloading node, namely the node for providing calculation service for the task vehicle.
This step is the last step in this embodiment, and specifically includes the following steps:
let the behavioral cost function be updated according to the following formula:
Figure BDA0003144027830000132
in the formula (17), R is a reward function,
Figure BDA0003144027830000133
in order to learn the rate of speed,
Figure BDA0003144027830000134
q (s, a) is the current behavioral merit function, Q*(s, a) is the updated behavioral cost function;
the optimal strategy can obtain the optimal behavior cost function through the formula (17), and the strategy is to select the action which can obtain the maximum behavior cost function value in the next state as the optimal action;
in the DQN algorithm, a neural network is used for fitting a Q value, and in each time slot, current condition information is input to a trained model, so that the Q value and corresponding actions, namely corresponding unloading nodes, can be obtained; and searching a new unloading node by using an epsilon-greedy search algorithm according to an epsilon probability, and selecting the unloading node according to a current Q value updating formula according to a 1-epsilon probability.
The above-mentioned embodiments are merely preferred embodiments of the present invention, and the scope of the present invention is not limited thereto, so that variations based on the shape and principle of the present invention should be covered within the scope of the present invention.

Claims (5)

1. A task unloading method based on energy consumption and time delay weighting releases computing resources of cooperative vehicles with idle computing resources in roads to provide computing services for task vehicles, and is characterized by comprising the following steps:
s1, constructing a system model according to communication between the task vehicle and the cooperative vehicle and communication between the task vehicle and an MEC server configured on a roadside unit arranged at the edge of the intersection road to obtain an objective function;
s2, judging whether the time for the task vehicle to solve the task of the task vehicle exceeds the maximum tolerance time delay, if so, performing local calculation, otherwise, entering the step S3;
s3, converting the task allocation process decision problem into a Markov decision process according to the system model constructed in the step S1 and the obtained objective function;
and S4, according to the Markov decision process obtained by conversion in the step S3, in the process, solving the problem of strategy selection in the Markov decision process by using a DQN algorithm, so as to obtain an optimal unloading node, namely the node for providing calculation service for the task vehicle.
2. The method for task offloading based on energy consumption and latency weighting as claimed in claim 1, wherein the step S1 comprises:
s1-1, carrying out scene modeling analysis, including dividing unloading modes into a vehicle-to-vehicle V2V unloading mode and an MEC unloading mode;
s1-2, building a mathematical model for the system model.
3. The method for task offloading based on energy consumption and latency weighting according to claim 2, wherein the step S1-2 specifically includes:
V2V unload mode:
in the unloading mode of V2V, assuming that each task vehicle can find the optimal cooperative vehicle and becomes an unloading pair with it, according to shannon's formula, the transmission rate between the task vehicle and the cooperative vehicle is:
Figure FDA0003144027820000011
in the formula (1), ω isV2VIs bandwidth, PiIn order to transmit the power, the power is transmitted,
Figure FDA0003144027820000012
for channel gain, N0Is noise;
in the unloading mode of V2V, the time delay comprises the time of data uploading and the time of data calculation in the cooperative vehicle, and the return time is ignored because the processed data of the task is small;
time delay of transmission process
Figure FDA0003144027820000021
And energy consumption
Figure FDA0003144027820000022
Respectively as follows:
Figure FDA0003144027820000023
Figure FDA0003144027820000024
in the formula (2), DiIs the size of the data of the task,
Figure FDA00031440278200000214
a set of task vehicles representing the inability to resolve their own tasks within a maximum tolerated delay;
collaborative vehicle computation latency
Figure FDA0003144027820000025
And energy consumption
Figure FDA0003144027820000026
Comprises the following steps:
Figure FDA0003144027820000027
Figure FDA0003144027820000028
in the formula (3), BiRepresenting the number of CPU cycles required to calculate one bit size data,
Figure FDA00031440278200000215
fjfor the computational capability of cooperating vehicles, PjFor coordinating the power consumption of the vehicles, Pj＝μ(fj)2μ is an energy parameter;
so the total delay in the unloading mode of V2V
Figure FDA0003144027820000029
And total energy consumption
Figure FDA00031440278200000210
Comprises the following steps:
Figure FDA00031440278200000211
Figure FDA00031440278200000212
considering the overhead of latency and energy together, the total cost function in the V2V offload mode is:
Figure FDA00031440278200000213
in the formula (5), α is a weight factor of time delay and energy consumption, and the closer α is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
MEC uninstallation mode:
when the MEC unloading mode is selected by the task vehicle, the task vehicle is communicated with the roadside unit through the cellular network and is communicated with an MEC server deployed on the roadside unit;
according to the shannon formula, the transmission rate between the mission vehicle and the MEC server is:
Figure FDA0003144027820000031
in the formula (6), ωMECIs the bandwidth;
in the MEC unloading mode, the time delay and the energy consumption are formed by two parts of task uploading to an MEC server and MEC server calculation;
time delay of task uploading part
Figure FDA0003144027820000032
And energy consumption
Figure FDA0003144027820000033
Respectively, as follows:
Figure FDA0003144027820000034
Figure FDA0003144027820000035
the time delay and the energy consumption calculated by the MEC server are respectively expressed as follows:
Figure FDA0003144027820000036
in the formula (8), fMECFor the computing power of the MEC server:
Figure FDA0003144027820000037
so the total delay in MEC offload mode
Figure FDA0003144027820000038
And energy consumption
Figure FDA0003144027820000039
Expressed as:
Figure FDA00031440278200000310
Figure FDA00031440278200000311
the time delay and energy overhead are comprehensively considered, so the total cost function in the MEC offload mode is:
Figure FDA00031440278200000312
in the formula (11), α is a weight factor of time delay and energy consumption; the closer alpha is to 1, the more emphasis is placed on time delay, otherwise, the more emphasis is placed on energy consumption;
then the cost function for the entire task is:
Figure FDA00031440278200000313
in equation (12), β ═ {0, 1} is an unloading decision parameter, and if the task vehicle selects the V2V unloading mode to unload, β ═ 0, and otherwise, β ═ 1;
after the cost function, the objective function becomes as follows:
Figure FDA0003144027820000041
C1：
Figure FDA0003144027820000042
in equation (13), C1 represents a constraint in the sense that neither V2V nor MEC offload mode computations can exceed the maximum tolerable latency of a task.
4. The method for unloading tasks based on energy consumption and time delay weighting as claimed in claim 1, wherein in step S3, the step of converting the task allocation process decision problem into a markov decision process specifically includes:
when a task vehicle carries out task unloading, tasks of the task vehicle are continuously unloaded to an MEC server or a cooperative vehicle, the state of a system is composed of the data size of the current task, the task complexity, the maximum tolerance time delay and the communication state between the vehicle and an unloading point, and the state space of the system is expressed as follows:
St＝{Ti′V2V1，V2V2，...，V2Vj，MEC1，MEC2，...，MECm} (14)
in formula (14), TiFor task type, V2V1，V2V2，...，V2VjFor the vehicle to communicate in the V2V unloaded mode, MEC1，MEC2，...，MECmPerforming a credit communication state in an MEC unloading mode for the vehicle;
and the motion space is composed of selectable unloading points, and the motion space is expressed as:
At＝{V1，V2，，...，Vj，M1，M2，...，Mm} (15)
in the formula (15), V1，V2，，...，VjFor cooperative vehicles, M1，M2，...，MmIs an MEC server;
the task vehicle selects one optimal unloading node according to a strategy and also selects the optimal unloading nodeIs from AtSelecting an action to perform the current task;
the reward function is set as
Figure FDA0003144027820000043
s is the current state, a is the action, the reward function is set as the negative total cost function, in order to make the reward of the whole process maximum, namely the smaller the total cost function;
the unloading strategy pi represents that the vehicle is in the state space StThe basis for taking action is as follows; the merits of a policy are judged by a policy-based merit function, which is divided into: a state cost function and a behavior cost function; wherein the content of the first and second substances,
the state cost function:
Figure FDA0003144027820000051
a behavioral cost function: qπ(s，a)＝E(Gt|St＝s，At＝a)；
Through the Bellman equation, the state cost function transforms to:
Figure FDA0003144027820000052
in formula (16), s is the current state, r(s) is the reward function in the current state, γ is the discount factor γ ∈ [0, 1], s ' is the next state, P (s ' | s) is the transition probability of the current state transitioning to the next state, and V (s ') is the value of the next state;
from the Bellman equation, it can be seen that the value of the current state is equal to the reward for reaching the current state plus the value of the next state.
5. The method for unloading tasks based on energy consumption and time delay weighting as claimed in claim 4, wherein in the step S4, the method for solving the problem of strategy selection in the Markov decision process by applying the DQN algorithm specifically comprises:
let the behavioral cost function be updated according to the following formula:
Figure FDA0003144027820000053
in the formula (17), R is a reward function,
Figure FDA0003144027820000054
in order to learn the rate of speed,
Figure FDA0003144027820000055
q (s, a) is the current behavioral merit function, Q*(s, a) is the updated behavioral cost function;
the optimal strategy can obtain the optimal behavior cost function through the formula (17), and the strategy is to select the action which can obtain the maximum behavior cost function value in the next state as the optimal action;
in the DQN algorithm, a neural network is used for fitting a Q value, and in each time slot, current condition information is input to a trained model, so that the Q value and corresponding actions, namely corresponding unloading nodes, can be obtained; and searching a new unloading node by using an epsilon-greedy search algorithm according to an epsilon probability, and selecting the unloading node according to a current Q value updating formula according to a 1-epsilon probability.
CN202110749538.2A 2021-07-01 2021-07-01 Task unloading method based on energy consumption and time delay weighting Pending CN113542376A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110749538.2A CN113542376A (en) 2021-07-01 2021-07-01 Task unloading method based on energy consumption and time delay weighting

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110749538.2A CN113542376A (en) 2021-07-01 2021-07-01 Task unloading method based on energy consumption and time delay weighting

Publications (1)

Publication Number Publication Date
CN113542376A true CN113542376A (en) 2021-10-22

Family

ID=78097636

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110749538.2A Pending CN113542376A (en) 2021-07-01 2021-07-01 Task unloading method based on energy consumption and time delay weighting

Country Status (1)

Country Link
CN (1) CN113542376A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114264220A (en) * 2021-12-23 2022-04-01 湖南大学 Method for accurately sensing and detecting relative displacement of mobile equipment
CN115243217A (en) * 2022-07-07 2022-10-25 中山大学 DDQN-based end edge cloud collaborative scheduling method and system in Internet of vehicles edge environment
CN115334076A (en) * 2022-07-08 2022-11-11 电子科技大学 Service migration method and system of edge server and edge server equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111885155A (en) * 2020-07-22 2020-11-03 大连理工大学 Vehicle-mounted task collaborative migration method for vehicle networking resource fusion
CN111918245A (en) * 2020-07-07 2020-11-10 西安交通大学 Multi-agent-based vehicle speed perception calculation task unloading and resource allocation method
US20210007023A1 (en) * 2020-09-17 2021-01-07 Intel Corporation Context aware handovers
CN112235835A (en) * 2020-09-09 2021-01-15 广州大学 Dynamic unloading method for time-varying mobile edge computing network
CN112822234A (en) * 2020-12-29 2021-05-18 华北电力大学 Task unloading method based on deep reinforcement learning in Internet of vehicles
CN112995289A (en) * 2021-02-04 2021-06-18 天津理工大学 Internet of vehicles multi-target computing task unloading scheduling method based on non-dominated sorting genetic strategy

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111918245A (en) * 2020-07-07 2020-11-10 西安交通大学 Multi-agent-based vehicle speed perception calculation task unloading and resource allocation method
CN111885155A (en) * 2020-07-22 2020-11-03 大连理工大学 Vehicle-mounted task collaborative migration method for vehicle networking resource fusion
CN112235835A (en) * 2020-09-09 2021-01-15 广州大学 Dynamic unloading method for time-varying mobile edge computing network
US20210007023A1 (en) * 2020-09-17 2021-01-07 Intel Corporation Context aware handovers
CN112822234A (en) * 2020-12-29 2021-05-18 华北电力大学 Task unloading method based on deep reinforcement learning in Internet of vehicles
CN112995289A (en) * 2021-02-04 2021-06-18 天津理工大学 Internet of vehicles multi-target computing task unloading scheduling method based on non-dominated sorting genetic strategy

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JINKYO JEONG 等: "Deep Reinforcement Learning-based Task Offloading Decision in the Time Varying Channel", 《2021 INTERNATIONAL CONFERENCE ON ELECTRONICS, INFORMATION, AND COMMUNICATION (ICEIC)》 *
童钊 等: "移动边缘计算中多约束下的任务卸载和资源分配算法", 《计算机工程与科学》 *
赵海涛等: "基于DQN的车载边缘网络任务分发卸载算法", 《通信学报》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114264220A (en) * 2021-12-23 2022-04-01 湖南大学 Method for accurately sensing and detecting relative displacement of mobile equipment
CN115243217A (en) * 2022-07-07 2022-10-25 中山大学 DDQN-based end edge cloud collaborative scheduling method and system in Internet of vehicles edge environment
CN115243217B (en) * 2022-07-07 2023-07-18 中山大学 DDQN-based terminal Bian Yun cooperative scheduling method and system in Internet of vehicles edge environment
CN115334076A (en) * 2022-07-08 2022-11-11 电子科技大学 Service migration method and system of edge server and edge server equipment

Similar Documents

Publication Publication Date Title
Liu et al. Resource allocation for edge computing in IoT networks via reinforcement learning
CN113542376A (en) Task unloading method based on energy consumption and time delay weighting
Zhang et al. A new task offloading algorithm in edge computing
Li et al. Energy-aware task offloading with deadline constraint in mobile edge computing
CN114650228B (en) Federal learning scheduling method based on calculation unloading in heterogeneous network
CN116541106B (en) Computing task unloading method, computing device and storage medium
Jiang et al. A reinforcement learning-based computing offloading and resource allocation scheme in F-RAN
Liu et al. Energy-efficient joint computation offloading and resource allocation strategy for isac-aided 6g v2x networks
Qi et al. Deep reinforcement learning based task scheduling in edge computing networks
Luo et al. Fog-enabled joint computation, communication and caching resource sharing for energy-efficient IoT data stream processing
Li et al. DQN-enabled content caching and quantum ant colony-based computation offloading in MEC
Dong et al. Content caching-enhanced computation offloading in mobile edge service networks
Qin et al. User‐Edge Collaborative Resource Allocation and Offloading Strategy in Edge Computing
Chen et al. Twin delayed deep deterministic policy gradient-based intelligent computation offloading for IoT
Lin et al. A deep reinforcement learning based computation offloading with mobile vehicles in vehicular edge computing
Wang Edge artificial intelligence-based affinity task offloading under resource adjustment in a 5G network
KR102385087B1 (en) Method for latency minimization in a fuzzy-based mobile edge orchestrator and system using same
Chuang et al. A popularity-aware and energy-efficient offloading mechanism in fog computing
Shi et al. Energy-efficient UAV-enabled computation offloading for industrial internet of things: a deep reinforcement learning approach
Li et al. Joint offloading and resource allocation with partial information for multi-user edge computing
CN114741191A (en) Multi-resource allocation method for compute-intensive task relevance
Cui et al. Resource-Efficient DNN Training and Inference for Heterogeneous Edge Intelligence in 6G
Ren et al. An Approximation Algorithm for Joint Data Uploading and Task Offloading in IoV
Zhao et al. DRL Connects Lyapunov in Delay and Stability Optimization for Offloading Proactive Sensing Tasks of RSUs
Li et al. Edge computing offloading strategy based on dynamic non-cooperative games in D-IoT

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20211022

RJ01 Rejection of invention patent application after publication