CN116360883A - Combined optimization method for unloading of Internet of vehicles computing tasks - Google Patents
Combined optimization method for unloading of Internet of vehicles computing tasks Download PDFInfo
- Publication number
- CN116360883A CN116360883A CN202310227797.8A CN202310227797A CN116360883A CN 116360883 A CN116360883 A CN 116360883A CN 202310227797 A CN202310227797 A CN 202310227797A CN 116360883 A CN116360883 A CN 116360883A
- Authority
- CN
- China
- Prior art keywords
- task
- unloading
- internet
- vehicles
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 238000005457 optimization Methods 0.000 title claims abstract description 34
- 238000004364 calculation method Methods 0.000 claims abstract description 34
- 230000005540 biological transmission Effects 0.000 claims abstract description 29
- 230000008569 process Effects 0.000 claims abstract description 17
- 238000005516 engineering process Methods 0.000 claims abstract description 10
- 238000005070 sampling Methods 0.000 claims abstract description 6
- 230000006870 function Effects 0.000 claims description 24
- 239000011159 matrix material Substances 0.000 claims description 24
- 230000009471 action Effects 0.000 claims description 23
- 238000004891 communication Methods 0.000 claims description 15
- 238000001228 spectrum Methods 0.000 claims description 12
- 230000001413 cellular effect Effects 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000013144 data compression Methods 0.000 claims description 6
- 230000006855 networking Effects 0.000 claims description 6
- 230000008447 perception Effects 0.000 claims description 6
- 238000013468 resource allocation Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 5
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000002787 reinforcement Effects 0.000 claims description 4
- 101001093748 Homo sapiens Phosphatidylinositol N-acetylglucosaminyltransferase subunit P Proteins 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 239000003795 chemical substances by application Substances 0.000 claims description 3
- 238000000354 decomposition reaction Methods 0.000 claims description 3
- 238000005259 measurement Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 230000018109 developmental process Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44594—Unloading
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a combined optimization method for unloading a calculation task of the Internet of vehicles; firstly, deploying an end-side-cloud cooperative task unloading framework in an edge environment of the Internet of vehicles, defining a task parameter set, and establishing a task unloading model; based on the task execution time delay calculation model, a task unloading strategy is determined; determining an objective function and constraint conditions of the optimization problem according to the service delay; the compressed sensing technology is introduced to reduce the influence of the task unloading data amount on the task execution time delay, and a large amount of redundant data is eliminated through compressed sampling; converting the optimization problem into a Markov decision process, solving the Markov decision process by using a deep Q network algorithm, and solving an optimal unloading decision of the task; the method is suitable for the Internet of vehicles environment with low time delay, high reliability, high efficiency and other requirements, provides better support for high-bandwidth and low-time delay services, solves the problems of remote transmission of cloud computing and limited edge computing resources, and avoids the problems of overlarge state space, overlarge task data unloading amount and the like.
Description
Technical Field
The invention relates to the technical field of internet of vehicles edge computing task unloading, in particular to a joint optimization method for internet of vehicles computing task unloading.
Background
In recent years, rapid developments in artificial intelligence technology and cellular networks have prompted the explosive development of the internet of vehicles, which are transitioning from vehicles to intelligent terminals. As a product of the modern industry and the rapid development of mobile communication, the internet of vehicles technology can provide various intelligent services for users, such as vehicle-mounted entertainment services, emergency avoidance, path planning, collision early warning and the like. Meanwhile, in order to cope with complex road environments, vehicles need to frequently perform information interaction, and meanwhile, a large amount of data is required to be collected through an on-board visual sensor to sense the surrounding environment. However, the limited computing and storage resources of the vehicle-mounted terminal cannot meet the requirements of low latency and high reliability of the vehicle-mounted application.
Mobile edge computing (Mobile Edge Computing, MEC) technology was introduced into the internet of vehicles as an effective way to address the above problems. The mobile edge computing technology sinks computing and storage resources to one side close to a user, the vehicle-mounted terminal processes the computing intensive and time delay sensitive services or data by unloading the computing intensive and time delay sensitive services or data to the edge server, and the computing result is transmitted back to the vehicle-mounted terminal after the processing is completed, so that the execution time delay of a task can be effectively reduced.
At present, a great deal of research is applied to the edge calculation in the scene of the internet of vehicles, and different mechanisms are adopted in the research to optimize the unloading scheme of the calculation task, so that the system overhead is effectively reduced. However, these studies have not focused on the impact of the amount of task data on the execution delay.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a combined optimization method for unloading the computing tasks of the Internet of vehicles, which aims to solve the problem that a large amount of perceived data cannot be processed due to limited computing resources in the existing vehicle edge network.
A combined optimization method for unloading of internet of vehicles computing tasks specifically comprises the following steps:
firstly, providing a task unloading framework of an end-side-cloud cooperative service, wherein the framework is divided into three layers, and consists of a perception layer, a decision layer and an execution layer; the sensing layer performs data compression on video data acquired by the vehicle-mounted camera through sparse conversion to obtain compressed data; the decision layer determines the optimal unloading strategy of the compressed data through the exploration of the environment and an experience playback mechanism by the agent; the execution layer carries out data reconstruction on the received compressed data, restores the original data, and analyzes and understands the video content; then, the processing result is returned to the vehicle-mounted terminal;
the joint optimization method for the task offloading of the Internet of vehicles comprises the following steps of:
step 1: deploying a task unloading framework of an end-side-cloud cooperative service in an edge environment of the Internet of vehicles, establishing a task unloading model, and defining an unloading task parameter set;
the task unloading model is established, and an unloading task parameter set is defined; the method specifically comprises the following steps:
first defining unloading task as a quaternary feature group
Representing computing task Ta n The size of the input data of (a); s is S n Indicating completion of task Ta n The amount of computing resources required; />
Representing task Ta n Executing the maximum time delay allowed by completion; alpha represents task Ta n Is a decision to unload;
defining a communication link between the vehicle-mounted terminal and the RSU through a DSRC technology, uploading a calculation task to the RSU server by using orthogonal spectrum resources by the vehicle, representing the available spectrum resources as a set K= {1,2, &.. n Maximum transmission rate Ra offloaded to a roadside unit v,r The method comprises the following steps:
wherein B is r K is the total bandwidth of the available spectrum between the vehicle and the RSU r P is the available orthogonal sub-channel v,r For the transmission power of the vehicle during the task unloading, h v,r Sigma for channel gain of mission vehicles and RSU 2 Power for white gaussian noise;
defining a communication link between an on-board terminal and a central cloud through a cellular network, and defining a task Ta n Maximum transmission rate Ra offloaded to the central cloud v,c The method comprises the following steps:
wherein B is c K is the total bandwidth of available frequency spectrum between vehicle and cloud server c P is the available orthogonal sub-channel v,c The transmission power h of the vehicle-mounted terminal when the task is unloaded v,c For the channel gain, sigma, of the wireless transmission channel between the mission vehicle and the cloud server 2 Power for white gaussian noise;
step 2: based on the task unloading model, a service delay calculation model is established, and a task unloading strategy is determined;
the task unloading model is based, a service delay calculation model is established, and a task unloading strategy is established, comprising the following steps:
definition task Ta n Is to be executed locally
The method comprises the following steps:
wherein F is v Representing the computing power of the vehicle-mounted terminal;
definition task Ta n Offloading transmission delay to RSU server
The method comprises the following steps:
backhaul for defining RSU computing modelThe time delay is as follows:
0＜β 1 ＜1；
definition task Ta n Computation latency generated at RSU server
The method comprises the following steps:
wherein F is r Representing the computing power of the RSU server;
the total time delay of the RSU calculation model is defined as follows:
definition task Ta n Transmission delay generated by offloading to cloud server via cellular network
The method comprises the following steps:
the return delay of the cloud computing model is defined as follows:
0＜β 2 ＜1；
definition task Ta n Calculation time delay generated by calculation at cloud server
The method comprises the following steps:
the total time delay of the cloud computing model is defined as follows:
the task unloading situation is represented by a 0-1 variable, and the task unloading strategy is determined as follows: the task requirement of each Internet of vehicles user can only be unloaded to one of the local, edge server and center cloud for execution;
defining an offloading decision to satisfy: alpha v ,α r ,α c ∈(0,1),α v +α r +α c ＝1，α v =1 means that the task performs local computation, α r =1 means task offloading to RSU server for computation, α c =1, the task is offloaded to the cloud server for calculation;
step 3: determining an objective function and constraint conditions of the optimization problem according to the service delay calculation model and the task unloading strategy; the objective function is:
the constraint conditions are as follows:
C3:α v ,α r ,α c ∈(0,1),α v +α r +α c ＝1；C4:β 1 ，β 2 ∈(0,1)；C5：P v,r ,P v,c ≤P max ；
step 4: sparse transformation is carried out on video data acquired by a vehicle vision sensor in a task unloading architecture perception layer to obtain compressed data;
when the video data is not sparse or not sparse enough, using a data dictionary to perform sparse representation; taking the video data as a sample set, and learning the sample to obtain an overcomplete dictionary D reflecting the characteristics of the sample, and enabling the representation of the overcomplete dictionary D to be more sparse;
the mathematical expression of K-SVD dictionary learning is:
Y＝[y 1 ,y 2 ,y 3 ,...,y N ]each y of (3) i Represents one sample, X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]As a sparse coefficient matrix, T represents sparsity;
the K-SVD dictionary learning specifically comprises the following steps:
initializing: randomly selecting a part of samples from the sample set, initializing an overcomplete dictionary, and setting iteration times J=1;
the following procedure is repeated until
If not, continuing the iteration process;
sparse coding: using OMP algorithm to calculate sparse coefficient to obtain sparse vector X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]For each sample y i The following formula is satisfied:
dictionary updating: the sparse vector X is fixed and updated column by column for each atom in D as follows:
wherein d j For the j-th column of the overcomplete dictionary D,
column j, E of X k To remove d k The error after that;
the dictionary updating comprises the following specific steps:
defining atoms in a sample
Using the formula
To calculate the value of the error matrix;
from error matrix E k Is selected to be only equal to omega k Corresponding columns, obtain E' k ；
Using formula E' k ＝U∑V T To E' k Singular value decomposition is performed, and then atom d in the dictionary is updated by selecting the first column in the selection U k =u (·, 1), and let
Update j=j+1;
video data acquired by a vehicle-mounted camera in a time slot t are expressed as an m X n-dimensional matrix X:
the column vector represents the time sequence of each visual sensor, and the row vector represents the data acquired by a plurality of visual sensors in the same time slot;
to facilitate matrix measurement, the elements of matrix X are represented in column order, yielding an n×1 (n=m·n) dimensional column vector:
vec(X)＝[x 11 ,x 12 ,...,x 1m ,x 21 ,x 22 ,...,x nm ] T
multiplying the observation matrix phi with the data vec (X) to complete data compression and obtain compressed data y i The expression is:
y i ＝Φ·vec(X)
step 5: converting the optimization problem into a Markov decision process at a task unloading architecture decision layer, and solving the Markov decision process by using a deep Q network to obtain an optimal unloading decision of a computing task;
determining the environment, state, action space and rewards of the Markov decision process;
the environment comprises Internet of vehicles information, task amount, communication resources and computing resources of a server;
the state comprises information such as computing resources, communication resources, task unloading data amount, task maximum allowable time delay, vehicle transmitting power, task maximum transmission rate, RSU computing capacity, RSU coverage, cloud server computing capacity, cloud server coverage and the like of all Internet of vehicles users in an environment in a specific time slot t;
the action space is provided with a task unloading decision, a computing resource allocation strategy and a communication resource allocation strategy decision;
the rewards include rewards for performing actions in a target direction and a non-target direction, and an objective function of the internet of vehicles system is optimized by using a DQN algorithm of deep reinforcement learning; the utilizing DQN algorithm specifically comprises the following steps:
step S1: initializing an action-cost function Q(s) of a size of an empirical playback pool storage space and a random weight θ t ,a t The method comprises the steps of carrying out a first treatment on the surface of the θ) and a target action-cost function Q (s t ,a t ；θ * ) Wherein θ is * ＝θ；
Step S2: after receiving the task unloading request of the vehicle-mounted terminal, the vehicle networking system collects the state information of the system, namely, inputs the information such as the task data volume, the maximum allowable time delay, the task transmission rate and the like into the network to obtain the initial state s of the system t ；
Step S3: for each time slot t, an action a is selected with epsilon probability that maximizes the Q value t Picking the next random action a 'with a probability of 1-epsilon' t ；
Step S4: according to action a t Real-time interaction with the environment to enter the next state s t+1 And is obtained according to the objective functionTo prize value r t ；
Step S5: will offload decision action a t System state s t And prize value r t Storing in an experience playback pool, uniformly and randomly sampling from the experience playback pool by utilizing a depth Q network, constructing an error function, and updating a network parameter theta by adopting a back propagation algorithm; the deep Q network iterates until the network parameters theta converge to obtain an optimal unloading decision;
step 6: at a task unloading architecture execution layer, the sparse data transmitted by the vehicle-mounted terminal is reconstructed to be original data, and the original data is transmitted to an RSU or a cloud server for analyzing and understanding video content;
the execution layer knows the observation matrix phi, the sparse basis phi and the compressed data y i The execution layer only needs to multiply the sparse basis ψ with the sparse Θ to obtain the original data X, which is expressed as: x=ψΘ;
the original data X is solved and converted into: min θ 0 s.t.y＝ΦψΘ。
The invention has the beneficial effects that:
the invention discloses a joint optimization method for unloading of a vehicle networking computing task, which comprises the steps of deploying an end-side-cloud cooperative service unloading framework in a vehicle networking edge environment, defining a task parameter set, and establishing a task unloading model; based on the task unloading model, a service delay calculation model is established, and a task unloading strategy is determined; determining an objective function and constraint conditions of the optimization problem according to the service delay; in order to reduce the influence of the task unloading data amount on the task execution time delay, a compressed sensing technology is introduced, and a large amount of redundant data is eliminated through compressed sampling; and converting the optimization problem into a Markov decision process, solving the Markov decision process by using a deep Q network algorithm, and solving the optimal unloading decision of the task.
The invention constructs an end-side-cloud cooperative service unloading architecture and a service delay calculation model, and eliminates a large amount of redundant data in a sampling stage by introducing a compressed sensing technology; by converting the optimization problem into a Markov decision process and solving using the DQN algorithm, the algorithms presented herein improve the performance of the short bursts by about 7%, 17% and 25% over the DQN, Q learning algorithms and random off-load when changing the task data size. The method is not only suitable for the car networking environment with low time delay, high reliability, high efficiency and the like, provides better support for high-bandwidth and low-time delay service, solves the defects of remote transmission of cloud computing and limited edge computing resources, and avoids the problems of overlarge state space, overlarge task data unloading amount and the like in an algorithm.
Drawings
FIG. 1 is a schematic flow diagram of a combined optimization method for unloading a computing task of the Internet of vehicles according to an embodiment of the invention;
fig. 2 is a schematic diagram of an end-side-cloud cooperative internet of vehicles task offloading architecture according to an embodiment of the present invention;
FIG. 3 is a flow chart of a DQN algorithm provided by an embodiment of the invention;
FIG. 4 is a graph comparing total overhead to task data size for various algorithm systems.
Detailed Description
The invention is further described below with reference to the drawings and examples;
a combined optimization method for unloading of internet of vehicles computing tasks specifically comprises the following steps:
firstly, providing a task unloading framework of an end-side-cloud cooperative service, wherein the framework is divided into three layers, and consists of a perception layer, a decision layer and an execution layer; the sensing layer performs data compression on video data acquired by the vehicle-mounted camera through sparse conversion to obtain compressed data; the decision layer determines the optimal unloading strategy of the compressed data through the exploration of the environment and an experience playback mechanism by the agent; the execution layer carries out data reconstruction on the received compressed data, restores the original data, and analyzes and understands the video content; then, the processing result is returned to the vehicle-mounted terminal;
the joint optimization method for unloading the computing tasks of the Internet of vehicles is realized based on the task unloading architecture of the end-side-cloud cooperative service, as shown in fig. 1, and specifically comprises the following steps:
s10, deploying a task unloading framework of the end-side-cloud cooperative service in an edge environment of the Internet of vehicles, defining a task parameter set as shown in fig. 2, and establishing a task unloading model.
Defining offloading tasks as a quad
Representing computing task Ta n The size of the input data of (a); s is S n Indicating completion of task Ta n The amount of computing resources required; />
Representing task Ta n Executing the maximum time delay allowed by completion; alpha represents task Ta n Is a decision to unload.
Defining a communication link between the vehicle-mounted terminal and the RSU through a DSRC technology, uploading a calculation task to the RSU server by using orthogonal spectrum resources by the vehicle, representing the available spectrum resources as a set K= {1,2, &.. n Maximum transmission rate Ra offloaded to a roadside unit v,r The method comprises the following steps:
B r k is the total bandwidth of the available spectrum between the vehicle and the RSU r P is the available orthogonal sub-channel v,r For the transmission power of the vehicle during the task unloading, h v,r Channel gains for the mission vehicle and RSU. Sigma (sigma) 2 Is the power of gaussian white noise.
Defining a communication link between an on-board terminal and a central cloud through a cellular network, and defining a task Ta n Maximum transmission rate Ra offloaded to the central cloud v,c The method comprises the following steps:
B c k is the total bandwidth of available frequency spectrum between vehicle and cloud server c P is the available orthogonal sub-channel v,c The transmission power h of the vehicle-mounted terminal when the task is unloaded v,c The channel gain of the wireless transmission channel between the task vehicle and the cloud server is obtained. Sigma (sigma) 2 Is the power of gaussian white noise.
S20, based on the task unloading model, an execution time delay calculation model is established, and a task unloading strategy is determined;
definition task Ta n Is to be executed locally
Wherein F is v The computing capability of the vehicle-mounted terminal is represented, and specifically:
definition task Ta n Offloading transmission delay to RSU server
The method comprises the following steps:
the return delay of the RSU calculation model is defined as follows:
0＜β 1 ＜1；
definition task Ta n Computation latency generated at RSU server
Wherein F is r Representing the computing power of the RSU server; the method comprises the following steps:
the total time delay of the RSU calculation model is defined as follows:
definition task Ta n Transmission delay generated by offloading to cloud server via cellular network
The method comprises the following steps:
the return delay of the cloud computing model is defined as follows:
0＜β 2 ＜1；
definition task Ta n Calculation time delay generated by calculation at cloud server
The method comprises the following steps:
the total time delay of the cloud computing model is defined as follows:
the task unloading situation is represented by a 0-1 variable, and the task unloading strategy is determined as follows: the task demand of each Internet of vehicles user can be only unloaded to one of the local, edge server and center cloud for execution;
defining the offloading decision should satisfy: alpha v ,α r ,α c ∈(0,1),α v +α r +α c ＝1，α v =1 means that the task performs local computation, α r =1 meansTask unloading to RSU server for calculation, alpha c =1, the task is offloaded to the cloud server for calculation;
s30, determining an objective function and constraint conditions of the optimization problem according to the service delay calculation model and the task unloading strategy; the objective function is:
the constraint conditions are as follows:
C3:α v ,α r ,α c ∈(0,1),α v +α r +α c ＝1；C4:β 1 ，β 2 ∈(0,1)；C5：P v,r ,P v,c ≤P max ；
s40, in the perception layer, sparse transformation is carried out on the collected video data by the vehicle vision sensor, and sparse data are obtained. Fig. 2 provides a schematic diagram of an end-side-cloud collaborative internet of vehicles task offloading architecture, where the signal is not sparse or sparse enough, it is common practice to use a data dictionary for sparse representation.
And learning the video data training sample to obtain an overcomplete dictionary D capable of reflecting the characteristics of the training sample, and enabling the signals to be sparser in representation.
The mathematical expression of K-SVD dictionary learning is:
Y＝[y 1 ,y 2 ,y 3 ,...,y N ]each y of (3) i Represents one sample, X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]For the sparse coefficient matrix, T represents sparsity.
The K-SVD dictionary learning specifically comprises the following steps:
initializing: randomly selecting a part of samples from the sample set, initializing an overcomplete dictionary, and setting j=1;
the following procedure is repeated until
Small enough, otherwise, another iterative process is entered.
Sparse coding: using OMP algorithm to calculate sparse coefficient to obtain sparse vector X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]For each sample y i The following formula is satisfied:
dictionary updating: the sparse vector X is fixed and updated column by column for each atom in D as follows:
d j for the j-th column of the overcomplete dictionary D,
column j, E of X k To remove d k The error after that;
the dictionary updating comprises the following specific steps:
defining atoms in a sample
Using the formula
To calculate the value of the error matrix;
from error matrix E k Is selected to be only equal to omega k Corresponding columns, obtain E' k ；
Using formula E' k ＝U∑V T To E' k Singular value decomposition is performed, and then atom d in the dictionary is updated by selecting the first column in the selection U k =u (·, 1), and let
Update j=j+1;
video data acquired by a vehicle vision sensor in a time slot t are expressed as an m X n dimensional matrix X:
the column vector represents the time series of each visual sensor and the row vector represents the data collected by multiple visual sensors during the same time slot.
To facilitate matrix measurement, the elements of matrix X are represented in column order, yielding an n×1 (n=m·n) dimensional column vector:
vec(X)＝[x 11 ,x 12 ,...,x 1m ,x 21 ,x 22 ,...,x nm ] T
multiplying the observation matrix phi with the data vec (X) to complete data compression and obtain compressed data y i The expression is:
y i ＝Φ·vec(X)
s50, converting the optimization problem into a Markov decision process in a decision layer, and solving the Markov decision process by using a deep Q network to obtain the optimal unloading decision of the calculation task. Fig. 3 provides a deep Q network schematic.
Determining the environment, state, action space and rewards of the Markov decision process;
the environment comprises Internet of vehicles information, task amount, communication resources and computing resources of a server;
the state comprises information such as computing resources, communication resources, task unloading data amount, task maximum allowable time delay, vehicle transmitting power, task maximum transmission rate, RSU computing capacity, RSU coverage, cloud server computing capacity, cloud server coverage and the like of all Internet of vehicles users in an environment in a specific time slot t;
the action space is provided with a task unloading decision, a computing resource allocation strategy and a communication resource allocation strategy decision;
the rewards include rewards for performing actions towards target and non-target directions, and the objective function of the internet of vehicles system is optimized by using a DQN algorithm of deep reinforcement learning, and the DQN algorithm specifically comprises the following steps:
a. initializing an action-cost function Q(s) of a size of an empirical playback pool storage space and a random weight θ t ,a t The method comprises the steps of carrying out a first treatment on the surface of the θ) and a target action-cost function Q (s t ,a t ；θ * ) Wherein θ is * ＝θ；
b. After receiving the task unloading request of the vehicle-mounted terminal, the vehicle networking system collects the state information of the system, namely, inputs the information such as the task data volume, the maximum allowable time delay, the task transmission rate and the like into the network to obtain the initial state s of the system t ；
c. For each time slot t, an action a is selected with epsilon probability that maximizes the Q value t Picking the next random action a 'with a probability of 1-epsilon' t ；
d. According to action a t Real-time interaction with the environment to enter the next state s t+1 And obtains a reward value r according to the objective function t ；
e. Will offload decision action a t System state s t And prize value r t Storing in an experience playback pool, uniformly and randomly sampling from the experience playback pool by utilizing a depth Q network, constructing an error function, and updating a network parameter theta by adopting a back propagation algorithm;
f. and iterating the deep Q network until the network parameters theta are converged to obtain an optimal unloading decision.
And solving the optimization problem of the objective function by solving the optimal values of the vehicle task unloading strategy and the RSU cooperative strategy. The problem is a mixed integer nonlinear NP-Hard problem, and as the number of vehicle terminals and the task size increase, the computational complexity increases rapidly. It is relatively difficult to solve this problem using conventional numerical optimization methods. The DON algorithm core idea is to use a neural network to replace a Q table to store information, so that the DON algorithm is more suitable for the high-dimensional situation, and the time delay sum of the vehicle in the system for completing one application is minimized.
DON uses neural network to approximate action-cost function, and uses Q network to traverse various actions under current state to make real-time interaction with environment, and its action, state value and rewarding value are stored in the dry experience playback pool. And repeatedly training the Q network through a plurality of iterative processes by the Q learning algorithm, and finally obtaining the optimal unloading strategy. Meanwhile, the DON introduces a target Q network on the basis of the original Q network, and the DON has the same initial weight as the original Q network structure, but the original Q network is updated every iteration, and the target Q network is updated at intervals, so that the correlation between the target Q value and the current Q value is reduced, and the algorithm efficiency is improved. The optimal problem shown by the objective function is solved by using the DQN algorithm in the deep reinforcement learning, the optimal unloading and cooperative strategy is obtained, the strategy self-updating can be realized according to the past experience in the space-time dynamic change environment, the total time delay of all tasks and the failure rate of calculation are minimized on the premise of meeting the task processing time delay constraint, the application request completion time is effectively reduced, and the use experience of a vehicle terminal user is improved.
S60, at an execution layer, the sparse data transmitted by the vehicle-mounted terminal is reconstructed to be original data, and the original data is transmitted to an RSU or a cloud server for execution. The execution layer knows the observation matrix phi, the sparse basis phi and the compressed data y i The execution layer only needs to multiply the sparse basis ψ with the sparse Θ to obtain the original data X, which is expressed as: x=ψΘ; the original data X is solved and converted into: min θ 0 s.t.y＝ΦψΘ。
As shown in fig. 4, the method of the invention is proved to have about 7%, 17% and 25% improvement in short-term performance compared with DQN, Q learning algorithm and random unloading, which are the relation diagrams of the total overhead of the system and the task data volume.
Claims (10)
1. The combined optimization method for the task unloading of the Internet of vehicles is characterized in that a task unloading framework of an end-side-cloud cooperative service is provided, and the framework is divided into three layers, namely a perception layer, a decision layer and an execution layer; the sensing layer performs data compression on video data acquired by the vehicle-mounted camera through sparse conversion to obtain compressed data; the decision layer determines the optimal unloading strategy of the compressed data through the exploration of the environment and an experience playback mechanism by the agent; the execution layer carries out data reconstruction on the received compressed data, restores the original data, and analyzes and understands the video content; and then returning the processing result to the vehicle-mounted terminal.
2. The joint optimization method for unloading the internet of vehicles computing tasks according to claim 1, which is characterized by comprising the following steps:
step 1: deploying a task unloading framework of an end-side-cloud cooperative service in an edge environment of the Internet of vehicles, establishing a task unloading model, and defining an unloading task parameter set;
step 2: based on the task unloading model, a service delay calculation model is established, and a task unloading strategy is determined;
step 3: determining an objective function and constraint conditions of the optimization problem according to the service delay calculation model and the task unloading strategy;
step 4: sparse transformation is carried out on video data acquired by a vehicle vision sensor in a task unloading architecture perception layer to obtain compressed data;
when the video data is not sparse or not sparse enough, using a data dictionary to perform sparse representation; taking the video data as a sample set, and learning the sample to obtain an overcomplete dictionary D reflecting the characteristics of the sample, and enabling the representation of the overcomplete dictionary D to be more sparse;
step 5: converting the optimization problem into a Markov decision process at a task unloading architecture decision layer, and solving the Markov decision process by using a deep Q network to obtain an optimal unloading decision of a computing task;
step 6: and at the task unloading architecture execution layer, the sparse data transmitted by the vehicle-mounted terminal is reconstructed to be original data, and the original data is transmitted to an RSU or a cloud server for analyzing and understanding the video content.
3. The joint optimization method for task offloading of internet of vehicles according to claim 2, wherein in step 1, a task offloading model is built and an offloading task parameter set is defined; the method specifically comprises the following steps:
first defining unloading task as a quaternary feature group
Representing computing task Ta n The size of the input data of (a); s is S n Indicating completion of task Ta n The amount of computing resources required; />
Representing task Ta n Executing the maximum time delay allowed by completion; alpha represents task Ta n Is a decision to unload;
defining a communication link between the vehicle-mounted terminal and the RSU through a DSRC technology, uploading a calculation task to the RSU server by using orthogonal spectrum resources by the vehicle, representing the available spectrum resources as a set K= {1,2, &.. n Maximum transmission rate Ra offloaded to a roadside unit v,r The method comprises the following steps:
wherein B is r K is the total bandwidth of the available spectrum between the vehicle and the RSU r P is the available orthogonal sub-channel v,r For the transmission power of the vehicle during the task unloading, h v,r Sigma for channel gain of mission vehicles and RSU 2 Power for white gaussian noise;
defining a communication link between an on-board terminal and a central cloud through a cellular network, and defining a task Ta n Maximum transmission rate Ra offloaded to the central cloud v,c The method comprises the following steps:
wherein B is c K is the total bandwidth of available frequency spectrum between vehicle and cloud server c P is the available orthogonal sub-channel v,c The transmission power h of the vehicle-mounted terminal when the task is unloaded v,c For the channel gain, sigma, of the wireless transmission channel between the mission vehicle and the cloud server 2 Is the power of gaussian white noise.
4. The joint optimization method for task offloading of internet of vehicles according to claim 2, wherein the step 2 of establishing a service delay calculation model and establishing a task offloading policy based on the task offloading model includes:
definition task Ta n Is to be executed locally
The method comprises the following steps:
wherein F is v Representing the computing power of the vehicle-mounted terminal;
definition task Ta n Offloading transmission delay to RSU server
The method comprises the following steps:
the return delay of the RSU calculation model is defined as follows:
definition task Ta n Computation latency generated at RSU server
The method comprises the following steps:
wherein F is r Representing the computing power of the RSU server;
the total time delay of the RSU calculation model is defined as follows:
definition task Ta n Transmission delay generated by offloading to cloud server via cellular network
The method comprises the following steps:
the return delay of the cloud computing model is defined as follows:
definition task Ta n Calculation time delay generated by calculation at cloud server
The method comprises the following steps:
the total time delay of the cloud computing model is defined as follows:
the task unloading situation is represented by a 0-1 variable, and the task unloading strategy is determined as follows: the task requirement of each Internet of vehicles user can only be unloaded to one of the local, edge server and center cloud for execution;
defining an offloading decision to satisfy: alpha v ,α r ,α c ∈(0,1),α v +α r +α c ＝1，α v =1 means that the task performs local computation, α r =1 means task offloading to RSU server for computation, α c =1 means that the task is offloaded to the cloud server for calculation.
5. The joint optimization method for unloading internet of vehicles computing tasks according to claim 2, wherein the objective function in step 3 is:
the constraint conditions are as follows:
C3:α v ,α r ,α c ∈(0,1),α v +α r +α c ＝1；C4:β 1 ，β 2 ∈(0,1)；C5：P v,r ,P v,c ≤P max 。
6. the joint optimization method for unloading the internet of vehicles computing tasks according to claim 2, wherein the mathematical expression of the learning of the 4K-SVD dictionary is:
Y＝[y 1 ,y 2 ,y 3 ,...,y N ]each y of (3) i Represents one sample, X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]As a sparse coefficient matrix, T represents sparsity;
the K-SVD dictionary learning specifically comprises the following steps:
initializing: randomly selecting a part of samples from the sample set, initializing an overcomplete dictionary, and setting iteration times J=1;
the following procedure is repeated until
If not, continuing the iteration process;
sparse coding: using OMP algorithm to calculate sparse coefficient to obtain sparse vector X= [ X ] 1 ,x 2 ,x 3 ,...,x N ]For each sample y i The following formula is satisfied:
dictionary updating: the sparse vector X is fixed and updated column by column for each atom in D as follows:
wherein d j For the j-th column of the overcomplete dictionary D,
column j, E of X k To remove d k The error after that;
video data acquired by a vehicle-mounted camera in a time slot t are expressed as an m X n-dimensional matrix X:
the column vector represents the time sequence of each visual sensor, and the row vector represents the data acquired by a plurality of visual sensors in the same time slot;
to facilitate matrix measurement, the elements of matrix X are represented in column order, yielding an n×1 (n=m·n) dimensional column vector:
vec(X)＝[x 11 ,x 12 ,...,x 1m ,x 21 ,x 22 ,...,x nm ] T
multiplying the observation matrix phi with the data vec (X) to complete data compression and obtain compressed data y i The expression is:
y i ＝Φ·vec(X)。
7. the joint optimization method for unloading internet of vehicles computing tasks according to claim 6, wherein the dictionary updating comprises the following specific steps:
defining atoms in a sample
Using the formula
To calculate the value of the error matrix;
from error matrix E k Is selected to be only equal to omega k Corresponding columns, obtain E' k ；
Using formula E' k ＝U∑V T To E' k Singular value decomposition is performed, and then atom d in the dictionary is updated by selecting the first column in the selection U k =u (·, 1), and let
Update j=j+1.
8. The joint optimization method for unloading the internet of vehicles computing tasks according to claim 2, wherein the step 5 is specifically:
determining the environment, state, action space and rewards of the Markov decision process;
the environment comprises Internet of vehicles information, task amount, communication resources and computing resources of a server;
the state comprises information such as computing resources, communication resources, task unloading data amount, task maximum allowable time delay, vehicle transmitting power, task maximum transmission rate, RSU computing capacity, RSU coverage, cloud server computing capacity, cloud server coverage and the like of all Internet of vehicles users in an environment in a specific time slot t;
the action space is provided with a task unloading decision, a computing resource allocation strategy and a communication resource allocation strategy decision;
the rewards include rewards for performing actions in both target and non-target directions, and the objective function of the internet of vehicles system is optimized using a deep reinforcement learning DQN algorithm.
9. The joint optimization method for unloading computing tasks of the internet of vehicles according to claim 8, wherein the method for utilizing the DQN algorithm specifically comprises the following steps:
step S1: initializing an action-cost function Q(s) of a size of an empirical playback pool storage space and a random weight θ t ,a t The method comprises the steps of carrying out a first treatment on the surface of the θ) and a target action-cost function Q (s t ,a t ；θ * ) Wherein θ is * ＝θ；
Step S2: after receiving the task unloading request of the vehicle-mounted terminal, the vehicle networking system collects the state information of the system, namely, inputs the information such as the task data volume, the maximum allowable time delay, the task transmission rate and the like into the network to obtain the initial state s of the system t ；
Step S3: for each time slot t, an action a is selected with epsilon probability that maximizes the Q value t Picking the next random action a 'with a probability of 1-epsilon' t ；
Step S4: according to action a t Real-time interaction with the environment to enter the next state s t+1 And obtains a reward value r according to the objective function t ；
Step S5: will offload decision action a t System state s t And prize value r t Storing in an experience playback pool, uniformly and randomly sampling from the experience playback pool by utilizing a depth Q network, constructing an error function, and updating a network parameter theta by adopting a back propagation algorithm; and iterating the deep Q network until the network parameters theta are converged to obtain an optimal unloading decision.
10. The joint optimization method for unloading the internet of vehicles computing tasks according to claim 2, wherein the step 6 is specifically:
the execution layer knows the observation matrix phi, the sparse basis phi and the compressed data y i The execution layer only needs to multiply the sparse basis ψ with the sparse Θ to obtain the original data X, which is expressed as: x=ψΘ;
the original data X is solved and converted into: min θ 0 s.t.y＝ΦψΘ。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310227797.8A CN116360883A (en) | 2023-03-10 | 2023-03-10 | Combined optimization method for unloading of Internet of vehicles computing tasks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310227797.8A CN116360883A (en) | 2023-03-10 | 2023-03-10 | Combined optimization method for unloading of Internet of vehicles computing tasks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116360883A true CN116360883A (en) | 2023-06-30 |
Family
ID=86911092
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310227797.8A Pending CN116360883A (en) | 2023-03-10 | 2023-03-10 | Combined optimization method for unloading of Internet of vehicles computing tasks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116360883A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117042051A (en) * | 2023-08-29 | 2023-11-10 | 燕山大学 | Task unloading strategy generation method, system, equipment and medium in Internet of vehicles |
| CN117295199A (en) * | 2023-11-14 | 2023-12-26 | 深圳市铭灏天智能照明设备有限公司 | Internet of things garage illumination system using edge side calculation |
-
2023
- 2023-03-10 CN CN202310227797.8A patent/CN116360883A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117042051A (en) * | 2023-08-29 | 2023-11-10 | 燕山大学 | Task unloading strategy generation method, system, equipment and medium in Internet of vehicles |
| CN117042051B (en) * | 2023-08-29 | 2024-03-08 | 燕山大学 | Task unloading strategy generation method, system, equipment and medium in Internet of vehicles |
| CN117295199A (en) * | 2023-11-14 | 2023-12-26 | 深圳市铭灏天智能照明设备有限公司 | Internet of things garage illumination system using edge side calculation |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112351503B (en) | Task prediction-based multi-unmanned aerial vehicle auxiliary edge computing resource allocation method | |
| CN116360883A (en) | Combined optimization method for unloading of Internet of vehicles computing tasks | |
| CN112817653A (en) | Cloud-side-based federated learning calculation unloading computing system and method | |
| CN111800828B (en) | Mobile edge computing resource allocation method for ultra-dense network | |
| CN114143346B (en) | Joint optimization method and system for task unloading and service caching of Internet of vehicles | |
| CN114142907B (en) | Channel screening optimization method and system for communication terminal equipment | |
| KR20200052182A (en) | Method and apparatus for compressing/decompressing deep learning model | |
| CN113645273B (en) | Internet of vehicles task unloading method based on service priority | |
| CN115034390B (en) | Deep learning model reasoning acceleration method based on cloud edge-side cooperation | |
| CN112512070B (en) | Multi-base-station cooperative wireless network resource allocation method based on graph attention mechanism reinforcement learning | |
| CN115134242B (en) | Vehicle-mounted computing task unloading method based on deep reinforcement learning strategy | |
| CN112929849B (en) | Reliable vehicle-mounted edge calculation unloading method based on reinforcement learning | |
| CN116016538A (en) | Dynamic environment-oriented side collaborative reasoning task unloading optimization method and system | |
| CN114723057A (en) | Neural network collaborative reasoning method for multi-access edge computing system | |
| CN114521002A (en) | Edge computing method for cloud edge and end cooperation | |
| CN116367231A (en) | Edge computing Internet of vehicles resource management joint optimization method based on DDPG algorithm | |
| CN115037751A (en) | Unmanned aerial vehicle-assisted heterogeneous Internet of vehicles task migration and resource allocation method | |
| CN115278693A (en) | CVN (continuously variable transmission) spectrum scheduling method and system based on driving state priority and scene simulation | |
| CN115052262A (en) | Potential game-based vehicle networking computing unloading and power optimization method | |
| CN114828095A (en) | Efficient data perception layered federated learning method based on task unloading | |
| CN116781141A (en) | LEO satellite cooperative edge computing and unloading method based on deep Q network | |
| CN115065683B (en) | Vehicle edge network task allocation and unloading method based on vehicle clustering | |
| CN115766478A (en) | Unloading method of air-ground cooperative edge computing server | |
| CN115967430A (en) | Cost-optimal air-ground network task unloading method based on deep reinforcement learning | |
| CN114513814A (en) | Edge network computing resource dynamic optimization method based on unmanned aerial vehicle auxiliary node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |