CN116257335A - UAV-assisted MEC system joint task scheduling and motion trajectory optimization method - Google Patents

Abstract

The invention discloses a UAV-assisted MEC system joint task scheduling and motion trajectory optimization method, fully considering the change of the position of the mobile device in a dynamic environment (under the condition that the ground terminal equipment and the UAV are both in dynamic movement) In the case of partial unloading of calculation tasks, through joint optimization of user scheduling and selection of calculation task offloading methods, UAV motion parameters (including flight angle, flight speed) and power distribution, the joint optimization of communication, calculation and motion effectively reduces the terminal load. The computing task processing delay of the device improves the unloading efficiency of the UAV-assisted edge computing system. And compared with DQN and other baseline algorithms in the simulation experiment, it is found that the DDPG algorithm has a significant improvement in processing delay.

Description

Joint task scheduling and motion trajectory optimization method for UAV-assisted MEC system

Technical Field

The present invention relates to the technical field of privacy protection for Internet of Vehicles services, and in particular to a method for joint task scheduling and motion trajectory optimization of a drone-assisted MEC system.

Background Art

With the development of the fifth generation of mobile communication technology, new computing-intensive and delay-sensitive applications have emerged rapidly, such as automatic navigation, face recognition, and online games. However, the computing power of terminal devices is often low and it is difficult to handle a large number of computing tasks; traditional cloud computing methods concentrate computing resources in the cloud, and the access to intensive computing tasks and the distance between the cloud and the terminal devices will lead to large transmission delays. Mobile Edge Computing (MEC) can conveniently provide computing services to handle terminal-intensive computing tasks by deploying computing servers at the edge of the network close to the user side, thereby effectively reducing computing processing delays and improving user service experience. Therefore, the emergence of the mobile edge computing model has greatly alleviated the pressure on network bandwidth and cloud computing centers, and optimized the responsiveness of computing and storage services.

Existing mobile edge computing servers/computing centers often use fixed deployment to provide services to users. However, in the case of sparse field communication facilities and sudden disasters, fixed infrastructure often finds it difficult to provide effective services. Unmanned Aerial Vehicles (UAVs) are flexible and easy to deploy. By establishing a line of sight (LOS) connection with ground terminals, they can be used to support ground communication relays and edge computing services.

In the scenario where drones are used to support ground communications, drones often remain stationary and act as aerial base stations, which can be applied to scenarios such as damaged infrastructure or sudden communication traffic at large events. In addition, using the computing power of drones, drones can provide computing resources on demand to ground terminal devices, effectively improving the user's service experience.

At present, academia and industry are conducting research on UAV-assisted MEC systems. In the literature [1] (T. Ren et al., "Enabling Efficient Scheduling in Large-Scale UAV-Assisted Mobile-Edge Computing via Hierarchical Reinforcement Learning," in IEEE Internet of Things Journal, vol. 9, no. 10, pp. 7095-7109, 15 May 15, 2022, doi: 10.1109/JIOT.2021.3071531.), the authors decompose the scheduling problem into two sub-problems for the UAV motion planning and ground terminal device computing offloading scheduling problem, and use the hierarchical reinforcement learning algorithm for alternating optimization to obtain a real-time scheduling strategy in a dynamic environment.

However, existing technologies mainly consider drones as communication relay nodes to provide communication services, or drones as edge computing nodes to provide computing services for terminal devices, and rarely consider solutions that coordinate the two service modes. Due to the limited computing power of drones, it is often impossible to meet the computing task offloading needs of terminal devices.

Summary of the invention

In view of the problem that the computing power of existing drones is limited and cannot meet the computing task offloading needs of terminal devices, the present invention proposes a drone-assisted MEC system joint task scheduling and motion trajectory optimization method, which not only considers the situation where the drone acts as an edge computing node to provide computing services to users, but also considers the situation where the drone acts as a communication relay node to forward computing tasks to the edge computing server on the base station side, thereby jointly optimizing user scheduling, computing task offloading selection, drone flight parameters and communication resource allocation, effectively reducing the computing task processing delay of ground terminal equipment and improving the offloading efficiency of the drone-assisted edge computing system.

In order to achieve the above object, the present invention provides the following technical solutions:

A method for joint task scheduling and motion trajectory optimization of a drone-assisted MEC system takes minimizing the completion of all terminal computing task processing as the optimization goal, takes drone transmission power, task offloading variables, drone flight speed, drone flight angle, drone's own energy, drone's moving area and terminal device's moving area as constraints, and adopts a deep deterministic policy gradient algorithm to solve the optimization goal.

Furthermore, the optimization objectives and constraints are expressed as follows:

C1:

C2:

C3:

C4:0≤β(n)≤2π

C5:

C6:{x _u (n)∈[0,L],y _u (n)∈[0,W]}

C7:{x _m (n)∈[0,L],y _m (n)∈[0,W]}

Where t _sum (n) is the total delay to complete the computing task processing in the time slot T, expressed as:

为地面终端设备将计算任务全部卸载给无人机的传输时延，

为无人机将计算任务部分卸载给基站的传输时延，t_mu(n)为无人机计算任务时延，t_mb(n)为基站部署MEC服务器侧完成计算任务所需时间；in,

The transmission delay for ground terminal equipment to offload all computing tasks to drones,

is the transmission delay of the UAV offloading part of the computing task to the base station, t _mu (n) is the computing task delay of the UAV, and t _mb (n) is the time required for the base station to deploy the MEC server side to complete the computing task;

的约束，C3是对无人机飞行速度v_u(n)的约束，C4是对无人机飞行角度β(n)的约束，C5是对无人机自身能量的约束，C6和C7分别是对无人机和终端设备的移动区域的限制；E_fly＝φ||v_u(n)||²为无人机的飞行能耗，其中，φ＝0.5M_UAVt_fly，M是无人机质量，t_fly为飞行时间，E_mu(n)＝γ_u(f_u)³t_mu(n)，E_mu(n)为无人机本地计算能耗，f_u为无人机的计算能力，γ_u为芯片结构对CPU处理的影响因子，E^U为无人机自带电量。C1 is the constraint on the UAV transmission power P _u (n), and C2 is the task offloading variable

, C3 is the constraint on the UAV's flight speed v _u (n), C4 is the constraint on the UAV's flight angle β(n), C5 is the constraint on the UAV's own energy, C6 and C7 are the restrictions on the moving areas of the UAV and the terminal device respectively; E _fly =φ||v _u (n)|| ² is the flight energy consumption of the UAV, where φ=0.5M _UAV t _fly , M is the mass of the UAV, t _fly is the flight time, E _mu (n)=γ _u (f _u ) ³ t _mu (n), E _mu (n) is the local computing energy consumption of the UAV, _fu is the computing power of the UAV, γ _u is the influence factor of the chip structure on CPU processing, and ^EU is the battery capacity of the UAV.

Furthermore, the process of solving the optimization objective using the deep deterministic policy gradient algorithm is:

其中q(n)代表无人机的位置信息，p_m(n)代表地面终端设备的位置信息，D_m(n)代表剩余计算任务数据量的大小，

代表无人机剩余的电量；The state space is represented as

Where q(n) represents the location information of the UAV, p _m (n) represents the location information of the ground terminal equipment, and D _m (n) represents the amount of data for the remaining computing task.

Represents the remaining battery power of the drone;

其中v_u(n)为无人机的速度大小，β(n)为无人机的飞行角度，P_u(n)为无人机的传输功率，

为任务卸载变量；The action space is represented as

Where v _u (n) is the speed of the UAV, β(n) is the flight angle of the UAV, and P _u (n) is the transmission power of the UAV.

Unload variables for tasks;

The reward function is expressed as r _n =-t _sum (n), where the reward function is the negative total delay;

Use Q-table to record and update the state-action value, i.e. Q(s,a), and use critic and actor networks for updating, as shown below:

θ ^Q' ←ρθ ^Q +(1-ρ)θ ^Q'

θ ^μ' ←ρθ ^μ +(1-ρ)θ ^μ'

Among them, θ ^Q is the parameter of the critic neural network, ρ is a constant, and θ ^μ is the parameter of the actor neural network.

Furthermore, the critic network is updated as follows:

L(θ ^Q )＝E _μ' [(y _n -Q(s _n ,a _n |θ ^Q )) ² ]

Where E _μ' is the averaging function, _yn is the Q target value, _yn = _rn +γQ(sn ₊₁ ,μ( _sn+1 )| ^θQ ), γ is the discount factor, and _rn is the reward function.

Furthermore, the Actor network is updated as follows:

为求策略梯度函数，μ为策略网络，N为从经验池随机抽取的大小。in,

To find the policy gradient function, μ is the policy network and N is the size randomly drawn from the experience pool.

Compared with the prior art, the present invention has the following beneficial effects:

The proposed method for joint task scheduling and motion trajectory optimization of the UAV-assisted MEC system takes full account of the change in the position of the mobile device and the partial unloading of the computing task in a dynamic environment (when both the ground terminal device and the UAV are in dynamic movement). By jointly optimizing the user scheduling and the selection of the computing task unloading method, the UAV motion parameters (including flight angle, flight speed) and power allocation, the communication, computing and motion are jointly optimized, which effectively reduces the computing task processing delay of the terminal device and improves the unloading efficiency of the UAV-assisted edge computing system. In the simulation experiment, it is compared with the baseline algorithms such as DQN, and it is found that the DDPG algorithm has a significant improvement in processing delay.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

Figure 1 is a model diagram of the drone-assisted edge system.

FIG2 is a system architecture diagram of a method for joint task scheduling and motion trajectory optimization of a drone-assisted MEC system provided in an embodiment of the present invention.

Figure 3 shows the comparison results of DDPG and DQN in processing latency.

Figure 4 shows the comparison results of DDPG with the Local baseline algorithm and the Offload baseline algorithm in terms of processing latency.

DETAILED DESCRIPTION

In order to better understand the technical solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

1) System Model

基站b的坐标为l_b(n)＝[x_b(n),y_b(n),h]^T。Figure 1 shows a UAV-assisted MEC system diagram, which includes a UAV, multiple ground terminal devices (MDs) and base stations. Consider that the time slots can be divided into T = {1, 2, ..., N}. In time slot n, the position coordinates of terminal device m are p _m (n) = [x _m (n), y _m (n), 0] ^T , and it moves randomly at a speed v _m ; the position coordinates of the UAV are

The coordinates of the base station b are l _b (n) = [x _b (n), y _b (n), h] ^T .

Considering that the UAV is deployed in the air, the transmission between the UAV and MDs is mainly based on line-of-sight. The channel gain between the UAV and MDs can be expressed as:

为无人机和MDs之间的距离。Where _g0 represents the channel gain at a reference distance of 1m.

is the distance between the UAV and MDs.

The channel gain between the drone and the base station can be expressed as:

为无人机和基站之间的距离。in,

is the distance between the drone and the base station.

In time slot n, the computational task carried by MD m can be expressed as:

W _m (n)={D _m (n), C _m (n), T _m (n)}

Where D _m (n) is the data size of the computing task, C _m (n) is the CPU cycle required to process each bit of data, and T _m (n) is the maximum allowed time to process the computing task.

2) Communication Model

(1) Communication model between MDs and drones

其中D_m(n)为MD所携带的计算任务数据量大小，

为MD和无人机之间的传输速率。The transmission delay of MDs to offload all computing tasks to drones is

Where D _m (n) is the amount of computing task data carried by MD,

is the transmission rate between MD and UAV.

是白高斯噪声，P_nlos是非视距传输损耗，f(n)为二值函数，且f(n)∈{0,1}。(f(n)＝0表示在无人机和MDs之间没有遮挡，f(n)＝1代表在无人机和MDs之间有遮挡)Where _Bu is the available bandwidth for communication between UAVs and MDs, _Pm is the transmission power of MDs,

is white Gaussian noise, _Pnlos is non-line-of-sight transmission loss, f(n) is a binary function, and f(n)∈{0,1}. (f(n)＝0 means there is no occlusion between the drone and MDs, and f(n)＝1 means there is occlusion between the drone and MDs)

Therefore, the transmission energy consumption can be expressed as:

(2) Communication model between drone and base station

The transmission delay when the drone offloads part of the computing task to the base station is:

为任务卸载变量，

为无人机和基站之间的传输速率。in

Unload variables for tasks,

is the transmission rate between the drone and the base station.

是无人机的传输功率，

是无人机最大传输功率，

是白高斯噪声。Where _Bk is the available bandwidth for communication between the drone and the base station,

is the transmission power of the drone,

is the maximum transmission power of the drone,

is white Gaussian noise.

Therefore, after obtaining the transmission delay between the drone and the base station, the transmission energy consumption can be expressed as:

(3) Drone mobility model

In time slot n, the drone flies from q(n) to the new position coordinates which can be expressed as:

q(n+1)＝[x _u (n)+l _b (n)cosβ(n),y _u (n)+l _b (n)sinβ(n)]

Wherein, l _b (n) = v _u (n) t _fly is the flight distance of the UAV, t _fly is the flight time, and β(n) is the flight angle of the UAV.

Therefore, the flight energy consumption of the UAV can be expressed as:

E _fly =φ||v _u (n)|| ²

Where, φ = 0.5M _UAV t _fly , and M is the mass of the UAV.

3) Computational model

First, the UAV calculation task delay is:

Where f _u is the computing power of the drone, and the unit is the number of CPU cycles per second.

At this time, the energy consumption generated by the UAV computing task is:

E _mu (n)=γ _u ( _fu ) ³ t _mu (n)

Among them, _γu is the impact factor of chip structure on CPU processing.

In addition, the time required for the base station to deploy the MEC server side to complete the computing task is:

Where _fb is the computing power of the base station, and the unit is the number of CPU cycles per second.

As shown in Figure 2, in the drone-assisted edge computing system, after the terminal randomly generates computing tasks and transmits them to the drone, there are two processing methods: the drone acts as an edge computing node to complete the offloaded computing task processing, or the drone is regarded as a relay forwarding and completes the computing task processing together with the edge node (base station).

The UAV-assisted MEC system joint task scheduling and motion trajectory optimization method proposed in the present invention is as follows.

Optimization goals:

Considering that the computing task processing is completed within the time slot T, the total delay can be expressed as:

in,

The present invention aims to minimize the processing of all terminal computing tasks, considers the ground terminal equipment and computing task offloading selection adjustment, UAV motion parameters (including flight angle, flight speed) and transmission power joint optimization, and models the joint optimization problem as follows:

C1:

C2:

C3:

C4:0≤β(n)≤2π

C5:

C6:{x _u (n)∈[0,L],y _u (n)∈[0,W]}

C7:{x _m (n)∈[0,L],y _m (n)∈[0,W]}

Among them, C1 is the constraint on the transmission power of the UAV, C2 is the constraint on the task offloading variable, C3 is the constraint on the flight speed of the UAV, C4 is the constraint on the flight angle of the UAV, C5 is the constraint on the UAV's own energy, and C6 and C7 are restrictions on the moving areas of the UAV and terminal equipment.

In view of the above optimization problem, the present invention transforms the non-convex optimization problem into a Markov decision process (MDP). Considering that the channel conditions and device states of the system are dynamic and time-varying, the deep deterministic policy gradient (DDPG) algorithm is used to solve the problem, which can effectively solve the optimization problem with a continuous action space.

The process of solving the optimization objective is:

其中q(n)代表无人机的位置信息，p_m(n)代表地面终端设备的位置信息，D_m(n)代表剩余计算任务数据量的大小，

代表无人机剩余的电量；The state space is represented as

Where q(n) represents the location information of the UAV, p _m (n) represents the location information of the ground terminal equipment, and D _m (n) represents the amount of data for the remaining computing task.

Represents the remaining battery power of the drone;

其中v_u(n)为无人机的速度大小，β(n)为无人机的飞行角度，P_u(n)为无人机的传输功率，

为任务卸载变量；The action space is represented as

Where v _u (n) is the speed of the UAV, β(n) is the flight angle of the UAV, and P _u (n) is the transmission power of the UAV.

Unload variables for tasks;

其中奖励函数为负的总时延。The reward function is expressed as

The reward function is the negative total delay.

In traditional reinforcement learning, Q-learning has been widely used. It uses Q-table to record and update state-action values, i.e., Q(s,a). However, it is difficult for Q-learning to extract and generalize features from previous experience, and it is ineffective in dealing with large-dimensional problems. DDPG can not only predict Q values by continuously training from previous experience, but also use critic and actor networks for updating. In the DDPG algorithm, the critic network can be updated as:

L(θ ^Q )＝E _u' [(y _n -Q(s _n ,a _n |θ ^Q )) ² ]

Where E _μ' is the averaging function, _yn is the Q target value, _yn = _rn +γQ(sn ₊₁ ,μ( _sn+1 )| ^θQ ), γ is the discount factor, and _rn is the reward function.

The Actor network is updated to:

为求策略梯度函数，μ为策略网络。in,

To find the policy gradient function, μ is the policy network.

The DDPG soft update critic and actor target network process is as follows:

θ ^Q' ←ρθ ^Q +(1-ρ)θ ^Q'

θ ^μ' ←ρθ ^μ +(1-ρ)θ ^μ'

Among them, θ ^Q is the parameter of the neural network, ρ is a constant, and θ ^μ is the parameter of the actor neural network.

The DDPG algorithm is as follows:

Randomly initialize the critic and actor networks;

Construct critic and actor target networks;

Initialize the experience replay pool;

For episode = 1, M do;

Initialize a random noise for exploration;

Initial state S ₁ ;

For t = 1, T do;

Randomly select an action a _t according to the current strategy;

After executing the action, the environment provides timely reward r _t+1 and new state s _t+1 ;

Add transitions(s _t , a _t , r _t+1 , s _t+1 ) to the experience pool;

Randomly extract a small batch of transitions (s _t , a _t , r _t+1 , s _t+1 ) from the experience pool;

更新critic网络参数；Calculate Q value based on mean square loss

Update critic network parameters;

Update actor network;

Finally, the critic and actor target networks are soft-updated.

While fully considering the situation that the position of mobile devices changes and the partial unloading of computing tasks in a dynamic environment, the present invention effectively reduces the computing task processing delay of terminal devices by jointly optimizing user scheduling and computing task unloading mode selection, drone motion parameters (including flight angle, flight speed) and power allocation, and effectively improves the unloading efficiency of drone-assisted edge computing systems. And compared with baseline algorithms such as DQN in simulation experiments, it is found that the DDPG algorithm has a significant improvement in processing delay. Among them, the comparison results of DDPG and DQN in processing delay (as shown in Figure 3) show that DDPG is nearly 20% higher than DQN. As shown in Figure 4, DDPG has improved nearly 60% in processing delay compared with the Local baseline algorithm (computational tasks are only calculated locally), and DDPG has improved nearly 34% in processing delay compared with the Offload baseline algorithm (computational tasks are all unloaded to the base station for edge computing).

The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (5)

1. The unmanned aerial vehicle auxiliary MEC system joint task scheduling and motion trail optimizing method is characterized in that all terminal calculation task processing is minimized to be completed to be an optimizing target, unmanned aerial vehicle transmission power, task unloading variables, unmanned aerial vehicle flight speed, unmanned aerial vehicle flight angle, unmanned aerial vehicle self energy, a moving area of an unmanned aerial vehicle and a moving area of terminal equipment are taken as constraints, and a depth deterministic strategy gradient algorithm is adopted to solve the optimizing target.

2. The unmanned aerial vehicle assisted MEC system joint task scheduling and motion trajectory optimization method of claim 1, wherein the optimization objectives and constraints are expressed as follows:

C4:0≤β(n)≤2π

C6:{x _u (n)∈[0,L],y _u (n)∈[0,W]}

C7:{x _m (n)∈[0,L],y _m (n)∈[0,W]}

wherein ,t_sum (n) is the total delay to complete the processing of the computing task in time slot T, expressed as:

wherein ,

offloading all computing tasks to a drone for ground terminal equipmentTransmission delay->

Transmission delay for unloading calculation task part to base station for unmanned aerial vehicle, t _mu (n) calculating task time delay for unmanned aerial vehicle, t _mb (n) time required for the base station to deploy MEC server side to complete the calculation task;

c1 is the transmission power P to the unmanned aerial vehicle _u (n) constraint, C2 is a variable for task offloading

C3 is the constraint on the unmanned aerial vehicle flight speed v _u (n) constraint, C4 is constraint on unmanned plane flight angle β (n), C5 is constraint on unmanned plane self energy, and C6 and C7 are limits on unmanned plane and terminal equipment movement area, respectively; e (E) _fly ＝φ||v _u (n)|| ² Is the flight energy consumption of the unmanned aerial vehicle, wherein phi=0.5M _UAV t _fly M is unmanned aerial vehicle mass, t _fly For time of flight, E _mu (n)＝γ _u (f _u ) ³ t _mu (n)，E _mu (n) locally calculating energy consumption for unmanned aerial vehicle, f _u Is the calculation capability of unmanned aerial vehicle, gamma _u For influencing the CPU processing by the chip structure, E ^U Is self-charged quantity of the unmanned aerial vehicle.

3. The unmanned aerial vehicle assisted MEC system joint task scheduling and motion trail optimization method according to claim 1, wherein the process of solving the optimization target by adopting the depth deterministic strategy gradient algorithm is as follows:

the state space is expressed as

Wherein q (n) represents positional information of the unmanned aerial vehicle, p _m (n) represents the position information of the ground terminal equipment, D _m (n) represents the size of the remaining calculation task data amount,

representing the residual electric quantity of the unmanned aerial vehicle;

the action space is expressed as

wherein v_u (n) is the speed of the unmanned aerial vehicle, beta (n) is the flight angle of the unmanned aerial vehicle, and P _u (n) is the transmission power of the unmanned aerial vehicle, < >>

Unloading variables for the task;

the bonus function is denoted as r _n ＝-t _sum (n) wherein the reward function is a negative total delay;

the state-action values, i.e., Q (s, a), are recorded and updated using the Q-table, with updates using the critic and actor networks, as follows:

θ ^Q' ←ρθ ^Q +(1-ρ)θ ^Q'

θ ^μ' ←ρθ ^μ +(1-ρ)θ ^μ'

wherein ,θ^Q Is a parameter of the critic neural network, ρ is a constant, θ ^μ Is a parameter of the actor neural network.

4. The unmanned aerial vehicle assisted MEC system joint task scheduling and motion trajectory optimization method of claim 3, wherein the critic network updates as:

L(θ ^Q )＝E _μ' [(y _n -Q(s _n ,a _n |θ ^Q )) ² ]

wherein ,E_μ' To calculate the mean value function, y _n For Q target value, y _n ＝r _n +γQ(s _n+1 ,μ(s _n+1 )|θ ^Q ) Gamma is the discount factor, r _n Is a bonus function.

5. The unmanned aerial vehicle assisted MEC system joint task scheduling and motion trail optimization method of claim 3, wherein the Actor network updates as:

wherein ,

for the policy gradient function, μ is the policy network and N is the size randomly extracted from the experience pool. />

Images