CN113950066B

CN113950066B - Single server part calculation unloading method, system and equipment under mobile edge environment

Info

Publication number: CN113950066B
Application number: CN202111060966.0A
Authority: CN
Inventors: 安玲玲; 张星雨; 廖鹏; 单颖欣; 岳佳豪; 马晓亮; 王泉
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2021-09-10
Filing date: 2021-09-10
Publication date: 2023-01-17
Anticipated expiration: 2041-09-10
Also published as: CN113950066A

Abstract

The invention belongs to the technical field of task unloading and resource allocation of mobile edge computing, and discloses a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment, wherein the method for partial computation unloading of the single server in the mobile edge environment comprises the following steps: a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink; a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm. The invention can dynamically make reasonable task unloading decision and allocate bandwidth and computing resource for each user in each time slice, thereby effectively reducing the weighted cost in two aspects of task processing total time delay and mobile equipment total energy consumption.

Description

Single server part calculation unloading method, system and equipment under mobile edge environment

Technical Field

The invention belongs to the technical field of task unloading and resource allocation of mobile edge computing, and particularly relates to a method, a system and equipment for unloading partial computing of a single server in a mobile edge environment.

Background

At present, the internet of things technology is continuously popularized and the light speed in the field of mobile communication is rapidly developed, a large number of intelligent mobile devices are produced at present, a large number of calculation-intensive tasks are mostly generated by massive intelligent devices and application, the data traffic is increased explosively, and higher requirements are brought to the data calculation capacity and the battery endurance capacity of mobile terminal equipment. Mobile Edge Computing (MEC) enables a terminal device to complete a computing task in a low-delay and low-energy-consumption manner, a Mobile user unloads the task to an Edge base station for processing, and the time delay and energy consumption of a local device can be effectively reduced by utilizing the computing power and energy reserve of an Edge server. The basic idea of edge computing is to change the computing task generated on the mobile device from being offloaded to the cloud end to being offloaded to the network edge end, so that the requirement of computation-intensive applications such as real-time online games and augmented reality on low delay is met. Different task offloading schemes have a large impact on task completion latency and mobile device energy consumption. Therefore, how to make the most reasonable task offloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved in the current computing offloading framework research, and a large number of task offloading strategies based on different algorithms and optimization objectives are also proposed. According to the optimization target of the algorithm, the calculation unloading strategy can be divided into the following three types:

the first category of strategies is mainly to reduce task processing delays. Bi and Zhang et al consider a multi-user MEC system supported by Wireless power transmission in "Computing Rate multiplication for Wireless Powered Mobile-Edge Computing with Binary Computing Offloading" (IEEE Transactions on Wireless Communications,2018,17 (6): 4177-4190), study a Binary Computing Offloading strategy in a multi-user MEC network, and solve the problems of Computing Rate weighting and Maximization of all Wireless devices in each time range by using an alternating direction multiplier algorithm, which has the disadvantages: when computing offloading is performed, energy consumption at one end of the mobile terminal device cannot be considered, the terminal device may have a situation that an offloading policy cannot operate normally due to insufficient electric energy, the mobile device is limited in terms of computing resources, and data is generally processed at the expense of high latency and high device energy consumption.

The second type of strategy is mainly to reduce the device energy consumption. Kuang and Shi et al propose an offload Game mechanism in the "Multi-User Offloading Game ranking in OFDM Mobile Computing System" (IEEE Transactions on Vehicular Technology,2019,68 (12): 12190-12201) to achieve the energy saving requirement by Offloading tasks for each device in an orthogonal frequency division multiple Access communication System using a Game theory approach. The method has the following defects: mobile devices need to be responsible for handling more and more compute intensive tasks such as high definition video live and face recognition. However, due to limited computing resources and battery endurance, these devices may not be able to process all tasks locally with low latency.

The third type of strategy is mainly time delay and energy consumption trade-off. In consideration of a system architecture including a plurality of access points, an Edge server and a Mobile device and an application scenario in which tasks can be left in a local device for execution and offload computation can be performed to a server or a remote cloud, an offload decision and Resource management scheme Based on an SARSA algorithm in Reinforcement Learning is designed to reduce the cost of the system in terms of energy consumption and computation delay. The disadvantages of the method are: in the actual unloading process, different systems may have different performance requirements instead of being limited to only time delay and energy consumption, and how to make the most reasonable task unloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved urgently in the research of the computing unloading framework.

Through the above analysis, the problems and defects of the prior art are as follows:

(1) In the existing calculation unloading strategy for reducing task processing delay, energy consumption at one end of mobile terminal equipment during calculation unloading cannot be considered, so that the total execution time delay of a model task is longer, and the total energy consumption of the mobile equipment is higher.

(2) In the existing calculation unloading strategy for reducing the energy consumption of the equipment, a user more hopes that the sum of the time consumption and the energy consumption of the system can be minimized to reduce the overall consumption of the system, or the time consumption and the energy consumption are balanced.

(3) In the existing calculation unloading strategy with compromise between time delay and energy consumption, different systems may have different performance requirements in the actual unloading process rather than being limited to time delay and energy consumption.

The difficulty in solving the above problems and defects is: in the process of task data transmission, enough wireless network bandwidth is needed, and the edge server also has limited computing resources, so that making the most reasonable task offloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved urgently in the research of the computing offloading framework nowadays. The device resources are limited by the conflict between the high-performance task processing requirements, but with the continuous increase of the task unloading scale, the power consumption generated by executing the tasks rises sharply, and the benefit of the MEC system is seriously influenced.

The significance of solving the problems and the defects is as follows: in a more complex environment of a single MEC server with multiple mobile devices, a partial computation unloading scheme taking total time delay of task execution and total energy consumption of the mobile devices as a common optimization index is urgently needed.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment.

The invention is realized in such a way that a single server part computation and unloading method under a mobile edge environment comprises the following steps:

constructing an application scene of a single base station and a single MEC server of a multi-user terminal to realize unloading decision of a mobile terminal task;

designing a network communication model, and sending each user task input data to an MEC server through a base station uplink to reasonably distribute network bandwidth resources;

step three, a local processing and edge unloading parallel task execution module is set up, and the feedback efficiency of the application service is improved;

step four, performing target modeling according to network communication time delay and task execution energy consumption, and realizing problem planning of the total cost of the MEC system in a single-base-station multi-user scene;

and fifthly, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing overall performance optimization based on time delay and energy consumption.

Further, in the step one, the construction of the application scenario of the single MEC server for multiple end users includes the following steps:

1) Modeling a single MEC scene of a plurality of mobile users in a cell, constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the serial number of mobile equipment as follows:

N＝{1,2,...}

and each user device n will generate a divisible compute-intensive task:

wherein D is _n Representing task A _n Size of the uploaded data amount, L _n Representing the execution of a computing task A _n The amount of computing resources that are required,

for processing task A _n Tolerable maximum time delay requirements;

2) Computing an offload decision vector, task A on user equipment n _n The execution ratio offloaded to the MEC server is denoted x _n ＝[0,1]And N belongs to N, the proportion of executing the task on the local equipment N is 1-x _n The two parts of operations of local execution of the subtask and calculation and unloading of the MEC server are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

the final computational offload decision vector x for all user tasks can be expressed as:

x＝{x ₁ ，x ₂ ，...，x _|N| }；

3) Setting a task waiting execution cache queue, and supposing that the time of the whole MEC system is divided into a plurality of time lengths which are tau ₀ And a buffer queue I of tasks to be executed, which are executed in a first-in first-out sequence, is set for each user terminal n in view of the problem of limited resources of mobile users _n ，K _n Is queue I _n The total amount of task calculation to be executed is calculated, and each time a new time slice t +1 starts, the total amount K of task calculation to be executed of the mobile terminal n _n The dynamically updated formula of (a) is:

wherein, K _n (t + 1) is the amount of tasks to be executed on the n buffer queue of the t +1 th time slice mobile terminal, x _n (t) indicating task A on the nth user terminal within time slice t _n (t) decision variables for local processing or edge offload, L _n (t) to perform task A _n (t) the amount of computational resources required,

representing the local execution task amount of the mobile user n in the t time slice;

4) The method solves the problems of data transmission and calculation, and aims at the problems of data transmission and calculation during task unloading at a server side, the information of tasks related to each mobile user is firstly forwarded to an MEC server through a base station, then the MEC server allocates corresponding calculation resources to execute the tasks, and the model also needs to consider a real-time allocation scheme of bandwidth and calculation resources.

Further, in the second step, the design of the network communication model includes the following steps:

defining the uplink network bandwidth resource of a base station as a fixed value W, all mobile users in a cell need to share the bandwidth resource of the base station, and making a reasonable network bandwidth resource allocation decision B = { B ] for a system in each time slice ₁ ，B ₂ ，...，B _|N| }; the proportion of the bandwidth resources allocated to the mobile user by the base station in the time slice t is B _n ，B _n ∈[0,1]And satisfy

Then according to Shannon's formula, when multiple terminals in a cell simultaneously offload tasks to the MEC server, the uplink task transmission rate r between user n and the server _n Can be expressed as:

wherein, P _n Is the transmission power of the nth user equipment, and g _n Representing the gain of the radio transmission channel between user N and the base station over time slice t, N ₀ Is the power spectral density of white gaussian channel noise.

Further, in the third step, the construction of the parallel task execution model includes the following steps:

calculating an unloading model classification according to a task unloading decision x = { x = { [ x ] ₁ ，x ₂ ，...，x _|N| Each mobile terminal can process the task locally and can unload part of the task to the MEC server for execution; the calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is prolonged;

a local execution model, wherein the local execution model processes task data and obtains an execution result through the computing power of the mobile equipment, and mainly relates to local execution delay

And device execution energy consumption

Two parts of overhead; defining the local CPU calculation frequency of the nth mobile terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x) _n )L _n As task A _n The number of CPU cycles required for the local execution section; in addition, for each user terminal, the sub-tasks processed locally should also take into account the waiting time of the current time slice on the task buffer queue waiting to be executed locally

Therefore, the total delay of the nth user terminal to process the sub-tasks locally is defined as:

in the edge unloading model, the edge server-side processing subtask generally needs to be divided into three steps; firstly, a mobile user transmits task related data to a base station through a wireless link; when the base station forwards the subtask unloaded by the mobile user n to the MEC server, the MEC server allocates the computing resource for the MEC server, and the computing resource allocation decision is represented as:

F＝{F ₁ ,F ₂ ,...,F _n ,...,F _|N| }；

and satisfy the conditions

Finally, the step ofThe MEC server feeds back the execution result to the mobile equipment, and the return data volume of the task execution result is usually very small and far lower than the uploaded task data, and the return rate of a wireless network is very high, so that the time delay and the energy consumption cost for returning the task result to the user are ignored; the required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

corresponding offload data x _n D _n The energy consumption during transmission is:

wherein, P _n The uplink transmission power of the nth mobile user; after the data transmission is completed, task A _n The computation time of the offload portion on the edge server is:

wherein, F _n Proportion of computing resources allocated to the nth mobile user for the MEC server, f ^C Computing power for the CPU of the MEC server; the total processing delay of the tasks unloaded to the MEC server is the sum of the transmission delay and the execution delay of the tasks, and is represented as follows:

in summary, for the time slice t, since the user local and the server in the MEC system can execute the task in parallel, the cell movesTask A for active user n _n The total processing delay is:

complete task A _n The total cost of energy consumption required is:

further, in the fourth step, the MEC system total cost problem planning includes the following steps:

aiming at the problems of task part unloading and network bandwidth and MEC computing resource distribution, a weighting factor omega of time delay and energy consumption is introduced ₁ And omega ₂ For adjusting the weight ratio of time and energy consumption costs according to user-specific preferences, and ₁ +ω ₂ =1, and the objective function is formulated as follows:

as shown in the above equation, according to the task execution model,

and

adopting task unloading decision x = { x) under t time slices respectively ₁ ，x ₂ ，...，x _|N| Bandwidth and computational resource allocation policy B = { B = } ₁ ,B ₂ ,...,B _n ,...,B _|N| And F = { F = } ₁ ,F ₂ ,...,F _n ,...,F _|N| Task processing delay and energy consumption of the electronic device; to pair

Maximizing aims to reduce task A as much as possible _n ToThe utility maximization of the whole MEC system model is realized by the weighted sum of the feed time and the energy consumption of the local mobile equipment;

the constraint conditions in C1-C3 respectively represent that the unloading proportion allocated to all mobile users in the cell is not more than 1, and the sum of the allocated bandwidth resources and the proportion of the calculated resources is less than or equal to 1; c4 then represents task A _n The time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

And C5 and C6 ensure that the time and energy cost required to employ the present computational offload scheme is no greater than the time delay and energy consumption of the full local execution.

Further, in the fifth step, the joint optimization of the time delay and the energy consumption includes the following steps:

giving next action decision based on the current state, and using quadruplet M = (S, A) _s ,P _ss',a ,R _s,a ) To describe this process;

wherein S is a limited set of states, A is a limited set of actions, S is the system state under the current time slice and S belongs to S, S 'is the next state of the system and S' belongs to S, a is the selected action and a belongs to A, P _ss',a Representing the probability of a transition from the current state s to the next new state s' when performing action a, R _s,a The instant direct reward obtained by converting the state s into s' after the action a is executed;

in addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of a future time slice has, i.e., the impact of actions taken based on the state of a later time slice on the overall reward value gradually decays, and the weighted sum G (t) of all reward values resulting from the selection of a set of actions on time slice t in the markov decision process is described as:

wherein, γ ⁱ R (t + i + 1) is time slice t + i +1 value performance of the reward obtained over time slice t;

and (3) function three-element design, namely respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-terminal user on the basis of a system model:

the state is as follows: the state space firstly ensures that all information in the environment can be contained, and the change of the environment in each time slice is fully reflected; the present system state is therefore defined as:

the system state is composed of four parts, namely the data volume D (t) of the mobile user arriving at the task in the cell on the current time slice t, the required computing resource number L (t), the task volume K (t) to be executed on the task cache queue and the data transmission rate r (t) between the mobile user and the server;

the actions are as follows: according to the data volume D (t) of a new arriving task of a user on each time slice in the state s (t), the number L (t) of required computing resources and the task volume K (t) to be executed on a task cache queue, the Agent of the Agent is a task A _n Making an offload proportion decision x (t), a ratio of bandwidth resources allocated per mobile user B (t), and a server-calculated resource allocation ratio F (t), i.e.:

a(t)＝[x ₁ (t),x ₂ (t),...,x _|N| (t),B ₁ (t),B ₂ (t),...,B _|N| (t),F ₁ (t),F ₂ (t),...,F _|N| (t)]；

the reward function: the instantaneous reward return obtained by the system on the time slice t is set as the objective function value in the formula, namely

The greater the reward R (t) obtained by performing action a (t), the greater the cost of time for all user tasks to perform at the current time slice t

And energy consumption cost

The smaller the weighted sum;

optimizing a depth deterministic strategy gradient (DDPG) algorithm, and dynamically solving a task unloading decision and resource allocation scheme under each time slice by using the DDPG algorithm so as to minimize a target function and reduce the weighted total cost of time delay and energy consumption;

the DDPG network structure is provided with an Actor and a Critic module and comprises four neural networks which are an Actor current network, an Actor target network, a Critic current network and a Critic target network respectively;

the action network module is used for selecting an action and delivering the action to the Agent of the intelligent Agent to execute the action, and the criticic module is used for evaluating a Q value according to a state s (t) and an action a (t); the experience playback unit stores a state transition data sample obtained by interaction with the environment for later sampling;

the objective function of the Critic module in the DDPG algorithm is a time difference TD-error which is used for representing the difference between the current action and the expected action, and the loss function of the Critic network is defined as the square of the TD-error:

where m is the number of state transition samples { s (t), a (t), R (t), s (t + 1) } randomly drawn from the empirical playback unit, R (t) being the reward resulting from performing action a (t) over a time slice t, Q' (s (t + 1), a (t + 1), θ (t) } ^Q ') then represents the evaluation value of the state s (t + 1) and action a (t + 1) pair on the next time slice t +1 given by the Critic object network, θ ^Q 、θ ^Q' The weighting parameters, Q (s (t), a (t), θ, of the current and target networks, respectively ^Q ) Then the value of the state s (t) and the action a (t) corresponding to the current moment t is judged through the Critic current network, and gamma is a discount factor;

the Actor module in the DDPG algorithm is decreased through gradientIn the method of (1), the network is updated with parameters, and an action a (t) = μ (s (t), θ) that maximizes the evaluation value is selected for the purpose of selecting the action ^μ ) The loss function is:

wherein, theta ^μ Is the weight parameter of the Actor's current network;

the DDPG-based multi-user single-server joint task unloading and resource allocation steps are as follows:

(1) input task request set { A ₁ ,A ₂ ,...,A _|N| }, uplink network bandwidth W and MEC server computing capacity f ^C The experience playback unit M is initialized, and the current network mu (s (t), theta) of the Actor is initialized randomly ^μ ) And criticic current network Q (s (t), a (t), θ ^Q ) Weight of theta ^μ And theta ^Q ；

(2) Randomly initializing m state transition data for action detection, and receiving an initial state s (0);

(3) generating an action a (t) = μ (s (t), θ, according to the current strategy and probing noise loop traversal ^μ )+noise(t)；

(4) Performing the offloading decision and resource allocation of action a (t) and obtaining a reward R (t) and a status s (t + 1) of a next time slice;

(5) randomly selecting M state conversion tuples { s (t), a (t), R (t), s (t + 1) } from an empirical playback unit M;

(6) updating the target networks of Critic and Actor by using a soft updating mode based on a loss function

(7) Repeating the step (3) to the step (6) for T times, wherein T is the number of time slices;

(8) repeating the step (2) to the step (7) for E times, wherein E is the number of epsilon;

(9) and outputting a task unloading decision x, a bandwidth resource allocation scheme B and a computing resource allocation scheme F.

Another object of the present invention is to provide a system for offloading computation of a single server portion in a mobile edge environment, which implements the method for offloading computation of a single server portion in a mobile edge environment, the system for offloading computation of a single server portion in a mobile edge environment comprising:

the MEC service scene construction module is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of mobile users in a single cell;

the network communication module is used for assuming the condition that a single base station deployed in a cell provides wireless network service for users, and sending each user task input data to the MEC server through a base station uplink;

the task execution module is used for dividing a calculation unloading model of a task into a local execution model and an edge unloading model, and executing the local processing part and the edge unloading part simultaneously in parallel so as to reduce the task processing time delay and improve the service feedback time;

the MEC system total cost problem planning module is used for improving the experience quality of each mobile user in a cell, measuring the performance of a calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the MEC system total cost in a single-cell multi-mobile-user scene according to the network communication and task execution model;

and the delay and energy consumption combined optimization module is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization in the MEC scene of multiple mobile users under all the time slices is realized.

It is a further object of the invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:

a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink;

a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm.

It is another object of the present invention to provide a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

The invention provides a single-server partial computation unloading strategy in a mobile edge environment, which is mainly applied to the technical field of task unloading and resource allocation problems of mobile edge computation and mainly solves the problems of task execution delay and equipment energy consumption in an application scene of a multi-user single MEC server. Aiming at the scene and the task type, a dynamic partial task unloading and resource allocation algorithm based on a depth deterministic gradient algorithm is designed, and finally, a partial computation unloading model taking minimization of total delay of all task computation and total energy consumption of terminal equipment as optimization targets is realized. The partial calculation unloading strategy based on the DDPG is compared with the partial calculation unloading strategy based on the DQN through a numerical simulation experiment, the partial calculation unloading strategy based on the DDPG is obtained through comparison research, the performance is optimal, and the time delay and energy consumption long-term weighting total overhead of the whole multi-user single-server MEC system can be effectively reduced.

Compared with the prior art, the invention also has the following advantages: under the partial unloading condition of a multi-mobile-equipment single MEC server, a problem planning model under the MEC system is constructed by taking the total task execution time delay and the total mobile-equipment energy consumption as common optimization indexes, a depth deterministic strategy gradient (DDPG) algorithm in a depth reinforcement learning theory is applied to the calculation unloading problem, and a partial calculation unloading scheme based on the DDPG is provided. The strategy allows the execution of tasks to be completed on the mobile equipment and the single MEC server, can dynamically make a reasonable task unloading decision and allocate bandwidth and computing resources for each user in each time slice, and effectively reduces the weighted cost in two aspects of the total time delay of task processing and the total energy consumption of the mobile equipment.

Drawings

Fig. 1 is a flowchart of a single-server partial computation offload method in a mobile edge environment according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a single-server part computation offload policy implementation process in a mobile edge environment according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of a single-server partial computing offload system in a mobile edge environment according to an embodiment of the present invention;

in fig. 3: 1. an MEC service scene construction module; 2. a network communication module; 3. a task execution module; 4. an MEC system total cost problem planning module; 5. and a time delay and energy consumption combined optimization module.

Fig. 4 is a diagram of a deep deterministic policy gradient network architecture provided by an embodiment of the present invention.

Fig. 5 is a diagrammatic view of the influence of the computing power of the MEC server on the overall system benefit in the simulation experiment result of the embodiment of the present invention.

Fig. 6 is a diagrammatic view of the influence of the number of mobile devices on the total system benefit in the simulation experiment result of the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Aiming at the problems of intensive real-time computing tasks, weak computing power of a mobile terminal, low running efficiency of a core network and the like in the technical field of the current Internet of things and 5G networks, the invention provides a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment, and the invention is described in detail by combining the following drawings, and is oriented to application scenes such as virtual reality games, wearable equipment, intelligent home and the like.

Those skilled in the art can also implement the method for offloading computation of a single server portion in a mobile edge environment by using other steps, and the method for offloading computation of a single server portion in a mobile edge environment provided by the present invention in fig. 1 is only one specific embodiment.

As shown in fig. 1, the single server part computation offload method in the mobile edge environment provided by the present invention includes the following steps:

s001, constructing an application scene of a single base station and a single MEC server of a multi-user terminal, and realizing unloading decision of a mobile terminal task;

s002, designing a network communication model, sending each user task input data to an MEC server through a base station uplink, and reasonably distributing network bandwidth resources;

s003, a local processing and edge unloading parallel task execution module is built, and the feedback efficiency of the application service is improved;

s004, performing target modeling according to network communication time delay and task execution energy consumption, and realizing problem planning of the total cost of the MEC system in a single-base-station multi-user scene;

and S005, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing overall performance optimization based on time delay and energy consumption.

As shown in fig. 2, the method for offloading computation of a single server portion in a mobile edge environment provided by the present invention specifically includes the following steps:

1) And (3) application scene description, and single base station and multi-user terminal single MEC service scenes are constructed. The method is based on the practical life application of the Internet of things, and a partial computation unloading model for the application scene of a single MEC server of a multi-terminal user is constructed.

2) A network communication model is constructed. Each user task input data is sent to the MEC server via the base station uplink. A reasonable network bandwidth resource allocation decision should be made for the system at each time slice.

3) And constructing a task execution model. The calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is greatly prolonged.

4) And planning the total cost of the MEC system. And measuring the performance of the calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the overall cost of the MEC system in the scene of single cell and multiple mobile users according to the network communication and task execution model.

5) And (4) jointly optimizing time delay and energy consumption. A combined dynamic task unloading and resource allocation strategy is designed based on a deep deterministic strategy gradient method, and an optimal compromise scheme capable of simultaneously reducing time delay and energy consumption is found according to changes of a real-time environment.

In step 1) provided by the embodiment of the present invention, constructing an application scenario of a single MEC server for multiple end users comprises the following steps:

1.1 Single MEC scenario modeling of multiple mobile users in a cell

Constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the serial number of the mobile equipment as follows:

N＝{1,2,...}；

and each user device n will generate a divisible compute-intensive task:

for processing task A _n Tolerable maximum time delay requirements.

1.2 Compute offload decision vectors

Task A on user equipment n _n The execution rate offloaded to the MEC server is denoted as x _n ＝[0,1]And N belongs to N, the proportion of executing the task on the local equipment N is 1-x _n The two parts of operations of local execution of the subtask and calculation and unloading of the MEC server are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

x＝{x ₁ ，x ₂ ，...，x _|N| }；

1.3 Set up pending execution task buffer queue

Suppose that the whole MEC system time is divided into a plurality of time lengths tau ₀ And a buffer queue I for waiting execution tasks to be executed in a first-in first-out sequence is set for each user terminal n aiming at the problem of limited mobile user resources _n ，K _n Is queue I _n The total amount of task calculation to be executed is calculated, and each time a new time slice t +1 starts, the total amount K of task calculation to be executed of the mobile terminal n _n The dynamically updated formula of (a) is:

wherein, K _n (t + 1) is the amount of tasks to be executed on the n buffer queue of the t +1 th time slice mobile terminal, x _n (t) indicating task A on the nth user terminal within time slice t _n (t) decision variables for local processing or edge offload, L _n (t) to perform task A _n (t) the amount of computing resources required,

1.4 To solve data transmission and computation problems

For data transmission and calculation problems during task unloading at a server side, the related task information of each mobile user needs to be forwarded to an MEC server through a base station, and then the MEC server allocates corresponding calculation resources to execute tasks. Therefore, the model also needs to consider a real-time allocation scheme of bandwidth and computing resources.

In step 2) provided by the embodiment of the present invention, the design of the network communication model includes the following steps:

defining the uplink network bandwidth resource of the base station as a fixed value W, all mobile users in a cell need to share the bandwidth resource of the base station, and making a reasonable network bandwidth resource allocation decision B = { B } for the system at each time slice ₁ ，B ₂ ，...，B _|N| }. The proportion of the bandwidth resources allocated to the mobile user by the base station in the time slice t is B _n ，B _n ∈[0,1]And satisfy

wherein r is _n Is the transmission power of the nth user equipment, and g _n Representing the gain of the radio transmission channel between user N and the base station over time slice t, N ₀ Is the power spectral density of white gaussian channel noise.

In step 3) provided by the embodiment of the present invention, the construction of the parallel task execution model includes the following steps:

3.1 Classification of computational offload models

Depending on task offload decision x = { x = } ₁ ，x ₂ ，...，x _|N| Each mobile terminal can process its task locally and can also offload a part of the task to be executed on the MEC server. Therefore, the calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is greatly prolonged.

3.2 ) native execution model

The local execution model processes task data and obtains an execution result through self computing capacity of the mobile equipment, and mainly relates to local execution time delay

And equipment execution energy consumption

Two parts overhead. Defining the local CPU calculation frequency of the nth mobile terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x) _n )L _n As task A _n Number of CPU cycles required to execute a portion locally. In addition, for each user terminal, the sub-tasks processed locally should also take into account the waiting time of the current time slice on the task buffer queue waiting to be executed locally

Therefore, the total delay of the nth ue for locally processing the subtasks is defined as:

3.3 Edge unload model

The edge server side processing subtasks typically need to be divided into three steps. First, the mobile user transmits task related data to the base station through a wireless link. When the base station forwards the subtasks unloaded by the mobile user n to the MEC server, the MEC server allocates the computing resources for the MEC server, and the computing resource allocation decision is represented as:

F＝{F ₁ ,F ₂ ,...,F _n ,...,F _|N| }；

and satisfy the conditions

And finally, the MEC server feeds back the execution result to the mobile equipment, and the return data volume of the task execution result is usually very small and is far lower than the uploaded task data, and the return rate of the wireless network is very high, so the method ignores the time delay and the energy consumption cost for returning the task result to the user. The required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

wherein, P _n The uplink transmission power of the nth mobile user. After the data transmission is completed, task A _n The computation time of the offload portion on the edge server is:

wherein, F _n Is MEC clothesProportion of computing resources allocated by the server to the nth mobile subscriber, f ^C The CPU computing power of the MEC server. The total processing delay of the tasks unloaded to the MEC server is the sum of the transmission delay and the execution delay of the tasks, and is represented as follows:

in summary, for the time slice t, since the user local and the server in the MEC system can execute the task in parallel, the task a of the mobile user n in the cell _n The total processing delay is:

complete task A _n The total cost of the energy consumption required is:

in step 4) provided by the embodiment of the invention, the overall cost problem planning of the MEC system comprises the following steps:

aiming at the problems of partial unloading of the task and allocation of network bandwidth and MEC (centralized computation center) computing resources, a weighting factor omega of time delay and energy consumption is introduced ₁ And omega ₂ For adjusting the weight ratio of time and energy consumption costs according to user-specific preferences, and ₁ +ω ₂ =1, and the objective function is formulated as follows:

as shown in the above equation, according to the task execution model,

and

adopting task unloading decision x = { x) under t time slices respectively ₁ ，x ₂ ，...，x _|N| Bandwidth and computational resource allocation policy B = { B = } ₁ ,B ₂ ,...,B _n ,...,B _|N| } and F = { F ₁ ,F ₂ ,...,F _n ,...,F _|N| The task processing time delay and energy consumption are reduced; for is to

Maximizing aims to reduce task A as much as possible _n And the weighted sum of the feedback time and the local mobile equipment energy consumption realizes the maximum utility of the whole MEC system model.

The constraint conditions in C1-C3 respectively represent that the unloading proportion distributed by all mobile users in the cell is not more than 1, and the sum of the proportion of the distributed bandwidth resources and the calculated resources is less than or equal to 1; c4 then represents task A _n The time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

In step 5) provided by the embodiment of the present invention, the joint optimization of time delay and energy consumption includes the following steps:

5.1 ) next action decision

Based on the current state, a next step action decision is given, specifically, a quadruplet M = (S, a) may be utilized _s ,P _ss',a ,R _s,a ) This process is described.

Wherein S is a limited set of states, A is a limited set of actions, S is the system state under the current time slice and S belongs to S, S 'is the next state of the system and S' belongs to S, a is the selected action and a belongs to A, P _ss',a Representing the probability of a transition from the current state s to the next new state s' when performing action a, R _s,a To be heldThe action a is then converted to an immediate direct award acquired by s' via state s.

In addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of future time slices has, i.e., the impact of actions taken based on the state of later time slices on the overall reward value gradually decays, and the weighted sum G (t) of all reward values resulting from the set of actions taken on time slice t in the markov decision process is described as:

wherein, γ ⁱ R (t + i + 1) is the value expression of the reward obtained by the time slice t + i +1 on the time slice t.

5.2 ) function three-factor design

Based on the system model, respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-terminal user:

5.2.1 State): the state space firstly ensures that all information in the environment can be contained, and the change of the environment in each time slice is fully reflected; the present system state is therefore defined as:

s(t)＝[D ₁ (t),D ₂ (t),...,D _|N| (t),L ₁ (t),L ₂ (t),...,L _|N| (t),K ₁ (t),K ₂ (t),...,K _|N| (t),r ₁ (t),r ₂ (t),...,r _|N| (t)]；

the system state comprises four parts, namely the data volume D (t) of the mobile user arriving at the task in the cell on the current time slice t, the number L (t) of the required computing resources, the task volume K (t) to be executed on the task cache queue and the data transmission rate r (t) between the mobile user and the server;

5.2.2 Actions): according to the data volume D (t) of the newly arrived task of the user on each time slice in the state s (t), the number L (t) of required computing resources and the task volume K (t) to be executed on the task cache queue, the Agent of the intelligent Agent is a task A _n Making an offload proportion decision x (t), a ratio of bandwidth resources allocated per mobile user B (t), and server calculationsThe resource allocation ratio F (t), i.e.:

5.2.3 A reward function: the instantaneous reward obtained by the system on the time slice t is set as the objective function value in the formula, namely

The greater the reward R (t) obtained by performing the action a (t), the greater the cost of time for all user tasks to perform at the current time slice t

And cost of energy consumption

The smaller the weighted sum.

5.3 Optimization of depth-deterministic policy gradient (DDPG) algorithm

And dynamically solving a task unloading decision and resource allocation scheme under each time slice by using a depth deterministic strategy gradient algorithm so as to minimize an objective function and reduce the weighted total cost of time delay and energy consumption.

5.3.1 Deep deterministic policy gradient network architecture

The DDPG network structure has two modules of Actor and Critic, and includes four neural networks, which are an Actor current network, an Actor target network, a Critic current network, and a Critic target network, respectively, and the deep deterministic policy gradient network architecture is shown in fig. 5.

The Actor network module is used for selecting an action and giving the action to the Agent of the Agent to execute the action, and the Critic module is used for evaluating a Q value according to the state s (t) and the action a (t). The empirical playback unit stores samples of state transition data obtained from interaction with the environment for later sampling.

5.3.2 Depth deterministic policy gradient algorithm objective function

The objective function of the Critic module in the DDPG algorithm is a timing difference TD-error, which is used for representing the difference between the current action and the expected action, and the loss function of the Critic network is defined as the square of the TD-error:

where m is the number of state transition samples { s (t), a (t), R (t), s (t + 1) } randomly drawn from the empirical playback unit, R (t) being the reward resulting from performing action a (t) over a time slice t, Q' (s (t + 1), a (t + 1), θ ^Q' ) Then represents the evaluation value, θ, of the state s (t + 1) and action a (t + 1) pair at the next time slice t +1 given by the Critic target network ^Q 、θ ^Q' The weighting parameters, Q (s (t), a (t), θ, of the current and target networks, respectively ^Q ) Then the value of the state s (t) and the action a (t) corresponding to the current moment t is judged through the Critic current network, and gamma is a discount factor;

the Actor module in the DDPG algorithm updates parameters of the network in a gradient descending mode, and aims to select an action a (t) = mu (s (t), theta (t)) which can enable the evaluation value to be maximum as far as possible ^μ ) The loss function is:

wherein, theta ^μ Is the weight parameter of the Actor's current network;

the multi-user single-server joint task unloading and resource allocation method based on the DDPG comprises the following steps:

(1) input task request set { A ₁ ,A ₂ ,...,A _N }, uplink network bandwidth W and MEC server computing capacity f ^C Initializing the experience playback unit M and simultaneously randomly initializing the current network mu (s (t), theta) of the Actor ^μ ) And criticic current network Q (s (t), a (t), θ ^Q ) Weight of theta ^μ And theta ^Q ；

(3) generating an action a (t) = μ (s (t), θ, according to current strategy and detection noise loop traversal ^μ )+noise(t)；

(4) Executing the unloading decision and resource allocation of the action a (t) and obtaining a report R (t) and a state s (t + 1) of the next time slice;

As shown in fig. 3, the single-server partial computation offload system in the mobile edge environment provided by the present invention specifically includes:

the MEC service scene constructing module 1 is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of mobile users in a single cell.

And the network communication module 2 is used for sending each user task input data to the MEC server through the uplink of the base station under the condition that the single base station deployed in the cell provides wireless network service for the users.

And the task execution module 3 is used for dividing the calculation unloading model of the task into a local execution model and an edge unloading model, and simultaneously executing the local processing part and the edge unloading part in parallel so as to reduce the task processing time delay and greatly improve the service feedback time.

And the MEC system total cost problem planning module 4 improves the experience quality of each mobile user in the cell, measures the performance of the calculation unloading model from two aspects of time delay and energy consumption, and performs problem planning on the MEC system total cost in a single-cell multi-mobile user scene according to the network communication and task execution model.

And the delay and energy consumption combined optimization module 5 is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization under the scene of multiple mobile users and single server MEC under all the time slices is realized.

The technical effects of the present invention will be described in detail with reference to simulation experiments.

1. Experimental setup

In order to verify the performance of the proposed partial computation unloading algorithm based on the depth certainty strategy gradient, the method adopts python 3.6 to carry out a simulation experiment, and the integrated development environment is JetBrains Pycharm.

2. Content of the experiment

Because the proposed model is different from models in existing edge computing task unloading documents, in order to verify the performance of the proposed algorithm, the cited partial computation unloading algorithm based on the depth deterministic strategy gradient is compared with three baseline algorithms and a partial computation unloading algorithm based on a Depth Q Network (DQN).

(1) Local execution (AL): the tasks generated by the mobile user in each time slice are all performed locally and there is no need to allocate bandwidth resources and computational resources at this time.

(2) Offload execution and Proportional allocation of resources (AOAPF): and unloading all tasks to the MEC server for execution and distributing bandwidth resources and computing resources according to the data size of each mobile user task and the required computing resource number.

(3) Offload execution and Random Fair (AOARF): the tasks are all offloaded to the MEC server execution and bandwidth resources and computing resources are randomly allocated.

(4) Partial computation offload algorithm based on DQN: the same DDPG-based partial computation offload algorithm states, actions, and reward functions as proposed by the present invention, but the action space of the DQN algorithm is discrete. For each mobile user n, a proportional decision alpha is made for its task offloading _n Bandwidth resource allocation ratio decision B _n And computing resource allocation proportion decision F _n The setting level is level =6, and can be respectively expressed as:

then is being satisfied

And

under the constraint of (2), each user-selectable action space is: alpha is alpha _n,level ×B _n,level ×F _n,level 。

3. Experimental results and performance analysis

As shown in FIG. 5, 1000 epicodes are trained for the proposed DDPG-based partial computation offload algorithm in the experiment, each epicode in the learning process contains 1000 time slices, and a weight factor ω of time delay and energy consumption is set ₁ And omega ₂ All of which are 0.5. First, table 1 shows the MEC server computing power f ^C The impact of offloading policy performance is calculated for a lower part of the proposed multi-user single-server MEC system. Experiments show that with MEC server computing power f ^C The proposed DDPG-based computational offload algorithm performs best.

TABLE 1 Effect of MEC Server computing capacity on Total System benefit

As shown in fig. 6, simulation experiments were performed on parameters of the number of terminal devices being 2,3,4,5, and 6, respectively, and table 2 shows the variation trend of the total benefit of the system with the increase of the number of mobile devices. Experiments show that under the condition that the number of mobile users is continuously increased, the total system benefits of the five algorithms, namely DDPG, AL, AOAPF, AOARF and DQN, are in a descending trend, the partial unloading strategy performance based on DDPG is optimal, the time delay and energy consumption long-term weighting total overhead of the whole multi-user single-server MEC system can be effectively reduced, and certain feasibility is achieved.

TABLE 2 influence of number of mobile devices on the overall benefit of the system

It should be noted that embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. It will be appreciated by those skilled in the art that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, for example such code provided on a carrier medium such as a diskette, CD-or DVD-ROM, a programmable memory such as read-only memory (firmware) or a data carrier such as an optical or electronic signal carrier. The apparatus and its modules of the present invention may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of hardware circuits and software, e.g., firmware.

The above description is only for the purpose of illustrating the embodiments of the present invention, and the scope of the present invention should not be limited thereto, and any modifications, equivalents and improvements made by those skilled in the art within the technical scope of the present invention as disclosed in the present invention should be covered by the scope of the present invention.

Claims

1. A single server part calculation unloading method in a mobile edge environment is characterized in that the single server part calculation unloading method in the mobile edge environment specifically comprises the following steps:

constructing an application scene of a single base station and a single MEC server of a multi-user terminal to realize the unloading decision of a user terminal task;

designing a network communication model, and sending task input data of each user terminal to an MEC server through a base station uplink to allocate network bandwidth resources;

step three, a local processing and edge unloading parallel task execution module is constructed, and the feedback efficiency of the application service is improved;

fourthly, performing target modeling according to network communication time delay and task execution energy consumption to realize problem planning of the total cost of the MEC system in a single-base-station multi-user terminal scene;

step five, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing combined performance optimization based on time delay and energy consumption;

in the first step, the construction of the multi-user terminal single MEC server application scene comprises the following steps:

1) Modeling a single MEC scene of a plurality of user terminals in a cell, constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the sequence number of user terminal equipment as follows:

N＝{1,2,...}

and each user terminal n will generate a divisible compute-intensive task:

wherein D is _n Representing task A _n Size of uploaded data amount, L _n Representing the execution of a computing task A _n The amount of computing resources that are required,

for processing task A _n Tolerable maximum time delay requirements;

2) Calculating an offload decision vector, task A on user terminal n _n The execution ratio offloaded to the MEC server is denoted x _n ＝[0,1]And N belongs to N, the ratio of the locally executed tasks on the user terminal N is 1-x _n The two parts of operations of task local execution and MEC server calculation unloading are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

the final computed offload decision vector x for all user terminal tasks can be expressed as:

x＝{x ₁ ，x ₂ ，...，x _|N| }；

3) Setting a task waiting execution cache queue, and supposing that the time of the whole MEC system is divided into a plurality of time lengths which are tau ₀ And a buffer queue I of tasks to be executed, which are executed in a first-in first-out sequence, is set for each user terminal n in view of the problem of limited user terminal resources _n ，K _n Is queue I _n The total amount of task computation to be executed is calculated, and each time a new time slice t +1 begins, the user terminal n waits for the total amount of task computation K to be executed _n The dynamically updated formula of (a) is:

wherein, K _n (t + 1) is the amount of tasks to be executed on the buffer queue of the t +1 th time slice user terminal n, x _n (t) indicates the task on the nth user terminal within a time slice t, A _n (t) is a decision variable for local processing or edge offload, L _n (t) to perform task A _n (t) the amount of computing resources required,

representing the local execution task amount of the user terminal n in the t time slice;

4) The method comprises the steps of solving the problems of data transmission and calculation, firstly forwarding the relevant task information of each user terminal to an MEC server through a base station aiming at the problems of data transmission and calculation when a server-side task is unloaded, and then distributing corresponding calculation resources to execute the task by the MEC server;

in the second step, the design of the network communication model comprises the following steps:

defining the uplink network bandwidth resource of a base station as a fixed value W, all user terminals in a cell need to share the bandwidth resource of the base station, and making a network bandwidth resource allocation decision B = { B ] for a system in each time slice ₁ ，B ₂ ，...，B _|N| }; the ratio of the bandwidth resources allocated to the user terminal by the base station in the time slice t is B _n ，B _n ∈[0,1]And satisfy

Then according to Shannon's formula, when multiple terminals in a cell simultaneously offload tasks to the MEC server, the uplink task transmission rate r between the user terminal n and the server _n Can be expressed as:

wherein, P _n Is the transmission power of the nth user terminal, and g _n Representing the gain of the radio transmission channel between the user terminal N and the base station over a time slice t, N ₀ Is the power spectral density of white gaussian channel noise;

in the third step, the construction of the parallel task execution module comprises the following steps:

calculating an unloading model classification according to task unloading decision x = { x = ₁ ，x ₂ ，...，x _|N| Each user terminal can process the task locally and can unload part of the task to the MEC server for execution; the calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing part and the edge unloading part are simultaneously executed in parallel, so that the time delay of task processing is reduced, and the service is improvedService feedback time;

a local execution model, which processes task data and obtains execution result by user terminal through its own computing power, mainly relating to local execution time delay

And device execution energy consumption

Two parts of overhead; defining the local CPU calculation frequency of the nth user terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x) _n )L _n As task A _n The number of CPU cycles required for the local execution section; in addition, for each user terminal, the locally processed subtasks should also adopt the waiting time of the current time slice on the task cache queue which is locally waiting to be executed

an edge server side needs to be divided into three steps for processing a subtask; firstly, a user terminal transmits task related data to a base station through a wireless link; when the base station forwards the subtask unloaded by the user terminal n to the MEC server, the MEC server allocates the computing resource for the MEC server, and the computing resource allocation decision is represented as:

F＝{F ₁ ,F ₂ ,...,F _n ,...,F _|N| }；

and satisfy the conditions

Finally, the MEC server feeds back the execution result to the user terminal, and the return data volume of the task execution result is usually very small and far lower than the uploaded task data, and the return rate of the wireless network is very high, so that the time delay and the energy consumption cost for returning the task result to the user terminal are ignored; the required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

wherein, P _n The uplink transmission power of the nth user terminal; after the data transmission is completed, task A _n The computation time of the offload part on the edge server is:

wherein, F _n Proportion of computing resources allocated to the nth user terminal for the MEC server, f ^C Computing power for the CPU of the MEC server; the total processing time delay of the tasks unloaded to the MEC server end is the sum of the transmission time delay and the execution time delay of the tasks, and is represented as follows:

for time slice t, since the local user terminal and the server in the MEC system can execute tasks in parallel, task a of user terminal n in the cell _n The total processing delay is:

complete task A _n The total cost of the energy consumption required is:

in the fourth step, the overall cost problem planning of the MEC system comprises the following steps:

aiming at the problems of partial unloading of the task and allocation of network bandwidth and MEC (centralized computation center) computing resources, a weighting factor omega of time delay and energy consumption is introduced ₁ And omega ₂ For adjusting the weight ratio of time and energy consumption costs according to user terminal specific preferences, and ω ₁ +ω ₂ =1, and the objective function is formulated as follows:

as shown in the above formula, according to the task executionThe line model is a model of the line,

and

respectively adopting task unloading decision x = { x ] under t time slices ₁ ，x ₂ ，...，x _|N| And bandwidth resource allocation policy B = { B = } ₁ ,B ₂ ,...,B _n ,...,B _|N| } and the computation resource allocation policy F = { F = ₁ ,F ₂ ,...,F _n ,...,F _|N| The task processing time delay and energy consumption are reduced; to pair

Maximizing aims to reduce task A as much as possible _n The weighted sum of the feedback time and the energy consumption of the local user terminal realizes the maximum utility of the whole MEC system model,

indicating the time delay of the overall local execution,

represents the energy consumption of all local executions;

constraint conditions in C1-C3 respectively represent that the unloading proportion distributed by all user terminals in a cell is not more than 1, and the sum of the proportion of the distributed bandwidth resources and the calculated resources is less than or equal to 1; c4 then represents task A _n The time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

And C5 and C6 ensure that the time and energy consumption costs required to employ the present computational offload scheme are no greater than the time delay and energy consumption of all local executions;

in the fifth step, the joint optimization of time delay and energy consumption comprises the following steps:

giving next action decision based on the current state, and using the quadruplet M = (S, A) _s ,P _ss',a ,R _s,a ) To describe this process;

in addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of the future time slice has, i.e., the effect of actions taken based on the state of the delayed time slice on the overall reward value gradually decays, and the weighted sum G (t) of all reward values resulting from the action of a set of actions taken on time slice t in the markov decision process is described as:

wherein, γ ⁱ R (t + i + 1) is the value expression of the reward obtained by the time slice t + i +1 on the time slice t;

and (3) function three-element design, namely respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-user terminal on the basis of a system model:

the system state comprises four parts, namely data volume D (t) of the user terminal to reach the task in the cell on the current time slice t, the number L (t) of required computing resources, the task volume K (t) to be executed on the task cache queue and the data transmission rate r (t) between the user terminal and the server;

the method comprises the following steps: according to the data volume D (t) of the newly arrived task of the user terminal on each time slice in the state s (t), the number L (t) of required computing resources and the task volume K (t) to be executed on the task cache queue, the Agent of the Agent is the task A _n Making an offload proportion decision x (t), a bandwidth resource ratio B (t) allocated to each ue, and a server computing resource allocation ratio F (t), i.e.:

the reward function: the instantaneous reward obtained by the system on the time slice t is set as the objective function value in the formula, namely

The larger the reward R (t) obtained by executing the action a (t) is, the time cost for executing all the user terminal tasks in the current time slice t is shown

And energy consumption cost

The smaller the weighted sum;

optimizing a depth certainty strategy gradient DDPG algorithm, and dynamically solving a task unloading decision and resource allocation scheme under each time slice by using the depth certainty strategy gradient algorithm so as to minimize a target function and reduce the weighted total cost of time delay and energy consumption;

the Actor network module is used for selecting an action and delivering the action to the Agent of the intelligent Agent to execute the action, and the Critic module is used for evaluating a Q value according to a state s (t) and an action a (t); the experience playback unit stores a state transition data sample obtained by interaction with the environment for later sampling;

the objective function of the criticc module in the DDPG algorithm is a time sequence difference TD-error and is used for representing the difference between the current action and the expected action, and the loss function of the criticc network is defined as the square of the TD-error:

wherein, theta ^μ Is the weight parameter of the Actor's current network;

the joint task unloading and resource allocation method based on the DDPG comprises the following steps of:

(1) input task request set { A ₁ ,A ₂ ,...,A _|N| And upstreamNetwork bandwidth W and MEC server computing power f ^C Initializing the experience playback unit M and simultaneously randomly initializing the current network mu (s (t), theta) of the Actor _μ ) And criticic current network Q (s (t), a (t), θ ^Q ) Weight of theta ^μ And theta ^Q ；

(6) updating the Critic and Actor target networks by using a soft updating mode based on the loss function;

2. A single server portion computing offload system in a mobile edge environment for implementing the single server portion computing offload method in the mobile edge environment of claim 1, wherein the single server portion computing offload system in the mobile edge environment comprises:

an MEC service scene construction module, which is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of user terminals in a single cell;

a network communication module, configured to assume a situation that a single base station deployed in a cell provides a wireless network service for a user terminal, and send task input data of each user terminal to an MEC server through a base station uplink;

the task execution module is used for dividing a calculation unloading model of a task into a local execution model and an edge unloading model, and simultaneously executing a local processing part and an edge unloading part in parallel so as to reduce the task processing time delay and improve the service feedback time;

the MEC system total cost problem planning module is used for improving the experience quality of each user terminal in a cell, measuring the performance of a calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the MEC system total cost in a single-cell multi-user terminal scene according to the network communication and task execution model;

and the delay and energy consumption combined optimization module is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization in the multi-user terminal single server MEC scene under all the time slices is realized.

3. A computer arrangement comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the method of single server portion computing offload in a mobile edge environment of claim 1.

4. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the single server portion computing offload method in a mobile edge environment of claim 1.