CN113950066A - Single server part calculation unloading method, system and equipment under mobile edge environment - Google Patents

Abstract

The invention belongs to the technical field of task unloading and resource allocation of mobile edge computing, and discloses a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment, wherein the method for partial computation unloading of the single server in the mobile edge environment comprises the following steps: a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink; a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm. The invention can dynamically make a reasonable task unloading decision and allocate bandwidth and computing resources for each user in each time slice, thereby effectively reducing the weighted cost in the aspects of the total time delay of task processing and the total energy consumption of the mobile equipment.

Description

Single server part calculation unloading method, system and equipment under mobile edge environment

Technical Field

The invention belongs to the technical field of task unloading and resource allocation of mobile edge computing, and particularly relates to a method, a system and equipment for unloading partial computing of a single server in a mobile edge environment.

Background

At present, the internet of things technology is continuously popularized and the light speed in the field of mobile communication is rapidly developed, a large number of intelligent mobile devices are produced at present, a large number of calculation-intensive tasks are mostly generated by massive intelligent devices and application, the data traffic is increased explosively, and higher requirements are brought to the data calculation capacity and the battery endurance capacity of mobile terminal equipment. Mobile Edge Computing (MEC) enables a terminal device to complete a computing task in a low-delay and low-energy-consumption manner, a Mobile user unloads the task to an Edge base station for processing, and the time delay and energy consumption of a local device can be effectively reduced by utilizing the computing power and energy reserve of an Edge server. The basic idea of edge computing is to change the computing task generated on the mobile device from being offloaded to the cloud end to being offloaded to the network edge end, so that the requirement of computation-intensive applications such as real-time online games and augmented reality on low delay is met. Different task offloading schemes have a large impact on both task completion latency and mobile device energy consumption. Therefore, how to make the most reasonable task offloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved in the current computing offloading framework research, and a large number of task offloading strategies based on different algorithms and optimization objectives are also proposed. According to the optimization target of the algorithm, the calculation unloading strategy can be divided into the following three types:

the first category of strategies is mainly to reduce task processing delays. Bi and Zhang et al consider a multi-user MEC system supported by Wireless power transmission in "Computing Rate Maximization for Wireless Power Mobile-Edge Computing with Binary Computing Offloading" (IEEE Transactions on Wireless Communications,2018,17 (6): 4177 and 4190), study a Binary Computing Offloading strategy in a multi-user MEC network, and solve the problems of Computing Rate weighting and Maximization of all Wireless devices in each time range by using an alternating direction multiplier algorithm, which has the following disadvantages: the energy consumption of one end of the mobile terminal device cannot be considered when computation offloading is performed, the terminal device may have a situation that an offloading policy cannot normally operate due to insufficient electric energy, the mobile device is limited in terms of computation resources, and data is generally processed at the expense of high latency and high device energy consumption.

The second category of strategies is mainly to reduce the device energy consumption. Kuang and Shi et al propose an offload Game mechanism in the "Multi-User offload Game Structure in OFDMAMobile Computing System" (IEEE Transactions on Vehicular Technology,2019,68(12): 12190-. The method has the following defects: mobile devices need to be responsible for handling more and more compute intensive tasks such as high definition video live and face recognition. However, due to limited computing resources and battery endurance, these devices may not be able to process all tasks locally with low latency.

The third type of strategy is mainly time delay and energy consumption trade-off. T. Alfakih and M.Hassan et al, "Task off flow and Resource Allocation for Mobile Edge Computing by Deep retrieval implementation on SARSA" (IEEEAccess,2020,8: 54074-. The method has the following defects: in the actual unloading process, different systems may have different performance requirements instead of being limited to only time delay and energy consumption, and how to make the most reasonable task unloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved urgently in the research of the computing unloading framework.

Through the above analysis, the problems and defects of the prior art are as follows:

(1) in the existing calculation unloading strategy for reducing task processing delay, energy consumption at one end of a mobile terminal device during calculation unloading cannot be considered, so that the total time delay of model task execution is longer, and the total energy consumption of the mobile device is higher.

(2) In the existing calculation unloading strategy for reducing the energy consumption of the equipment, a user more hopes that the sum of the time consumption and the energy consumption of the system can be minimized to reduce the overall consumption of the system, or the time consumption and the energy consumption are balanced.

(3) In the existing calculation unloading strategy with compromise between time delay and energy consumption, different systems may have different performance requirements in the actual unloading process rather than being limited to time delay and energy consumption.

The difficulty in solving the above problems and defects is: in the process of task data transmission, enough wireless network bandwidth is needed, and the edge server also has limited computing resources, so that making the most reasonable task offloading decision and resource allocation scheme according to the terminal device and the surrounding environment factors is a difficult problem to be solved urgently in the research of the computing offloading framework nowadays. The device resources are limited by the conflict between the high-performance task processing requirements, but with the continuous increase of the task unloading scale, the power consumption generated by the execution of the tasks rises sharply, and the benefit of the MEC system is seriously influenced.

The significance of solving the problems and the defects is as follows: under the more complex environment of a single MEC server with multiple mobile devices, a partial computation unloading scheme taking the total time delay of task execution and the total energy consumption of the mobile devices as common optimization indexes is urgently needed.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment.

The invention is realized in such a way that a single server part computation and unloading method under a mobile edge environment comprises the following steps:

constructing an application scene of a single base station and a single MEC server of a multi-user terminal to realize unloading decision of a mobile terminal task;

designing a network communication model, and sending each user task input data to an MEC server through a base station uplink to reasonably distribute network bandwidth resources;

step three, a local processing and edge unloading parallel task execution module is set up, and the feedback efficiency of the application service is improved;

fourthly, performing target modeling according to network communication time delay and task execution energy consumption to realize problem planning of the total cost of the MEC system in a single-base-station multi-user scene;

and fifthly, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing overall performance optimization based on time delay and energy consumption.

Further, in the step one, the constructing the application scenario of the single MEC server for multiple end users includes the following steps:

1) modeling a single MEC scene of a plurality of mobile users in a cell, constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the serial number of mobile equipment as follows:

N＝{1,2,...}

and each user device n will generate a divisible compute-intensive task:

wherein D is_nRepresenting task A_nSize of the uploaded data amount, L_nRepresenting the execution of a computing task A_nThe amount of computing resources that are required,

for processing task A_nTolerable maximum time delay requirements;

2) computing an offload decision vector, task A on user equipment n_nThe execution ratio offloaded to the MEC server is denoted x_n＝[0,1]And N belongs to N, the proportion of the tasks executed on the local equipment N is 1-x_nThe two parts of operations of local execution of the subtask and calculation and unloading of the MEC server are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

the final computational offload decision vector x for all user tasks can be expressed as:

x＝{x₁，x₂，...，x_|N|}；

3) setting a buffer queue for waiting to execute tasks, and supposing that the time of the whole MEC system is divided into a plurality of time lengths which are tau₀And a buffer queue I of tasks to be executed, which are executed in a first-in first-out sequence, is set for each user terminal n in view of the problem of limited resources of mobile users_n，K_nIs queue I_nThe total amount of task calculation to be executed is calculated, and each time a new time slice t +1 starts, the total amount K of task calculation to be executed of the mobile terminal n_nThe dynamically updated formula of (a) is:

wherein, K_n(t +1) is the amount of tasks to be executed on the n buffer queue of the t +1 th time slice mobile terminal, x_n(t) indicating task A on the nth user terminal within time slice t_n(t) decision variables for local processing or edge offload, L_n(t) to perform task A_n(t) the amount of computing resources required,

representing the local execution task amount of the mobile user n in the t time slice;

4) the method solves the problems of data transmission and calculation, and aims at the problems of data transmission and calculation during task unloading at a server side, the information of tasks related to each mobile user is firstly forwarded to an MEC server through a base station, then the MEC server allocates corresponding calculation resources to execute the tasks, and the model also needs to consider a real-time allocation scheme of bandwidth and calculation resources.

Further, in the second step, the design of the network communication model includes the following steps:

defining the uplink network bandwidth resource of the base station as a fixed value W, all mobile users in a cell need to share the bandwidth resource of the base station, and making a reasonable network bandwidth resource allocation decision B ═ B for the system at each time slice₁，B₂，...，B_|N|}; the proportion of the bandwidth resources allocated to the mobile user by the base station in the time slice t is B_n，B_n∈[0,1]And satisfy

Then according to Shannon's formula, when multiple terminals in a cell simultaneously offload tasks to the MEC server, the uplink task transmission rate r between user n and the server_nCan be expressed as:

wherein, P_nIs the transmission power of the nth user equipment, and g_nRepresenting the gain of the radio transmission channel between user N and the base station over time slice t, N₀Is the power spectral density of white gaussian channel noise.

Further, in the third step, the construction of the parallel task execution model includes the following steps:

calculating unloading model classification, and deciding x as { x according to task unloading₁，x₂，...，x_|N|Each mobile terminal can process the task locally and can unload part of the task to the MEC server for execution; dividing the calculation unloading model of the task into a local execution model and an edge unloading model, and simultaneously executing the local processing part and the edge unloading part in parallelThe task processing time delay is reduced, and the service feedback time is prolonged;

a local execution model, wherein the local execution model processes task data and obtains an execution result through the computing power of the mobile equipment, and mainly relates to local execution delay

And device execution energy consumption

Two parts of overhead; defining the local CPU calculation frequency of the nth mobile terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x)_n)L_nAs task A_nThe number of CPU cycles required for the local execution section; in addition, for each user terminal, the sub-tasks processed locally should also take into account the waiting time of the current time slice on the task buffer queue waiting to be executed locally

Therefore, the total delay of the nth ue for locally processing the subtasks is defined as:

in the edge unloading model, the edge server-side processing subtask generally needs to be divided into three steps; firstly, a mobile user transmits task related data to a base station through a wireless link; when the base station forwards the subtasks unloaded by the mobile user n to the MEC server, the MEC server allocates the computing resources for the MEC server, and the computing resource allocation decision is represented as:

F＝{F₁,F₂,...,F_n,...,F_|N|}；

and satisfy the conditions

Finally, the MEC server feeds back the execution result to the mobile equipment, and as the return data volume of the task execution result is usually very small and is far lower than the uploaded task data, and the return rate of the wireless network is very high, the time delay and the energy consumption cost for returning the task result to the user are ignored; the required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

corresponding offload data x_nD_nThe energy consumption during transmission is:

wherein, P_nThe uplink transmission power of the nth mobile user; after the data transmission is completed, task A_nThe computation time of the offload portion on the edge server is:

wherein, F_nProportion of computing resources allocated to the nth mobile user for the MEC server, f^CComputing power for the CPU of the MEC server; the total processing delay of the tasks unloaded to the MEC server is the sum of the transmission delay and the execution delay of the tasks, and is represented as follows:

in summary, for the time slice t, since the user local and the server in the MEC system can execute the task in parallel, the task a of the mobile user n in the cell_nThe total processing delay is:

complete task A_nThe total cost of energy consumption required is:

further, in the fourth step, the MEC system total cost problem planning includes the following steps:

aiming at the problems of task part unloading and network bandwidth and MEC computing resource distribution, a weighting factor omega of time delay and energy consumption is introduced₁And omega₂For adjusting the weight ratio of time and energy consumption costs according to user-specific preferences, and₁+ω ₂1, and the objective function is formulated as follows:

as shown in the above equation, according to the task execution model,

and

adopting task unloading decision x ═ x under t time slices respectively₁，x₂，...，x_|N|Bandwidth and computing resource allocation policy B ═ B₁,B₂,...,B_n,...,B_|N|F ═ F₁,F₂,...,F_n,...,F_|N|The task processing time delay and energy consumption are reduced; to pair

Maximizing aims to reduce task A as much as possible_nThe feedback time and the weighted sum of the energy consumption of the local mobile equipment realize the utility maximization of the whole MEC system model;

the constraint conditions in C1-C3 respectively represent that the unloading proportion allocated to all mobile users in the cell is not more than 1, and the sum of the allocated bandwidth resources and the calculation resource proportion is less than or equal to 1; c4 represents task A_nThe time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

And C5 and C6 ensure that the time and energy costs required to employ the present computing offloading scheme are no greater than the time delay and energy consumption of a full local execution.

Further, in the fifth step, the joint optimization of the time delay and the energy consumption includes the following steps:

and giving a next action decision based on the current state, and using the quadruple M ═ S, A_s,P_ss',a,R_s,a) To describe this process;

wherein S is a limited set of states, A is a limited set of actions, S is the system state under the current time slice and S belongs to S, S 'is the next state of the system and S' belongs to S, a is the selected action and a belongs to A, P_ss',aRepresenting the probability of a transition from the current state s to the next new state s' when performing action a, R_s,aImmediate direct access to be obtained via a state s-to-s' after execution of action aA reward;

in addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of future time slices has, i.e., the impact of actions taken based on the state of later time slices on the overall reward value gradually decays, and the weighted sum g (t) of all reward values resulting from the selection of a set of actions on time slice t in the markov decision process is described as:

wherein, γⁱR (t + i +1) is the value expression of the reward obtained by the time slice t + i +1 on the time slice t;

and (3) function three-element design, namely respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-terminal user on the basis of a system model:

the state is as follows: the state space firstly ensures that all information in the environment can be contained, and the change of the environment in each time slice is fully reflected; the present system state is therefore defined as:

s(t)＝[D₁(t),D₂(t),…,D_|N|(t),L₁(t),L₂(t),…,L_|N|(t),K₁(t),K₂(t),…,K_|N|(t),r₁(t),r₂(t),…,r_|N|(t)]；

the system state comprises four parts, namely data volume D (t) of tasks reached by the mobile user in a cell on a current time slice t, required computing resource number L (t), task volume K (t) to be executed on a task cache queue and data transmission rate r (t) between the mobile user and a server;

the actions are as follows: according to the data volume D (t) of the new arriving task of the user on each time slice in the state s (t), the required computing resource number L (t) and the task volume K (t) to be executed on the task cache queue, the Agent of the Agent is the task A_nMaking an offload proportion decision x (t), a ratio of bandwidth resources allocated by each mobile user B (t), and a server-calculated resource allocation ratio F (t), i.e.:

a(t)＝[x₁(t),x₂(t),…,x_|N|(t),B₁(t),B₂(t),…,B_|N|(t),F₁(t),F₂(t),…,F_|N|(t)]；

the reward function: the instantaneous reward return obtained by the system on the time slice t is set as the objective function value in the formula, namely

The larger the reward R (t) obtained by executing the action a (t), the higher the time cost for executing all the user tasks in the current time slice t is

And cost of energy consumption

The smaller the weighted sum;

optimizing a depth deterministic strategy gradient (DDPG) algorithm, and dynamically solving a task unloading decision and resource allocation scheme under each time slice by using the DDPG algorithm so as to minimize a target function and reduce the weighted total cost of time delay and energy consumption;

the DDPG network structure is provided with an Actor and a Critic module and comprises four neural networks which are an Actor current network, an Actor target network, a Critic current network and a Critic target network respectively;

the action network module is used for selecting an action and delivering the action to the Agent of the intelligent Agent to execute the action, and the criticic module is used for evaluating a Q value according to a state s (t) and an action a (t); the experience playback unit stores a state transition data sample obtained by interaction with the environment for later sampling;

the objective function of the criticc module in the DDPG algorithm is a time sequence difference TD-error and is used for representing the difference between the current action and the expected action, and the loss function of the criticc network is defined as the square of the TD-error:

wherein m is the number of state transition samples { s (t), a (t), R (t), s (t +1) } randomly drawn from the empirical playback unit, R (t) is the reward resulting from performing action a (t) over time slice t, Q '(s (t +1), a (t +1), θ (T }) is the reward resulting from performing action a (t) over time slice t, Q' (s (t +1), a (t +1), θ (T +1), and (T })^Q') Then represents the evaluation value, θ, of the state s (t +1) and action a (t +1) pair at the next time slice t +1 given by the Critic target network^Q、θ^Q'The weighting parameters of the current and target networks, Q (s (t), a (t), θ^Q) Then the value of the corresponding state s (t) and the action a (t) at the current moment t is judged through the Critic current network, and gamma is a discount factor;

the Actor module in the DDPG algorithm updates parameters of the network in a gradient descending mode, and aims to select an action a (t) ═ mu (s (t)), theta (t) capable of enabling evaluation value to be maximum as far as possible^μ) The loss function is:

wherein, theta^μIs the weight parameter of the Actor's current network;

the multi-user single-server joint task unloading and resource allocation method based on the DDPG comprises the following steps:

input task request set { A₁,A₂,...,A_|N|}, uplink network bandwidth W and MEC server computing capacity f^CInitializing the experience playback unit M and simultaneously randomly initializing the current network mu (s (t), theta) of the Actor^μ) And Critic Current network Q (s (t), a (t), θ^Q) Weight of theta^μAnd theta^Q；

Randomly initializing m state transition data for action detection, and receiving an initial state s (0);

traversing and generating action a (t) mu (s (t) and theta according to the current strategy and the detection noise^μ)+noise(t)；

Executing the unloading decision and resource allocation of the action a (t) and obtaining a report R (t) and a state s (t +1) of the next time slice;

randomly selecting M state conversion tuples { s (t), a (t), R (t), s (t +1) }fromthe empirical playback unit M;

updating Critic and Actor target network by using soft updating mode based on loss function

Seventhly, repeating the step III to the step III for T times, wherein T is the number of time slices;

eighthly, repeating the step (E) and the step (C), wherein E is the number of epicode;

and ninthly, outputting a task unloading decision x, a bandwidth resource allocation scheme B and a computing resource allocation scheme F.

Another object of the present invention is to provide a single server part computation offload system in a mobile edge environment for implementing the single server part computation offload method in the mobile edge environment, the single server part computation offload system in the mobile edge environment, comprising:

the MEC service scene construction module is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of mobile users in a single cell;

the network communication module is used for assuming the condition that a single base station deployed in a cell provides wireless network service for users, and sending each user task input data to the MEC server through a base station uplink;

the task execution module is used for dividing a calculation unloading model of a task into a local execution model and an edge unloading model, and executing the local processing part and the edge unloading part simultaneously in parallel so as to reduce the task processing time delay and improve the service feedback time;

the MEC system total cost problem planning module is used for improving the experience quality of each mobile user in a cell, measuring the performance of a calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the MEC system total cost in a single-cell multi-mobile user scene according to the network communication and task execution model;

and the delay and energy consumption combined optimization module is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization in the MEC scene of multiple mobile users under all the time slices is realized.

It is a further object of the invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:

a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink;

a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm.

It is another object of the present invention to provide a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink;

a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm.

The invention provides a single-server partial computation unloading strategy in a mobile edge environment, which is mainly applied to the technical field of task unloading and resource allocation problems of mobile edge computation and mainly solves the problems of task execution delay and equipment energy consumption in an application scene of a multi-user single MEC server. Aiming at the scene and the task type, a dynamic partial task unloading and resource allocation algorithm based on a depth deterministic gradient algorithm is designed, and finally, a partial computation unloading model taking minimization of total delay of all task computation and total energy consumption of terminal equipment as optimization targets is realized. The partial calculation unloading strategy based on the DDPG is compared with the partial calculation unloading strategy based on the DQN through a numerical simulation experiment, the partial calculation unloading strategy based on the DDPG is obtained through comparison research, the performance is optimal, and the time delay and energy consumption long-term weighting total overhead of the whole multi-user single-server MEC system can be effectively reduced.

Compared with the prior art, the invention also has the following advantages: under the partial unloading condition of a multi-mobile-equipment single MEC server, a problem planning model under the MEC system is constructed by taking the total task execution time delay and the total mobile-equipment energy consumption as common optimization indexes, a depth deterministic strategy gradient (DDPG) algorithm in a depth reinforcement learning theory is applied to the calculation unloading problem, and a partial calculation unloading scheme based on the DDPG is provided. The strategy allows the execution of tasks to be completed on the mobile equipment and the single MEC server, can dynamically make a reasonable task unloading decision and allocate bandwidth and computing resources for each user in each time slice, and effectively reduces the weighted cost in two aspects of the total time delay of task processing and the total energy consumption of the mobile equipment.

Drawings

Fig. 1 is a flowchart of a single-server partial computation offload method in a mobile edge environment according to an embodiment of the present invention.

Fig. 2 is a schematic diagram of a single-server partial computation offload policy implementation process in a mobile edge environment according to an embodiment of the present invention.

FIG. 3 is a block diagram of a single server component computing offload system in a mobile edge environment according to an embodiment of the present invention;

in fig. 3: 1. an MEC service scene construction module; 2. a network communication module; 3. a task execution module; 4. an MEC system total cost problem planning module; 5. and a time delay and energy consumption combined optimization module.

Fig. 4 is a diagram of a deep deterministic policy gradient network architecture provided by an embodiment of the present invention.

Fig. 5 is a diagrammatic view of the influence of the computing power of the MEC server on the total benefit of the system in the simulation experiment result of the embodiment of the present invention.

Fig. 6 is a diagrammatic view of the influence of the number of mobile devices on the total system benefit in the simulation experiment result of the embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

Aiming at the problems of intensive real-time computing tasks, weak computing power of a mobile terminal, low running efficiency of a core network and the like in the technical field of the current Internet of things and 5G networks, the invention provides a method, a system and equipment for partial computation unloading of a single server in a mobile edge environment, and the invention is described in detail by combining the following drawings, and is oriented to application scenes such as virtual reality games, wearable equipment, intelligent home and the like.

Those skilled in the art can also implement the method for offloading computation of a single server portion in a mobile edge environment by using other steps, and the method for offloading computation of a single server portion in a mobile edge environment provided by the present invention in fig. 1 is only one specific embodiment.

As shown in fig. 1, the single server part computation offload method in the mobile edge environment provided by the present invention includes the following steps:

s001, constructing an application scene of a single base station and a single MEC server of a multi-user terminal, and realizing unloading decision of a mobile terminal task;

s002, designing a network communication model, sending each user task input data to the MEC server through a base station uplink, and reasonably distributing network bandwidth resources;

s003, a local processing and edge unloading parallel task execution module is built, and the feedback efficiency of the application service is improved;

s004, performing target modeling by depending on network communication time delay and task execution energy consumption, and realizing problem planning of the total cost of the MEC system in a single-base-station multi-user scene;

and S005, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing overall performance optimization based on time delay and energy consumption.

As shown in fig. 2, the method for offloading computation of a single server portion in a mobile edge environment provided by the present invention specifically includes the following steps:

1) and (3) application scene description, and single base station and multi-user terminal single MEC service scenes are constructed. And standing in the angle of the actual life application of the Internet of things, and constructing a partial computation unloading model aiming at the application scene of a single MEC server of a plurality of terminal users.

2) A network communication model is constructed. Each user task input data is sent to the MEC server via the base station uplink. A reasonable network bandwidth resource allocation decision should be made for the system at each time slice.

3) And constructing a task execution model. The calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is greatly prolonged.

4) And planning the total cost of the MEC system. And measuring the performance of the calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the total cost of the MEC system in the single-cell multi-mobile-user scene according to the network communication and task execution model.

5) And (4) jointly optimizing time delay and energy consumption. A combined dynamic task unloading and resource allocation strategy is designed based on a depth certainty strategy gradient method, and an optimal compromise scheme capable of simultaneously reducing time delay and energy consumption is found according to changes of a real-time environment.

In step 1) provided by the embodiment of the present invention, constructing an application scenario of a single MEC server for multiple end users comprises the following steps:

1.1) Single MEC Scenario modeling of multiple Mobile subscribers in a cell

Constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the serial number of the mobile equipment as follows:

N＝{1,2,...}；

and each user device n will generate a divisible compute-intensive task:

wherein D is_nRepresenting task A_nSize of the uploaded data amount, L_nRepresenting the execution of a computing task A_nThe amount of computing resources that are required,

for processing task A_nTolerable maximum time delay requirements.

1.2) calculating an offload decision vector

Task A on user equipment n_nThe execution ratio offloaded to the MEC server is denoted x_n＝[0,1]And N belongs to N, the proportion of the tasks executed on the local equipment N is 1-x_nThe two parts of operations of local execution of the subtask and calculation and unloading of the MEC server are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

the final computational offload decision vector x for all user tasks can be expressed as:

x＝{x₁，x₂，...，x_|N|}；

1.3) setting a waiting task buffer queue

Suppose the whole MEC system time is divided into several time lengths tau₀And a buffer queue I of tasks to be executed, which are executed in a first-in first-out sequence, is set for each user terminal n in view of the problem of limited resources of mobile users_n，K_nIs queue I_nThe total amount of task calculation to be executed is calculated, and each time a new time slice t +1 starts, the total amount K of task calculation to be executed of the mobile terminal n_nThe dynamically updated formula of (a) is:

wherein, K_n(t +1) is the amount of tasks to be executed on the n buffer queue of the t +1 th time slice mobile terminal, x_n(t) is inIndicating task A on nth user terminal in interval t_n(t) decision variables for local processing or edge offload, L_n(t) to perform task A_n(t) the amount of computing resources required,

representing the local execution task amount of the mobile user n in the t time slice;

1.4) solving data transfer and computation problems

For data transmission and calculation problems during task unloading at a server side, the related task information of each mobile user needs to be forwarded to an MEC server through a base station, and then the MEC server allocates corresponding calculation resources to execute tasks. Therefore, the model also needs to consider a real-time allocation scheme of bandwidth and computing resources.

In step 2) provided by the embodiment of the present invention, the design of the network communication model includes the following steps:

defining the uplink network bandwidth resource of the base station as a fixed value W, all mobile users in a cell need to share the bandwidth resource of the base station, and making a reasonable network bandwidth resource allocation decision B ═ B for the system at each time slice₁，B₂，...，B_|N|}. The proportion of the bandwidth resources allocated to the mobile user by the base station in the time slice t is B_n，B_n∈[0,1]And satisfy

Then according to Shannon's formula, when multiple terminals in the cell simultaneously unload tasks to the MEC server, the uplink task transmission rate between user n and server

Can be expressed as:

wherein, P_nIs the transmission power of the nth user equipment, and g_nRepresenting the gain of the radio transmission channel between user N and the base station over time slice t, N₀Is the power spectral density of white gaussian channel noise.

In step 3) provided by the embodiment of the present invention, the construction of the parallel task execution model includes the following steps:

3.1) computational offload model classification

Depending on task offload decision x ═ { x ═ x₁，x₂，...，x_|N|And each mobile terminal can process the task locally and can unload part of the task to the MEC server for execution. Therefore, the calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is greatly prolonged.

3.2) local execution model

The local execution model processes task data and obtains an execution result through the computing capacity of the mobile equipment, and mainly relates to local execution delay

And device execution energy consumption

Two parts overhead. Defining the local CPU calculation frequency of the nth mobile terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x)_n)L_nAs task A_nThe number of CPU cycles required to execute the part locally. Furthermore, for each user terminal, it is localThe subtasks should also take into account the latency on the buffer queue of the task waiting to be executed locally in the current slot

Therefore, the total delay of the nth ue for locally processing the subtasks is defined as:

3.3) edge unloading model

The edge server side processing subtasks typically need to be divided into three steps. First, the mobile user transmits task related data to the base station through a wireless link. When the base station forwards the subtasks unloaded by the mobile user n to the MEC server, the MEC server allocates the computing resources for the MEC server, and the computing resource allocation decision is represented as:

F＝{F₁,F₂,...,F_n,...,F_|N|}；

and satisfy the conditions

And finally, the MEC server feeds back the execution result to the mobile equipment, and the return data volume of the task execution result is usually very small and is far lower than the uploaded task data, and the return rate of the wireless network is very high, so the method ignores the time delay and the energy consumption cost for returning the task result to the user. The required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

corresponding offload data x_nD_nThe energy consumption during transmission is:

wherein, P_nThe uplink transmission power of the nth mobile user. After the data transmission is completed, task A_nThe computation time of the offload portion on the edge server is:

wherein, F_nProportion of computing resources allocated to the nth mobile user for the MEC server, f^CThe CPU computing power of the MEC server. The total processing delay of the tasks unloaded to the MEC server is the sum of the transmission delay and the execution delay of the tasks, and is represented as follows:

in summary, for the time slice t, since the user local and the server in the MEC system can execute the task in parallel, the task a of the mobile user n in the cell_nThe total processing delay is:

complete task A_nThe total cost of energy consumption required is:

in step 4) provided by the embodiment of the present invention, the total cost problem planning of the MEC system includes the following steps:

aiming at the problems of task part unloading and network bandwidth and MEC computing resource distribution, a weighting factor omega of time delay and energy consumption is introduced₁And omega₂For adjusting the weight ratio of time and energy consumption costs according to user-specific preferences, and₁+ω ₂1, and the objective function is formulated as follows:

as shown in the above equation, according to the task execution model,

and

adopting task unloading decision x ═ x under t time slices respectively₁，x₂，...，x_|N|Bandwidth and computing resource allocation policy B ═ B₁,B₂,...,B_n,...,B_|N|F ═ F₁,F₂,...,F_n,...,F_|N|The task processing time delay and energy consumption are reduced; to pair

Maximizing aims to reduce task A as much as possible_nThe weighted sum of the feedback time and the local mobile device energy consumption of the mobile device realizes the maximum utility of the whole MEC system model.

The constraint conditions in C1-C3 respectively represent that the unloading proportion allocated to all mobile users in the cell is not more than 1, and the sum of the allocated bandwidth resources and the calculation resource proportion is less than or equal to 1; c4 represents task A_nThe time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

And C5 and C6 ensure the adoption of the calculation unloading schemeThe time and energy cost required is no more than the time delay and energy consumption of the full local execution.

In step 5) provided by the embodiment of the present invention, the joint optimization of the time delay and the energy consumption includes the following steps:

5.1) decision of next action

The next step of action decision is given based on the current state, specifically, the quadruple M ═ S, a can be utilized_s,P_ss',a,R_s,a) This process is described.

Wherein S is a limited set of states, A is a limited set of actions, S is the system state under the current time slice and S belongs to S, S 'is the next state of the system and S' belongs to S, a is the selected action and a belongs to A, P_ss',aRepresenting the probability of a transition from the current state s to the next new state s' when performing action a, R_s,aThe immediate direct reward is obtained for the transition from state s to s' after performing action a.

In addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of future time slices has, i.e., the impact of actions taken based on the state of later time slices on the overall reward value gradually decays, and the weighted sum g (t) of all reward values resulting from the selection of a set of actions on time slice t in the markov decision process is described as:

wherein, γⁱR (t + i +1) is the value expression of the reward obtained by the time slice t + i +1 on the time slice t.

5.2) function three-factor design

Based on the system model, respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-terminal user:

5.2.1) state: the state space firstly ensures that all information in the environment can be contained, and the change of the environment in each time slice is fully reflected; the present system state is therefore defined as:

s(t)＝[D₁(t),D₂(t),...,D_|N|(t),L₁(t),L₂(t),...,L_|N|(t),K₁(t),K₂(t),...,K_|N|(t),r₁(t),r₂(t),...,r_|N|(t)]；

the system state comprises four parts, namely data volume D (t) of tasks reached by the mobile user in a cell on a current time slice t, required computing resource number L (t), task volume K (t) to be executed on a task cache queue and data transmission rate r (t) between the mobile user and a server;

5.2.2) action: according to the data volume D (t) of the new arriving task of the user on each time slice in the state s (t), the required computing resource number L (t) and the task volume K (t) to be executed on the task cache queue, the Agent of the Agent is the task A_nMaking an offload proportion decision x (t), a ratio of bandwidth resources allocated by each mobile user B (t), and a server-calculated resource allocation ratio F (t), i.e.:

a(t)＝[x₁(t),x₂(t),...,x_|N|(t),B₁(t),B₂(t),...,B_|N|(t),F₁(t),F₂(t),...,F_|N|(t)]；

5.2.3) reward function: the instantaneous reward return obtained by the system on the time slice t is set as the objective function value in the formula, namely

The larger the reward R (t) obtained by executing the action a (t), the higher the time cost for executing all the user tasks in the current time slice t is

And cost of energy consumption

The smaller the weighted sum.

5.3) Deep Deterministic Policy Gradient (DDPG) algorithm optimization

And dynamically solving a task unloading decision and resource allocation scheme under each time slice by using a depth deterministic strategy gradient algorithm so as to minimize an objective function and reduce the weighted total cost of time delay and energy consumption.

5.3.1) deep deterministic policy gradient network architecture

The DDPG network structure has two modules of Actor and Critic, and includes four neural networks, which are an Actor current network, an Actor target network, a Critic current network, and a Critic target network, respectively, and the deep deterministic policy gradient network architecture is shown in fig. 5.

The Actor network module is used for selecting an action and delivering the action to the Agent of the Agent to execute the action, and the criticic module is used for evaluating the Q value according to the state s (t) and the action a (t). The empirical playback unit stores samples of state transition data obtained from interaction with the environment for later sampling.

5.3.2) depth-deterministic policy gradient Algorithm target function

The objective function of the Critic module in the DDPG algorithm is a time sequence difference TD-error, which is used for representing the difference between the current action and the expected action, and the loss function of the Critic network is defined as the square of the TD-error:

wherein m is the number of state transition samples { s (t), a (t), R (t), s (t +1) } randomly drawn from the empirical playback unit, R (t) is the reward resulting from performing action a (t) over time slice t, Q '(s (t +1), a (t +1), θ (T }) is the reward resulting from performing action a (t) over time slice t, Q' (s (t +1), a (t +1), θ (T +1), and (T })^Q') Then represents the evaluation value, θ, of the state s (t +1) and action a (t +1) pair at the next time slice t +1 given by the Critic target network^Q、θ^Q'The weighting parameters of the current and target networks, Q (s (t), a (t), θ^Q) Then the value of the corresponding state s (t) and the action a (t) at the current moment t is judged through the Critic current network, and gamma is a discount factor;

the Actor module in the DDPG algorithm updates parameters of the network in a gradient descending mode, and aims to select an action a (t) ═ mu (s (t)), theta (t) capable of enabling evaluation value to be maximum as far as possible^μ) The loss function is:

wherein, theta^μIs the weight parameter of the Actor's current network;

the multi-user single-server joint task unloading and resource allocation method based on the DDPG comprises the following steps:

input task request set { A₁,A₂,...,A_|N|}, uplink network bandwidth W and MEC server computing capacity f^CInitializing the experience playback unit M and simultaneously randomly initializing the current network mu (s (t), theta) of the Actor^μ) And Critic Current network Q (s (t), a (t), θ^Q) Weight of theta^μAnd theta^Q；

Randomly initializing m state transition data for action detection, and receiving an initial state s (0);

traversing and generating action a (t) mu (s (t) and theta according to the current strategy and the detection noise^μ)+noise(t)；

Executing the unloading decision and resource allocation of the action a (t) and obtaining a report R (t) and a state s (t +1) of the next time slice;

randomly selecting M state conversion tuples { s (t), a (t), R (t), s (t +1) }fromthe empirical playback unit M;

updating Critic and Actor target network by using soft updating mode based on loss function

Seventhly, repeating the step III to the step III for T times, wherein T is the number of time slices;

eighthly, repeating the step (E) and the step (C), wherein E is the number of epicode;

and ninthly, outputting a task unloading decision x, a bandwidth resource allocation scheme B and a computing resource allocation scheme F.

As shown in fig. 3, the single-server partial computation offload system in the mobile edge environment provided by the present invention specifically includes:

the MEC service scene constructing module 1 is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of mobile users in a single cell.

And the network communication module 2 is used for sending each user task input data to the MEC server through the uplink of the base station under the condition that the single base station deployed in the cell provides wireless network service for the users.

And the task execution module 3 is used for dividing the calculation unloading model of the task into a local execution model and an edge unloading model, and executing the local processing part and the edge unloading part simultaneously and parallelly so as to reduce the task processing time delay and greatly improve the service feedback time.

And the MEC system total cost problem planning module 4 improves the experience quality of each mobile user in the cell, measures the performance of the calculation unloading model from two aspects of time delay and energy consumption, and performs problem planning on the MEC system total cost in a single-cell multi-mobile user scene according to the network communication and task execution model.

And the delay and energy consumption combined optimization module 5 is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization in the multi-mobile-user single-server MEC scene under all the time slices is realized.

The technical effects of the present invention will be described in detail with reference to simulation experiments.

1. Experimental setup

In order to verify the performance of the proposed partial computation offload algorithm based on the depth deterministic strategy gradient, the method adopts python 3.6 to perform a simulation experiment, and the integrated development environment is JetBrains Pycarm.

2. Content of the experiment

Because the proposed model is different from models in existing edge computing task unloading documents, in order to verify the performance of the proposed algorithm, the cited partial computation unloading algorithm based on the Deep deterministic strategy gradient is compared with three baseline algorithms and a partial computation unloading algorithm based on a Deep Q Network (DQN).

(1) Local execution (AL): the tasks generated by the mobile user in each time slice are all performed locally and there is no need to allocate bandwidth resources and computational resources at this time.

(2) Offload execution and Proportional allocation of resources (AOAPF): and unloading all tasks to the MEC server for execution and distributing bandwidth resources and computing resources according to the data size of each mobile user task and the required computing resource number.

(3) Offload execution and Random Fair (AOARF): the tasks are all offloaded to the MEC server execution and bandwidth resources and computing resources are randomly allocated.

(4) Partial computation offload algorithm based on DQN: the same DDPG-based partial computation offload algorithm states, actions, and reward functions as proposed by the present invention, but the action space of the DQN algorithm is discrete. For each mobile user n, a proportional decision alpha is made for its task offloading_nBandwidth resource allocation ratio decision B_nAnd computing resource allocation proportion decision F_nThe setting level is 6, which can be respectively expressed as:

then is being satisfied

And

under the constraint of (2), each user-selectable action space is: alpha is alpha_n,level×B_n,level×F_n,level。

3. Experimental results and performance analysis

As shown in FIG. 5, 1000 epicodes are trained for the proposed DDPG-based partial computation offload algorithm in the experiment, each epicode in the learning process contains 1000 time slices, and a weight factor ω of time delay and energy consumption is set₁And ω₂Are all 0.5. First, table 1 shows the MEC server computing power f^CThe impact of offloading policy performance is calculated for a lower part of the proposed multi-user single-server MEC system. Experiments show that with MEC server computing power f^CThe proposed DDPG-based computational offload algorithm performs best.

TABLE 1 Effect of MEC Server computing capacity on Total System benefit

As shown in fig. 6, simulation experiments are performed on parameters of which the number of terminal devices is 2, 3, 4, 5, and 6, respectively, and table 2 shows a variation trend of the total system benefit with an increase in the number of mobile devices. Experiments show that under the condition that the number of mobile users is continuously increased, the total system benefits of the five algorithms of DDPG, AL, AOAPF, AOARF and DQN are in a descending trend, the partial unloading strategy performance based on DDPG is optimal, the time delay and the energy consumption long-term weighting total overhead of the whole multi-user single-server MEC system can be effectively reduced, and certain feasibility is achieved.

TABLE 2 influence of number of mobile devices on the overall benefit of the system

It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided on a carrier medium such as a disk, CD-or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier, for example. The apparatus and its modules of the present invention may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of hardware circuits and software, e.g., firmware.

The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.

Claims (9)

1. A single server part calculation unloading method in a mobile edge environment is characterized in that the single server part calculation unloading method in the mobile edge environment specifically comprises the following steps:

constructing an application scene of a single base station and a single MEC server of a multi-user terminal to realize unloading decision of a mobile terminal task;

designing a network communication model, and sending each user task input data to an MEC server through a base station uplink to reasonably distribute network bandwidth resources;

step three, a local processing and edge unloading parallel task execution module is set up, and the feedback efficiency of the application service is improved;

fourthly, performing target modeling according to network communication time delay and task execution energy consumption to realize problem planning of the total cost of the MEC system in a single-base-station multi-user scene;

and fifthly, dynamically optimizing a task unloading and resource allocation strategy by using a depth certainty strategy gradient method, and realizing overall performance optimization based on time delay and energy consumption.

2. The single-server partial computation offload method in mobile edge environment of claim 1, wherein in the first step, constructing a single-MEC server application scenario for multiple end-users comprises the following steps:

1) modeling a single MEC scene of a plurality of mobile users in a cell, constructing a system network architecture consisting of a single base station, a plurality of user terminals and a public high-performance MEC server, and expressing the serial number of mobile terminal equipment as follows:

N＝{1,2,...}

and each user device n will generate a divisible compute-intensive task:

wherein D is_nRepresenting task A_nSize of the uploaded data amount, L_nRepresenting the execution of a computing task A_nThe amount of computing resources that are required,

for processing task A_nTolerable maximum time delay requirements;

2) computing an offload decision vector, task A on user equipment n_nThe execution ratio offloaded to the MEC server is denoted x_n＝[0,1]And N belongs to N, the proportion of the tasks executed on the local equipment N is 1-x_nThe two parts of operations of local execution of the subtask and calculation and unloading of the MEC server are executed in a parallel mode, so that the total processing time delay of the task is reduced, and the service response time is prolonged;

the final computational offload decision vector x for all user tasks can be expressed as:

x＝{x₁，x₂，...，x_|N|}；

3) setting a buffer queue for waiting to execute tasks, and supposing that the time of the whole MEC system is divided into a plurality of time lengths which are tau₀And a buffer queue I of tasks to be executed, which are executed in a first-in first-out sequence, is set for each user terminal n in view of the problem of limited resources of mobile users_n，K_nIs queue I_nUpper part of the main shaftThe total amount of task calculation of a row, n, is the total amount of task calculation K to be performed by the mobile terminal whenever a new time slice t +1 starts_nThe dynamically updated formula of (a) is:

wherein, K_n(t +1) is the amount of tasks to be executed on the n buffer queue of the t +1 th time slice mobile terminal, x_n(t) indicating task A on the nth user terminal within time slice t_n(t) decision variables for local processing or edge offload, L_n(t) to perform task A_n(t) the amount of computing resources required,

representing the local execution task amount of the mobile user n in the t time slice;

4) the method solves the problems of data transmission and calculation, and aims at the problems of data transmission and calculation during task unloading at a server side, the information of tasks related to each mobile user is firstly forwarded to the MEC server through the base station, and then the MEC server distributes corresponding calculation resources to execute the tasks.

3. The method for offloading computation of a single server portion in a mobile edge environment of claim 1, wherein in the second step, the design of the network communication model comprises the steps of:

defining the uplink network bandwidth resource of the base station as a fixed value W, all mobile users in a cell need to share the bandwidth resource of the base station, and making a reasonable network bandwidth resource allocation decision B ═ B for the system at each time slice₁，B₂，...，B_|N|}; the proportion of the bandwidth resources allocated to the mobile user by the base station in the time slice t is B_n，B_n∈[0,1]And satisfy

Then according to Shannon's formula, when there are more than one cellUplink task transmission rate r between user n and server when terminal unloads task to MEC server simultaneously_nCan be expressed as:

wherein, P_nIs the transmission power of the nth user equipment, and g_nRepresenting the gain of the radio transmission channel between user N and the base station over time slice t, N₀Is the power spectral density of white gaussian channel noise.

4. The partial computation offload method for single server in mobile edge environment according to claim 1, wherein in the third step, the construction of the parallel task execution model includes the following steps:

calculating unloading model classification, and deciding x as { x according to task unloading₁，x₂，...，x_|N|Each mobile terminal can process the task locally and can unload part of the task to the MEC server for execution; the calculation unloading model of the task is divided into a local execution model and an edge unloading model, and the local processing and the edge unloading are simultaneously executed in parallel, so that the task processing time delay is reduced, and the service feedback time is prolonged;

a local execution model, wherein the local execution model processes task data and obtains an execution result through the computing power of the mobile equipment, and mainly relates to local execution delay

And device execution energy consumption

Two parts of overhead; defining the local CPU calculation frequency of the nth mobile terminal as

The computation delay of the locally executed part of the task

Can be expressed as:

wherein (1-x)_n)L_nAs task A_nThe number of CPU cycles required for the local execution section; in addition, for each user terminal, the locally processed subtasks should also adopt the waiting time of the current time slice on the task cache queue which is locally waiting to be executed

Therefore, the total delay of the nth ue for locally processing the subtasks is defined as:

in the edge unloading model, the edge server-side processing subtask generally needs to be divided into three steps; firstly, a mobile user transmits task related data to a base station through a wireless link; when the base station forwards the subtasks unloaded by the mobile user n to the MEC server, the MEC server allocates the computing resources for the MEC server, and the computing resource allocation decision is represented as:

F＝{F₁,F₂,...,F_n,...,F_|N|}；

and satisfy the conditions

Finally, the MEC server feeds back the execution result to the mobile equipment, and the task execution result is fed backThe data volume is usually very small and far lower than the uploaded task data, and the return rate of the wireless network is very high, so that the time delay and the energy consumption cost for returning the task result to the user are ignored; the required transmission time is delayed in the process of transmitting the unloading task data to the base station by the nth user terminal

Is defined as:

corresponding offload data x_nD_nThe energy consumption during transmission is:

wherein, P_nThe uplink transmission power of the nth mobile user; after the data transmission is completed, task A_nThe computation time of the offload portion on the edge server is:

wherein, F_nProportion of computing resources allocated to the nth mobile user for the MEC server, f^CComputing power for the CPU of the MEC server; the total processing delay of the tasks unloaded to the MEC server is the sum of the transmission delay and the execution delay of the tasks, and is represented as follows:

in summary, for the time slice t, since the user local and the server in the MEC system can execute the task in parallel, the mobile user n in the cellTask A_nThe total processing delay is:

complete task A_nThe total cost of energy consumption required is:

5. the method for offloading partial computation of a server in a mobile edge environment of claim 1, wherein in the fourth step, the MEC system total cost problem planning comprises the following steps:

aiming at the problems of task part unloading and network bandwidth and MEC computing resource distribution, a weighting factor omega of time delay and energy consumption is introduced₁And omega₂For adjusting the weight ratio of time and energy consumption costs according to user-specific preferences, and₁+ω₂1, and the objective function is formulated as follows:

s.t.

as shown in the above equation, according to the task execution model,

and

adopting task unloading decision x ═ x under t time slices respectively₁，x₂，...，x_|N|Bandwidth and computing resource allocation policy B ═ B₁,B₂,...,B_n,...,B_|N|F ═ F₁,F₂,...,F_n,...,F_|N|The task processing time delay and energy consumption are reduced; to pair

Maximizing aims to reduce task A as much as possible_nThe feedback time and the weighted sum of the energy consumption of the local mobile equipment realize the utility maximization of the whole MEC system model;

the constraint conditions in C1-C3 respectively represent that the unloading proportion allocated to all mobile users in the cell is not more than 1, and the sum of the allocated bandwidth resources and the calculation resource proportion is less than or equal to 1; c4 represents task A_nThe time delay required for the local execution part and the offload to the server-side part must not exceed its tolerable maximum deadline

And C5 and C6 ensure that the time and energy costs required to employ the present computing offloading scheme are no greater than the time delay and energy consumption of a full local execution.

6. The method for offloading computation of a single server portion in a mobile edge environment of claim 1, wherein in the fifth step, the joint optimization of latency and energy consumption comprises the following steps:

and giving a next action decision based on the current state, and using the quadruple M ═ S, A_s,P_ss',a,R_s,a) To describe this process;

wherein S is a limited set of states, A is a limited set of actions, S is the system state under the current time slice and S belongs to S, S 'is the next state of the system and S' belongs to S, a is the selected action and a belongs to A, P_ss',aRepresenting the probability of a transition from the current state s to the next new state s' when performing action a, R_s,aThe instant direct reward obtained by converting the state s into s' after the action a is executed;

in addition, discount factor γ ∈ [0,1] is used to measure the reward value that the state of future time slices has, i.e., the impact of actions taken based on the state of later time slices on the overall reward value gradually decays, and the weighted sum g (t) of all reward values resulting from the selection of a set of actions on time slice t in the markov decision process is described as:

wherein, γⁱR (t + i +1) is the value expression of the reward obtained by the time slice t + i +1 on the time slice t;

and (3) function three-element design, namely respectively designing the states, actions and reward functions as follows according to the application scene of the single MEC server of the multi-terminal user on the basis of a system model:

the state is as follows: the state space firstly ensures that all information in the environment can be contained, and the change of the environment in each time slice is fully reflected; the present system state is therefore defined as:

s(t)＝[D₁(t),D₂(t),...,D_|N|(t),L₁(t),L₂(t),...,L_|N|(t),K₁(t),K₂(t),...,K_|N|(t),r₁(t),r₂(t),...,r_|N|(t)]；

the system state comprises four parts, namely data volume D (t) of tasks reached by the mobile user in a cell on a current time slice t, required computing resource number L (t), task volume K (t) to be executed on a task cache queue and data transmission rate r (t) between the mobile user and a server;

the actions are as follows: according to the data volume D (t) of the new arriving task of the user on each time slice in the state s (t), the required computing resource number L (t) and the task volume K (t) to be executed on the task cache queue, the Agent of the Agent is the task A_nMaking an offload proportion decision x (t), a ratio of bandwidth resources allocated by each mobile user B (t), and a server-calculated resource allocation ratio F (t), i.e.:

a(t)＝[x₁(t),x₂(t),...,x_|N|(t),B₁(t),B₂(t),...,B_|N|(t),F₁(t),F₂(t),...,F_|N|(t)]；

the reward function: the instantaneous reward return obtained by the system on the time slice t is set as the objective function value in the formula, namely

The larger the reward R (t) obtained by executing the action a (t), the higher the time cost for executing all the user tasks in the current time slice t is

And cost of energy consumption

The smaller the weighted sum;

optimizing a depth deterministic strategy gradient (DDPG) algorithm, and dynamically solving a task unloading decision and resource allocation scheme under each time slice by using the DDPG algorithm so as to minimize a target function and reduce the weighted total cost of time delay and energy consumption;

the DDPG network structure is provided with an Actor and a Critic module and comprises four neural networks which are an Actor current network, an Actor target network, a Critic current network and a Critic target network respectively;

the action network module is used for selecting an action and delivering the action to the Agent of the intelligent Agent to execute the action, and the criticic module is used for evaluating a Q value according to a state s (t) and an action a (t); the experience playback unit stores a state transition data sample obtained by interaction with the environment for later sampling;

the objective function of the criticc module in the DDPG algorithm is a time sequence difference TD-error and is used for representing the difference between the current action and the expected action, and the loss function of the criticc network is defined as the square of the TD-error:

wherein m is the number of state transition samples { s (t), a (t), R (t), s (t +1) } randomly drawn from the empirical playback unit, R (t) is the reward resulting from performing action a (t) over time slice t, Q '(s (t +1), a (t +1), θ (T }) is the reward resulting from performing action a (t) over time slice t, Q' (s (t +1), a (t +1), θ (T +1), and (T })^Q') Then represents the evaluation value, θ, of the state s (t +1) and action a (t +1) pair at the next time slice t +1 given by the Critic target network^Q、θ^Q′The weighting parameters of the current and target networks, Q (s (t), a (t), θ^Q) Then the value of the corresponding state s (t) and the action a (t) at the current moment t is judged through the Critic current network, and gamma is a discount factor;

the Actor module in the DDPG algorithm updates parameters of the network in a gradient descending mode, and aims to select an action a (t) ═ mu (s (t)), theta (t) capable of enabling evaluation value to be maximum as far as possible^μ) The loss function is:

wherein, theta^μIs the weight parameter of the Actor's current network;

the multi-user single-server joint task unloading and resource allocation method based on the DDPG comprises the following steps:

input task request set { A₁,A₂,...,A_|N|}, uplink network bandwidth W and MEC server computing capacity f^CInitializing the experience playback unit M and simultaneously randomly initializing the current network mu (s (t), theta) of the Actor^μ) And Critic Current network Q (s (t), a (t), θ^Q) Weight of theta^μAnd theta^Q；

Randomly initializing m state transition data for action detection, and receiving an initial state s (0);

traversing and generating action a (t) mu (s (t) and theta according to the current strategy and the detection noise^μ)+noise(t)；

Executing the unloading decision and resource allocation of the action a (t) and obtaining a report R (t) and a state s (t +1) of the next time slice;

randomly selecting M state conversion tuples { s (t), a (t), R (t), s (t +1) }fromthe empirical playback unit M;

updating Critic and Actor target network by using soft updating mode based on loss function

Seventhly, repeating the step III to the step III for T times, wherein T is the number of time slices;

eighthly, repeating the step (E) and the step (C), wherein E is the number of epicode;

and ninthly, outputting a task unloading decision x, a bandwidth resource allocation scheme B and a computing resource allocation scheme F.

7. A single server part computation offload system in a mobile edge environment for implementing the single server part computation offload method in the mobile edge environment according to any one of claims 1 to 6, wherein the single server part computation offload system in the mobile edge environment comprises:

the MEC service scene construction module is used for abstracting an application scene that a single base station and an edge server are deployed around a plurality of mobile users in a single cell;

the network communication module is used for assuming the condition that a single base station deployed in a cell provides wireless network service for users, and sending each user task input data to the MEC server through a base station uplink;

the task execution module is used for dividing a calculation unloading model of a task into a local execution model and an edge unloading model, and executing the local processing part and the edge unloading part simultaneously in parallel so as to reduce the task processing time delay and improve the service feedback time;

the MEC system total cost problem planning module is used for improving the experience quality of each mobile user in a cell, measuring the performance of a calculation unloading model from two aspects of time delay and energy consumption, and performing problem planning on the MEC system total cost in a single-cell multi-mobile user scene according to the network communication and task execution model;

and the delay and energy consumption combined optimization module is used for solving the problem in each time slice, so that the delay and energy consumption combined optimization in the MEC scene of multiple mobile users under all the time slices is realized.

8. A computer device, characterized in that the computer device comprises a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to carry out the steps of:

a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink;

a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm.

9. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:

a single base station and multi-user terminal single MEC server scene is established, a network communication model is designed, and each user task input data is sent to the MEC server by depending on a base station uplink;

a local processing and edge unloading parallel task execution module is built, problem planning is carried out on the total cost of the MEC system under the single-cell multi-mobile-user scene by means of network communication delay and task execution energy consumption, and delay and energy consumption are jointly optimized by means of a depth deterministic gradient algorithm.

Images