CN116489711A - Task migration method of edge computing network based on deep reinforcement learning - Google Patents
Task migration method of edge computing network based on deep reinforcement learning Download PDFInfo
- Publication number
- CN116489711A CN116489711A CN202310455141.1A CN202310455141A CN116489711A CN 116489711 A CN116489711 A CN 116489711A CN 202310455141 A CN202310455141 A CN 202310455141A CN 116489711 A CN116489711 A CN 116489711A
- Authority
- CN
- China
- Prior art keywords
- task
- user terminal
- mec server
- migration
- delay
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005012 migration Effects 0.000 title claims abstract description 129
- 238000013508 migration Methods 0.000 title claims abstract description 129
- 238000000034 method Methods 0.000 title claims abstract description 80
- 230000002787 reinforcement Effects 0.000 title claims abstract description 32
- 238000004364 calculation method Methods 0.000 claims abstract description 59
- 238000005265 energy consumption Methods 0.000 claims abstract description 53
- 230000006870 function Effects 0.000 claims abstract description 50
- 230000008569 process Effects 0.000 claims abstract description 37
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 230000005540 biological transmission Effects 0.000 claims description 26
- 238000012545 processing Methods 0.000 claims description 15
- 230000009471 action Effects 0.000 claims description 7
- 230000002829 reductive effect Effects 0.000 claims description 5
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000011160 research Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0958—Management thereof based on metrics or performance parameters
- H04W28/0967—Quality of Service [QoS] parameters
- H04W28/0975—Quality of Service [QoS] parameters for reducing delays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
- G06F9/5088—Techniques for rebalancing the load in a distributed system involving task migration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/092—Reinforcement learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/101—Server selection for load balancing based on network conditions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1023—Server selection for load balancing based on a hash applied to IP addresses or costs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
- H04W28/09—Management thereof
- H04W28/0958—Management thereof based on metrics or performance parameters
- H04W28/0967—Quality of Service [QoS] parameters
- H04W28/0983—Quality of Service [QoS] parameters for optimizing bandwidth or throughput
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/502—Proximity
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Artificial Intelligence (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Computer Hardware Design (AREA)
- Multi Processors (AREA)
Abstract
The invention provides a task migration method of an edge computing network based on deep reinforcement learning. The method comprises the following steps: when the time of the local calculation task of the user terminal is larger than the maximum allowable delay, the user terminal sends a task migration request carrying task data to the MEC server; obtaining the total cost of the task migration process according to the total delay and the total energy consumption in the process of migrating the task of the user terminal to the MEC server; establishing an objective function by taking the total migration cost of the tasks of all user terminals as an optimization target, and solving the objective function by using an edge calculation migration algorithm based on deep reinforcement learning to obtain an optimal task migration strategy; and the user terminal uploads the task data to the appointed MEC server according to the optimal migration strategy, and the MEC server returns the execution result of the task to the user terminal. The method of the invention realizes the weighted sum minimization problem of the task completion time and the energy consumption based on the edge calculation migration strategy of the deep reinforcement learning, and effectively reduces the delay and the energy consumption.
Description
Technical Field
The invention relates to the technical field of mobile edge computing, in particular to a task migration method of an edge computing network based on deep reinforcement learning.
Background
Three-dimensional reconstruction and target recognition serve as important research directions in the field of computer vision, and are widely applied to the fields of intelligent robots, security monitoring, target tracking and the like. Compared with two-dimensional image data, the point cloud data records the position information of the target space, is not influenced by factors such as scale, rotation and illumination, and becomes an important data carrier in the fields of three-dimensional reconstruction and target recognition research. With the development of deep learning and reinforcement learning, especially the generation of a countermeasure network (Generative Adversarial Network, GAN) provides a theoretical basis for solving the problems of three-dimensional reconstruction and target recognition.
The novel intelligent unloading scheme combines artificial intelligent technologies such as deep learning, Q-learning, deep reinforcement learning, federal learning and the like, can better ensure the performance of computing and unloading services, and applies the artificial intelligent related technology to computing and unloading problems, so that the unloading process is more intelligent, thereby reducing energy loss and delay time in the unloading process.
There are a large number of schemes for mobile edge computing, most of which focus on task offloading policies in mobile edge computing, and these schemes often study quasi-static models, ignoring the mobility of the user terminal. In fact, in a real edge computing environment, mobility of the user terminal is definitely present, when considering that a more real user terminal has an edge computing scene of mobility, difficulty of research work is further improved, but mobility of the user terminal is one of important features actually present in mobile edge computing and is not negligible, further research needs to be carried out in the more real edge environment, so that related strategies of design are better applied to the real scene, and further improvement of edge computing research is promoted. For example, two user terminals having mobile devices with limited resources, when a computing task is generated, the user terminal offloads the computing task to the MEC server closest to the moment to execute, then the user terminal randomly moves among a plurality of MEC servers, the plurality of tasks are in competition relation with the computing resources of the MEC servers, the task always executes a strategy which is probably not optimal on the MEC server from which the task is offloaded at the beginning, and therefore, in order to ensure the service quality, the user terminal has to think about whether the task needs to dynamically migrate among the MEC servers along the moving track of the user terminal.
The task migration problem caused by the mobility of the user terminal in the mobile edge computing is not negligible, and especially aiming at the currently emerging mobile internet application such as unmanned, virtual reality, online games and the like, the application is often computationally intensive and highly sensitive to delay, and the task migration problem caused by the mobility is complex when the application request is met.
At present, most of research in mobile edge computing in the prior art is related to task offloading policy research under a static model, mobility-considered research work is less, most of the research work is related to a single-user terminal scene, and task types with strict deadline requests are not considered, in the scene of considering the mobility of a user terminal, the completion time of a task includes the transmission time of input and output results of the task, the execution time on an MEC server and the migration time between MEC servers are influenced by different task migration policies, so that it is very troublesome to ensure the completion deadline of the task. However, for many low-latency applications in an edge computing environment, such as autopilot, online gaming, etc., it is necessary to ensure that the task's completion deadline is one of the important indicators of the quality of service of the user terminal. Although the task migration strategy is fully optimized, the service quality of the user terminal can be improved, from the perspective of the MEC server system, migration energy consumption cost caused by frequent migration cannot be ignored, and further, the task migration strategy is designed under the constraint of migration energy consumption to be of great importance.
Disclosure of Invention
The embodiment of the invention provides a task migration method of an edge computing network based on deep reinforcement learning, which is used for realizing an optimal task migration strategy with lower computing resource requirements.
In order to achieve the above purpose, the present invention adopts the following technical scheme.
A task migration method of an edge computing network based on deep reinforcement learning comprises the following steps:
when the time of the local calculation task of the user terminal in the edge calculation network is larger than the maximum allowable delay, the user terminal sends a task migration request carrying task data to be calculated to the MEC server;
based on the total delay and total energy consumption in migrating the tasks of the user terminal to the MEC server execution,
obtaining the total cost of the task migration process;
establishing an objective function by taking the total migration cost of the tasks of all user terminals as an optimization target, and solving the objective function by using an edge calculation migration algorithm based on deep reinforcement learning to obtain an optimal task migration strategy;
after the optimal migration strategy is obtained through calculation, the user terminal executes the task locally or uploads task data to be processed to the designated MEC server according to the optimal migration strategy, the MEC server distributes corresponding calculation bandwidth resources to process the task data of the user terminal, and the execution result of the task is returned to the user terminal.
When the time of the local calculation task of the user terminal in the edge calculation network is greater than the maximum allowable delay, the user terminal sends a task migration request carrying task data to be calculated to the MEC server, which comprises the following steps:
when a task is needed to be calculated by a user terminal i in an edge computing network, determining the maximum allowable delay of the task and the time for locally computing the task by the user terminal i, if the time for locally computing the task by the user terminal i is larger than the maximum allowable delay, sending a task migration request carrying task data needed to be computed to an MEC server j by the user terminal i, and synchronizing information of the user terminal in an edge cloud by the MEC server j to generate the following task table:
wherein x is ij Indicating whether a computing task is executing locally or is migrated, lambda ij Representing the bandwidth duty cycle, beta, allocated by the MEC server j to the user terminal i ij Representing the computing resource duty cycle that MEC server j allocates to user terminal i,represents the maximum allowable delay of the user terminal i, D i Representing the size of task data to be calculated;
the edge cloud synthesizes the task forms of all MEC servers to obtain a total task set F, and the total task set F is issued to all MEC servers under the edge cloud;
F＝{F ij |i∈{1，2，…，N}，j∈{0，1，2，…，M}} (2)
the method further comprises the following steps:
when the local computation time is less than the maximum allowable delay of the task, the task is executed in the local computation, x ij =0, j=0, the local computation delay of the task is:
x ij x represents whether a computing task is executed locally or migrated to a MEC server ij =0 denotes that the task is executed locally, β ij Representing the CPU duty cycle allocated to the task by the local user terminal; f (f) i l Representing the computing power of the local user terminal, the energy consumption generated by the task i in the local computing is as follows:
E ij l ＝p i l T ij l (4)
wherein p is i l Representing the computing power of the local user terminal;
the total cost of the local calculation task of the user terminal i is as follows:
C ij l ＝∝T ij l +(1-∝)E ij l (5)
wherein, the weight of time and energy consumption cost of the task is respectively represented by the weight of the task, the weight of the task is increased for the delay sensitive task, and the weight of the task is increased for different task types corresponding to different task weights, namely, the weight of the task is increased for the delay sensitive task; for energy consumption sensitive tasks, the value of weight oc is reduced.
Preferably, the obtaining the total cost of the task migration process according to the total delay and the total energy consumption in the process of migrating the task of the user terminal to the MEC server includes:
when the local computation task time of the user terminal i is greater than its maximum allowable delay, then x ij When the value of j is not less than 1 and not less than 1, the user terminal i sends a task migration request carrying task data to be calculated to the MEC server j, and the uplink rate V from the user terminal i to the MEC server j ij up Expressed as:
wherein B is i Representing bandwidth, P, of MEC server j i up Representing transmission power of uploading data in unit time by user terminal i, H i Representing the channel gain of user terminal i in the wireless channel; n (N) o Represents the noise power of the channel g up Represents the target bit error rate, τ (g) up ) Representing the signal-to-noise margin introduced to meet the uplink target bit error rate, d (i, j) representing the distance between the user terminal i and the MEC server j, δ representing the loss index of the transmission channel path;
downlink rate T of MEC server j to user terminal i ij do Expressed as:
user terminal i in task migration calculationTransmission delay for transferring calculation tasks to MEC server jExpressed as:
according to transmission delayObtaining corresponding transmission energy consumption->
After the task data of the user terminal i is transmitted to the MEC server j, the MEC server j calculates the time T of the task ij e Expressed as:
wherein the method comprises the steps ofRepresenting the computing power, beta, of MEC server j ij Representing the computing resource duty cycle allocated to user terminal i by MEC server j;
after the MEC server j completes the task calculation of the user terminal i, the MEC server j sends the calculation result of the task to the transmission delay of the user terminal iThe method comprises the following steps:
wherein D is ij o Returning the data size of the calculation result to the MEC server j;
transmission delay for transmitting the calculation result of the task to the user terminal i according to the MEC server jObtaining the corresponding receiving energy consumption of the user terminal i>
Wherein p is i do A downlink transmission power per unit time for the user terminal i;
combining formulas (8), (10) and (11) to obtain total delay T in the execution process of task migration of user terminal i to MEC server j ij total The method comprises the following steps:
in the execution process of transferring the task of the user terminal i to the MEC server j, the local waiting energy consumption E of the user terminal i ij w The method comprises the following steps:
E ij w ＝p i w T ij total (14)
wherein p is i w For the power of the device during the waiting state of the user terminal i.
Combining formulas (9), (12) and (14), the total energy consumption consumed by the user terminal in the process of transferring the task of the user terminal i to the MEC server j is obtained as follows:
the total delay and total energy consumption in the process of transferring the task of the user terminal i to the MEC server j are combined to obtain the total cost C of the task transferring process ij total The method comprises the following steps:
C ij total ＝∝T ij total +(1-∝)E ij total (16)
oc represents the weight of the time for MEC server j to process the task, 1-oc: the weights of the energy consumption costs of the MEC server j processing tasks are respectively represented, and the number of the MEC servers j is oc epsilon [0,1].
Preferably, said establishing an objective function for the optimization objective with the objective of minimizing the total migration cost of the tasks of all the user terminals includes:
the following objective function is established with the objective of optimizing the migration total cost C of the task of minimizing all user terminals:
s.t.
Z ij ∈{0，1} (17-5)
in the above optimization problem, the objective function (17-1) is the sum of weights for minimizing task execution delay and energy consumption of all users, and is represented by the total cost C, the constraint (17-2) represents that neither the delay generated by selecting local calculation nor the delay generated by selecting migration calculation can be larger than the maximum delay which can be tolerated by the user terminal for task execution, and the constraint (17-3) represents the local user terminal or MEC server componentThe bandwidth duty cycle sum of the allocation tasks must be less than or equal to 1, constraint (17-4) represents that the sum of the CPU duty cycles of all the tasks migrated to the MEC server (or executed locally) is less than or equal to 1, constraint (17-5) represents the valued constraint of the variable, when Z ij =0 means that task i does not select node j for calculation, j is the local user terminal or MEC server, when Z ij =1 means task i selects j node to perform computation.
The method for solving the objective function by using the edge calculation migration algorithm based on the deep reinforcement learning to obtain an optimal task migration strategy comprises the following steps:
defining a state space of an edge computing migration algorithm based on deep reinforcement learning as follows:
S t ＝(C ij (t)) (18)
wherein C is ij (t) represents the weighted sum cost of the task completion time at time slot t and the energy consumption at the MEC server;
defining an action space of an edge computing migration algorithm based on deep reinforcement learning as follows:
A t ＝(∝ ij (t)，γ ij (t)，λ ij (t)，β ij (t)) (19)
after executing the action oc, the MEC server agent will obtain a reward value R (s, a) in a certain state s, defining a reward function of the edge computing migration algorithm based on deep reinforcement learning as:
where V is a prize value determined by the particular environment, r t-1 Is a reward in the previous state;
when the user side has the task unloading, the optimal migration strategy is obtained through the DRL-ECO algorithm;
the DQN algorithm builds a network-optimized loss function through a Q-learning algorithm, and updates the formula as follows:
Q(s，a)←Q(s，a)+α[r+γmaxQ(s′，a′)-Q(s，a)] (41)
defining a loss function as:
L(θ)＝E[(TargetQ-Q(s，a，θ)) 2 Q] (42)
θ is a weight parameter, and the target Q value is:
TargetQ＝r+γmaxQ(s′，a′，θ)] (43)
after obtaining the loss function, bringing the bandwidth, the computing capacity, the energy consumption in unit time, the computing resources and the energy consumption parameters required by the user terminal task of the MEC server into the objective function, solving the objective function by using a gradient descent method, solving the weight function theta of the loss function L (theta), and obtaining the optimal migration strategy of the corresponding task when convergence is achieved through iteration for many times;
and transmitting the task to the MEC server for execution according to the optimal migration strategy, distributing computing resources to perform task processing work by the MEC server according to task requirements, and returning a task processing result to the terminal equipment.
Preferably, the method is applied to a three-dimensional reconstruction task scene of a point cloud of a mobile edge computing environment.
According to the technical scheme provided by the embodiment of the invention, the method constructs the problem of weighted sum minimization of task completion time and energy consumption, and realizes efficient task migration calculation by jointly optimizing the allocation of CPU computing capacity and bandwidth. The edge computing migration system objective function and the objective network are fused, and the network efficiency is further improved through an experience playback mechanism and based on comprehensive optimization consideration of network resources of an optimal migration strategy.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a task migration model diagram of an edge computing network according to an embodiment of the present invention;
FIG. 2 is a process flow diagram of a task migration method for an edge computing network based on deep reinforcement learning according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an implementation of an edge computing migration algorithm (DRL-ECO) based on deep reinforcement learning according to an embodiment of the present invention.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
For the purpose of facilitating an understanding of the embodiments of the invention, reference will now be made to the drawings of several specific embodiments illustrated in the drawings and in no way should be taken to limit the embodiments of the invention.
The embodiment of the invention provides a migration method of a three-dimensional reconstruction task of a point cloud of a mobile edge computing environment based on deep reinforcement learning. The calculation migration of the mobile edge calculation mainly comprises the steps of task division, migration strategies, task uploading, MEC (Mobile Edge Computing, mobile edge calculation) server execution, result return and the like, wherein the task division and the migration strategies are the two most core links.
The task migration model diagram of the edge computing network provided by the embodiment of the invention is shown in fig. 1, and comprises a user terminal layer and an edge cloud layer.
The user terminal layer is composed of N user terminals (UsrEquipment, UE), such as smartphones, AR (Augmented Reality ) devices, etc. These user terminals are deployed in designated areas to perceive whether a task is generated. And when the task data to be processed in the user terminal exceeds the local computing capacity, sending a computing request carrying the task data to the edge cloud.
The edge cloud layer consists of M MEC servers. Each MEC server has two roles:
1) And generating a migration strategy, namely dividing the task into a plurality of subtasks according to the received task data when the MEC server receives the calculation request sent by the user terminal layer, generating an optimal migration strategy by combining the bandwidth of all nodes in the edge cloud and the distribution condition of calculation resources, and then sending the migration strategy to the user terminal corresponding to the user terminal layer.
2) Computing migration and data processing: when the user terminal layer user terminal receives the corresponding migration strategy, the data to be calculated is sent to the corresponding MEC server from the local according to the migration strategy, and the MEC server sends the corresponding result back to the corresponding user terminal through calculation.
Based on the task migration model shown in fig. 1, a process flow of the task migration method of the edge computing network based on deep reinforcement learning provided by the embodiment of the invention is shown in fig. 2, and the method comprises the following processing steps:
step S10: the user terminal sends a task migration request carrying task data to be calculated to the MEC server, the MEC server generates task tables, and the task tables of all the MEC servers are integrated to obtain a total task set.
When the user terminal has tasks to be calculated, firstly determining the maximum allowable delay of the tasks and the time of locally calculating the tasks, if the time of locally calculating the tasks is greater than the maximum allowable delay, sending a task migration request, and obtaining the task data size D to be calculated i And sending to the MEC server. The MEC server synchronizes the information of the user terminal in the edge cloud to generate a task table as follows:
wherein x is ij Indicating whether the computing task is executing locally or is migrated; lambda (lambda) ij Representing the bandwidth duty cycle, beta, allocated by the MEC server j to the user terminal i ij Representing the computing resource duty cycle allocated to user terminal i by MEC server j;represents the maximum allowable delay of the user terminal, D i Representing the size of the task data that needs to be calculated. The MEC server synchronously updates and maintains the task table under the edge cloud, and the synchronization only needs to update the table information after the MEC server makes a task migration strategy each time and broadcast the table information to all MEC servers under the same edge cloud. And the edge cloud synthesizes the task forms of all MEC servers to obtain a total task set F.
F＝{F ij |i∈{1，2，…，N}，j∈{0，1，2，…，M}} (2)
The aggregate task set F will form all MEC servers in the edge cloud. Based on the total task set F, the migration strategy of all the tasks can be obtained, so that the corresponding processing cost of each task is calculated.
When the local computation time is less than the maximum allowable delay for user terminal i, the task will be performed with the user terminal local computation (i.e., x ij =0, j=0), the local execution delay of task i is only related to the processing power of the local CPU. The local computation delay for the task is therefore:
wherein beta is ij Representing the CPU duty cycle allocated to the task by the local user terminal; f (f) i l Representing the computing power (i.e., number of cycles per second of CPU) of the local user terminal. The energy consumption of task i at the time of local computation is therefore:
E ij l ＝p i l T ij l (4)
wherein p is i l Representing the computational power of the local user terminal.
In combination with the local calculation delay formula (3) and the corresponding energy consumption formula (4), the total cost of the local calculation task can be expressed as:
C ij l ＝∝T ij l +(1-∝)E ij l (5)
wherein ∈0,1 ∈1 ∈0 ∈1 @ each represents the weight of the time and energy consumption costs of the task. Because the task weight of each user terminal may be different, different tasks should be assigned different weights according to task types, when the task is a delay sensitive task, the value of the weight is increased appropriately, when the task is an energy consumption sensitive task, the value of the weight is decreased appropriately, and the specific initialization value can be selected preferentially according to a large number of experiments.
Step S20: and obtaining the total cost of the task migration process according to the total delay and the total energy consumption in the process of migrating the task of the user terminal to the MEC server.
When the local computation time of user terminal task i is greater than its maximum allowable delay, then user terminal i chooses to perform the task by computational migration, i.e. when x ij When=1 or j is larger than or equal to 1, the user terminal i sends a data migration request to the edge cloud. Each task migrates to a different MEC server with a different uplink and downlink rate, assuming that the uplink rate of user terminal i to MEC server j is as follows:
wherein B is i Representing the bandwidth of the MEC server: p (P) i up Representing the transmission power of the user terminal i for uploading data in unit time; h i Representing the channel gain of user terminal i in the wireless channel; n (N) o Representing the noise power of the channel; g up Represents the target bit error rate, τ (g) up ) Representing the signal-to-noise margin introduced to meet the uplink target bit error rate: d (i, j) denotes the distance between the user terminal and the MEC server j: delta represents the loss index of the transmission channel path.
Assuming that the downlink and uplink of MEC server j to user terminal i have the same channel environment and noise, the downlink rate can be expressed as:
according to the above steps, in the task migration calculation, the user terminal i uploads the transmission delay of the calculation task to the MEC server jCan be expressed as:
based on the transmission delay, the corresponding transmission energy consumption can be obtained
Wherein pi is up The uplink transmission power per unit time is used for the user terminal i.
After the data is transmitted to the MEC server, the CPU resource allocated to the task by the MEC server is utilized to calculate, so that the calculation time T of the task of the user terminal i on the MEC server j can be obtained ij e The method comprises the following steps:
wherein the method comprises the steps ofRepresenting the computing power, beta, of MEC server j ij Representing the computational resource duty cycle that MEC server j allocates to user terminal i.
After the MEC server j completes the task calculation of the user terminal i, returning the calculation result to the user terminal i, wherein the transmission delay of the MEC server j sending the result to the user terminal i is as follows:
wherein D is ij o And returning the data size of the calculation result to the MEC server j.
Based on the MEC server; the transmission delay of the transmission result to the user terminal i can obtain the corresponding receiving energy consumption of the user terminal i:
wherein p is i do The transmission power of the downlink per unit time is the user terminal i.
Combining equations (8), (10) and (11), the overall delay in the execution of the task migration of the ue i to the MEC server j can be obtained as:
therefore, in the process of transferring the task of the user terminal i to the MEC server j, the local waiting energy consumption of the user terminal i is as follows:
E ij w ＝p i w T ij total (14)
wherein p is i w For the power of the device during the waiting state of the user terminal i.
By combining formulas (9), (12) and (14), it can be obtained that the total energy consumed by the user terminal in the process of migrating the task i to the MEC server j is:
finally, the task combined with the user terminal i is migrated to the MEC server; the total delay and total energy consumption in the execution process, namely formulas (13) and (15), can obtain the total cost of task migration in the execution process from the task migration of the user terminal i to the MEC server j as follows:
C ij total ＝∝T ij total +(1-∝)E ij total (16)
wherein ∈0,1 ∈1 ∈each represents the weight of the time and energy consumption cost of processing tasks by the MEC server. Similar to the above, since each MEC server may handle different task weights, the specific initialization value of the weights may be preferred based on a large number of experiments.
Step S30: an objective function is established for optimization purposes with a view to minimizing the overall cost of migration of tasks for all user terminals.
The time delay and the energy consumption serve as two core indexes for measuring the network performance, a specific optimization target is to minimize the weight sum of the task execution delay and the energy consumption of all user terminals, namely the total cost C serves as an optimization target, and an objective function is established. The method is realized by jointly optimizing migration strategies, bandwidth allocation and computing resource allocation, wherein tasks can be locally executed or migrated, and specific functions are constructed as follows:
s.t.
Z ij ∈{0，1} (17-5)
in the above optimization problem, the objective function (17-1) is the weight and expressed by the total cost C that minimizes the total task completion time and the energy consumption of the user terminal. Constraint (17-2) indicates that neither the delay caused by selecting the local calculation nor the delay caused by selecting the migration calculation is greater than the maximum delay tolerated by the user terminal for task execution. Constraint (17-3) indicates that the bandwidth duty cycle allocated to each task by the node (which may be the local user terminal or the MEC server) and must be less than or equal to 1, i.e. the bandwidth occupied by all user terminal tasks migrating to the MEC server and the bandwidth allocation of the local user terminal is less than or equal to the maximum bandwidth of the MEC server. Similarly, constraint (17-4) represents all migration to MEC servers (or locally)Executing) the sum of the duty CPU duty cycles is less than or equal to 1. Constraint (17-5) represents the value constraint of the variable, when Z ij =0 means that task i does not select node j for computation (j may be the local user terminal or MEC server), when Z ij =1 means task i selects j node to perform computation.
Step S40: and solving the objective function by using an edge computing migration algorithm (DRL-ECO) based on deep reinforcement learning to obtain an optimal task migration strategy.
The edge computing migration algorithm (DRL-ECO) based on the deep reinforcement learning is continuously learned in the MEC server based on the intelligent agent of the interaction between the observation and the environment, so that the optimal migration strategy is obtained.
The implementation principle of the edge computing migration algorithm (DRL-ECO) based on the deep reinforcement learning provided by the embodiment of the invention is shown in fig. 3, and the specific processing procedure comprises:
three key element states, actions, and rewards of a deep reinforcement learning based edge computing migration algorithm (DRL-ECO) are defined:
state space definition:
S t ＝(C ij (t)) (18)
wherein C is ij (t) represents the weighted sum cost of the task completion time at time slot t and the energy consumption at the MEC server.
Defining an action space:
A t ＝(∝ ij (t)，γ ij (t)，λ ij (t)，β ij (t)) (19)
wherein whether or not to migrate ≡ ij (t) local CPU occupancy β ij (t) Bandwidth occupancy γ ij (t) and external CPU occupancy lambda ij (t) formulating the following minimization problem.
The MEC server agent will obtain the prize value R (s, a) in a certain state s after performing every possible action oc. Since typically the reward function is related to the objective function, the objective of the optimization problem is to minimize the total cost of the sum of task execution delays and energy consumption weights for all user terminals, while the solution objective is to obtain the maximum reward function value, the magnitude of the reward function is inversely related to the total cost. Accordingly, the bonus function is defined as:
where V is a prize value determined by the particular environment, r t-1 Is awarded in the previous state.
The DQN algorithm integrated with the edge calculation migration model can better learn the cost function of the reinforcement learning task, and further provides a migration strategy with reference value for the MEC server agent, so that the optimal migration strategy is automatically made based on DON, and the system cost is further reduced.
The DQN algorithm integrated with the edge calculation migration model can better learn the cost function of the reinforcement learning task, and further provides a migration strategy with reference value for the MEC server agent, so that an optimal migration decision is automatically made based on DON, and the system cost is further reduced.
The DQN algorithm builds a network-optimized loss function through a Q-learning algorithm, and updates the formula as follows:
Q(s，a)←Q(s，a)+α[r+γmaxQ(s′，a′)-Q(s，a)] (41)
defining a loss function as:
L(θ)＝E[(TargetQ-Q(s，a，θ)) 2 Q] (42)
θ is a weight parameter, and the target Q value is:
TargetQ＝r+γmaxQ(s′，a′，θ)] (43)
after obtaining the loss function, solving the weight function theta of the loss function L (theta) by using a gradient descent method, and obtaining an optimal migration strategy when convergence is achieved through iteration for a plurality of times. In practical application, parameters such as bandwidth, computing capacity, energy consumption in unit time of the MEC server, computing resources required by a user terminal task, energy consumption and the like are brought in, and the optimal migration strategy of the corresponding task is solved through the neural network.
The optimal migration strategy is as follows: when the number of tasks is smaller at first, all local calculation is performed, and when the calculation task amount does not exceed the local calculation capacity, all local calculation is selected; if the local computing capacity is exceeded, selecting partial migration or whole migration according to the amount of resources required by the task and the method; in addition to all local computing, other cost weighted sums decrease as the number of MEC servers increases, reducing local energy consumption with more external computing resources.
Step S50, the MEC server transmits the optimal migration strategy to the user terminal i, and the user terminal uploads task data to be processed to the designated MEC server j through the wireless access network or the cellular mobile network according to the optimal migration strategy, and the MEC server j allocates corresponding calculation bandwidth resources to process the task data of the user terminal i. And finally, the MEC server j returns the execution result of the task to the user terminal i.
The task migration method can be applied to a three-dimensional reconstruction task scene of the point cloud of the mobile edge computing environment, and the processing process of the three-dimensional reconstruction task of the point cloud of the mobile edge computing environment calculated by the MEC server comprises the following processing steps:
step S1: acquiring an original image in a point cloud scene of a mobile edge computing environment;
s2, extracting image metadata and preprocessing corresponding images;
step S3, segmenting the preprocessed image data set;
step S4: transmitting the images of the subtasks corresponding to the segmentation results to corresponding containers;
step S5: each three-dimensional reconstruction container performs the same three-dimensional reconstruction step for the respective assigned image to generate a three-dimensional sub-model;
step S6: generating a digital surface model and an orthographic image;
step S7: and merging all corresponding three-dimensional sub-models, and performing splicing processing and image enhancement processing on the three-dimensional reconstructed orthographic images.
In summary, the problem of weighted sum minimization of task completion time and energy consumption in the edge cloud is solved by the edge calculation migration strategy based on deep reinforcement learning, delay and energy consumption are effectively reduced, and performance of the three-dimensional reconstruction system is greatly optimized. The algorithm considers the joint optimization of CPU occupation ratio and bandwidth allocation, rapidly realizes the optimal task migration strategy with lower calculation resource requirement, and obviously reduces the network cost.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (7)
1. The task migration method of the edge computing network based on the deep reinforcement learning is characterized by comprising the following steps of:
when the time of the local calculation task of the user terminal in the edge calculation network is larger than the maximum allowable delay, the user terminal sends a task migration request carrying task data to be calculated to the MEC server;
obtaining the total cost of the task migration process according to the total delay and the total energy consumption in the process of migrating the task of the user terminal to the MEC server;
establishing an objective function by taking the total migration cost of the tasks of all user terminals as an optimization target, and solving the objective function by using an edge calculation migration algorithm based on deep reinforcement learning to obtain an optimal task migration strategy;
after the optimal migration strategy is obtained through calculation, the user terminal uploads task data to be processed to the designated MEC server according to the optimal migration strategy, the MEC server distributes corresponding calculation bandwidth resources to process the task data of the user terminal, and the task execution result is returned to the user terminal.
2. The method of claim 1, wherein the time for the user terminal to locally calculate the task in the edge computing network is greater than the maximum allowable delay, and the user terminal sends a task migration request carrying task data to be calculated to the MEC server, comprising:
when a task is needed to be calculated by a user terminal i in an edge computing network, determining the maximum allowable delay of the task and the time for locally computing the task by the user terminal i, if the time for locally computing the task by the user terminal i is larger than the maximum allowable delay, sending a task migration request carrying task data needed to be computed to an MEC server j by the user terminal i, and synchronizing information of the user terminal in an edge cloud by the MEC server j to generate the following task table:
wherein x is ij Indicating whether a computing task is executing locally or is migrated, lambda ij Representing the bandwidth duty cycle, beta, allocated by the MEC server j to the user terminal i ij Representing the computing resource duty cycle that MEC server j allocates to user terminal i,represents the maximum allowable delay of the user terminal i, D i Representing the size of task data to be calculated;
the edge cloud synthesizes the task forms of all MEC servers to obtain a total task set F, and the total task set F is issued to all MEC servers under the edge cloud;
F＝{F ij |i∈{1，2，…，N}，j∈{0，1，2，…，M}} (2)。
3. the method of claim 1, wherein the method further comprises:
when the local computation time is less than the maximum allowable delay for the task, the task is executed at the local computation,
x ij =0, j denotes the number of the MEC server, j=0 when the task is calculated to be executed at the local user terminal
The local computation delay of the task is:
x ij indicating whether a computing task is executing locally or migratingTo MEC server execute, x ij =0 denotes that the task is executed locally, β ij Representing the CPU duty cycle allocated to the task by the local user terminal; f (f) i l Representing the computing power of the local user terminal, the energy consumption generated by the task i in the local computing is as follows:
wherein p is i l Representing the computing power of the local user terminal;
the total cost of the local calculation task of the user terminal i is as follows:
wherein, the weight of time and energy consumption cost of the task is respectively represented by the weight of the task, the weight of the task is increased for the delay sensitive task, and the weight of the task is increased for different task types corresponding to different task weights, namely, the weight of the task is increased for the delay sensitive task; for energy consumption sensitive tasks, the value of weight oc is reduced.
4. A method according to claim 3, wherein the obtaining the total cost of the task migration process based on the total delay and the total energy consumption in the task migration process of the user terminal to the MEC server comprises:
when the local computation task time of the user terminal i is greater than its maximum allowable delay, then x ij ＝1，
When j is more than or equal to 1, the user terminal i sends a task migration request carrying task data to be calculated to the MEC server j, and the uplink rate V from the user terminal i to the MEC server j ij up Expressed as:
wherein B is j Representing bandwidth, P, of MEC server j i up Representing transmission power of uploading data in unit time by user terminal i, H i Representing the channel gain of user terminal i in the wireless channel; n (N) o Represents the noise power of the channel g up Represents the target bit error rate, τ (g) up ) Representing the signal-to-noise margin introduced to meet the uplink target bit error rate, d (i, j) representing the distance between the user terminal i and the MEC server j, δ representing the loss index of the transmission channel path;
downlink rate V of MEC server j to user terminal i ij do Expressed as:
transmission delay of user terminal i to upload computing task to MEC server j in task migration computingExpressed as:
according to transmission delayObtaining corresponding transmission energy consumption->
After the task data of the user terminal i is transmitted to the MEC server j, the MEC server j calculates the time T of the task ij e Expressed as:
wherein f i e Representing the computing power, beta, of MEC server j ij Representing the computing resource duty cycle allocated to user terminal i by MEC server j;
after the MEC server j completes the task calculation of the user terminal i, the MEC server j sends the calculation result of the task to the transmission delay of the user terminal iThe method comprises the following steps:
wherein D is ij o Returning the data size of the calculation result to the MEC server j;
transmission delay for transmitting the calculation result of the task to the user terminal i according to the MEC server jObtaining the corresponding receiving energy consumption of the user terminal i>
Wherein p is i do A downlink transmission power per unit time for the user terminal i;
combining formulas (8), (10) and (11) to obtain total delay T in the execution process of task migration of user terminal i to MEC server j ij total The method comprises the following steps:
in the execution process of transferring the task of the user terminal i to the MEC server j, the local waiting energy consumption E of the user terminal i ij w The method comprises the following steps:
wherein p is i w The power of the equipment in the waiting state process for the user terminal i;
combining formulas (9), (12) and (14), the total energy consumption consumed by the user terminal in the process of transferring the task of the user terminal i to the MEC server j is obtained as follows:
the total delay and total energy consumption in the process of transferring the task of the user terminal i to the MEC server j are combined to obtain the total cost C of the task transferring process ij total The method comprises the following steps:
oc represents the weight of the time for the MEC server j to process the task, 1 oc represents the weight of the energy cost for the MEC server j to process the task, and oc epsilon 0,1, respectively.
5. The method of claim 4, wherein said establishing an objective function for optimization with a view to minimizing the total cost of migration of tasks for all user terminals comprises:
the following objective function is established with the objective of optimizing the migration total cost C of the task of minimizing all user terminals:
s.t.
Z ij ∈{0，1}(17-5)
in the objective function, the objective function (17-1) is the weight sum of minimizing task execution delay and energy consumption of all users, expressed by total cost C, the constraint (17-2) is the maximum delay which can not be more than the maximum delay which can be tolerated by the user terminal for task execution whether the delay generated by local calculation or the delay generated by migration calculation is selected, the constraint (17-3) is the bandwidth ratio sum of the local user terminal or MEC server allocated to each task must be less than or equal to 1, the constraint (17-4) is the sum of the CPU ratio sums of all tasks migrated to the MEC server or executed locally is less than or equal to 1, the constraint (17-5) is the value constraint of a variable, when Z ij =0 means that task i does not select node j for calculation, j is the local user terminal or MEC server, when Z ij =1 means task i selects j node to perform computation.
6. The method of claim 5, wherein solving the objective function using a depth reinforcement learning based edge computing migration algorithm to obtain an optimal task migration strategy comprises:
defining a state space of an edge computing migration algorithm based on deep reinforcement learning as follows:
S t ＝(C ij (t)) (18)
wherein C is ij (t) represents the weighted sum cost of the task completion time at time slot t and the energy consumption at the MEC server;
defining an action space of an edge computing migration algorithm based on deep reinforcement learning as follows:
A t ＝(∝ ij (t)，γ ij (t)，λ ij (t)，β ij (t))(19)
after executing the action oc, the MEC server agent will obtain a reward value R (s, a) in a certain state s, defining a reward function of the edge computing migration algorithm based on deep reinforcement learning as:
where V is a prize value determined by the particular environment, r t-1 Is a reward in the previous state;
when the user side has the task unloading, the optimal migration strategy is obtained through the DRL-ECO algorithm;
the DQN algorithm builds a network-optimized loss function through a Q-learning algorithm, and updates the formula as follows:
Q(s,a)←Q(s,a)+α[r+γmaxQ(s ′ ,a ′ )-Q(s,a)](41)
defining a loss function as:
L(θ)＝E[(TargetQ-Q(s,a,θ)) 2 Q] (42)
θ is a weight parameter, and the target Q value is:
TargetQ＝r+γmaxQ(s ′ ,a ′ ,θ)] (43)
after obtaining the loss function, bringing the bandwidth, the computing capacity, the energy consumption in unit time, the computing resources and the energy consumption parameters required by the user terminal task of the MEC server into the objective function, solving the objective function by using a gradient descent method, solving the weight function theta of the loss function L (theta), and obtaining the optimal migration strategy of the corresponding task when convergence is achieved through iteration for many times;
and transmitting the task to the MEC server for execution according to the optimal migration strategy, distributing computing resources to perform task processing work by the MEC server according to task requirements, and returning a task processing result to the terminal equipment.
7. The method according to any one of claims 1 to 6, wherein the method is applied to a three-dimensional reconstruction task scene of a point cloud of a mobile edge computing environment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310455141.1A CN116489711A (en) | 2023-04-25 | 2023-04-25 | Task migration method of edge computing network based on deep reinforcement learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310455141.1A CN116489711A (en) | 2023-04-25 | 2023-04-25 | Task migration method of edge computing network based on deep reinforcement learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116489711A true CN116489711A (en) | 2023-07-25 |
Family
ID=87219109
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310455141.1A Pending CN116489711A (en) | 2023-04-25 | 2023-04-25 | Task migration method of edge computing network based on deep reinforcement learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116489711A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111953759A (en) * | 2020-08-04 | 2020-11-17 | 国网河南省电力公司信息通信公司 | Collaborative computing task unloading and transferring method and device based on reinforcement learning |
| CN112134916A (en) * | 2020-07-21 | 2020-12-25 | 南京邮电大学 | Cloud edge collaborative computing migration method based on deep reinforcement learning |
| CN113010282A (en) * | 2021-03-03 | 2021-06-22 | 电子科技大学 | Edge cloud collaborative serial task unloading method based on deep reinforcement learning |
| CN113326002A (en) * | 2021-05-22 | 2021-08-31 | 清华大学 | Cloud edge cooperative control system based on computing migration and migration decision generation method |
| CN113950066A (en) * | 2021-09-10 | 2022-01-18 | 西安电子科技大学 | Single server part calculation unloading method, system and equipment under mobile edge environment |
| CN115190033A (en) * | 2022-05-22 | 2022-10-14 | 重庆科技学院 | Cloud edge fusion network task unloading method based on reinforcement learning |
| CN115914230A (en) * | 2022-12-05 | 2023-04-04 | 同济大学 | Adaptive mobile edge computing unloading and resource allocation method |
-
2023
- 2023-04-25 CN CN202310455141.1A patent/CN116489711A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112134916A (en) * | 2020-07-21 | 2020-12-25 | 南京邮电大学 | Cloud edge collaborative computing migration method based on deep reinforcement learning |
| CN111953759A (en) * | 2020-08-04 | 2020-11-17 | 国网河南省电力公司信息通信公司 | Collaborative computing task unloading and transferring method and device based on reinforcement learning |
| CN113010282A (en) * | 2021-03-03 | 2021-06-22 | 电子科技大学 | Edge cloud collaborative serial task unloading method based on deep reinforcement learning |
| CN113326002A (en) * | 2021-05-22 | 2021-08-31 | 清华大学 | Cloud edge cooperative control system based on computing migration and migration decision generation method |
| CN113950066A (en) * | 2021-09-10 | 2022-01-18 | 西安电子科技大学 | Single server part calculation unloading method, system and equipment under mobile edge environment |
| CN115190033A (en) * | 2022-05-22 | 2022-10-14 | 重庆科技学院 | Cloud edge fusion network task unloading method based on reinforcement learning |
| CN115914230A (en) * | 2022-12-05 | 2023-04-04 | 同济大学 | Adaptive mobile edge computing unloading and resource allocation method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Alfakih et al. | Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA | |
| CN109857546B (en) | Multi-server mobile edge computing unloading method and device based on Lyapunov optimization | |
| CN110662238B (en) | Reinforced learning scheduling method and device for burst request under edge network | |
| Zou et al. | A3C-DO: A regional resource scheduling framework based on deep reinforcement learning in edge scenario | |
| CN112422644B (en) | Method and system for unloading computing tasks, electronic device and storage medium | |
| CN116489708B (en) | Meta universe oriented cloud edge end collaborative mobile edge computing task unloading method | |
| CN113543176A (en) | Unloading decision method of mobile edge computing system based on assistance of intelligent reflecting surface | |
| CN114595632A (en) | Mobile edge cache optimization method based on federal learning | |
| Kang et al. | Cooperative UAV resource allocation and task offloading in hierarchical aerial computing systems: A MAPPO based approach | |
| CN113269461A (en) | Game-based edge computing resource management method | |
| CN114205353B (en) | Calculation unloading method based on hybrid action space reinforcement learning algorithm | |
| CN115190033B (en) | Cloud edge fusion network task unloading method based on reinforcement learning | |
| CN113867843B (en) | Mobile edge computing task unloading method based on deep reinforcement learning | |
| CN113626104A (en) | Multi-objective optimization unloading strategy based on deep reinforcement learning under edge cloud architecture | |
| CN116489712A (en) | Mobile edge computing task unloading method based on deep reinforcement learning | |
| CN116893861A (en) | Multi-agent cooperative dependency task unloading method based on space-ground cooperative edge calculation | |
| CN116489226A (en) | Online resource scheduling method for guaranteeing service quality | |
| CN114079882B (en) | Method and device for cooperative calculation and path control of multiple unmanned aerial vehicles | |
| Wang et al. | Digital twin-assisted knowledge distillation framework for heterogeneous federated learning | |
| Kang et al. | Uav-assisted dynamic avatar task migration for vehicular metaverse services: A multi-agent deep reinforcement learning approach | |
| Tao et al. | DRL-Driven Digital Twin Function Virtualization for Adaptive Service Response in 6G Networks | |
| CN116489711A (en) | Task migration method of edge computing network based on deep reinforcement learning | |
| CN112445617A (en) | Load strategy selection method and system based on mobile edge calculation | |
| CN116362327A (en) | Model training method and system and electronic equipment | |
| Yu et al. | Virtual reality in metaverse over wireless networks with user-centered deep reinforcement learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |