CN110798849A - Computing resource allocation and task unloading method for ultra-dense network edge computing - Google Patents

Computing resource allocation and task unloading method for ultra-dense network edge computing Download PDF

Info

Publication number
CN110798849A
CN110798849A CN201910959379.1A CN201910959379A CN110798849A CN 110798849 A CN110798849 A CN 110798849A CN 201910959379 A CN201910959379 A CN 201910959379A CN 110798849 A CN110798849 A CN 110798849A
Authority
CN
China
Prior art keywords
task
base station
computing
edge
time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910959379.1A
Other languages
Chinese (zh)
Inventor
刘家佳
郭鸿志
孙文
张海宾
周小艺
吕剑锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northwestern Polytechnical University
Original Assignee
Northwestern Polytechnical University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northwestern Polytechnical University filed Critical Northwestern Polytechnical University
Priority to CN201910959379.1A priority Critical patent/CN110798849A/en
Publication of CN110798849A publication Critical patent/CN110798849A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/02—Arrangements for optimising operational condition
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W24/00—Supervisory, monitoring or testing arrangements
    • H04W24/06—Testing, supervising or monitoring using simulated traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W72/00—Local resource management
    • H04W72/50—Allocation or scheduling criteria for wireless resources
    • H04W72/53—Allocation or scheduling criteria for wireless resources based on regulatory allocation policies

Abstract

A computing resource allocation and task unloading method for ultra-dense network edge computing comprises the following steps: step 1, establishing a system model based on an ultra-dense network edge computing network of an SDN (software defined network), and acquiring network parameters; step 2, obtaining parameters required by edge calculation: sequentially carrying out local calculation and unloading to an edge server of a macro base station and an edge server connected with a small base station s to obtain an uplink data rate for transmitting a calculation task; step 3, obtaining an optimal computing resource allocation and task unloading strategy by adopting a Q-learning scheme; and step 4, obtaining an optimal computing resource allocation and task unloading strategy by adopting the DQN scheme. It is applicable to dynamic systems by stimulating agents to find optimal solutions on the basis of learning variables. In the Reinforcement Learning (RL) algorithm, Q-Learning performs well in some time-varying networks. By combining the deep learning technology with Q-learning, a learning scheme based on a Deep Q Network (DQN) is provided, so that the benefits of mobile equipment and operators are optimized simultaneously in a time-varying environment, and the learning time is shorter and the convergence is faster than that of a method based on Q-learning. The method achieves the benefit of optimizing Mobile Devices (MDs) and operators simultaneously in a time-varying environment based on DQN.

Description

Computing resource allocation and task unloading method for ultra-dense network edge computing
Technical Field
The invention belongs to the technical field of intelligent computers, and particularly relates to a computing resource allocation and task unloading method for ultra-dense network edge computing.
Background
In today's society, ever increasing Mobile Devices (MDs) with innovative applications place unprecedented demands on user experience and network capacity expansion. The ultra-dense network (UDN) can provide enough baseband resources and ubiquitous connectivity for widely distributed mobile equipment, and the Mobile Edge Computing (MEC) can well meet the requirements of high computing resources and low delay of various novel Internet of things applications. Therefore, the combination of ultra-dense networks and mobile edge computing is considered a promising future technology that can significantly increase the capacity of the system and extend the cloud computing power to the nearest edge servers to meet the ever-increasing computing demands of mobile devices.
However, how to optimize the computing resource configuration to maximize the operator's operating revenue while reducing the cost of the mobile device while meeting the different computing requirements of the mobile device has become a challenge to be solved. In terms of mobile equipment, as a key technology in mobile edge computing, Mobile Edge Computing Offloading (MECO) is an effective scheme for improving the benefit of mobile equipment by selecting an optimal offloading strategy. For operators, the computing resources of the edge servers in different computing demand areas are reasonably configured, so that the operating cost (OPEX) can be remarkably reduced. However, most of the conventional optimization schemes focus on one-time optimization targets in certain scenes and situations, and it is difficult to achieve long-term unloading performance in a changing real-world environment.
Disclosure of Invention
The present invention aims to provide a method for allocating computing resources and unloading tasks for ultra-dense network edge computing, so as to solve the above problems.
In order to achieve the purpose, the invention adopts the following technical scheme:
a computing resource allocation and task unloading method for ultra-dense network edge computing comprises the following steps:
step 1, establishing a system model based on an ultra-dense network edge computing network of an SDN (software defined network), and acquiring network parameters: calculating the average task rate reaching the edge server of the macro base station and the average task rate reaching the edge server of the small base station according to the number of the mobile devices, the number of the macro base stations, the number of the wireless channels of the small base stations connected to the macro base station and the number of the wireless channels connected to the small base stations s in the scene;
step 2, obtaining parameters required by edge calculation: sequentially carrying out local calculation and unloading to an edge server of a macro base station and an edge server connected with a small base station s to obtain an uplink data rate for transmitting a calculation task;
step 3, obtaining an optimal computing resource allocation and task unloading strategy by adopting a Q-learning scheme;
and step 4, obtaining an optimal computing resource allocation and task unloading strategy by adopting the DQN scheme.
Further, in step 1, network parameters are obtained: number of mobile devices in scene C, set
Figure RE-GDA00023008649800000210
Represents; the number of macro base stations is 1, the number of small base stations is B, and the set is usedRepresents; the number of radio channels connected to the macro base station is WmThe number of radio channels connected to the small base station s is Ws(ii) a The computing task types are E in total, and are represented by epsilon {1,2, … E }, the arrival and processing processes of the tasks adopt an M/M/1 queuing model, and the average task rate of the edge server of the macro base station is
Figure RE-GDA0002300864980000021
The average task rate to the edge server of the small base station is
Figure RE-GDA0002300864980000022
The transmission power of mobile m is pm,nAnd a channel gain between the macro base station and the base station of
Figure RE-GDA0002300864980000023
And a small base station has a channel gain of
Figure RE-GDA0002300864980000024
The dependent variables define: c mobile devices are distributed randomly and covered by 1 macro base station and B small base stations, and the distance from each device to the macro base station and the small base stations is DmUse of sets
Figure RE-GDA0002300864980000025
Denotes the distance of all devices to all base stations, wherein
Figure RE-GDA0002300864980000026
α for n-type calculation task requested by Mobile Device (MD) mm,nRepresentation, computation task feature set ofWherein im,nSetting the size of the task to 300-; om,nSetting the CPU period required for processing the computing task as 100-1000 Megacycles;set to 0.5-3s for maximum allowable processing delay αm,nIs described as an offload decision set
Figure RE-GDA0002300864980000029
Wherein the content of the first and second substances,indicating that the task is to be computed locally,
Figure RE-GDA0002300864980000032
indicating that the task should be offloaded to an edge server connected to the macro base station,
Figure RE-GDA0002300864980000033
indicating that mobile device m selects to offload task αm,nEdge server to connected small base station, wherein
Figure RE-GDA0002300864980000034
σ2The background noise power is set to-100 dbm.
Further, define
Figure RE-GDA0002300864980000035
All tasks to be processed at time t; definition ofFor the total computing resources at all edge servers time t,
Figure RE-GDA0002300864980000037
wherein
Figure RE-GDA0002300864980000038
as(t) is the total resources of all edge servers connecting the macro base station and the small base station; definition of
Figure RE-GDA0002300864980000039
The computing resources of the seed are used at time t for all edge servers,
Figure RE-GDA00023008649800000310
wherein sigmam(t) and bs(t) for connecting macro base station and small base stationResources being used by all edge servers; definition of
Figure RE-GDA00023008649800000311
To allocate policies for computing resources to all edge servers,
Figure RE-GDA00023008649800000312
Figure RE-GDA00023008649800000313
further, in the step 2,
a. local compute task αm,nQueue condition v ofm,nSetting the queuing delay introduced per decision periodSecond, CPU cycle o required to process a computational taskm,nAnd computing resources q of a particular mobile devicem. Locally calculating total delay
Figure RE-GDA00023008649800000315
Calculated from the following formula:
Figure RE-GDA00023008649800000316
b. computation tasks α off-load to edge servers of macro base stationm,nQueue condition v ofm,nSetting the queuing delay introduced per decision periodSecond, calculate size of task im,nCPU cycles o required to process a computational taskm,nAnd the computing resource size q of the edge server connected with the macro base stationmBetween 16-32 GHz. Uplink data rate for transmitting computational tasksCalculated from the following formula:
Figure RE-GDA00023008649800000319
final total delay of the unloading mode
Figure RE-GDA00023008649800000320
Calculated from the following formula:
Figure RE-GDA00023008649800000321
c. computation tasks α off-loaded to edge servers connecting small cells sm,nQueue condition v ofm,nSetting the queuing delay introduced per decision period
Figure RE-GDA00023008649800000322
Second, calculate size of task im,nCPU cycles o required to process a computational taskm,nAnd the computing resource size q of the edge server connected with the macro base stationsBetween 4-8 GHz. Uplink data rate for transmitting computational tasks
Figure RE-GDA0002300864980000041
Calculated from the following formula
Figure RE-GDA0002300864980000042
Final total delay of the unloading mode
Figure RE-GDA0002300864980000043
Calculated from the following formula:
Figure RE-GDA0002300864980000044
the calculation time for which three calculation schemes can be obtained is
Figure RE-GDA0002300864980000045
Further, step 3 specifically includes:
1) initializing a Q table, setting all Q values to be 0, and setting a discount factor gamma and a learning rate α;
2) defining system states
Figure RE-GDA0002300864980000046
the system state at time t isα thereinm,n(t) is a calculation task feature, vm,n(t) is the task queuing state,
Figure RE-GDA0002300864980000048
computing resources for the total of all edge servers at time t-1;
Figure RE-GDA0002300864980000049
computing resources of the seeds are used for all the edge servers at the time t;
Figure RE-GDA00023008649800000410
the distance between the mobile equipment and all edge servers is collected;
3) defining actions
Figure RE-GDA00023008649800000411
: the set of actions at time t is
Figure RE-GDA00023008649800000412
Figure RE-GDA00023008649800000413
I.e. a computing resource allocation strategy for all edge servers;
4) defining a reward function
Figure RE-GDA00023008649800000414
Calculating processing time from edges
Figure RE-GDA00023008649800000415
And price function
Figure RE-GDA00023008649800000416
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2The price of computing resources in unit time unit for an edge server connected with a small base station is set as 1; pim,nThe size of the computing resources allocated to the respective computing task; the normalized conversion was defined as γ (x), and the normalized user benefit-expenditure utility was calculated as
Figure RE-GDA00023008649800000515
Rm,n(t) earnings after processing the calculation tasks; the reward value function at time t is
Figure RE-GDA0002300864980000052
Figure RE-GDA0002300864980000053
For the total number of processing tasks at time t,
Figure RE-GDA0002300864980000054
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station;
5) observing the current system state s, and executing corresponding action a according to the Q (s, a) value stored in the Q table, namely resource allocation; then observing the next system state s 'after the action a is executed, and according to the current system state s, the executed action a and the next system state s', obtaining the system state
Figure RE-GDA00023008649800000516
Obtaining the current Q value and storingStoring in a Q table; continuously executing the training process until the training is finished; and finally, obtaining the optimal computing resource allocation and task unloading strategy.
Further, step 4 specifically includes:
1) initializing, namely, evaluating the weight parameter of the network to be theta, the weight parameter of the target network to be theta', discounting factor gamma and learning rate α, and exploring the probability
Figure RE-GDA0002300864980000056
A priori playback
Figure RE-GDA0002300864980000057
2) Defining a system state S: the system state at time t is
Figure RE-GDA0002300864980000058
α thereinm,n(t) is a calculation task feature, vm,n(t) is the task queuing state,
Figure RE-GDA0002300864980000059
computing resources for the total of all edge servers at time t-1;computing resources of the seeds are used for all the edge servers at the time t;
Figure RE-GDA00023008649800000511
the distance between the mobile equipment and all edge servers is collected;
3) define action A: the set of actions at time t is
Figure RE-GDA00023008649800000512
Figure RE-GDA00023008649800000517
I.e. a computing resource allocation strategy for all edge servers;
4) defining a reward function R: calculating processing time from edges
Figure RE-GDA00023008649800000514
And price function
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2The price of computing resources in unit time unit for an edge server connected with a small base station is set as 1; pim,nThe size of the computing resources allocated to the respective computing task; the normalized conversion was defined as γ (x), and the normalized user benefit-expenditure utility could be calculated asRm,n(t) earnings after processing the calculation tasks; the reward value function at time t is
Figure RE-GDA0002300864980000063
Figure RE-GDA0002300864980000064
For the total number of processing tasks at time t,
Figure RE-GDA0002300864980000065
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station;
5) adopts an epsilon-greedy method to explore the probability
Figure RE-GDA0002300864980000066
Gradually decreases from 1 to 0.1; observing the current system state s (t), selecting a random number omega from 0 to 1 ifRandomly selecting an action from all possible actions to execute, namely allocating computing resources; if it is not
Figure RE-GDA0002300864980000068
According to a (t) argmaxaQ(s) (t; a (t); theta) selection action execution; after the corresponding action is executed, a reward function r (t) is calculated, the next system state s (t +1) is observed, and the transition states (s (t); a (t); r (t); s (t +1)) are stored in an a priori replay
Figure RE-GDA0002300864980000069
In (1),
Figure RE-GDA00023008649800000610
wherein Λ (t) ═ { s (t); a (t); r (t); s (t +1) }; randomly selecting MiniBatch from the prior experiment as a sample, and selecting y ═ r (n) + gamma maxa(n+1)Q (s (n + 1); a (n + 1); θ') sets a target network value y; then by a gradient decreasing function
Figure RE-GDA00023008649800000611
Figure RE-GDA00023008649800000612
Updating and evaluating a network weight parameter theta; continuously executing the process, and updating the target network weight parameter theta' to the current evaluation network weight parameter theta after J times; repeating the training process until the training is finished; and finally, obtaining the optimal computing resource allocation and task unloading strategy.
Compared with the prior art, the invention has the following technical effects:
in order to obtain a global view of a network and realize centralized management and scheduling, a Software Defined Network (SDN) is introduced into an architecture. By separating the control plane from the data plane, the SDN controller collects dynamic information from the mobile device and the network. Then, by continuously monitoring the system state, an optimal computing resource configuration strategy and a task unloading strategy can be generated.
Two optimal strategies for generating computing resource allocation and fast decision are proposed for a scene, one is a Q-learning-based method, and the other is a DQN-based method.
For the Q-learning based method, the following advantages are provided:
reinforcement Learning (RL) is an important branch of machine learning that is applicable to dynamic systems by stimulating an agent to find an optimal solution based on learning variables. In the Reinforcement Learning (RL) algorithm, Q-Learning performs well in some time-varying networks. The method achieves the benefit of optimizing Mobile Devices (MDs) and operators simultaneously in a time-varying environment based on Q-learning.
For DQN based methods, the following advantages are present:
on the basis of a Q-learning-based method, aiming at the problems of complex state and action information and long learning process in the Q-learning process, the method combines a deep learning technology with the Q-learning, and provides a learning scheme based on a Deep Q Network (DQN). The method has the advantages of the Q-learning-based method, realizes the effect of optimizing the mobile equipment and the operator simultaneously in the time-varying environment, and has shorter learning time and faster convergence compared with the Q-learning-based method.
Drawings
FIG. 1 is a schematic view of a scene model;
FIG. 2 is a flow chart of an algorithm for a Q-learning based method;
FIG. 3 is an algorithmic flow chart of a DQN-based method;
FIG. 4 is a graph of a training curve tracking weighted network utility comparison at different learning rates for a DQN-based method;
FIG. 5 is a comparison of training curves for DQN-based methods tracking average rewards at different learning rates;
FIG. 6 is a comparison of the convergence behavior of the DQN-based method and the Q-learning-based method with training segments;
FIG. 7 is a comparison of total processing delay with the number of Mobile Devices (MDs) under the DQN-based method, the Q-learning-based method, and the game theory method;
fig. 8 is a comparison of computational resource utilization and number of Mobile Devices (MDs) under the DQN-based method, the Q-learning-based method, and the game theory method.
Detailed Description
The invention is further described below with reference to the accompanying drawings:
step 1): as shown in fig. 1, the system model is established as an ultra-dense network edge computing network based on SDN, and is a scenario of multiple users, multiple task types and multiple MEC servers, a user local server and an edge server can only process one computing task at most simultaneously, and an MEC base station supports multi-user access
Acquiring network parameters: number of mobile devices in scene C, setRepresents; the number of macro base stations is 1, the number of small base stations is B, and the set is used
Figure RE-GDA0002300864980000082
Represents; the number of radio channels connected to the macro base station is WmThe number of radio channels connected to the small base station s is Ws(ii) a The computing task types are E in total, and are represented by epsilon {1,2, … E }, the arrival and processing processes of the tasks adopt an M/M/1 queuing model, and the average task rate of the edge server of the macro base station is
Figure RE-GDA0002300864980000083
The average task rate to the edge server of the small base station is
Figure RE-GDA0002300864980000084
The transmission power of Mobile Device (MD) m is pm,nAnd a channel gain between the macro base station and the base station of
Figure RE-GDA0002300864980000085
And a small base station has a channel gain of
The related variables in the invention are defined as follows: c mobile devices are distributed randomly and covered by 1 macro base station and B small base stations, and the distance from each device to the macro base station and the small base stations is DmUse of setsDenotes the distance of all devices to all base stations, wherein
Figure RE-GDA0002300864980000088
Figure RE-GDA0002300864980000089
α for n types of computing tasks requested by mobile device mm,nRepresentation, computation task feature set of
Figure RE-GDA00023008649800000810
Wherein im,nSetting the size of the task to 300-; om,nSetting the CPU period required for processing the computing task as 100-1000 Megacycles;
Figure RE-GDA00023008649800000811
set to 0.5-3s for maximum allowable processing delay αm,nIs described as an offload decision set
Figure RE-GDA00023008649800000812
Wherein the content of the first and second substances,
Figure RE-GDA00023008649800000813
indicating that the task is to be computed locally,
Figure RE-GDA00023008649800000816
indicating that the task should be offloaded to an edge server connected to the macro base station,
Figure RE-GDA00023008649800000815
indicating moving equipmentBackup m selection offload task αm,nEdge server to connected small base station, wherein
Figure RE-GDA0002300864980000091
σ2The background noise power is set to-100 dbm.
Because this patent needs to carry out many times iterative learning, define the following variable for convenient representation: definition of
Figure RE-GDA0002300864980000092
All tasks to be processed at time t; definition of
Figure RE-GDA0002300864980000093
For the total computing resources at all edge servers time t,
Figure RE-GDA0002300864980000094
Figure RE-GDA0002300864980000095
whereinas(t) is the total resources of all edge servers connecting the macro base station and the small base station; definition of
Figure RE-GDA0002300864980000097
The computing resources of the seed are used at time t for all edge servers,
Figure RE-GDA0002300864980000098
wherein sigmam(t) and bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station; definition of
Figure RE-GDA00023008649800000910
To allocate policies for computing resources to all edge servers,
Figure RE-GDA00023008649800000911
step 2): acquiring parameters required by edge calculation:
a. local compute task αm,nQueue condition v ofm,nSetting the queuing delay introduced per decision period
Figure RE-GDA00023008649800000912
Second, CPU cycle o required to process a computational taskm,nAnd computing resources q of a particular mobile devicem. Locally calculating total delay
Figure RE-GDA00023008649800000913
Calculated from the following formula:
b. computation tasks α off-load to edge servers of macro base stationm,nQueue condition v ofm,nSetting the queuing delay introduced per decision period
Figure RE-GDA00023008649800000915
Second, calculate task size im,nCPU cycles o required to process a computational taskm,nAnd the computing resource size q of the edge server connected with the macro base stationmBetween 16-32 GHz. Uplink data rate for transmitting computational tasks
Figure RE-GDA00023008649800000916
Calculated from the following formula:
Figure RE-GDA00023008649800000917
final total delay of the unloading mode
Figure RE-GDA00023008649800000918
Is represented by the formulaAnd calculating to obtain:
Figure RE-GDA00023008649800000919
c. computation tasks α off-loaded to edge servers connecting small cells sm,nQueue condition v ofm,nSetting the queuing delay introduced per decision periodSecond, calculate task size im,nCPU cycles o required to process a computational taskm,nAnd the computing resource size q of the edge server connected with the macro base stationsBetween 4-8 GHz. Uplink data rate for transmitting computational tasks
Figure RE-GDA00023008649800000921
Calculated from the following formula
Figure RE-GDA00023008649800000922
Final total delay of the unloading modeCalculated from the following formula:
Figure RE-GDA0002300864980000102
the calculation time for which three calculation schemes can be obtained is
Figure RE-GDA0002300864980000103
Step 3): as shown in fig. 2, a Q-learning scheme is adopted to obtain an optimal computing resource allocation and task offloading strategy:
1. initialization is to initialize the Q table, make all Q values 0, set the discount factor γ and the learning rate α.
2. Defining system states
Figure RE-GDA00023008649800001015
: the system state at time t is
Figure RE-GDA0002300864980000104
α thereinm,n(t) is a calculation task feature, vm,n(t) is the task queuing state,
Figure RE-GDA0002300864980000106
the total computing resources at time t-1 are for all edge servers.The seed computing resources are used at time t for all edge servers.
Figure RE-GDA0002300864980000108
Is the set of distances of the mobile device to all edge servers.
3. Defining actions
Figure RE-GDA0002300864980000109
: the set of actions at time t is
Figure RE-GDA00023008649800001010
Figure RE-GDA00023008649800001011
I.e., a policy for the allocation of computing resources to all edge servers.
4. Defining a reward function
Figure RE-GDA00023008649800001012
Calculating processing time from edges
Figure RE-GDA00023008649800001013
And price function
Figure RE-GDA00023008649800001014
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2The price of computing resources in unit time unit for an edge server connected with a small base station is set as 1; pim,nIs the size of the computing resource allocated to the respective computing task. The normalized conversion was defined as γ (x), and the normalized user benefit-expenditure utility could be calculated as
Figure RE-GDA0002300864980000111
Rm,n(t) is the revenue after processing the computing task. the reward value function at time t is
Figure RE-GDA0002300864980000112
Figure RE-GDA0002300864980000113
For the total number of processing tasks at time t,
Figure RE-GDA0002300864980000114
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resource being used by all edge servers connecting the macro base station and the small base station.
5. And observing the current system state s, and executing corresponding action a, namely resource allocation according to the Q (s, a) value stored in the Q table. Then observing the next system state s 'after the action a is executed, and according to the current system state s, the executed action a and the next system state s', obtaining the system state
Figure RE-GDA0002300864980000115
The current Q value is obtained and stored in the Q table. The training process is continuously executed until the training is finished. And finally, obtaining the optimal computing resource allocation and task unloading strategy.
Step 4): as shown in fig. 3, an optimal computation resource allocation and task offloading strategy is obtained by using a DQN scheme:
1. initializing, namely, evaluating the weight parameter of the network to be theta, the weight parameter of the target network to be theta', discounting factor gamma and learning rate α, and exploring the probability
Figure RE-GDA0002300864980000116
A priori playback
2. Defining a system state S: the system state at time t is
Figure RE-GDA0002300864980000118
Figure RE-GDA0002300864980000119
α thereinm,n(t) is a calculation task feature, vm,n(t) is the task queuing state,
Figure RE-GDA00023008649800001110
the total computing resources at time t-1 are for all edge servers.
Figure RE-GDA00023008649800001111
The seed computing resources are used at time t for all edge servers.
Figure RE-GDA00023008649800001117
Is the set of distances of the mobile device to all edge servers.
3. Define action a,: the set of actions at time t is
Figure RE-GDA00023008649800001113
Figure RE-GDA00023008649800001114
I.e., a policy for the allocation of computing resources to all edge servers.
4. Defining a reward function R: calculating processing time from edges
Figure RE-GDA00023008649800001115
And price function
Figure RE-GDA00023008649800001116
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2The price of computing resources in unit time unit for an edge server connected with a small base station is set as 1; pim,nIs the size of the computing resource allocated to the respective computing task. The normalized conversion was defined as γ (x), and the normalized user benefit-expenditure utility could be calculated as
Figure RE-GDA0002300864980000121
Rm,n(t) is the revenue after processing the computing task. the reward value function at time t is
Figure RE-GDA0002300864980000122
For the total number of processing tasks at time t,
Figure RE-GDA0002300864980000124
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resource being used by all edge servers connecting the macro base station and the small base station.
5. Adopts an epsilon-greedy method to explore the probability
Figure RE-GDA0002300864980000125
Gradually decreasing from 1 to 0.1. Observing the current system state s (t), selecting a random number omega from 0 to 1 if
Figure RE-GDA0002300864980000126
Randomly selecting an action from all possible actions to execute, namely allocating computing resources; if it is not
Figure RE-GDA0002300864980000127
According to a (t) argmaxaQ(s) (t; a (t); theta) is performed by a selection action. After the corresponding action is executed, a reward function r (t) is calculated, the next system state s (t +1) is observed, and the transition states (s (t); a (t); r (t); s (t +1)) are stored in an a priori replay
Figure RE-GDA00023008649800001212
In (1),
Figure RE-GDA0002300864980000129
Figure RE-GDA00023008649800001210
wherein Λ (t) ═ { s (t); a (t); r (t); s (t +1) }. Randomly selecting MiniBatch from the prior experiment as a sample, and selecting y ═ r (n) + gamma maxa(n+1)Q (s (n + 1); a (n + 1); θ') sets the target network value y. Then by a gradient decreasing function
Figure RE-GDA00023008649800001211
And updating the evaluation network weight parameter theta. And continuously executing the process, and updating the target network weight parameter theta' to the current evaluation network weight parameter theta after J times of operation. The training process is repeated until the training is finished. And finally, obtaining the optimal computing resource allocation and task unloading strategy.
Fig. 4 shows that the training curve based on the DQN method tracks the utility contrast of the weighting network under different learning rates, and the learning rate is an important parameter influencing the convergence performance. It can be seen that the training process does not converge within the set number of cycles when the learning rate is set to 0.1, and converges much slower when the learning rate is set to 0.001. In contrast, with a learning rate of 0.01, the training process converges faster, reaching higher practicability.
Fig. 5 shows that the training curve based on the DQN method tracks the average reward contrast at different learning rates. It can be seen in this figure that the curve at a learning rate of 0.01 converges most quickly around a period of 400. Although the bonus rate achieved at the learning rate of 0.001 is slightly reduced at 0.01, it converges slowly at about a period of 750. In addition, in the case where the learning rate is 0.1, the algorithm can obtain a higher reward than the other two learning rates at some time, but cannot ensure convergence. Therefore, we chose the learning rate of 0.01 as a simulation parameter for subsequent experiments.
Fig. 6 shows a comparison of the convergence behavior of the DQN-based method and the Q-learning-based method with training segments, which shows that both the DQN-based method and the Q-learning-based method can converge to near optimal utility values in short-term training. However, in contrast to the DQN-based method, which converges when the period is around 400, the Q-learning-based method converges when the period exceeds 1000. The improvement of DQN convergence speed is mainly benefited by dual Q network and memory space
Figure RE-GDA0002300864980000131
The use of (1).
As shown in fig. 7, which is a comparison of the total processing delay and the number of mobile devices under the DQN-based method, the Q-learning-based method and the game theory method, it can be seen from the figure that the DQN-based method and the Q-learning-based method can significantly reduce the processing delay compared to the game theory method. The reason for this phenomenon is that our proposed solution looks at the long-term offload performance of time-varying systems.
Fig. 8 shows the comparison of the computing resource utilization rate with the number of mobile devices under the DQN-based method, the Q-learning-based method and the game theory method, and it can be seen from the figure that the average computing resource utilization rate obtained by the Q-learning-based method and the DQN-based method is 74.58% and 77.37%, respectively. However, with the game theory approach, the resulting utilization is 61.38%, which is much lower than our proposed solution. By improving the utilization rate of computing resources, operators can effectively reduce the operation cost (OPEX) without reducing the unloading performance of the system.

Claims (6)

1. A computing resource allocation and task unloading method for ultra-dense network edge computing is characterized by comprising the following steps:
step 1, establishing a system model based on an ultra-dense network edge computing network of an SDN (software defined network), and acquiring network parameters: calculating the average task rate reaching the edge server of the macro base station and the average task rate reaching the edge server of the small base station according to the number of the mobile devices, the number of the macro base stations, the number of the wireless channels of the small base stations connected to the macro base station and the number of the wireless channels connected to the small base stations s in the scene;
step 2, obtaining parameters required by edge calculation: sequentially carrying out local calculation and unloading to an edge server of a macro base station and an edge server connected with a small base station s to obtain an uplink data rate for transmitting a calculation task;
step 3, obtaining an optimal computing resource allocation and task unloading strategy by adopting a Q-learning scheme;
and step 4, obtaining an optimal computing resource allocation and task unloading strategy by adopting the DQN scheme.
2. The method for computing resource allocation and task offloading in ultra-dense network edge computing according to claim 1, wherein in step 1, the network parameters are obtained: number of mobile devices in scene C, set
Figure RE-FDA0002300864970000018
Represents; the number of macro base stations is 1, the number of small base stations is B, and the set is used
Figure RE-FDA0002300864970000019
Represents; the number of radio channels connected to the macro base station is WmThe number of radio channels connected to the small base station s is Ws(ii) a The computing task types are E in total, and are represented by epsilon {1, 2.. E }, the arrival and processing processes of the tasks adopt an M/M/1 queuing model, and the average task rate reaching the edge server of the macro base station is
Figure RE-FDA0002300864970000011
The average task rate to the edge server of the small base station is
Figure RE-FDA0002300864970000012
The transmission power of mobile m is pm，nAnd a channel gain between the macro base station and the base station ofAnd a small base station has a channel gain of
Figure RE-FDA0002300864970000014
The dependent variables define: c mobile devices are distributed randomly and covered by 1 macro base station and B small base stations, and the distance from each device to the macro base station and the small base stations is DmUse of sets
Figure RE-FDA0002300864970000015
Denotes the distance of all devices to all base stations, wherein
Figure RE-FDA0002300864970000016
α for n types of computing tasks requested by mobile device mm，nRepresentation, computation task feature set ofWherein im，nSetting the size of the task to 300-; om，nSetting the CPU period required for processing the computing task as 100-1000 Megacycles;
Figure RE-FDA0002300864970000021
set to 0.5-3s for maximum allowable processing delay αm，nUnloading blockThe policy set is described as
Figure RE-FDA0002300864970000022
Wherein the content of the first and second substances,
Figure RE-FDA0002300864970000023
Figure RE-FDA0002300864970000024
indicating that the task is to be computed locally,indicating that the task should be offloaded to an edge server connected to the macro base station,
Figure RE-FDA0002300864970000026
indicating that mobile device m selects to offload task αm，nEdge server to connected small base station, wherein
Figure RE-FDA0002300864970000027
σ2The background noise power is set to-100 dbm.
3. The method of claim 2, wherein defining the computing resource allocation and task off-loading for ultra-dense network edge computing
Figure RE-FDA0002300864970000028
All tasks to be processed at time t; definition of
Figure RE-FDA0002300864970000029
For the total computing resources at all edge servers time t,whereinas(t) is the total resources of all edge servers connecting the macro base station and the small base station; definition of
Figure RE-FDA00023008649700000212
The computing resources of the seed are used at time t for all edge servers,
Figure RE-FDA00023008649700000213
wherein sigmam(t) and bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station; definition ofTo allocate policies for computing resources to all edge servers,
Figure RE-FDA00023008649700000215
4. the method for computing resource allocation and task offloading in ultra-dense network edge computing as claimed in claim 1, wherein in step 2,
a. local compute task αm，nQueue condition v ofm，nSetting the queuing delay introduced per decision period
Figure RE-FDA00023008649700000217
Second, CPU cycle o required to process a computational taskm，nAnd computing resources q of a particular mobile devicem(ii) a Locally calculating total delay
Figure RE-FDA00023008649700000218
Calculated from the following formula:
Figure RE-FDA00023008649700000219
b. computation tasks α off-load to edge servers of macro base stationm，nQueue condition v ofm，nSetting the queuing delay introduced per decision period
Figure RE-FDA00023008649700000220
Second, calculate size of task im，nCPU cycles o required to process a computational taskm，nAnd the computing resource size q of the edge server connected with the macro base stationmBetween 16-32 GHz; uplink data rate for transmitting computational tasks
Figure RE-FDA00023008649700000221
Figure RE-FDA00023008649700000222
Calculated from the following formula:
Figure RE-FDA00023008649700000223
final total delay of the unloading mode
Figure RE-FDA00023008649700000224
Calculated from the following formula:
Figure RE-FDA0002300864970000031
c. computation tasks α off-loaded to edge servers connecting small cells sm，nQueue condition v ofm，nSetting the queuing delay introduced per decision period
Figure RE-FDA0002300864970000032
Second, calculate size of task im，nCPU cycles o required to process a computational taskm，nAnd edge server computation resources connecting macro base stationsSource size qsBetween 4-8 GHz; uplink data rate for transmitting computational tasks
Figure RE-FDA0002300864970000033
Calculated from the following formula
Final total delay of the unloading mode
Figure RE-FDA0002300864970000036
Calculated from the following formula:
Figure RE-FDA0002300864970000037
the calculation time for which three calculation schemes can be obtained is
5. The method for computing resource allocation and task offloading in ultra-dense network edge computing according to claim 1, wherein step 3 specifically includes:
1) initializing a Q table, setting all Q values to be 0, and setting a discount factor gamma and a learning rate α;
2) defining system states
Figure RE-FDA0002300864970000039
the system state at time t is
Figure RE-FDA00023008649700000310
α thereinm，n(t) is a calculation task feature, vm，n(t) is the task queuing state,
Figure RE-FDA00023008649700000311
computing resources for the total of all edge servers at time t-1;
Figure RE-FDA00023008649700000312
computing resources of the seeds are used for all the edge servers at the time t;
Figure RE-FDA00023008649700000313
the distance between the mobile equipment and all edge servers is collected;
3) defining actions
Figure RE-FDA00023008649700000314
the set of actions at time t is
Figure RE-FDA00023008649700000315
Figure RE-FDA00023008649700000316
I.e. a computing resource allocation strategy for all edge servers;
4) defining a reward function
Figure RE-FDA00023008649700000317
Calculating processing time from edges
Figure RE-FDA00023008649700000318
And price function
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2For connecting small base stationsThe price of the edge server per unit time unit computing resource is set to be 1; pim，nThe size of the computing resources allocated to the respective computing task; defining the normalized conversion as gamma (x), and calculating the normalized user profit-expenditure utility asRm，n(t) earnings after processing the calculation tasks; the reward value function at time t is
Figure RE-FDA0002300864970000043
Figure RE-FDA0002300864970000044
For the total number of processing tasks at time t,
Figure RE-FDA0002300864970000045
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station;
5) observing the current system state s, and executing corresponding action a according to the Q (s, a) value stored in the Q table, namely resource allocation; then observing the next system state s 'after the action a is executed, and according to the current system state s, the executed action a and the next system state s', obtaining the system stateObtaining a current Q value and storing the current Q value in a Q table; continuously executing the training process until the training is finished; and finally, obtaining the optimal computing resource allocation and task unloading strategy.
6. The method for computing resource allocation and task offloading in ultra-dense network edge computing according to claim 1, wherein step 4 specifically includes:
1) initialization: the weight parameter of the evaluation network is theta, and the weight parameter of the target networkNumber θ', discount factor γ and learning rate α, probability of exploration
Figure RE-FDA0002300864970000047
A priori playback
Figure RE-FDA0002300864970000048
2) Defining a system state S: the system state at time t is
Figure RE-FDA0002300864970000049
α thereinm，n(t) is a calculation task feature, vm，n(t) is the task queuing state,computing resources for the total of all edge servers at time t-1;
Figure RE-FDA00023008649700000411
computing resources of the seeds are used for all the edge servers at the time t;
Figure RE-FDA00023008649700000414
the distance between the mobile equipment and all edge servers is collected;
3) define action A: the set of actions at time t is
Figure RE-FDA00023008649700000413
I.e. a computing resource allocation strategy for all edge servers;
4) defining a reward function R: calculating processing time from edges
Figure RE-FDA0002300864970000051
And price function
Figure RE-FDA0002300864970000052
Wherein mu1The price of computing resources per time unit for edge servers connected to macro base stations is set to 0.7, mu2The price of computing resources in unit time unit for an edge server connected with a small base station is set as 1; pim，nThe size of the computing resources allocated to the respective computing task; defining the normalized transformation as γ (x), the normalized user benefit-expenditure utility can be calculated as
Figure RE-FDA0002300864970000053
Rm，n(t) earnings after processing the calculation tasks; the reward value function at time t is
Figure RE-FDA0002300864970000054
Figure RE-FDA0002300864970000055
For the total number of processing tasks at time t,
Figure RE-FDA0002300864970000056
as(t) Total resources, σ, of all edge servers connecting the macro base station and the small base stationm(t)，bs(t) is the resources being used by all edge servers connecting the macro base station and the small base station;
5) adopts an epsilon-greedy method to explore the probability
Figure RE-FDA00023008649700000513
Gradually decreases from 1 to 0.1; observing the current system state s (t), selecting a random number omega from 0 to 1 if
Figure RE-FDA00023008649700000511
Randomly selecting an action from all possible actions to execute, namely allocating computing resources; if it is not
Figure RE-FDA00023008649700000512
According to a (t) argmaxaQ(s) (t; a (t); theta) selection action execution; after the corresponding action is executed, a reward function r (t) is calculated, the next system state s (t +1) is observed, and the transition states (s (t); a (t); r (t); s (t +1)) are stored in an a priori replayIn (1),wherein Λ (t) ═ { s (t); a (t); r (t); s (t +1) }; randomly selecting MiniBatch from the prior experiment as a sample, and selecting y ═ r (n) + gamma maxa(n+1)Q (s (n + 1); a (n + 1); θ') sets a target network value y; then by a gradient decreasing function
Figure RE-FDA00023008649700000510
Updating and evaluating a network weight parameter theta; continuously executing the process, and updating the target network weight parameter theta' to the current evaluation network weight parameter theta after J times; repeating the training process until the training is finished; and finally, obtaining the optimal computing resource allocation and task unloading strategy.
CN201910959379.1A 2019-10-10 2019-10-10 Computing resource allocation and task unloading method for ultra-dense network edge computing Pending CN110798849A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910959379.1A CN110798849A (en) 2019-10-10 2019-10-10 Computing resource allocation and task unloading method for ultra-dense network edge computing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910959379.1A CN110798849A (en) 2019-10-10 2019-10-10 Computing resource allocation and task unloading method for ultra-dense network edge computing

Publications (1)

Publication Number Publication Date
CN110798849A true CN110798849A (en) 2020-02-14

Family

ID=69439277

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910959379.1A Pending CN110798849A (en) 2019-10-10 2019-10-10 Computing resource allocation and task unloading method for ultra-dense network edge computing

Country Status (1)

Country Link
CN (1) CN110798849A (en)

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111405568A (en) * 2020-03-19 2020-07-10 三峡大学 Computing unloading and resource allocation method and device based on Q learning
CN111400001A (en) * 2020-03-09 2020-07-10 清华大学 Online computing task unloading scheduling method facing edge computing environment
CN111524034A (en) * 2020-05-12 2020-08-11 华北电力大学 High-reliability low-time-delay low-energy-consumption power inspection system and inspection method
CN111556461A (en) * 2020-04-29 2020-08-18 南京邮电大学 Vehicle-mounted edge network task distribution and unloading method based on deep Q network
CN111586696A (en) * 2020-04-29 2020-08-25 重庆邮电大学 Resource allocation and unloading decision method based on multi-agent architecture reinforcement learning
CN111585637A (en) * 2020-04-17 2020-08-25 长沙理工大学 Unmanned aerial vehicle task unloading and resource allocation method based on edge computing system
CN111641681A (en) * 2020-05-11 2020-09-08 国家电网有限公司 Internet of things service unloading decision method based on edge calculation and deep reinforcement learning
CN111683381A (en) * 2020-04-28 2020-09-18 南京邮电大学 End-to-end network slice resource allocation algorithm based on deep reinforcement learning
CN111726826A (en) * 2020-05-25 2020-09-29 上海大学 Online task unloading method in base station intensive edge computing network
CN111800828A (en) * 2020-06-28 2020-10-20 西北工业大学 Mobile edge computing resource allocation method for ultra-dense network
CN111813539A (en) * 2020-05-29 2020-10-23 西安交通大学 Edge computing resource allocation method based on priority and cooperation
CN111918339A (en) * 2020-07-17 2020-11-10 西安交通大学 AR task unloading and resource allocation method based on reinforcement learning in mobile edge network
CN111970718A (en) * 2020-07-22 2020-11-20 西北工业大学 Deep learning-based power distribution method in energy collection untrusted relay network
CN111988805A (en) * 2020-08-28 2020-11-24 重庆邮电大学 End edge cooperation method for reliable time delay guarantee
CN112039950A (en) * 2020-08-03 2020-12-04 威胜信息技术股份有限公司 Edge computing network task scheduling and resource allocation method and edge computing system
CN112084018A (en) * 2020-08-11 2020-12-15 河海大学 Method for rapidly evaluating average time delay of non-uniform access users in edge computing network
CN112188631A (en) * 2020-10-09 2021-01-05 上海海事大学 Ocean fog node resource management method based on gradient estimation
CN112218337A (en) * 2020-09-04 2021-01-12 暨南大学 Cache strategy decision method in mobile edge calculation
CN112511336A (en) * 2020-11-05 2021-03-16 上海大学 Online service placement method in edge computing system
CN112887435A (en) * 2021-04-13 2021-06-01 中南大学 Method for improving task unloading cooperation rate in edge calculation
CN112905315A (en) * 2021-01-29 2021-06-04 北京邮电大学 Task processing method, device and equipment in Mobile Edge Computing (MEC) network
CN113377547A (en) * 2021-08-12 2021-09-10 南京邮电大学 Intelligent unloading and safety guarantee method for computing tasks in 5G edge computing environment
CN113452625A (en) * 2021-06-28 2021-09-28 重庆大学 Deep reinforcement learning-based unloading scheduling and resource allocation method
CN113490219A (en) * 2021-07-06 2021-10-08 香港中文大学（深圳） Dynamic resource allocation method for ultra-dense networking
CN113573363A (en) * 2021-07-27 2021-10-29 西安热工研究院有限公司 MEC calculation unloading and resource allocation method based on deep reinforcement learning
CN113747507A (en) * 2021-08-16 2021-12-03 北京信息科技大学 Computing resource management method and device for 5G ultra-dense network
CN113747504A (en) * 2021-07-29 2021-12-03 山东师范大学 Method and system for multi-access edge computing combined task unloading and resource allocation
CN114339819A (en) * 2020-11-06 2022-04-12 北京航空航天大学 Calculation unloading method based on optimal resource allocation amount and search algorithm
CN114490057A (en) * 2022-01-24 2022-05-13 电子科技大学 MEC unloaded task resource allocation method based on deep reinforcement learning

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170353260A1 (en) * 2016-06-02 2017-12-07 Samsung Electronics Co., Ltd Network configuration
CN108901046A (en) * 2018-06-14 2018-11-27 北京大学 Cotasking unloading algorithm and system design scheme towards mobile edge calculations
CN109121151A (en) * 2018-11-01 2019-01-01 南京邮电大学 Distributed discharging method under the integrated mobile edge calculations of cellulor
CN109302709A (en) * 2018-09-14 2019-02-01 重庆邮电大学 The unloading of car networking task and resource allocation policy towards mobile edge calculations
CN109639760A (en) * 2018-11-02 2019-04-16 西北工业大学 It is a kind of based on deeply study D2D network in cache policy method
CN109803292A (en) * 2018-12-26 2019-05-24 佛山市顺德区中山大学研究院 A method of the mobile edge calculations of more secondary user's based on intensified learning
CN109976909A (en) * 2019-03-18 2019-07-05 中南大学 Low delay method for scheduling task in edge calculations network based on study
CN110113190A (en) * 2019-04-24 2019-08-09 西北工业大学 Time delay optimization method is unloaded in a kind of mobile edge calculations scene

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170353260A1 (en) * 2016-06-02 2017-12-07 Samsung Electronics Co., Ltd Network configuration
CN108901046A (en) * 2018-06-14 2018-11-27 北京大学 Cotasking unloading algorithm and system design scheme towards mobile edge calculations
CN109302709A (en) * 2018-09-14 2019-02-01 重庆邮电大学 The unloading of car networking task and resource allocation policy towards mobile edge calculations
CN109121151A (en) * 2018-11-01 2019-01-01 南京邮电大学 Distributed discharging method under the integrated mobile edge calculations of cellulor
CN109639760A (en) * 2018-11-02 2019-04-16 西北工业大学 It is a kind of based on deeply study D2D network in cache policy method
CN109803292A (en) * 2018-12-26 2019-05-24 佛山市顺德区中山大学研究院 A method of the mobile edge calculations of more secondary user's based on intensified learning
CN109976909A (en) * 2019-03-18 2019-07-05 中南大学 Low delay method for scheduling task in edge calculations network based on study
CN110113190A (en) * 2019-04-24 2019-08-09 西北工业大学 Time delay optimization method is unloaded in a kind of mobile edge calculations scene

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李季: "基于深度强化学习的移动边缘计算中的计算卸载与资源分配算法研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (50)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111400001A (en) * 2020-03-09 2020-07-10 清华大学 Online computing task unloading scheduling method facing edge computing environment
CN111400001B (en) * 2020-03-09 2022-09-23 清华大学 Online computing task unloading scheduling method facing edge computing environment
CN111405568B (en) * 2020-03-19 2023-01-17 三峡大学 Computing unloading and resource allocation method and device based on Q learning
CN111405568A (en) * 2020-03-19 2020-07-10 三峡大学 Computing unloading and resource allocation method and device based on Q learning
CN111585637B (en) * 2020-04-17 2022-02-11 长沙理工大学 Unmanned aerial vehicle task unloading and resource allocation method based on edge computing system
CN111585637A (en) * 2020-04-17 2020-08-25 长沙理工大学 Unmanned aerial vehicle task unloading and resource allocation method based on edge computing system
CN111683381A (en) * 2020-04-28 2020-09-18 南京邮电大学 End-to-end network slice resource allocation algorithm based on deep reinforcement learning
CN111586696B (en) * 2020-04-29 2022-04-01 重庆邮电大学 Resource allocation and unloading decision method based on multi-agent architecture reinforcement learning
CN111586696A (en) * 2020-04-29 2020-08-25 重庆邮电大学 Resource allocation and unloading decision method based on multi-agent architecture reinforcement learning
CN111556461A (en) * 2020-04-29 2020-08-18 南京邮电大学 Vehicle-mounted edge network task distribution and unloading method based on deep Q network
CN111641681A (en) * 2020-05-11 2020-09-08 国家电网有限公司 Internet of things service unloading decision method based on edge calculation and deep reinforcement learning
CN111524034A (en) * 2020-05-12 2020-08-11 华北电力大学 High-reliability low-time-delay low-energy-consumption power inspection system and inspection method
CN111524034B (en) * 2020-05-12 2023-11-03 华北电力大学 High-reliability low-time-delay low-energy-consumption power inspection system and inspection method
CN111726826A (en) * 2020-05-25 2020-09-29 上海大学 Online task unloading method in base station intensive edge computing network
CN111726826B (en) * 2020-05-25 2024-03-19 上海大学 Online task unloading method in base station intensive edge computing network
CN111813539A (en) * 2020-05-29 2020-10-23 西安交通大学 Edge computing resource allocation method based on priority and cooperation
CN111800828A (en) * 2020-06-28 2020-10-20 西北工业大学 Mobile edge computing resource allocation method for ultra-dense network
CN111800828B (en) * 2020-06-28 2023-07-18 西北工业大学 Mobile edge computing resource allocation method for ultra-dense network
CN111918339B (en) * 2020-07-17 2022-08-05 西安交通大学 AR task unloading and resource allocation method based on reinforcement learning in mobile edge network
CN111918339A (en) * 2020-07-17 2020-11-10 西安交通大学 AR task unloading and resource allocation method based on reinforcement learning in mobile edge network
CN111970718A (en) * 2020-07-22 2020-11-20 西北工业大学 Deep learning-based power distribution method in energy collection untrusted relay network
CN111970718B (en) * 2020-07-22 2022-03-11 西北工业大学 Deep learning-based power distribution method in energy collection untrusted relay network
CN112039950B (en) * 2020-08-03 2021-11-30 威胜信息技术股份有限公司 Edge computing network task scheduling and resource allocation method and edge computing system
WO2022027776A1 (en) * 2020-08-03 2022-02-10 威胜信息技术股份有限公司 Edge computing network task scheduling and resource allocation method and edge computing system
CN112039950A (en) * 2020-08-03 2020-12-04 威胜信息技术股份有限公司 Edge computing network task scheduling and resource allocation method and edge computing system
CN112084018A (en) * 2020-08-11 2020-12-15 河海大学 Method for rapidly evaluating average time delay of non-uniform access users in edge computing network
CN112084018B (en) * 2020-08-11 2022-06-24 河海大学 Method for rapidly evaluating average time delay of non-uniform access users in edge computing network
CN111988805A (en) * 2020-08-28 2020-11-24 重庆邮电大学 End edge cooperation method for reliable time delay guarantee
CN111988805B (en) * 2020-08-28 2022-03-29 重庆邮电大学 End edge cooperation method for reliable time delay guarantee
CN112218337B (en) * 2020-09-04 2023-02-28 暨南大学 Cache strategy decision method in mobile edge calculation
CN112218337A (en) * 2020-09-04 2021-01-12 暨南大学 Cache strategy decision method in mobile edge calculation
CN112188631A (en) * 2020-10-09 2021-01-05 上海海事大学 Ocean fog node resource management method based on gradient estimation
CN112511336B (en) * 2020-11-05 2022-11-18 上海大学 Online service placement method in edge computing system
CN112511336A (en) * 2020-11-05 2021-03-16 上海大学 Online service placement method in edge computing system
CN114339819B (en) * 2020-11-06 2023-10-03 北京航空航天大学 Computing unloading method based on optimal resource allocation amount and search algorithm
CN114339819A (en) * 2020-11-06 2022-04-12 北京航空航天大学 Calculation unloading method based on optimal resource allocation amount and search algorithm
CN112905315A (en) * 2021-01-29 2021-06-04 北京邮电大学 Task processing method, device and equipment in Mobile Edge Computing (MEC) network
CN112887435A (en) * 2021-04-13 2021-06-01 中南大学 Method for improving task unloading cooperation rate in edge calculation
CN113452625B (en) * 2021-06-28 2022-04-15 重庆大学 Deep reinforcement learning-based unloading scheduling and resource allocation method
CN113452625A (en) * 2021-06-28 2021-09-28 重庆大学 Deep reinforcement learning-based unloading scheduling and resource allocation method
CN113490219B (en) * 2021-07-06 2022-02-25 香港中文大学（深圳） Dynamic resource allocation method for ultra-dense networking
CN113490219A (en) * 2021-07-06 2021-10-08 香港中文大学（深圳） Dynamic resource allocation method for ultra-dense networking
CN113573363A (en) * 2021-07-27 2021-10-29 西安热工研究院有限公司 MEC calculation unloading and resource allocation method based on deep reinforcement learning
CN113573363B (en) * 2021-07-27 2024-01-23 西安热工研究院有限公司 MEC calculation unloading and resource allocation method based on deep reinforcement learning
CN113747504A (en) * 2021-07-29 2021-12-03 山东师范大学 Method and system for multi-access edge computing combined task unloading and resource allocation
CN113377547A (en) * 2021-08-12 2021-09-10 南京邮电大学 Intelligent unloading and safety guarantee method for computing tasks in 5G edge computing environment
CN113747507A (en) * 2021-08-16 2021-12-03 北京信息科技大学 Computing resource management method and device for 5G ultra-dense network
CN113747507B (en) * 2021-08-16 2024-01-09 北京信息科技大学 5G ultra-dense network-oriented computing resource management method and device
CN114490057A (en) * 2022-01-24 2022-05-13 电子科技大学 MEC unloaded task resource allocation method based on deep reinforcement learning
CN114490057B (en) * 2022-01-24 2023-04-25 电子科技大学 MEC offloaded task resource allocation method based on deep reinforcement learning

Similar Documents

Publication Publication Date Title
CN110798849A (en) Computing resource allocation and task unloading method for ultra-dense network edge computing
CN110377353B (en) System and method for unloading computing tasks
CN110418416B (en) Resource allocation method based on multi-agent reinforcement learning in mobile edge computing system
CN113950066B (en) Single server part calculation unloading method, system and equipment under mobile edge environment
Nath et al. Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems
CN108920280B (en) Mobile edge computing task unloading method under single-user scene
Fadlullah et al. HCP: Heterogeneous computing platform for federated learning based collaborative content caching towards 6G networks
CN109857546B (en) Multi-server mobile edge computing unloading method and device based on Lyapunov optimization
Li et al. Deep reinforcement learning based computation offloading and resource allocation for MEC
CN109684075B (en) Method for unloading computing tasks based on edge computing and cloud computing cooperation
Lai et al. Intelligent secure mobile edge computing for beyond 5G wireless networks
CN111800828B (en) Mobile edge computing resource allocation method for ultra-dense network
CN111405569A (en) Calculation unloading and resource allocation method and device based on deep reinforcement learning
CN109600178B (en) Optimization method for energy consumption, time delay and minimization in edge calculation
Nath et al. Multi-user multi-channel computation offloading and resource allocation for mobile edge computing
CN109246761B (en) Unloading method based on alternating direction multiplier method considering delay and energy consumption
CN112118287A (en) Network resource optimization scheduling decision method based on alternative direction multiplier algorithm and mobile edge calculation
Yang et al. Deep reinforcement learning in cache-aided MEC networks
CN113543342B (en) NOMA-MEC-based reinforcement learning resource allocation and task unloading method
CN110519849B (en) Communication and computing resource joint allocation method for mobile edge computing
Nath et al. Dynamic computation offloading and resource allocation for multi-user mobile edge computing
CN114697333B (en) Edge computing method for energy queue equalization
CN113573363B (en) MEC calculation unloading and resource allocation method based on deep reinforcement learning
CN115473896A (en) Electric power internet of things unloading strategy and resource configuration optimization method based on DQN algorithm
Chen et al. Joint optimization of task offloading and resource allocation via deep reinforcement learning for augmented reality in mobile edge network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200214