CN110427261A - A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree - Google Patents

A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree Download PDF

Info

Publication number
CN110427261A
CN110427261A CN201910741439.2A CN201910741439A CN110427261A CN 110427261 A CN110427261 A CN 110427261A CN 201910741439 A CN201910741439 A CN 201910741439A CN 110427261 A CN110427261 A CN 110427261A
Authority
CN
China
Prior art keywords
resource allocation
search
task
edge calculations
mobile
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910741439.2A
Other languages
Chinese (zh)
Inventor
陈杰男
陈思宇
李帅
王琪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201910741439.2A priority Critical patent/CN110427261A/en
Publication of CN110427261A publication Critical patent/CN110427261A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Biomedical Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Strategic Management (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Economics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Development Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Marketing (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Game Theory and Decision Science (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

The application discloses a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree, to support optimization of the Edge Server to resource allocation.Edge Server regard mobile edge calculations system mode as input, Edge Server scheduling of resource module exports optimal resource allocation scheme by deeply learning algorithm, and mobile device terminal carries out task unloading according to optimal resource allocation scheme and executes task together with Edge Server.The deeply learning algorithm cooperates completion by DNN, MCTS, LSTM, and compared with greedy search and DQN algorithm, algorithm proposed in this paper is substantially improved in terms of optimization service time delay and the service capacity consumption of optimization mobile terminal.

Description

A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree
Technical field
It is the present invention relates to a kind of intelligence communication field, in particular to a kind of based on the edge of depth Monte Carlo tree search Calculate method for allocating tasks.
Background technique
Have some algorithms at present to be applied in the optimization distribution of mobile edge calculations resource.First method uses line Property planning algorithm, optimizes computing resource and bandwidth resources, and Lai Tigao system maximum throughput simultaneously reduces service response and prolongs When, to promote mobile limbic system performance, but this method is unable to adjust the unloading rate of task.Second is based on Lyapunov Optimization method is a kind of algorithm of dynamic adjustment calculating task unloading rate, can reduce the time of calculating task completion, disadvantage It is that the task complexity that can be handled is lower, the higher distribution task of complexity cannot be handled.And both resource allocations are excellent Linear programming algorithm, the Lyapunov algorithm used in change method is all discovery learning, and the experience of the mankind is needed to instruct It could complete.In addition, mobile number of devices is significantly increased under 5G scenes of internet of things, the demand of mobile subscriber terminal calculating task is more Sample, optimization problem become complicated, and existing method is difficult to handle the optimization problem of high complexity.
Summary of the invention
It is an object of the invention to overcome the above-mentioned deficiency in the presence of the prior art, provide a kind of based on depth Meng Teka The edge calculations method for allocating tasks of Luo Shu search becomes in demand multiplicity, the optimization problem of mobile subscriber terminal calculating task When complicated, it is also able to achieve the optimization to environmental resource distribution.In order to achieve the above-mentioned object of the invention, the present invention provides following technologies Scheme:
When mobile subscriber terminal generates calculating task, Edge Server updates mobile edge calculations system status information； The mobile edge calculations system includes mobile device terminal, radio communication base station, Edge Server；The mobile edge calculations System status information includes the computing capability of Edge Server, the wireless bandwidth resource of radio communication base station, times of mobile device Business solicited message, the task request message includes the historical channel gain information of each mobile device terminal and base station, currently The data volume size of task to be processed, cpu clock periodicity needed for completing current task, the local of mobile device terminal Cpu clock frequency；
Mobile edge calculations system status information is transferred to DNN (deep neural network) by Edge Server, MCTS (is covered special Ka Luoshu) and LSTM (shot and long term memory network)；The LSTM increases according to the channel of mobile device terminal and radio communication base station Benefit predicts channel gain in future, sends MCTS and DNN for obtained channel gain prediction data；DNN is according to movement Edge calculations system status information and channel estimating data, obtain resource allocation actions prior probability, the resource allocation that will be obtained Movement prior probability is sent to MCTS；
MCTS combines mobile edge calculations system status information, and channel gain prediction data and resource allocation actions priori are general Rate searches for optimal resource allocation scheme and optimal resource allocation scheme is sent to mobile device terminal after MCTS is searched for, and moves Task is offloaded to mobile edge calculations system by dynamic device end, and mobile edge calculations system execution module is according to optimal resource point Optimal resource allocation behavior is executed with scheme.
Wherein MCTS carries out the optimal resource allocation scheme that emulation search is found to task status, is searched for using the MCTS In obtained optimal resource allocation scheme deposit experience pond, experience pond size is fixed, and is deleted when data are filled with and is deposited earliest Then the data entered are stored in new data.And the data in use experience pond train the DNN at regular intervals, to improve The predictablity rate for stating DNN makes DNN export updated resource allocation actions prior probability and preferably MCTS is instructed to be searched Rope, to optimize the edge calculations method for allocating tasks.The better search result of MCTS, which can preferably update DNN again, simultaneously makes It is more acurrate to obtain its prediction.
The present invention provides MCTS search routine is as follows:
S1: according to the root node of mobile edge calculations system mode initialization MCTS；
S2: search starting point is set by root node and starts to be searched for next time；
S3: judge whether to complete the search of pre-determined number, if so, step s9 is executed, if it is not, thening follow the steps s4；
S4: judging whether present node is leaf node, if so, step s5 is executed, if it is not, thening follow the steps s6；
S5: after entering leaf node, i.e., after the completion of computational resource allocation, assessing the Resource Allocation Formula, return to award, The state of all nodes on the paths is updated according to award；
S6: judge whether present node is full expanding node, if it is not, step s7 is executed, if so, thening follow the steps s8；
S7: all child nodes of present node are extended according to the prior probability of the output of DNN, and according to formulaNext node is selected, wherein Q (v 'k) be defined as saving Point v 'kJackpot prize value, N (v 'k) it is defined as node v 'kAccess times, e be balanced algorithm development and exploration proportionality coefficient, p(v′k|sk) indicate next node prior probability；
S8: choosing search from all possible resource allocation actions and be worth highest movement, execute the movement of selection into Enter next layer of node；
S9: the most path of output access number is optimal Resource Allocation Formula.
Above-mentioned optimal distributing scheme but also as DNN training set, to promote search performance again.
The acquisition methods of reward value r are as follows:
Optimal execution time t is setbestTo use minimum time used in history Resource Allocation Formula completion task, just Initial value is infinity, and wherein t indicates that Current resource allocation plan is completed the time of required by task, and σ is setting value and σ > 1.
The present invention is as follows using LSTM prediction channel gain principle:
If Fig. 1 is LSTM network structure:
LSTM network is according to historical juncture channel gain hτ-p+1, hτ-p+2..., hτTo predict future time channel gain hτ+1, it is by LSTM net definitions
hτ+1=gθ(hτ-p+1,hτ-p+2,...,hτ)
Here, θ is defined as the weight parameter of LSTM network, and LSTM Web vector graphic cell factory stores long term state, It is mainly controlled by three doors, i.e. input gate, forgets door and out gate.It is illustrated in figure 2 LSTM eucaryotic cell structure figure:
Pass through to each permission header length, this is by sigmoid function and the nervous layer of point-by-point multiplying Lai real Existing, chief component can be summarized as follows.
Input gate: it is determined that the input of current network to be saved in the quantity of location mode, the realization of input gate is shown in (1) and in (2), h is currently inputtedτWith previous LSTM cell stateIt is used as the input of input gate, then weight matrix Which information determination is needed to update by the result of multiplication, recently enters sigmoid layers or tanh layers.
Forget door: it determines to forget how many current input on network, and remaining output is then saved in current list First state.Door inputs h from currentτWith previous LSTM cell stateInformation is obtained, and exports 0 to 1 probability, wherein 1 Expression is fully retained, and 0 indicates to abandon completely.Relevant equations are as follows
Out gate: it will export new LSTM cell state.Firstly, sigmoid layers by determine need lead-out unit which A little parts.Then tanh layers are sent to export the probability value between [- 1,1] by location mode.Finally, probability value multiplied by Sigmoid layers of output.Correlate equation is as follows:
Pass through optimization object function J (θ) Lai Xunlian LSTM parameter θ
HereIt is the data label obtained from mobile edge calculations network, regularization term ξ ‖ θ ‖2Avoid overfitting. From the channel gain at MEC network collection continuous p+1 moment, input of the preceding p data as LSTM is then used, and use the P+1 data train LSTM network as label.
The DNN that the present invention uses be it is trained in advance, training method is as follows: according to the shape of mobile edge calculations system State, generates a simulated environment, and Monte Carlo first carries out equal proportion search, the resource ratio of as same task distribution or It the ratios such as is between the different schemes of task unloading ratio, depending on specific ratio setting according to circumstances, search process and above-mentioned Resource allocation methods are similar, and the search plan for being different in the training process is only simulated in simulated environment, search result It is only used as the training set of DNN, search result does not have to execute in true mobile edge calculations system.It will search after search As a result it is sent to DNN, DNN is using search result as training set, because DNN has Function Fitting performance, one not exported to MCTS A little states can also export prior probability, therefore, when having mobile edge calculations system status information and channel gain prediction data defeated Fashionable, DNN can export resource allocation actions prior probability to MCTS.
At this point, MTCS can be scanned in true environment according to the prior probability, optimal Resource Allocation Formula is obtained (search result).Then, we carry out reverse train update to DNN using MTCS search result obtained in true environment, By the real-time training of MTCS search result, DNN is just continuously updated, optimizes, thus export more accurate prior probability to MCTS, MCTS continue to optimize the Resource Allocation Formula of its output according to updated prior probability.
It is DNN structure of the present invention as shown in Figure 3.By last several layers of separation of deep neural network, with building Neural network with sublayer, to export the behavior of the resource allocation of multitask.DNN receives task statusAnd export the behavior a for distributing each resourcex={ aX, l(l=0,1 ..., q-1) prior probabilityDNN includes an input layer Hi, n public hidden layer { H1..., HnAnd q sublayer, each sublayer packet Containing m sub- hidden layersEvery layer of neuron number is expressed asThe parameter of every layer of neural network isWherein i indicates that input layer, s indicate son Hidden layer, O indicate that output layer, W indicate weight, and b indicates biasing.In the training stage, the training set of DNN is generated by MCTS, and is made With the parameter of RMSProp (root mean square backpropagation) algorithm training DNN.Each RMSProp optimizer exports mutually independent damage Lose function Wherein Δ ‖ θ ‖2It is parameter regularization term, to avoid Over-fitting.By q RMSProp optimizer, solve so that the smallest DNN parameter of the value of loss function, for example, first of optimization Device is according to resource allocation policyUpdate the parameter of DNN The l+1 optimizer uses the θ updated laterl, according to strategyTo update θl+1.Therefore, in public hidden layer Parameter (W1:n,b1:n) can be by the behavior label of all sublayersIt is updated.In forecast period, DNN provides environment Source and mission bit stream As input, the prior probability distribution of each sublayer is exportedWherein x indicates xth item task.
Edge Server completes the calculating task of mobile subscriber terminal, multiple neighbouring sides using co-operating mode Edge server shares computing resource, and partial task is offloaded to other adjacent domains when handling local task by Edge Server Idle Edge Server.
Compared with prior art, beneficial effects of the present invention:
Edge calculations method for allocating tasks of the invention can be completed in the case where no human intervention (or artificially labelling) Study to resource allocation policy；
Invention defines variable tasks to unload ratio, moves Edge Server according to the calculating energy of mobile subscriber terminal Power and the remaining computing resource of each Edge Server determine the scale that calculating task is completed for user, mobile subscriber Terminal upload server permits the calculating task part of processing, and rest part is voluntarily handled by mobile subscriber terminal.
Edge calculations method for allocating tasks of the invention is used by cooperating between DNN, MCTS, LSTM mobile When demand multiplicity, the optimization problem of family terminal calculating task become complexity, it is also able to achieve the optimization to environmental resource distribution.DNN It instructs MCTS to scan for prior probability, beta pruning is carried out to MCTS, so that MCTS is done less returns low search, such MCTS Can be with less searching times come search performance the same when reaching no DNN, the even more than not no performance of DNN, property 50% or so can be promoted compared with DQN.The channel estimating module based on LSTM is devised, to the channel gain of user's future time It is predicted, the movement of mobile subscriber is that consecutive variations are regular, and the channel estimating module is used in MCTS to future time instance Task calculated using the channel gain of prediction when scanning for so that the acquisition of reward is more acurrate.
The DNN that the present invention uses is multitask neural network: being to be mutually related between multiple subtasks, with a nerve Network one sub- task state of corresponding output cannot learn to the relationship between multiple subtasks, therefore the multitask nerve used Network energy boosting algorithm constringency performance.
The algorithm that the present invention uses has flexibility: if optimization aim changes, only need to change reward function (reward) Setting achieve that new optimization aim, without redesign algorithm.(example optimization aim as in the previous is that optimization is minimum Change time delay, be now to be changed to minimize energy consumption, then only needing to change the set-up mode of reward function at this time)
After the algorithm off-line training that the present invention uses is good, it can also be collected simultaneously training data during on-line operation, To promote search performance again.
Detailed description of the invention:
Fig. 1 is LSTM network structure.
Fig. 2 show LSTM eucaryotic cell structure figure.
Fig. 3 show the DNN structure of invention use.
Fig. 4 is a kind of showing for edge calculations method for allocating tasks based on the search of depth Monte Carlo tree of the present invention It is intended to.
Fig. 5 show the flow chart of MCTS search of the present invention.
Specific embodiment
Below with reference to test example and specific embodiment, the present invention is described in further detail.But this should not be understood It is all that this is belonged to based on the technology that the content of present invention is realized for the scope of the above subject matter of the present invention is limited to the following embodiments The range of invention.
Embodiment 1:
It is a kind of based on deeply study edge method for allocating tasks, whole distribution method as shown in figure 4, include with Lower step:
Edge Server refreshes mobile edge calculations system status information；The mobile edge calculations system status information packet Include the computing resource situation of Edge Server, the communication resource situation of radio communication base station, the task request message of mobile device, The task request message includes the historical channel gain information of each mobile device terminal and base station, currently pending task Data volume size, complete current task needed for cpu clock periodicity, the local cpu clock frequency of mobile device terminal；
Mobile edge calculations system status information is transferred to DNN, MCTS and LSTM by Edge Server；Edge Server connects Receive N number of task requests T from N number of mobile device terminalc={ T0, T1..., TN-1, while τ+1 at the time of be following, τ+ 2 ... M virtual task T is generated at randomv={ T0, T1..., TM-1, wherein virtual M task be for following task it is reserved Then resource allocates resources to X=M+N task T={ T0..., TN..., TX-1, LSTM is according to mobile device terminal and base station Channel gain the channel gain of the following M task is predicted, will obtain channel gain prediction data be sent to MCTS and It is general that DNN, DNN further according to mobile edge calculations system status information and channel gain prediction data obtain resource allocation actions priori Obtained prior probability is sent to MCTS by rate；
MCTS combines mobile edge calculations system status information and channel gain prediction data, generates a simulated environment, The node of Monte Carlo tree is expressed as to the distribution condition of last distribution movement, the task that this task is completed in last movement is unloaded The prior probability of the distribution of load ratio or resource ratio, MCTS combination DNN output scans for；
Search process flow chart such as Fig. 5, the specific steps are as follows:
S1, the root node that MCTS is initialized according to mobile edge calculations system mode, root node information are expressed as s0=(Fe, B, T) and channel gain prediction data, wherein FeIndicate the computing resource situation of Edge Server, B indicates mobile edge calculations system The communication resource situation of system, T indicate task request message；Wherein T contains the channel gain h of mobile device terminal and base station, The period c and cpu frequency f of the size of data d of task, mobile device terminal CPU completion required by taskl；
S2, it sets search starting point to root node and starts to be searched for next time, when search first searches for first task It is offloaded to the ratio of Edge Server, then search is the communication resource ratio of first task distribution, is searched again for as first item The computing resource ratio of task distribution, first task are assigned, and search Section 2 task is also appointed according to above-mentioned for first item The sequence of business distribution executes, and searches X item task always and completes resource allocation, so search depth, that is, path leaf node Place number of stories m ax_depth=3*X；
S3, judge i > max_search, if the search of pre-determined number is completed, if so, step S9 is executed, if it is not, then holding Row step S4, wherein i is that searching times, max_search have been presetting searching times；
S4, judge whether k > max_depth, present node are leaf node, if so, step S5 is executed, if it is not, then holding Row step S6, k are the number of plies where present node；
S5, after entering leaf node, i.e., after the completion of computational resource allocation, Resource Allocation Formula is sent on the ring of simulation Border executes, and the time parameter of required by task is completed according to the Resource Allocation Formula, obtains award r, is updated on the paths according to r The state of all nodes, more new formula have N (sk, ak)=N (sk, ak)+1, (Q (sk, ak)*N(sk, ak)+r/(N(sk, ak)+1), Wherein s indicates state, a expression movement, N (sk, ak) indicate side (sk, ak) searching times, Q (sk, ak) indicate state-movement pair Side (sk, ak) value；
S6, judge whether present node is full expanding node, if it is not, step S7 is executed, if so, thening follow the steps S8；
S7, all child nodes that present node is extended according to the prior probability of the output of DNN, and according to formulaSelect next node；
S8, from all possible distribution movement according to formula It chooses search and is worth highest movement, the movement for executing selection enters next layer of node；
S9, the most path of output access number, i.e., optimal Resource Allocation Formula；
Because the most path of searching times is exactly to be worth highest path, the path pair after multiple selection circulation The ratio for the distribution resource answered is exactly optimal allocation proportion, i.e. the path is exactly optimal Resource Allocation Formula, refers to edge To the computing resource ratio of distribution of computation tasks, Edge Server and mobile device terminal are carried out wireless communication and are used server Communication resource ratio, the calculating task of mobile device terminal is offloaded to the ratio of Edge Server.
Edge Server described in above-described embodiment 1 refers to local Edge Server, and mobile device terminal refers to local shifting Dynamic device end.
Embodiment 2:
The calculating that multiple neighbouring Edge Servers can complete mobile subscriber terminal using co-operating mode is appointed Business, comprising the following steps:
Local Edge Server refreshes mobile edge calculations system status information；The mobile edge calculations system mode letter Computing resource situation, the collaboration edge service that breath includes the computing resource situation of local Edge Server, cooperates with Edge Server The address of device, the wireless bandwidth resource of radio communication base station and mobile device task request message, the task request message Historical channel gain information including each mobile device terminal and base station, the data volume size of currently pending task are complete At cpu clock periodicity needed for current task, the local cpu clock frequency of mobile device terminal；
Mobile edge calculations system status information is transferred to DNN, MCTS and LSTM by local Edge Server；Edge service Mobile edge calculations system status information is transferred to DNN, MCTS and LSTM by device；Edge Server is received from N number of movement N number of task requests T of device endc={ T0, T1..., TN-1, while τ+1 at the time of be following, τ+2 ... generate M at random Virtual task Tv={ T0, T1..., TM-1, wherein virtual M task is to reserve resource for following task, then by resource Distribute to X=M+N task T={ T0..., TN..., TX-1, LSTM is according to the channel gain of mobile device terminal and base station to not The channel gain for carrying out M task is predicted, will be obtained channel gain prediction data and is sent to MCTS and DNN, DNN is further according to shifting Dynamic edge calculations system status information and channel gain prediction data obtain resource allocation actions prior probability, the priori that will be obtained Probability is sent to MCTS；
MCTS combines mobile edge calculations system status information and channel gain prediction data, generates a simulated environment, The node of Monte Carlo tree is expressed as to the distribution condition of last distribution movement, the collaboration side of this task is completed in last movement The selection of edge server, task unload the distribution of ratio or resource ratio, and the prior probability of MCTS combination DNN output is searched Rope；
Search process flow chart such as Fig. 5, the specific steps are as follows:
S1, the root node that MCTS is initialized according to mobile edge calculations system mode, root node information are expressed as s0=(Fe, B, T) and channel estimating data, wherein FeIndicate the calculating of Edge Server (local Edge Server and collaboration Edge Server) Ability and resource situation, B indicate the state of each wireless channel, such as fading profiles and noise size, and T indicates task requests letter Breath；Wherein T contains the communication resource h of mobile device terminal and base station, and the size of data d of task, mobile device terminal CPU are complete At the period c and cpu frequency f of required by taskl；
S2, it sets search starting point to root node and starts to be searched for next time, when search first searches for first task It is offloaded to the ratio of local Edge Server, then search is the communication resource ratio of first task distribution, searches again for local Edge Server is the computing resource ratio of first task distribution, searches for the Edge Server address of collaboration, is offloaded to collaboration The ratio of the first task of Edge Server, the computing resource ratio that collaboration Edge Server is distributed to first task, the One task is assigned, and search Section 2 task is offloaded to the ratio of local Edge Server, is then searched for and is appointed for Section 2 The communication resource ratio of business distribution searches again for the computing resource ratio that local Edge Server is the distribution of Section 2 task, search Collaboration Edge Server address (complete front task do not distribute front collaboration Edge Server resource, then after Collaboration Edge Server before continuous use is then searched for newly if having distributed the resource of the collaboration Edge Server of front Cooperate with Edge Server), it is offloaded to the ratio of the Section 2 task of collaboration Edge Server, cooperates with Edge Server to Section 2 The computing resource ratio of task distribution, executes according to above-mentioned allocation order, searches X item task always and completes resource point Match, so number of stories m ax_depth=6*X where the leaf node of search depth, that is, path；
S3, judge i >=max_search, if the search of pre-determined number is completed, if so, step S9 is executed, if it is not, then holding Row step S4, wherein i is that searching times, max_search have been the searching times of setting；
S4, judge whether k > max_depth, present node are leaf node, if so, step S5 is executed, if it is not, then executing Step S6, k are present node, and max_depth is leaf node；
S5, after entering leaf node, i.e., after the completion of resource allocation, required by task is completed according to the Resource Allocation Formula Time parameter obtains award r, the state of all nodes on the paths is updated according to r, more new formula has N (sk, ak)=N (sk, ak)+1, (Q (sk, ak)*N(sk, ak)+r/(N(sk, ak)+1), wherein s indicates state, a expression movement, N (sk, ak) indicate side (sk, ak) searching times, Q (sk,ak) indicate state-movement opposite side (sk,ak) value；
S6, judge whether present node is full expanding node, if it is not, step S7 is executed, if so, thening follow the steps S8；
S7, all child nodes that present node is extended according to the prior probability of the output of DNN, and according to formulaSelect next node；
S8, from all possible distribution movement according to formula It chooses search and is worth highest movement, the movement for executing selection enters next layer of node；
S9, the most path of output access number, i.e., optimal Resource Allocation Formula；
Because the most path confidence level of searched number is higher, and the path is corresponding after multiple selection circulation The ratio for distributing resource is exactly optimal Resource Allocation Formula.Optimal Resource Allocation Formula includes being offloaded to local edge service The task ratio of device, communication resource ratio, the computing resource ratio of local server distribution, the ground of the Edge Server of collaboration Location is offloaded to the task ratio of collaboration Edge Server, the computing resource ratio of collaboration Edge Server distribution.
Above embodiments are only to illustrate the present invention and not limit the technical scheme described by the invention, although this explanation The present invention has been described in detail referring to above-mentioned each embodiment for book, but the present invention is not limited to above-mentioned specific implementation Mode, therefore any couple of present invention modifies or equivalent replacement；And the technical side of all spirit and scope for not departing from invention Case and its improvement, are intended to be within the scope of the claims of the invention.

Claims (10)

1. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree, which is characterized in that the method packet It includes:
Step 1: Edge Server updates mobile edge calculations system status information；The mobile edge calculations system mode letter Breath includes: the computing resource situation of Edge Server, the communication resource situation of radio communication base station and mobile terminal, and mobile The task request message of equipment；The task request message includes: the channel of each mobile device terminal and radio communication base station Cpu clock week needed for gain information, the data volume size of currently pending task, mobile device terminal complete current task Issue and mobile device terminal cpu clock frequency；
Step 2: mobile edge calculations system status information is transferred to DNN, MCTS and LSTM by Edge Server；The LSTM Future channel gain is predicted according to the channel gain of mobile device terminal and radio communication base station, and obtained channel gain is pre- Measured data is sent to MCTS and DNN；The DNN is obtained according to mobile edge calculations system status information and channel gain prediction data To resource allocation actions prior probability, obtained resource allocation actions prior probability is sent to MCTS；
Step 3: MCTS is dynamic according to the mobile edge calculations system status information, channel gain prediction data and resource allocation It is scanned for as prior probability, obtains optimal resource allocation scheme；And the optimal resource allocation scheme is sent to movement and is set Task is offloaded to mobile edge calculations system execution module by standby terminal, mobile device terminal, and mobile edge calculations system executes Module executes optimal resource allocation behavior according to optimal resource allocation scheme.
2. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the DNN is trained in advance, make its satisfaction: when thering is mobile edge calculations system status information and channel to increase When beneficial prediction data inputs, resource allocation actions prior probability can be exported to MCTS.
3. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, Be characterized in that, described in step 3 search the following steps are included:
S1:MCTS initializes root node according to mobile edge calculations system mode；
S2: search starting point is set by root node and is searched for next time；
S3: judge whether to complete the search of pre-determined number, if so, step s9 is executed, if it is not, thening follow the steps s4；
S4: judging whether present node is leaf node, if so, step s5 is executed, if it is not, thening follow the steps s6；
S5: after entering leaf node, i.e., after the completion of computational resource allocation, assessing the Resource Allocation Formula, return to award, according to Award updates the state of all nodes on the paths；
S6: judge whether present node is full expanding node, if it is not, step s7 is executed, if so, thening follow the steps s8；
S7: all child nodes of present node are extended according to the resource allocation actions prior probability of the output of DNN, and according to formulaNext node is selected, wherein Q (v 'k) be defined as saving Point v 'kJackpot prize value, N (v 'k) it is defined as node v 'kAccess times, e be balanced algorithm development and exploration proportionality coefficient, p(v′k|sk) indicate next node prior probability, k indicate present node the number of plies；
S8: choosing search from all possible resource allocation actions and be worth highest movement, under the movement entrance for executing selection One layer of node；
S9: the most path of output access number is optimal Resource Allocation Formula.
4. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the optimal resource allocation scheme training DNN searched for using MCTS in the step 3, described in improving The predictablity rate of DNN makes DNN export updated resource allocation actions prior probability and preferably MCTS is instructed to scan for, Optimize the edge calculations method for allocating tasks.
5. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the DNN includes an input layer Hi, n public hidden layer { H1,…,HnAnd q division sublayer, each Dividing sublayer includes m sub- hidden layersEvery layer of neuron number is expressed asThe parameter of every layer of neural network isWherein i indicates that input layer, s indicate Sub- hidden layer, O indicate that output layer, W indicate weight, and b indicates biasing.
6. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the LSTM is according to historical juncture channel gain hτ-p+1, hτ-p+2..., hτPredict future time channel gain hτ+1, It is by LSTM net definitions
hτ+1=gθ(hτ-p+1,hτ-p+2..., hτ)
Wherein, θ is the weight parameter of LSTM, and LSTM Web vector graphic cell factory stores long term state, by input gate, forgets door With three door controls of out gate.
7. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 3, It is characterized in that, the acquisition methods of the step s5 reward value r in described search are as follows:
Wherein tbestFor using minimum time used in history Resource Allocation Formula completion task, initial value is infinity, t is indicated Current resource allocation plan is completed the time of required by task, and σ is setting value and σ > 1.
8. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the optimal Resource Allocation Formula includes: that the calculating task of mobile device terminal is offloaded to local edge service The ratio of device, local Edge Server are whole to the computing resource ratio and Edge Server and mobile device of distribution of computation tasks End carries out wireless communication used communication resource ratio.
9. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, Edge Server completes the calculating task of mobile subscriber terminal, multiple neighbouring sides using co-operating mode Edge server shares computing resource, and it is neighbouring can be offloaded to other when handling local task by Edge Server for partial task The Edge Server of region free time.
10. a kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree according to claim 1, It is characterized in that, the optimal Resource Allocation Formula includes the task ratio for being offloaded to local Edge Server, communication resource ratio Example, the computing resource ratio of local server distribution, the address of the Edge Server of collaboration are offloaded to collaboration Edge Server Task ratio, the computing resource ratio of collaboration Edge Server distribution.
CN201910741439.2A 2019-08-12 2019-08-12 A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree Pending CN110427261A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910741439.2A CN110427261A (en) 2019-08-12 2019-08-12 A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910741439.2A CN110427261A (en) 2019-08-12 2019-08-12 A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree

Publications (1)

Publication Number Publication Date
CN110427261A true CN110427261A (en) 2019-11-08

Family

ID=68415599

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910741439.2A Pending CN110427261A (en) 2019-08-12 2019-08-12 A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree

Country Status (1)

Country Link
CN (1) CN110427261A (en)

Cited By (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110971706A (en) * 2019-12-17 2020-04-07 大连理工大学 Approximate optimization and reinforcement learning-based task unloading method in MEC
CN110989614A (en) * 2019-12-18 2020-04-10 电子科技大学 Vehicle edge calculation transfer scheduling method based on deep reinforcement learning
CN111104298A (en) * 2019-11-22 2020-05-05 国网黑龙江省电力有限公司信息通信公司 LSTM-based power grid server running state prediction device
CN111126594A (en) * 2019-11-25 2020-05-08 北京邮电大学 Neural network model dynamic segmentation method and device based on edge calculation
CN111324839A (en) * 2020-02-20 2020-06-23 盈嘉互联（北京）科技有限公司 Building big data caching method and device
CN111367657A (en) * 2020-02-21 2020-07-03 重庆邮电大学 Computing resource collaborative cooperation method based on deep reinforcement learning
CN111414252A (en) * 2020-03-18 2020-07-14 重庆邮电大学 Task unloading method based on deep reinforcement learning
CN111459670A (en) * 2020-03-30 2020-07-28 中科边缘智慧信息科技（苏州）有限公司 Method for performing cooperative processing at different levels of edge calculation
CN111813538A (en) * 2020-05-27 2020-10-23 西安交通大学 Edge computing resource allocation method
CN112328914A (en) * 2020-11-06 2021-02-05 辽宁工程技术大学 Task allocation method based on space-time crowdsourcing worker behavior prediction
CN112416578A (en) * 2020-11-05 2021-02-26 中山大学 Container cloud cluster resource utilization optimization method based on deep reinforcement learning
CN112464611A (en) * 2020-12-11 2021-03-09 电子科技大学 Automatic PCB wiring system based on cloud-end collaborative intelligent processing
CN112486690A (en) * 2020-12-11 2021-03-12 重庆邮电大学 Edge computing resource allocation method suitable for industrial Internet of things
CN112596892A (en) * 2020-11-23 2021-04-02 中标慧安信息技术股份有限公司 Data interaction method and system of multi-node edge computing equipment
CN112699560A (en) * 2020-12-31 2021-04-23 青岛海尔科技有限公司 Position recognition method and device, storage medium and electronic device
CN112738851A (en) * 2020-12-16 2021-04-30 北京邮电大学 Intelligent resource allocation system based on prediction mobile edge network and allocation method thereof
CN112764936A (en) * 2021-01-29 2021-05-07 北京邮电大学 Edge calculation server information processing method and device based on deep reinforcement learning
CN112862083A (en) * 2021-04-06 2021-05-28 南京大学 Deep neural network inference method and device under edge environment
CN112911016A (en) * 2021-02-25 2021-06-04 北京邮电大学 Edge-side collaborative computing unloading method and system, electronic equipment and storage medium
CN112948116A (en) * 2021-03-05 2021-06-11 电子科技大学 Edge computing cooperative computing resource allocation method based on online excitation
CN113010282A (en) * 2021-03-03 2021-06-22 电子科技大学 Edge cloud collaborative serial task unloading method based on deep reinforcement learning
CN113055721A (en) * 2019-12-27 2021-06-29 中国移动通信集团山东有限公司 Video content distribution method and device, storage medium and computer equipment
CN113139644A (en) * 2021-03-24 2021-07-20 北京科技大学顺德研究生院 Information source navigation method and device based on deep Monte Carlo tree search
CN113220459A (en) * 2021-05-26 2021-08-06 杭州海康威视数字技术股份有限公司 Task processing method and device
CN113315700A (en) * 2020-02-26 2021-08-27 中国电信股份有限公司 Computing resource scheduling method, device and storage medium
CN113382383A (en) * 2021-06-11 2021-09-10 浙江工业大学 Method for unloading calculation tasks of public transport vehicle based on strategy gradient
CN113407464A (en) * 2020-03-16 2021-09-17 中移(上海)信息通信科技有限公司 Data processing method, device, equipment and medium
US20210342198A1 (en) * 2020-05-01 2021-11-04 Dell Products L. P. System for allocating task processing between an iot device and an edge device
CN113673672A (en) * 2021-07-08 2021-11-19 哈尔滨工业大学 Curling game strategy generation method based on Monte Carlo reinforcement learning
CN113905070A (en) * 2021-09-29 2022-01-07 上海兴容信息技术有限公司 Service providing method and system
CN114116050A (en) * 2021-11-16 2022-03-01 天津市英贝特航天科技有限公司 Selective unloading method and system for edge calculation
CN115396955A (en) * 2022-08-24 2022-11-25 广西电网有限责任公司 Resource allocation method and device based on deep reinforcement learning algorithm
CN117521576A (en) * 2024-01-08 2024-02-06 深圳鸿芯微纳技术有限公司 Computing resource sharing method, device, equipment and medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160205122A1 (en) * 2013-04-10 2016-07-14 Gabriel Bassett System and Method for Cyber Security Analysis and Human Behavior Prediction
CN108809713A (en) * 2018-06-08 2018-11-13 中国科学技术大学 Monte Carlo tree searching method based on optimal resource allocation algorithm
CN109002358A (en) * 2018-07-23 2018-12-14 厦门大学 Mobile terminal software adaptive optimization dispatching method based on deeply study
CN109857532A (en) * 2019-01-22 2019-06-07 杭州电子科技大学 DAG method for scheduling task based on the search of Monte Carlo tree
CN109976915A (en) * 2019-04-02 2019-07-05 中国联合网络通信集团有限公司 The optimization method and system of side cloud coordination requirement based on edge calculations

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160205122A1 (en) * 2013-04-10 2016-07-14 Gabriel Bassett System and Method for Cyber Security Analysis and Human Behavior Prediction
CN108809713A (en) * 2018-06-08 2018-11-13 中国科学技术大学 Monte Carlo tree searching method based on optimal resource allocation algorithm
CN109002358A (en) * 2018-07-23 2018-12-14 厦门大学 Mobile terminal software adaptive optimization dispatching method based on deeply study
CN109857532A (en) * 2019-01-22 2019-06-07 杭州电子科技大学 DAG method for scheduling task based on the search of Monte Carlo tree
CN109976915A (en) * 2019-04-02 2019-07-05 中国联合网络通信集团有限公司 The optimization method and system of side cloud coordination requirement based on edge calculations

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JIENAN CHEN，SIYU CHEN等: "iRAF: A Deep Reinforcement Learning Approach for Collaborative Mobile Edge Computing IoT Networks", 《IEEE INTERNET OF THINGS JOURNAL》 *
SHUAI YU，RAMI LANGAR 等: "Collaborative Computation Offloading for Multi-access Edge Computing", 《IEEE CONFERENCE PUBLICATION》 *
YANG DU，PENGYU GAO 等: "Monte-Carlo Tree Search Aided Contextual Online Learning Approach for Wireless Caching", 《IEEE CONFERENCE PUBLICATION》 *
徐涵: "《大数据、人工智能和网络舆情管理》", 31 October 2018, 武汉大学出版社 *

Cited By (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111104298A (en) * 2019-11-22 2020-05-05 国网黑龙江省电力有限公司信息通信公司 LSTM-based power grid server running state prediction device
CN111126594A (en) * 2019-11-25 2020-05-08 北京邮电大学 Neural network model dynamic segmentation method and device based on edge calculation
CN111126594B (en) * 2019-11-25 2023-08-04 北京邮电大学 Neural network model dynamic segmentation method and device based on edge calculation
CN110971706A (en) * 2019-12-17 2020-04-07 大连理工大学 Approximate optimization and reinforcement learning-based task unloading method in MEC
CN110989614A (en) * 2019-12-18 2020-04-10 电子科技大学 Vehicle edge calculation transfer scheduling method based on deep reinforcement learning
CN113055721B (en) * 2019-12-27 2022-12-09 中国移动通信集团山东有限公司 Video content distribution method and device, storage medium and computer equipment
CN113055721A (en) * 2019-12-27 2021-06-29 中国移动通信集团山东有限公司 Video content distribution method and device, storage medium and computer equipment
CN111324839A (en) * 2020-02-20 2020-06-23 盈嘉互联（北京）科技有限公司 Building big data caching method and device
CN111324839B (en) * 2020-02-20 2021-07-27 盈嘉互联（北京）科技有限公司 Building big data caching method and device
CN111367657A (en) * 2020-02-21 2020-07-03 重庆邮电大学 Computing resource collaborative cooperation method based on deep reinforcement learning
CN113315700A (en) * 2020-02-26 2021-08-27 中国电信股份有限公司 Computing resource scheduling method, device and storage medium
CN113315700B (en) * 2020-02-26 2022-06-28 中国电信股份有限公司 Computing resource scheduling method, device and storage medium
CN113407464A (en) * 2020-03-16 2021-09-17 中移(上海)信息通信科技有限公司 Data processing method, device, equipment and medium
CN113407464B (en) * 2020-03-16 2023-10-27 中移(上海)信息通信科技有限公司 Data processing method, device, equipment and medium
CN111414252A (en) * 2020-03-18 2020-07-14 重庆邮电大学 Task unloading method based on deep reinforcement learning
CN111459670A (en) * 2020-03-30 2020-07-28 中科边缘智慧信息科技（苏州）有限公司 Method for performing cooperative processing at different levels of edge calculation
US11620167B2 (en) * 2020-05-01 2023-04-04 Dell Products L.P. System for allocating task processing between an IoT device and an edge device
US20210342198A1 (en) * 2020-05-01 2021-11-04 Dell Products L. P. System for allocating task processing between an iot device and an edge device
CN111813538B (en) * 2020-05-27 2024-03-29 西安交通大学 Edge computing resource allocation method
CN111813538A (en) * 2020-05-27 2020-10-23 西安交通大学 Edge computing resource allocation method
CN112416578B (en) * 2020-11-05 2023-08-15 中山大学 Container cloud cluster resource utilization optimization method based on deep reinforcement learning
CN112416578A (en) * 2020-11-05 2021-02-26 中山大学 Container cloud cluster resource utilization optimization method based on deep reinforcement learning
CN112328914A (en) * 2020-11-06 2021-02-05 辽宁工程技术大学 Task allocation method based on space-time crowdsourcing worker behavior prediction
CN112596892B (en) * 2020-11-23 2021-08-31 中标慧安信息技术股份有限公司 Data interaction method and system of multi-node edge computing equipment
CN112596892A (en) * 2020-11-23 2021-04-02 中标慧安信息技术股份有限公司 Data interaction method and system of multi-node edge computing equipment
CN112486690B (en) * 2020-12-11 2024-01-30 重庆邮电大学 Edge computing resource allocation method suitable for industrial Internet of things
CN112486690A (en) * 2020-12-11 2021-03-12 重庆邮电大学 Edge computing resource allocation method suitable for industrial Internet of things
CN112464611A (en) * 2020-12-11 2021-03-09 电子科技大学 Automatic PCB wiring system based on cloud-end collaborative intelligent processing
CN112464611B (en) * 2020-12-11 2022-09-20 电子科技大学 Automatic PCB wiring system based on cloud-end collaborative intelligent processing
CN112738851B (en) * 2020-12-16 2022-05-20 北京邮电大学 Intelligent resource allocation system based on predictive mobile edge network and allocation method thereof
CN112738851A (en) * 2020-12-16 2021-04-30 北京邮电大学 Intelligent resource allocation system based on prediction mobile edge network and allocation method thereof
CN112699560A (en) * 2020-12-31 2021-04-23 青岛海尔科技有限公司 Position recognition method and device, storage medium and electronic device
CN112764936B (en) * 2021-01-29 2022-06-14 北京邮电大学 Edge calculation server information processing method and device based on deep reinforcement learning
CN112764936A (en) * 2021-01-29 2021-05-07 北京邮电大学 Edge calculation server information processing method and device based on deep reinforcement learning
CN112911016A (en) * 2021-02-25 2021-06-04 北京邮电大学 Edge-side collaborative computing unloading method and system, electronic equipment and storage medium
CN113010282A (en) * 2021-03-03 2021-06-22 电子科技大学 Edge cloud collaborative serial task unloading method based on deep reinforcement learning
CN112948116B (en) * 2021-03-05 2023-06-13 电子科技大学 Edge computing cooperation computing resource allocation method based on online excitation
CN112948116A (en) * 2021-03-05 2021-06-11 电子科技大学 Edge computing cooperative computing resource allocation method based on online excitation
CN113139644B (en) * 2021-03-24 2024-02-09 北京科技大学顺德研究生院 Information source navigation method and device based on deep Monte Carlo tree search
CN113139644A (en) * 2021-03-24 2021-07-20 北京科技大学顺德研究生院 Information source navigation method and device based on deep Monte Carlo tree search
CN112862083A (en) * 2021-04-06 2021-05-28 南京大学 Deep neural network inference method and device under edge environment
CN112862083B (en) * 2021-04-06 2024-04-09 南京大学 Deep neural network inference method and device in edge environment
CN113220459A (en) * 2021-05-26 2021-08-06 杭州海康威视数字技术股份有限公司 Task processing method and device
CN113220459B (en) * 2021-05-26 2023-10-10 杭州海康威视数字技术股份有限公司 Task processing method and device
CN113382383A (en) * 2021-06-11 2021-09-10 浙江工业大学 Method for unloading calculation tasks of public transport vehicle based on strategy gradient
CN113673672B (en) * 2021-07-08 2024-03-29 哈尔滨工业大学 Curling competition strategy generation method based on Monte Carlo reinforcement learning
CN113673672A (en) * 2021-07-08 2021-11-19 哈尔滨工业大学 Curling game strategy generation method based on Monte Carlo reinforcement learning
CN113905070A (en) * 2021-09-29 2022-01-07 上海兴容信息技术有限公司 Service providing method and system
CN113905070B (en) * 2021-09-29 2023-12-01 兴容(上海)信息技术股份有限公司 Service providing method and system
CN114116050A (en) * 2021-11-16 2022-03-01 天津市英贝特航天科技有限公司 Selective unloading method and system for edge calculation
CN115396955A (en) * 2022-08-24 2022-11-25 广西电网有限责任公司 Resource allocation method and device based on deep reinforcement learning algorithm
CN117521576A (en) * 2024-01-08 2024-02-06 深圳鸿芯微纳技术有限公司 Computing resource sharing method, device, equipment and medium
CN117521576B (en) * 2024-01-08 2024-04-26 深圳鸿芯微纳技术有限公司 Computing resource sharing method, device, equipment and medium

Similar Documents

Publication Publication Date Title
CN110427261A (en) A kind of edge calculations method for allocating tasks based on the search of depth Monte Carlo tree
CN112134916B (en) Cloud edge collaborative computing migration method based on deep reinforcement learning
Chai et al. Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT
CN111556461B (en) Vehicle-mounted edge network task distribution and unloading method based on deep Q network
Chen et al. Efficiency and fairness oriented dynamic task offloading in internet of vehicles
Li et al. NOMA-enabled cooperative computation offloading for blockchain-empowered Internet of Things: A learning approach
CN111953758B (en) Edge network computing unloading and task migration method and device
Liu et al. A reinforcement learning-based resource allocation scheme for cloud robotics
Vemireddy et al. Fuzzy reinforcement learning for energy efficient task offloading in vehicular fog computing
Chen et al. Multiuser computation offloading and resource allocation for cloud–edge heterogeneous network
CN111367657B (en) Computing resource collaborative cooperation method based on deep reinforcement learning
CN113794494B (en) Edge computing system and computing unloading optimization method for low-orbit satellite network
Li et al. Adaptive service function chaining mappings in 5G using deep Q-learning
Chen et al. Edge intelligence computing for mobile augmented reality with deep reinforcement learning approach
CN115271099A (en) Self-adaptive personalized federal learning method supporting heterogeneous model
Gao et al. Fast adaptive task offloading and resource allocation via multiagent reinforcement learning in heterogeneous vehicular fog computing
CN115033359A (en) Internet of things agent multi-task scheduling method and system based on time delay control
Yan et al. Two-dimensional task offloading for mobile networks: An imitation learning framework
Wang et al. Improving the performance of tasks offloading for internet of vehicles via deep reinforcement learning methods
Rodoshi et al. Deep reinforcement learning based dynamic resource allocation in cloud radio access networks
CN117436485A (en) Multi-exit point end-edge-cloud cooperative system and method based on trade-off time delay and precision
Chen et al. Traffic prediction-assisted federated deep reinforcement learning for service migration in digital twins-enabled MEC networks
Tang et al. Optimization search strategy for task offloading from collaborative edge computing
Huang et al. Mobility-aware computation offloading with load balancing in smart city networks using MEC federation
Cui et al. Resource-Efficient DNN Training and Inference for Heterogeneous Edge Intelligence in 6G

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20191108