CN113708969A - Cooperative embedding method of cloud data center virtual network based on deep reinforcement learning - Google Patents

Cooperative embedding method of cloud data center virtual network based on deep reinforcement learning Download PDF

Info

Publication number
CN113708969A
CN113708969A CN202110995220.2A CN202110995220A CN113708969A CN 113708969 A CN113708969 A CN 113708969A CN 202110995220 A CN202110995220 A CN 202110995220A CN 113708969 A CN113708969 A CN 113708969A
Authority
CN
China
Prior art keywords
network
virtual
node
cpu
mapping
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110995220.2A
Other languages
Chinese (zh)
Other versions
CN113708969B (en
Inventor
王廷
杨芃
王志浩
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
East China Normal University
Original Assignee
East China Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by East China Normal University filed Critical East China Normal University
Priority to CN202110995220.2A priority Critical patent/CN113708969B/en
Publication of CN113708969A publication Critical patent/CN113708969A/en
Application granted granted Critical
Publication of CN113708969B publication Critical patent/CN113708969B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/145—Network analysis or design involving simulating, designing, planning or modelling of a network
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/46—Interconnection of networks
    • H04L12/4641—Virtual LANs, VLANs, e.g. virtual private networks [VPN]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Signal Processing (AREA)
  • Molecular Biology (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Security & Cryptography (AREA)
  • Medical Informatics (AREA)
  • Geometry (AREA)
  • Computer Hardware Design (AREA)
  • Probability & Statistics with Applications (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention discloses a deep reinforcement learning-based collaborative embedding method for a cloud data center virtual network, which specifically comprises the following steps: s1, completing modeling of the virtual network embedding problem; s2, completing modeling of the computing resource CPU and the network resource bandwidth in the underlying network; and S3, constructing a neural network, and processing the request of the virtual network to realize the embedding of the virtual network into the underlying network. The invention is based on the deep reinforcement learning technology, and dynamically learns the resource allocation strategy under a series of constraint conditions according to the existing available resources and the virtual network request of the tenant, so as to improve the resource utilization rate and the request allocation success rate. Meanwhile, the invention combines a resource abstraction model Blocking Island, which can abstractly express the structure and available resources of each part of the underlying network, thereby reducing the search space and improving the calculation efficiency and the algorithm performance.

Description

Cooperative embedding method of cloud data center virtual network based on deep reinforcement learning
Technical Field
The invention relates to the technical field of cloud computing, in particular to a cooperative embedding method for processing a virtual network request in a cloud data center and realizing mapping from a virtual network to a bottom network based on deep reinforcement learning.
Background
The cloud computing data center is used as a highly-multiplexed shared environment, and is configured with a large number of physical servers and virtual machines running on the physical servers, so that resources such as high-reliability computing, storage and network are provided for multiple tenants in a pay-as-you-go mode. How to effectively share underlying physical network resources among multiple tenants with different network characteristics and requirements is a key issue. Therefore, a Network Virtualization (Network Virtualization) technology has become an effective resource sharing technology, and multi-tenant multiplexing of computing resources (such as CPUs) and Network resources (such as bandwidth) in a data center is effectively realized. The Network structure customized by the tenant is generally called Virtual Network (VN). How to use the existing and dynamically-changing limited underlying physical resources to serve the Virtual Network requests of multiple tenants and effectively allocate resources, and maximizing the benefits is the Virtual Network Embedding (VNE) problem.
In the VNE problem, the physical Network owned by a cloud computing data center (or ISP) is called an underlying Network (infrastructure Network); the Network customized by the tenant is called a Virtual Network (Virtual Network). In an underlying network, each compute node (e.g., host) owns a certain amount of available computing resources, such as a CPU count or a processor core count; the computing nodes are connected by physical links, each of which can carry a certain amount of network resources, such as bandwidth. Virtual networks are essentially tenants' requirements for their network structure and resources. In a virtual network, each node represents a virtual operation node, which needs a certain amount of computing resources; each link connecting nodes represents a virtual link, which similarly requires a certain amount of network bandwidth resources. VNE is to map one or more virtual networks onto an existing underlay network while ensuring that resources are utilized efficiently and requests are handled as successfully as possible. The "mapping", i.e., resource allocation, allocates resources on the underlying network to the tenant's virtual network. Specifically, the computing resources of the computing nodes in the underlying network are allocated to the virtual nodes as needed, and the network resources of each link on the underlying network path are allocated to the virtual links as needed.
The mapping of virtual networks to shared physical resources by allocating resources of an underlying network to virtual nodes and virtual links of the virtual network and satisfying a series of constraints is a resource allocation and scheduling problem with extremely high computational complexity, and the VNE problem proved by existing studies is the NP-hard problem, and the method for the optimal solution of this kind of problem has reached exponential order complexity. In a real scene, a network architecture of a cloud data center is large and complex in scale, and a great amount of time is consumed for making a decision on resource allocation through the methods, which is unrealistic and difficult to meet the requirements of low time delay and high efficiency of a modern cloud data center. Therefore, many studies on VNE have been seeking approximately optimal solutions through some heuristic, deterministic method. These methods usually consist of a set of manually written, static, fixed processes that are directly sorted, filtered, and finally mapped according to the characteristics of the virtual network and the underlying network only. Therefore, the method cannot dynamically adjust the mapping process according to the feedback or the income obtained in actual operation, is difficult to learn some resource allocation rules from historical data, and cannot be adjusted and optimized while falling into a local optimal solution. With the development of artificial intelligence related fields in recent years, reinforcement learning is widely applied to the fields of science, engineering, art and the like as a machine learning method. Generally, in the reinforcement learning method, there is one agent interacting with the environment: at each time slice, the agent receives the current state from the environment as input, and selects an action to interact with the environment according to the policy of the agent, so as to enter a new state and obtain a reward (reward). The reinforcement learning method can continuously adjust the strategy of agent selection action according to the obtained reward size and the learning target so as to efficiently obtain an approximately optimal solution in some complex problems. The nature of reinforcement learning also makes it applicable to solving VNE problems. In fact, there have been studies that began to introduce methods such as Q-Learning, Policy Network, etc., while Learning strategies for allocating underlying Network resources in conjunction with other machine Learning models. However, there is relatively little work in this area and there is much room for improvement and optimization in the work. Therefore, how to combine machine learning to solve the VNE problem is still a popular research topic in academic and industrial circles, and has high scientific research value.
Disclosure of Invention
Aiming at the problems that the model is complex, the calculation requirement is high, or only the characteristics related to the underlying network nodes are simply extracted as input and the like in the prior art, the invention aims to provide a cooperative embedding method of a cloud data center virtual network based on deep reinforcement learning so as to improve the resource utilization rate and the request allocation success rate.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a collaborative embedding method of a cloud data center virtual network based on deep reinforcement learning comprises the following specific steps:
s1, completing the modeling of virtual network embedding;
firstly, the underlying network in the actually existing data center is modeled into a weighted undirected graph Gs＝(Ns，Es) Wherein the network nodes and links respectively form a point set N in the undirected graphsAnd edge set EsPoint n in the figures∈NsWeight C (n) ofv) And edge es∈EsWeight B (e) ofs) Respectively representing the computing resource CPU of the network node and the network resource bandwidth of the link; modeling the virtual network requested by the tenant into a weighted undirected graph G in the same wayv＝(Nv，Ev)；
Virtual network embedding is then translated into finding a graph G that represents the virtual networkvTo graph G representing the underlying networksA set of mapping relation functions of:
Figure BDA0003233634250000021
wherein
Figure BDA0003233634250000022
And ensure that:
Figure BDA0003233634250000023
Figure BDA0003233634250000024
meanwhile, both the CPU resource and the bandwidth resource in the virtual network need to be met by the mapped network:
Figure BDA0003233634250000025
Figure BDA0003233634250000026
wherein the function C (n)v) Representing a node nvCPU computing resources available at the top, function B (e)v) Represents a link evBandwidth resources available on;
on the basis, linear combination of the total CPU requirement and the total bandwidth requirement in the virtual network request is used
Figure BDA0003233634250000031
Figure BDA0003233634250000032
To represent the expected gain when the request is successfully processed, where α is a weighting factor; summing the cost of computing resources and the cost of network resources when processing requests
Figure BDA0003233634250000033
Figure BDA0003233634250000034
To represent the total overhead of processing the request, wherein
Figure BDA0003233634250000035
Representing edges e for composing virtual networksvUnderlying network path of
Figure BDA0003233634250000036
The number of middle edges;
embedding the virtual network is definitely determined to maximize the benefit and minimize the cost on the basis of searching a mapping relation function meeting the conditions;
s2, completing modeling of the computing resource CPU and the network resource bandwidth in the underlying network; first, defining CPU available quantity upper limit CPUmaxAnd the bandwidth availability upper limit Bandmax(ii) a Then a fixed set of values cpu is presetstridesAnd bandstridesAnd will [0, CPUmax]And [0, Bandmax]Two intervals are equally divided into cpustridesAnd bandstridesA segment; respectively recording the end points of each equal part as
Figure BDA0003233634250000037
And
Figure BDA0003233634250000038
wherein
Figure BDA0003233634250000039
i∈[0，cpustrides)，j∈[0，bandstrides) (ii) a Finally, for each node n in the underlay networksSequentially constructing a beta-BI model of the CPU resource
Figure BDA00032336342500000310
And beta-BI model of bandwidth resources
Figure BDA00032336342500000311
Wherein
Figure BDA00032336342500000312
Refers to a slave node nsAvailable CPU reachable from start is not lower than
Figure BDA00032336342500000313
Of the node(s) of (a),
Figure BDA00032336342500000314
refers to a slave node nsStarting from available bandwidth not less than
Figure BDA00032336342500000315
A set of edge-reachable points of (c);
s3, constructing a neural network; various virtual network requests are then processed: for received virtual network G with the configuration described in step 1v＝(Nv，Ev) Request, neural network traversal NvFor each node n traversed tovAfter being processed by the neural network, the virtual node n is givenvMapping to the probability distribution of different bottom nodes, and then sampling according to the probability distribution to obtain a bottom node to complete a virtual node nvEmbedding; to be treated as a whole virtual node set NvAfter traversing and completing embedding, completing the virtual link E by BVLO and HVLM algorithmsvThereby realizing the embedding of the whole virtual network; and finally, finishing the back propagation of the neural network according to the error between the reward value reward after the virtual network request is mapped and the predicted value of the neural network in the forward propagation process, and updating the model parameters in the neural network.
Step S2 beta-BI model of CPU resource
Figure BDA00032336342500000316
And beta-BI model of bandwidth resources
Figure BDA00032336342500000317
By constructing an undirected graph G based on an underlying networks＝(Ns，Es) The above method of breadth-first search is used.
The step S3 specifically includes:
s31, constructing a neural network;
firstly, extracting the characteristics of an input layer; traversing underlying network node set NsEach underlying network node n insSequentially acquiring the beta-BI model of the CPU resource calculated in the step S2
Figure BDA00032336342500000318
And beta-BI model of bandwidth resources
Figure BDA00032336342500000319
From the beta-BI model corresponding to the underlying network node and the virtual network node n to be mapped currentlyvExtracting features from the information as neural network input; in particular for the underlying network node nsEach of (1)
Figure BDA0003233634250000041
And
Figure BDA0003233634250000042
extracting from it
Figure BDA0003233634250000043
Maximum value, minimum value, median and average value of CPU available quantity, degree, maximum value, minimum value, median and average value of bandwidth available quantity in adjacent edges, virtual node n to be mapped currentlyvAnd n is the CPU computing resource demandvAnd the bottom node nsThe difference of the available computing resource amounts of (a) as an input feature;
then constructing the rest part of the network, wherein the rest part of the network comprises 4 layers; specifically, the rest part is a one-dimensional convolution layer comprising two convolution kernels, a ReLU layer, a one-dimensional convolution layer of a single kernel and a softmax layer in sequence;
s32, the neural network receives the virtual network Gv＝(Nv，Ev) The request is processed by first completing the virtual node set NvMapping of (2);
traverse NvFor each virtual network node n thereinvAcquiring the input features in S31;
firstly, processing input features by a convolution layer in one-dimensional direction, wherein the convolution layer comprises two convolution kernels; the input features are recorded as
Figure BDA0003233634250000044
Where m is the number of bottom nodes, f is the number of features, and the two convolution kernels are each
Figure BDA0003233634250000045
Figure BDA0003233634250000046
The output of the convolutional layer is M.w1，w2]；
After passing through the convolutional layer with 2 convolutional kernels, the input features are processed through a ReLU layer to avoid overfitting;
the ReLU layer is followed by a layer of single-core one-dimensional convolutional layer processing; specifically, a score is calculated for each underlying network node from the learning results of the previous two layers, and the current virtual node n is quantitatively shownvMapping to the quality of each underlying network node; the result output by the layer is recorded as v ═ v1，v2，...，vm]Then for the underlying network nodes
Figure BDA0003233634250000047
With a calculated score of vi；
Finally, the scoring results calculated by the previous layer for each node are converted into probabilities and output through processing of a Softmax layer; specifically, the output result of the Softmax layer is recorded as S ═ S1，s2，...，sm]Wherein s is1，s2，...，smCorresponding to the bottom nodes one to one, the probability distribution is expressed as:
Figure BDA0003233634250000048
where exp () is an exponent based on a natural constantA function; for the underlying network nodes with higher corresponding scores in the upper layer, the higher the probability value output in the layer is, the more the current virtual node n is selected asvThe greater the probability of the mapped point of (a); then, on the basis of output results of the Softmax layer, firstly filtering the available computing resources which cannot meet the virtual node nvCalculating the resource requirement or the number of the nodes which are mapped to the nodes and reach a certain threshold value, and then sampling according to the probability output by the Softmax layer; current virtual node nvMapping the obtained data to the sampled underlying network nodes;
s33, after the mapping of the nodes in the virtual network request is completed through the above step S32, the link E is completedvMapping of (2);
firstly, sequencing virtual links to be mapped by using a BVLO algorithm; specifically, let virtual link evThe bottom network nodes mapped by the two end points are n ″)sAnd n'sThe symbol also includes n ″)sAnd n'sBandwidth resource beta-BI model
Figure BDA0003233634250000049
Of which the largest beta value is LCFsn(n″sn″′s) Beta value is expressed as
Figure BDA00032336342500000410
The BVLO algorithm will sequence the virtual links in order according to the following keys:
1)
Figure BDA0003233634250000051
smaller virtual links are ranked more forward;
2)LCFsn(n″s，n″′s) The smaller the number of nodes, the more forward the virtual link is ordered;
3) the higher the required bandwidth is, the higher the virtual link sequence is;
4) if the two virtual links are in equal sequence in 1), 2) and 3), randomly sequencing;
then virtual link mapping is performed based on the HVLM algorithm:
1) corresponding to underlying network nodes n' from two end points of the virtual linksAnd n'sLCF ofsn(n″s，n″′s) In (1), seek from n ″)sAnd n'sEach edge of the set of paths needs to meet the bandwidth requirement of the virtual link;
2) selecting a path with the least number of edges from the paths obtained in the step 1);
3) selecting a path with the maximum minimum value of available network resources in a passing edge from the paths with the minimum number of edges;
4) if a plurality of feasible paths still exist after the steps 1), 2) and 3), randomly selecting one path;
s34, completing point set NvAnd edge set EvI.e. the virtual network G is completedv＝(Nv，Ev) Processing the request, and then updating the neural network parameters according to reward value reward mapped by the virtual network;
the reward is calculated as follows:
1) if the mapping of the virtual node fails, that is, the mapping corresponding to a certain virtual node is not found, then n is applied to each virtual node of the virtual network requestvThe method comprises the following steps:
Figure BDA0003233634250000052
that is, in the forward propagation process corresponding to the virtual node, the neural network predicts the available CPU number of the underlying network node mapped by the maximum probability and the virtual node nvDifference in required CPU number from the virtual node nvThe ratio of the number of CPUs required;
2) if the virtual link ev＝(n″v，n″′v) Mapping fails, where n ″)vAnd n'vIs evI.e., does not find a suitable, feasible underlying path for mapping, first uses binary enumeration to find all the nodes in the tree
Figure BDA0003233634250000053
To
Figure BDA0003233634250000054
On the path of (2), the maximum value of the available bandwidth of the minimum edge is recorded as
Figure BDA0003233634250000055
Then for each virtual node n of the virtual network requestvThe method comprises the following steps:
Figure BDA0003233634250000056
the ratio of the difference between the maximum value and the required bandwidth of the virtual link to the required bandwidth of the virtual link;
3) virtual network Gv＝(Nv，Ev) The request mapping is successful; for each virtual node nvCalculating the weighted sum of the computing resources of the virtual node and the network resources of the virtual link connected with the virtual node, and dividing the weighted sum by the sum of the resources consumed by mapping the points and the links; using the obtained quotient as a mapping node nvThe corresponding return is sent back to the user,
Figure BDA0003233634250000057
wherein e isvTo connect node nvThe virtual link of (2);
after calculating the reward value reward corresponding to the virtual network request, calculating the error of forward propagation according to the predicted value of forward propagation each time and the reward corresponding to the forward propagation, and performing backward propagation to update the model parameters of the neural network.
The invention has the beneficial effects that:
after experimental comparison is carried out on the yield-cost ratio, the request success rate, the yield and the average mapping overhead by comparing the method in the invention with a Presto algorithm (a heuristic method for solving VNE problems based on a BI model) and an RLA algorithm (a method for learning virtual network mapping strategies based on Policy Gradient), the following results can be seen: the method provided by the invention is higher than Presto and RLA in the benefit-overhead ratio, request success rate and benefit; on average mapping overhead, this method is again lower than the other two comparison methods. Therefore, the resource allocation strategy is dynamically learned based on the deep reinforcement learning technology, and the resource utilization rate and the request allocation success rate are effectively improved. Meanwhile, the resource abstraction model combined by the method effectively reduces the search space and improves the calculation efficiency and the algorithm performance. The method has better performance in terms of resource utilization, overhead saving and income acquisition.
Drawings
FIG. 1 is a diagram of a neural network architecture constructed in the present invention;
FIG. 2 is a graph comparing the present invention with the Presto, RLA algorithm in terms of benefit-to-cost ratio;
FIG. 3 is a graph comparing the performance of the present invention with Presto, RLA algorithms on the success rate of virtual network requests;
FIG. 4 is a graph comparing the performance of the present invention with that of Presto, RLA algorithms on average mapping overhead;
FIG. 5 is a graph of the performance of the present invention versus the Presto, RLA algorithm in terms of yield.
Detailed Description
The invention is described in detail below with reference to the figures and examples. It will be clear that the examples given are only intended to illustrate the invention and are not intended to limit the scope of the invention.
The invention relates to a deep reinforcement learning-based collaborative embedding method for a cloud data center virtual network, which comprises the following steps:
s1, completing the modeling of virtual network embedding;
firstly, the underlying network in the actually existing data center is modeled into a weighted undirected graph Gs＝(Ns，Es) Wherein the network nodes and links respectively form a point set N in the undirected graphsAnd edge set EsPoint n in the figures∈NsWeight C (n) ofv) And edge es∈EsWeight B (e) ofs) Respectively representing the computing resource CPU of the network node and the network resource bandwidth of the link; modeling the virtual network requested by the tenant into a weighted undirected graph G in the same wayv＝(Nv，Ev)；
Virtual network embedding is then translated into finding a graph G that represents the virtual networkvTo graph G representing the underlying networksA set of mapping relation functions of:
Figure BDA0003233634250000061
wherein
Figure BDA0003233634250000062
And ensure that:
Figure BDA0003233634250000063
Figure BDA0003233634250000064
meanwhile, both the CPU resource and the bandwidth resource in the virtual network need to be met by the mapped network:
Figure BDA0003233634250000065
Figure BDA0003233634250000066
wherein the function C (n)v) Representing a node nvCPU computing resources available at the top, function B (e)v) Represents a link evBandwidth resources available on;
on the basis, linear combination of the total CPU requirement and the total bandwidth requirement in the virtual network request is used
Figure BDA0003233634250000071
Figure BDA0003233634250000072
To represent the expected gain when the request is successfully processed, where α is a weighting factor; summing the cost of computing resources and the cost of network resources when processing requests
Figure BDA0003233634250000073
Figure BDA0003233634250000074
To represent the total overhead of processing the request, wherein
Figure BDA0003233634250000075
Representing edges e for composing virtual networksvUnderlying network path of
Figure BDA0003233634250000076
And the number of edges;
embedding the virtual network is definitely determined to maximize the benefit and minimize the cost on the basis of searching a mapping relation function meeting the conditions;
s2, completing modeling of the computing resource CPU and the network resource bandwidth in the underlying network; first, defining CPU available quantity upper limit CPUmaxAnd the bandwidth availability upper limit Bandmax(ii) a Then a fixed set of values cpu is presetstridesAnd bandstridesAnd will [0, CPUmax]And [0, Bandmax]Two intervals are equally divided into cpustridesAnd bandstridesA segment; respectively recording the end points of each equal part as
Figure BDA0003233634250000077
And
Figure BDA0003233634250000078
wherein
Figure BDA0003233634250000079
i∈[0，cpustrides)，j∈[0，bandstrides) (ii) a Finally, for each node n in the underlay networksSequentially constructing the beta-BI model of the CPU resource by a breadth-first search method
Figure BDA00032336342500000710
And beta-BI model of bandwidth resources
Figure BDA00032336342500000711
Wherein
Figure BDA00032336342500000712
Refers to a slave node nsAvailable CPU reachable from start is not lower than
Figure BDA00032336342500000713
Of the node(s) of (a),
Figure BDA00032336342500000714
refers to a slave node nsStarting from available bandwidth not less than
Figure BDA00032336342500000715
A set of edge-reachable points of (c);
s3, constructing a neural network based on the policy gradient method, then processing the request of the virtual network, respectively completing the mapping of the virtual nodes and the virtual links to realize the embedding of the virtual network into the underlying network, and completing the updating of parameters in the neural network according to the reward value obtained by processing the request; the method specifically comprises the following substeps:
s31, constructing a neural network with the structure shown in the figure 1;
firstly, extracting the characteristics of an input layer; traversing underlying network node set NsEach underlying network node n insSequentially acquiring the beta-BI model of the CPU resource calculated in the step S2
Figure BDA00032336342500000716
And beta-BI model of bandwidth resources
Figure BDA00032336342500000717
From the beta-BI model corresponding to the underlying network node and the virtual network node n to be mapped currentlyvExtracting features from the information as neural network input; in particular for the underlying network node nsEach of (1)
Figure BDA00032336342500000718
And
Figure BDA00032336342500000719
extracting from it
Figure BDA00032336342500000720
Maximum value, minimum value, median and average value of CPU available quantity, degree, maximum value, minimum value, median and average value of bandwidth available quantity in adjacent edges, virtual node n to be mapped currentlyvAnd n is the CPU computing resource demandvAnd the bottom node nsThe difference of the available computing resource amounts of (a) as an input feature;
then constructing the rest part of the network, wherein the rest part of the network comprises 4 layers; specifically, the rest part is a one-dimensional convolution layer comprising two convolution kernels, a ReLU layer, a one-dimensional convolution layer of a single kernel and a softmax layer in sequence;
s32, the neural network receives the virtual network Gv＝(Nv，Ev) The request is processed by first completing the virtual node set NvMapping of (2);
traverse NvFor each virtual network node n thereinvAcquiring the input features in S31;
firstly, processing input features by a convolution layer in one-dimensional direction, wherein the convolution layer comprises two convolution kernels; the input features are recorded as
Figure BDA0003233634250000081
Where m is the number of bottom nodes, f is the number of features, and the two convolution kernels are each
Figure BDA0003233634250000082
Figure BDA0003233634250000083
The output of the convolutional layer is M.w1，w2]；
After passing through the convolutional layer with 2 convolutional kernels, the input features are processed through a ReLU layer to avoid overfitting;
the ReLU layer is followed by a layer of single-core one-dimensional convolutional layer processing; specifically, from the frontCalculating a grade for each underlying network node in the two-layer learning result, and quantitatively showing the current virtual node nvMapping to the quality of each underlying network node; the result output by the layer is recorded as v ═ v1，v2，...，vm]Then for the underlying network nodes
Figure BDA0003233634250000088
With a calculated score of vi；
Finally, the scoring results calculated by the previous layer for each node are converted into probabilities and output through a Soffmax layer; specifically, the output result of the Soffmax layer is S ═ S1，s2，...，sm]Wherein s is1，s2，...，smCorresponding to the bottom nodes one to one, the probability distribution is expressed as:
Figure BDA0003233634250000084
here exp () is an exponential function with a natural constant as the base; for the underlying network nodes with higher corresponding scores in the upper layer, the higher the probability value output in the layer is, the more the current virtual node n is selected asvThe greater the probability of the mapped point of (a); then, on the basis of the output result of the Soffmax layer, firstly filtering available computing resources which cannot meet the requirement of the virtual node nvCalculating the resource requirement or the number of the nodes which are mapped to the nodes and reach a certain threshold value, and then sampling according to the probability output by the Softmax layer; current virtual node nvMapping the obtained data to the sampled underlying network nodes;
s33, after the mapping of the nodes in the virtual network request is completed through the above step S32, the link E is completedvMapping of (2);
firstly, sequencing virtual links to be mapped by using a BVLO algorithm; specifically, let virtual link evThe bottom network nodes mapped by the two end points are n ″)sAnd n'sThe symbol also includes n ″)sAnd n'sBandwidth resource beta-BI model
Figure BDA0003233634250000085
Of which the largest beta value is LCFsn(n″s，n″′s) Beta value is expressed as
Figure BDA0003233634250000086
The BVLO algorithm will sequence the virtual links in order according to the following keys:
1)
Figure BDA0003233634250000087
smaller virtual links are ranked more forward;
2)LCFsn(n″s，n″′s) The smaller the number of nodes, the more forward the virtual link is ordered;
3) the higher the required bandwidth is, the higher the virtual link sequence is;
4) if the two virtual links are in equal sequence in 1), 2) and 3), randomly sequencing;
then virtual link mapping is performed based on the HVLM algorithm:
1) corresponding to underlying network nodes n' from two end points of the virtual linksAnd n'sLCF ofsn(n″s，n″′s) In (1), seek from n ″)sAnd n'sEach edge of the set of paths needs to meet the bandwidth requirement of the virtual link;
2) selecting a path with the least number of edges from the paths obtained in the step 1);
3) selecting a path with the maximum minimum value of available network resources in a passing edge from the paths with the minimum number of edges;
4) if a plurality of feasible paths still exist after the steps 1), 2) and 3), randomly selecting one path;
s34, completing point set NvAnd edge set EvI.e. the virtual network G is completedv＝(Nv，Ev) Processing the request, and then updating the neural network parameters according to reward value reward mapped by the virtual network;
the reward is calculated as follows:
1) if the mapping of the virtual node fails, that is, the mapping corresponding to a certain virtual node is not found, then n is applied to each virtual node of the virtual network requestvThe method comprises the following steps:
Figure BDA0003233634250000091
that is, in the forward propagation process corresponding to the virtual node, the neural network predicts the available CPU number of the underlying network node mapped by the maximum probability and the virtual node nvDifference in required CPU number from the virtual node nvThe ratio of the number of CPUs required;
2) if the virtual link ev＝(n″v，n″′v) Mapping fails, where n ″)vAnd n'vIs evI.e., does not find a suitable, feasible underlying path for mapping, first uses binary enumeration to find all the nodes in the tree
Figure BDA0003233634250000092
To
Figure BDA0003233634250000093
On the path of (2), the maximum value of the available bandwidth of the minimum edge is recorded as
Figure BDA0003233634250000094
Then for each virtual node n of the virtual network requestvThe method comprises the following steps:
Figure BDA0003233634250000095
the ratio of the difference between the maximum value and the required bandwidth of the virtual link to the required bandwidth of the virtual link;
3) virtual network Gv＝(Nv，Ev) The request mapping is successful; for each virtual node nvCalculating the weighted sum of the computing resources of the virtual node and the network resources of the virtual link connected with the virtual node, and dividing the weighted sum by the sum of the resources consumed by mapping the points and the links; using the obtained quotient as a mapping node nvThe corresponding return is sent back to the user,
Figure BDA0003233634250000096
wherein e isvTo connect node nvThe virtual link of (2);
after calculating the reward value reward corresponding to the virtual network request, calculating the error of forward propagation according to the predicted value of forward propagation each time and the reward corresponding to the forward propagation, and performing backward propagation to update the model parameters of the neural network.
From the test results, it can be found that as shown in fig. 2, where ICE refers to the present invention, the mapping strategy of the present invention has an average gain-cost ratio higher than that of Presto algorithm by 3.7%, the highest by approximately 5%, and an average gain-cost ratio higher than that of RLA algorithm by 6.3%, and the highest by approximately 7.8%. Further, as shown in FIG. 3, where ICE refers to the present invention; the mapping strategy of the invention also has better performance than that of Presto algorithm and RLA algorithm in the success rate of virtual network requests, and the highest performance is respectively higher than 9.4 percent and 14.4 percent. As the number of virtual network requests increases, the success rate of each algorithm will decrease, but the rate of decrease is slowest with the present invention. As shown in fig. 4, wherein ICE refers to the present invention; on the overhead of mapping, the present invention saves resource consumption by 4.9% on average compared to Presto algorithm and 8.5% on average compared to RLA algorithm, and gains 9.2% and 12.7% more gains than the two algorithms respectively at the highest, as shown in fig. 5, where ICE refers to the present invention; the invention is mainly because the invention is a cooperative virtual network embedding method based on a beta-BI model, which fully considers the available resource amount and corresponding topological information of each node, edge and each part of an underlying network when mapping nodes, learns a point set mapping strategy from the information through a neural network, and selects a proper edge set mapping step to process an incoming virtual network request.

Claims (3)

1. A collaborative embedding method of a cloud data center virtual network based on deep reinforcement learning is characterized by comprising the following specific steps:
s1, completing the modeling of virtual network embedding;
firstly, the seed is put into practical useThe underlying network in the data center that exists is modeled as a weighted undirected graph Gs＝(Ns，Es) Wherein the network nodes and links respectively form a point set N in the undirected graphsAnd edge set EsPoint n in the figures∈NsWeight C (n) ofv) And edge es∈EsWeight B (e) ofs) Respectively representing the computing resource CPU of the network node and the network resource bandwidth of the link; modeling the virtual network requested by the tenant into a weighted undirected graph G in the same wayv＝(Nv，Ev)；
Virtual network embedding is then translated into finding a graph G that represents the virtual networkvTo graph G representing the underlying networksA set of mapping relation functions of:
Figure FDA0003233634240000011
wherein
Figure FDA0003233634240000012
And ensure that:
Figure FDA0003233634240000013
Figure FDA0003233634240000014
meanwhile, both the CPU resource and the bandwidth resource in the virtual network need to be met by the mapped network:
Figure FDA0003233634240000015
Figure FDA0003233634240000016
wherein the function C (n)v) Representing a node nvCPU computing resources available at the top, function B (e)v) Represents a link evBandwidth resources available on;
on the basis, linear combination of the total CPU requirement and the total bandwidth requirement in the virtual network request is used
Figure FDA0003233634240000017
Figure FDA0003233634240000018
To represent the expected gain when the request is successfully processed, where α is a weighting factor; summing the cost of computing resources and the cost of network resources when processing requests
Figure FDA0003233634240000019
Figure FDA00032336342400000110
To represent the total overhead of processing the request, wherein
Figure FDA00032336342400000111
Representing edges e for composing virtual networksvUnderlying network path of
Figure FDA00032336342400000112
The number of middle edges;
embedding the virtual network is definitely determined to maximize the benefit and minimize the cost on the basis of searching a mapping relation function meeting the conditions;
s2, completing modeling of the computing resource CPU and the network resource bandwidth in the underlying network; first, defining CPU available quantity upper limit CPUmaxAnd the bandwidth availability upper limit Bandmax(ii) a Then a fixed set of values cpu is presetstridesAnd bandstridesAnd will [0, CPUmax]And [0, Bandmax]Two intervals are equally divided into cpustridesAnd bandstriaesA segment; respectively recording the end points of each equal part as
Figure FDA00032336342400000113
Medicine for curing cancer
Figure FDA00032336342400000114
Wherein
Figure FDA00032336342400000115
Finally, for each node n in the underlay networksSequentially constructing a beta-BI model of the CPU resource
Figure FDA00032336342400000116
And beta-BI model of bandwidth resources
Figure FDA00032336342400000117
Wherein
Figure FDA00032336342400000118
Refers to a slave node nsAvailable CPU reachable from start is not lower than
Figure FDA00032336342400000119
Of the node(s) of (a),
Figure FDA00032336342400000120
refers to a slave node nsStarting from available bandwidth not less than
Figure FDA00032336342400000121
A set of edge-reachable points of (c);
s3, constructing a neural network; various virtual network requests are then processed: for received virtual network G with the configuration described in step 1v＝(Nv，Ev) Request, neural network traversal NvFor each node n traversed tovAfter being processed by the neural network, the virtual node n is givenvMapping to the probability distribution of different bottom nodes, and then sampling according to the probability distribution to obtain a bottom node to complete a virtual node nvEmbedding; to be treated as a whole virtual node set NvAfter traversing and completing embedding, completing the virtual link E by BVLO and HVLM algorithmsvFromThe embedding of the whole virtual network is realized; and finally, finishing the back propagation of the neural network according to the error between the reward value reward after the virtual network request is mapped and the predicted value of the neural network in the forward propagation process, and updating the model parameters in the neural network.
2. The deep reinforcement learning-based collaborative embedding method for the cloud data center virtual network according to claim 1, wherein step S2 is implemented by using the β -BI model of the CPU resource
Figure FDA0003233634240000021
And beta-BI model of bandwidth resources
Figure FDA0003233634240000022
By constructing an undirected graph G based on an underlying networks＝(Ns，Es) The above method of breadth-first search is used.
3. The method for collaborative embedding of the cloud data center virtual network based on the deep reinforcement learning of claim 1, wherein the step S3 specifically includes:
s31, constructing a neural network;
firstly, extracting the characteristics of an input layer; traversing underlying network node set NsEach underlying network node n insSequentially acquiring the beta-BI model of the CPU resource calculated in the step S2
Figure FDA0003233634240000023
And beta-BI model of bandwidth resources
Figure FDA0003233634240000024
From the beta-BI model corresponding to the underlying network node and the virtual network node n to be mapped currentlyvExtracting features from the information as neural network input; in particular for the underlying network node nsEach of (1)
Figure FDA0003233634240000025
And
Figure FDA0003233634240000026
extracting from it
Figure FDA0003233634240000027
Maximum value, minimum value, median and average value of CPU available quantity, degree, maximum value, minimum value, median and average value of bandwidth available quantity in adjacent edges, virtual node n to be mapped currentlyvAnd n is the CPU computing resource demandvAnd the bottom node nsThe difference of the available computing resource amounts of (a) as an input feature;
then constructing the rest part of the network, wherein the rest part of the network comprises 4 layers; specifically, the rest part is a one-dimensional convolution layer comprising two convolution kernels, a ReLU layer, a one-dimensional convolution layer of a single kernel and a softmax layer in sequence;
s32, the neural network receives the virtual network Gv＝(Nv，Ev) The request is processed by first completing the virtual node set NvMapping of (2);
traverse NvFor each virtual network node n thereinvAcquiring the input features in S31;
firstly, processing input features by a convolution layer in one-dimensional direction, wherein the convolution layer comprises two convolution kernels; the input features are recorded as
Figure FDA0003233634240000028
Where m is the number of bottom nodes, f is the number of features, and the two convolution kernels are each
Figure FDA0003233634240000029
Figure FDA00032336342400000210
The output of the convolutional layer is M.w1，w2]；
After passing through the convolutional layer with 2 convolutional kernels, the input features are processed through a ReLU layer to avoid overfitting;
the ReLU layer is followed by a layer of single-core one-dimensional convolutional layer processing; specifically, a score is calculated for each underlying network node from the learning results of the previous two layers, and the current virtual node n is quantitatively shownvMapping to the quality of each underlying network node; the result output by the layer is recorded as v ═ v1，v2，...，vm]Then for the underlying network nodes
Figure FDA0003233634240000031
With a calculated score of vi；
Finally, the scoring results calculated by the previous layer for each node are converted into probabilities and output through processing of a Softmax layer; specifically, the output result of the Softmax layer is recorded as S ═ S1，s2，...，sm]Wherein s is1，s2，...，smCorresponding to the bottom nodes one to one, the probability distribution is expressed as:
Figure FDA0003233634240000032
here exp () is an exponential function with a natural constant as the base; for the underlying network nodes with higher corresponding scores in the upper layer, the higher the probability value output in the layer is, the more the current virtual node n is selected asvThe greater the probability of the mapped point of (a); then, on the basis of output results of the Softmax layer, firstly filtering the available computing resources which cannot meet the virtual node nvCalculating the resource requirement or the number of the nodes which are mapped to the nodes and reach a certain threshold value, and then sampling according to the probability output by the Softmax layer; current virtual node nvMapping the obtained data to the sampled underlying network nodes; s33, after the mapping of the nodes in the virtual network request is completed through the above step S32, the link E is completedvMapping of (2);
firstly, sequencing virtual links to be mapped by using a BVLO algorithm; more specifically, it relates toVirtual link evThe bottom network nodes mapped by the two end points are n ″)sAnd n'sThe symbol also includes n ″)sAnd n'sBandwidth resource beta-BI model
Figure FDA0003233634240000033
Of which the largest beta value is LCFsn(n″s，n″′s) Beta value is expressed as
Figure FDA0003233634240000034
The BVLO algorithm will sequence the virtual links in order according to the following keys:
1)
Figure FDA0003233634240000035
smaller virtual links are ranked more forward;
2)LCFsn(n″s，n″′s) The smaller the number of nodes, the more forward the virtual link is ordered;
3) the higher the required bandwidth is, the higher the virtual link sequence is;
4) if the two virtual links are in equal sequence in 1), 2) and 3), randomly sequencing;
then virtual link mapping is performed based on the HVLM algorithm:
1) corresponding to underlying network nodes n' from two end points of the virtual linksAnd n'sLCF ofsn(n″s，n″′s) In (1), seek from n ″)sAnd n'sEach edge of the set of paths needs to meet the bandwidth requirement of the virtual link;
2) selecting a path with the least number of edges from the paths obtained in the step 1);
3) selecting a path with the maximum minimum value of available network resources in a passing edge from the paths with the minimum number of edges;
4) if a plurality of feasible paths still exist after the steps 1), 2) and 3), randomly selecting one path;
s34, completing point set NvAnd edge set EvI.e. the virtual network G is completedv＝(Nv，Ev) Processing the request, and then updating the neural network parameters according to reward value reward mapped by the virtual network;
the reward is calculated as follows:
1) if the mapping of the virtual node fails, that is, the mapping corresponding to a certain virtual node is not found, then n is applied to each virtual node of the virtual network requestvThe method comprises the following steps:
Figure FDA0003233634240000041
that is, in the forward propagation process corresponding to the virtual node, the neural network predicts the available CPU number of the underlying network node mapped by the maximum probability and the virtual node nvDifference in required CPU number from the virtual node nvThe ratio of the number of CPUs required;
2) if the virtual link ev＝(n″v，n″′v) Mapping fails, where n ″)vAnd n'vIs evI.e., does not find a suitable, feasible underlying path for mapping, first uses binary enumeration to find all the nodes in the tree
Figure FDA0003233634240000042
To
Figure FDA0003233634240000043
On the path of (2), the maximum value of the available bandwidth of the minimum edge is recorded as
Figure FDA0003233634240000044
Then for each virtual node n of the virtual network requestvThe method comprises the following steps:
Figure FDA0003233634240000045
the ratio of the difference between the maximum value and the required bandwidth of the virtual link to the required bandwidth of the virtual link;
3) virtual network Gv＝(Nv，Ev) The request mapping is successful; for each virtual node nvCalculating the weighted sum of the computing resources of the virtual node and the network resources of the virtual link connected with the virtual node, and dividing the weighted sum by the sum of the resources consumed by mapping the points and the links; using the obtained quotient as a mapping node nvThe corresponding return is sent back to the user,
Figure FDA0003233634240000046
wherein e isvTo connect node nvThe virtual link of (2);
after calculating the reward value reward corresponding to the virtual network request, calculating the error of forward propagation according to the predicted value of forward propagation each time and the reward corresponding to the forward propagation, and performing backward propagation to update the model parameters of the neural network.
CN202110995220.2A 2021-08-27 2021-08-27 Collaborative embedding method of cloud data center virtual network based on deep reinforcement learning Active CN113708969B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110995220.2A CN113708969B (en) 2021-08-27 2021-08-27 Collaborative embedding method of cloud data center virtual network based on deep reinforcement learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110995220.2A CN113708969B (en) 2021-08-27 2021-08-27 Collaborative embedding method of cloud data center virtual network based on deep reinforcement learning

Publications (2)

Publication Number Publication Date
CN113708969A true CN113708969A (en) 2021-11-26
CN113708969B CN113708969B (en) 2023-09-29

Family

ID=78655942

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110995220.2A Active CN113708969B (en) 2021-08-27 2021-08-27 Collaborative embedding method of cloud data center virtual network based on deep reinforcement learning

Country Status (1)

Country Link
CN (1) CN113708969B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113037546A (en) * 2021-03-03 2021-06-25 中国石油大学(华东) Security virtual network mapping method based on deep reinforcement learning
CN114268552A (en) * 2021-12-16 2022-04-01 云南电网有限责任公司电力科学研究院 Complex network node prediction method
CN115001978A (en) * 2022-05-19 2022-09-02 华东师范大学 Cloud tenant virtual network intelligent mapping method based on reinforcement learning model
CN115361731A (en) * 2022-08-18 2022-11-18 东南大学 Multi-user power distribution method based on one-dimensional convolutional neural network
CN115941506A (en) * 2022-09-29 2023-04-07 重庆邮电大学 Multi-type service resource arrangement method based on strategy network reinforcement learning

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110088146A (en) * 2010-01-28 2011-08-03 한국과학기술원 Method and system for embedding resource for request for virtual network in mobile environment
CN110365514A (en) * 2019-05-24 2019-10-22 北京邮电大学 SDN multistage mapping method of virtual network and device based on intensified learning
CN110365568A (en) * 2019-06-18 2019-10-22 西安交通大学 A kind of mapping method of virtual network based on deeply study
CN113037546A (en) * 2021-03-03 2021-06-25 中国石油大学(华东) Security virtual network mapping method based on deep reinforcement learning

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20110088146A (en) * 2010-01-28 2011-08-03 한국과학기술원 Method and system for embedding resource for request for virtual network in mobile environment
CN110365514A (en) * 2019-05-24 2019-10-22 北京邮电大学 SDN multistage mapping method of virtual network and device based on intensified learning
CN110365568A (en) * 2019-06-18 2019-10-22 西安交通大学 A kind of mapping method of virtual network based on deeply study
CN113037546A (en) * 2021-03-03 2021-06-25 中国石油大学(华东) Security virtual network mapping method based on deep reinforcement learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
朱国晖;张茵;刘秀霞;孙天骜;: "基于两次优先级排序的虚拟网络映射算法", 计算机工程与科学, no. 05 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113037546A (en) * 2021-03-03 2021-06-25 中国石油大学(华东) Security virtual network mapping method based on deep reinforcement learning
CN114268552A (en) * 2021-12-16 2022-04-01 云南电网有限责任公司电力科学研究院 Complex network node prediction method
CN114268552B (en) * 2021-12-16 2023-10-13 云南电网有限责任公司电力科学研究院 Complex network node prediction method
CN115001978A (en) * 2022-05-19 2022-09-02 华东师范大学 Cloud tenant virtual network intelligent mapping method based on reinforcement learning model
CN115001978B (en) * 2022-05-19 2023-10-03 华东师范大学 Cloud tenant virtual network intelligent mapping method based on reinforcement learning model
CN115361731A (en) * 2022-08-18 2022-11-18 东南大学 Multi-user power distribution method based on one-dimensional convolutional neural network
CN115941506A (en) * 2022-09-29 2023-04-07 重庆邮电大学 Multi-type service resource arrangement method based on strategy network reinforcement learning

Also Published As

Publication number Publication date
CN113708969B (en) 2023-09-29

Similar Documents

Publication Publication Date Title
CN113708969A (en) Cooperative embedding method of cloud data center virtual network based on deep reinforcement learning
CN109710374A (en) The VM migration strategy of task unloading expense is minimized under mobile edge calculations environment
CN112286677B (en) Resource-constrained edge cloud-oriented Internet of things application optimization deployment method
CN109818786B (en) Method for optimally selecting distributed multi-resource combined path capable of sensing application of cloud data center
CN110264471A (en) A kind of image partition method, device, storage medium and terminal device
CN112380008A (en) Multi-user fine-grained task unloading scheduling method for mobile edge computing application
CN113098714A (en) Low-delay network slicing method based on deep reinforcement learning
CN112685138B (en) Multi-workflow scheduling method based on multi-population hybrid intelligent optimization in cloud environment
CN113485826B (en) Load balancing method and system for edge server
CN112149990B (en) Fuzzy supply and demand matching method based on prediction
CN116112563A (en) Dual-strategy self-adaptive cache replacement method based on popularity prediction
CN109388476A (en) Docker based on multiple target biogeography optimization algorithm polymerize laying method with virtual machine
Karimunnisa et al. Task Classification and Scheduling Using Enhanced Coot Optimization in Cloud Computing.
CN113608855B (en) Reinforced learning method for placing service function chains in edge calculation
CN118210603A (en) Cloud resource scheduling method based on enhanced growth optimizer
Dandachi et al. A robust monte-carlo-based deep learning strategy for virtual network embedding
Kang et al. Honeybee mating optimization algorithm for task assignment in heterogeneous computing systems
CN118012602A (en) Distributed cluster data equalization method based on balanced multi-way tree
Wu et al. Historical information-based differential evolution for dynamic optimization problem
Zhang et al. Information guiding and sharing enhanced simultaneous heat transfer search and its application to k-means optimization
CN116938323A (en) Satellite transponder resource allocation method based on reinforcement learning
Yu et al. Dynamic reproductive ant colony algorithm based on piecewise clustering
CN115361453A (en) Load fair unloading and transferring method for edge service network
CN114980216A (en) Dependent task unloading system and method based on mobile edge calculation
CN115001978A (en) Cloud tenant virtual network intelligent mapping method based on reinforcement learning model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant