CN113490254A - VNF migration method based on bidirectional GRU resource demand prediction in federal learning - Google Patents
VNF migration method based on bidirectional GRU resource demand prediction in federal learning Download PDFInfo
- Publication number
- CN113490254A CN113490254A CN202110919085.3A CN202110919085A CN113490254A CN 113490254 A CN113490254 A CN 113490254A CN 202110919085 A CN202110919085 A CN 202110919085A CN 113490254 A CN113490254 A CN 113490254A
- Authority
- CN
- China
- Prior art keywords
- vnf
- network
- physical node
- resource
- energy consumption
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000005012 migration Effects 0.000 title claims abstract description 48
- 238000013508 migration Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000002457 bidirectional effect Effects 0.000 title claims abstract description 13
- 238000005265 energy consumption Methods 0.000 claims abstract description 52
- 230000006870 function Effects 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims abstract description 17
- 230000002787 reinforcement Effects 0.000 claims abstract description 5
- YFPJFKYCVYXDJK-UHFFFAOYSA-N Diphenylphosphine oxide Chemical compound C=1C=CC=CC=1[P+](=O)C1=CC=CC=C1 YFPJFKYCVYXDJK-UHFFFAOYSA-N 0.000 claims abstract 4
- 238000004808 supercritical fluid chromatography Methods 0.000 claims description 45
- 238000012549 training Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 9
- 230000008859 change Effects 0.000 claims description 7
- 238000012935 Averaging Methods 0.000 claims description 5
- 230000009471 action Effects 0.000 claims description 2
- 230000002776 aggregation Effects 0.000 claims description 2
- 238000004220 aggregation Methods 0.000 claims description 2
- 238000012545 processing Methods 0.000 claims description 2
- 238000010295 mobile communication Methods 0.000 abstract description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 241000764238 Isis Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000002370 liquid polymer infiltration Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W48/00—Access restriction; Network selection; Access point selection
- H04W48/08—Access restriction or access information delivery, e.g. discovery data delivery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/0215—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices
- H04W28/0221—Traffic management, e.g. flow control or congestion control based on user or device properties, e.g. MTC-capable devices power availability or consumption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W28/00—Network traffic management; Network resource management
- H04W28/02—Traffic management, e.g. flow control or congestion control
- H04W28/08—Load balancing or load distribution
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a VNF migration method based on bidirectional GRU resource demand prediction in federal learning, and belongs to the technical field of mobile communication. The method comprises the following steps: s1: in a network slice scene, a VNF migration problem caused by time-varying network flow and a VNF migration delay problem caused by the fact that the resource demand of the VNF is lack of prediction are considered, and the resource demand of the VNF is predicted by adopting a FedBi-GRU algorithm; s2: according to the resource demand prediction result, calculating the resource utilization rate of the physical node, judging the physical node with overloaded resource use or underloaded resource use in the network system, and realizing system energy consumption optimization and load balance while ensuring network performance through VNF migration; s3: and obtaining the optimal decision of VNF migration by adopting a DPPO deep reinforcement learning method. The invention can reduce the migration times of the virtual network function and the energy consumption of the network system, and can ensure the load balance of the network system.
Description
Technical Field
The invention belongs to the technical field of mobile communication, and relates to a VNF migration method based on federal learning bidirectional GRU resource demand prediction.
Background
Software Defined Networking (SDN) and Network Function Virtualization (NFV) are two new architectures for managing network systems, the main idea behind SDN is to make the network directly programmable and separate the control plane from the data plane, provide a centralized view for the network, provide manageability for complex network systems, NFV converts physical layer resources into virtual resources, separates software instances from underlying dedicated hardware, and makes the network flexible.
By using the NFV technology, traditional network hardware resources can be virtualized into a plurality of virtual machines, various network element function software of an operator network can be instantiated into a Virtual Network Function (VNF), and the VNF can be deployed in a general server, so that the flexibility of network service deployment is greatly improved, the investment cost of network equipment of the operator is reduced, and the high-efficiency network resource utilization rate is realized. The dynamic change of the network flow causes that the available resources of the physical nodes are not matched with the resources required by the VNF, if the new VNF is re-instantiated, the state information of the original VNF is lost, the VNF can be migrated to solve the problem of resource mismatching, and in addition, the VNF is migrated by utilizing the VNF to realize the integration of the VNF, so that the energy consumption of a system can be reduced.
The existing VNF migration problem is that VNF migration is carried out based on real-time network information, few documents are used for carrying out resource demand prediction work of the VNF, a VNF migration plan is made in advance, and prediction methods adopted by the documents are all centralized machine learning. In addition, in the face of high-dimensional complexity of a VNF migration remapping space, an existing heuristic algorithm is not enough to find an optimal VNF migration scheme.
Disclosure of Invention
In view of this, the present invention provides a VNF migration method based on federal learning bidirectional GRU resource demand prediction, which reduces the training burden of the existing centralized prediction mode, improves the generalization of the prediction model, and simultaneously can realize network system energy consumption optimization and load balancing under the limitation of the bottom CPU, storage and bandwidth resources, and end-to-end network delay.
In order to achieve the purpose, the invention provides the following technical scheme:
a VNF migration method based on bidirectional GRU resource demand forecasting of federal learning specifically comprises the following steps:
s1: under a network slice scene, a Virtual Network Function (VNF) migration problem caused by time-varying network flow and a VNF migration delay problem caused by the lack of prediction of VNF resource demand are considered, and the resource demand of the VNF is predicted by adopting a Federal learning-based bidirectional gating cycle unit network (FedBi-GRU) algorithm;
s2: calculating the resource utilization rate of the physical node according to the resource demand prediction result obtained in the step S1, determining the physical node with overloaded resource use or underloaded resource use in the network system, and migrating through the VNF, thereby realizing system energy consumption optimization and load balancing while ensuring network performance;
s3: in consideration of the complexity and high dimensionality of the virtual network function migration decision problem, the optimal decision of VNF migration is obtained by adopting a deep reinforcement learning method of distributed near-end policy optimization (DPPO).
Further, in step S1, the network slice scene includes three layers of architectures: the physical infrastructure layer provides physical resources of the system and mainly comprises a plurality of physical nodes; the virtualization layer abstracts physical resources such as an access network, a core network, a frequency spectrum and the like into virtual resources and provides resource support for the slice layer; the network slice layer is used for processing user network services and is processed by SFC formed by arranging a series of VNFs according to a set sequence; wherein SFC represents a service function chain;
the VNF migration problem is selecting a remapped physical node for a VNF;
the resource requirements of the VNF are CPU, storage and bandwidth resources required by the VNF at a future time.
Further, in step S1, the FedBi-GRU algorithm performs local Bi-GRU resource demand prediction model training for VNFs on SFCs on respective deployed physical nodes, performs aggregation and averaging of prediction model parameters at the SDN controller, and then issues the prediction model parameters to each VNF for continuing model training, where Bi-GRU represents a bidirectional gating cycle unit, and SDN represents a software defined network; the method specifically comprises the following steps:
s11: issuing initial global parameters by the SDN controller;
s12: each VNF on the SFC receives the initial parameters to carry out local Bi-GRU resource demand prediction model training;
s13: uploading parameters of a local Bi-GRU resource demand prediction model by each VNF on the SFC;
s14: the SDN controller performs federal averaging on the collected parameters to obtain new global parameters;
s15: the VNF on the SFC receives the new global parameters and continues to train the Bi-GRU resource demand prediction model;
s16: and repeating the steps S11-S15 until the Bi-GRU resource demand forecasting models of all VNFs on the SFC converge.
Further, in step S2, the resource utilization rate of the physical node is a ratio of the sum of CPU resource requirements of all VNFs on the physical node to the CPU capacity of the physical node;
the resource utilization rate of the physical node with the resource use overload or the resource use underload is higher than the highest CPU resource use threshold value of the physical node or lower than the lowest CPU resource use threshold value;
the network performance is to satisfy the network service quality of the user.
Further, in step S2, the system energy consumption is optimized to minimize the network system energy consumption after VNF migration, where the network system energy consumption includes the operation state energy consumption and the state switching energy consumption of the bottom physical node; the operation state energy consumption comprises basic energy consumption of physical nodes capable of operating and load energy consumption of the nodes, the load energy consumption is in direct proportion to the utilization rate of CPU resources, and the operation energy consumption of the physical nodes at the moment t is represented as:
wherein Z isn(t) binary indicating whether physical node n is onVariables, physical node n remains on as long as there is a mapping of VNF to physical node n, at which time Zn(t) 1, otherwise the physical node enters the sleep state, Zn(t) ═ 0; is provided with
For the fundamental energy consumption of the operation of the physical node n,
the energy consumption is the energy consumption that the CPU resource of the physical node is occupied;
the CPU resource utilization rate of the physical node n at time t is expressed as:
wherein,
indicating whether VNF j in the ith SFC at time instant tth maps to a binary variable of physical node n,
indicating that VNF j maps onto physical node n; f is a set of SFCs,
all VNFs on the ith SFC are collected; cnRepresenting the computing resources owned by node n;
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) is represented as follows:
the switching energy consumption generated by the change of the working state of the node n at the moment t is expressed as
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system is expressed as:
wherein N isPIs a set of underlying physical nodes.
Further, in step S2, the load balancing is to minimize a variance of network resources, where the variance of network resources is a variance of utilization rates of CPUs and storage resources of all physical nodes of the network system;
assuming that at time t, VNF j on SFC i triggers a migration condition, after VNF j is migrated from physical node n to physical node m, load change of physical node m may be caused, and the load of physical node m at time t is represented as:
wherein,
and
respectively representing the CPU and storage resource loads of the physical node m at the moment t;
a binary variable indicating whether VNF j in the ith SFC maps to physical node n,
indicating that VNF j maps onto physical node n;
respectively representing the computing resource requirement and the storage resource requirement of the jth VNF on the ith SFC at time instant tth,
all VNFs on the ith SFC are collected; the network resource variance of a single physical node cannot measure the load balance of the whole network system, and the load balance of the whole network system can ensure good network performance. The mean value of the CPUs of the entire network system at time t
Mean value of storage load
Is represented as follows:
wherein N isPIs a set of underlying physical nodes, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m;
the CPU and storage resource variance of the entire network system is represented as:
the resource variance of the entire network system is expressed as:
further, in step S3, the complexity and high-dimensional property of the virtual network function migration decision problem is that the VNF mapping decision space is high-dimensional and complex due to the diversity of network traffic flows and the dynamic variability of the network traffic environment.
Further, in step S3, the deep reinforcement learning method for distributed near-end policy optimization (DPPO) has a structure jointly trained by a global network and a plurality of agent local networks, and the global network and the agent local networks have the same near-end policy optimization (PPO) network structure, and specifically includes the following steps:
s31: taking an SFC as an agent, and independently and synchronously carrying out PPO algorithm training on each agent in different threads to obtain different PPO network parameters; wherein PPO represents near-end policy optimization;
s32: each agent regularly pushes a loss function gradient to the global PPO network;
s33: when the gradient of each agent is accumulated to the updating frequency times of the global network by collecting the gradient of each agent, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
s34: pushing parameters obtained by the global network to each agent, enabling the agents to interactively update the gradient with the slice network system, pushing the parameters to the global network to update global network parameters, and repeating the steps until the DPPO algorithm is converged; where DPPO represents distributed near-end policy optimization.
Further, in step S3, the optimal decision of VNF migration is to find a VNF mapping policy that achieves approximately the minimum system energy consumption and network resource variance by using a DPPO algorithm.
The invention has the beneficial effects that: aiming at the problem of VNF migration delay caused by lack of effective resource prediction in the existing VNF migration problem, the invention provides a distributed Federal learning bidirectional gating unit resource prediction algorithm, the distributed prediction architecture relieves the problems of centralized prediction data training pressure and memory shortage, and an intelligent VNF migration algorithm based on distributed near-end strategy optimization is provided based on the resource demand prediction result, so that energy consumption optimization and load balance of a network system are realized.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.
Drawings
For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a VNF resource demand prediction flow diagram for a Federal bidirectional gated cycle Unit (FedBi-GRU) of the present invention;
fig. 2 is a flowchart of a VNF migration algorithm for distributed near-end policy optimization.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.
Referring to fig. 1-2, the physical layer of the network slice of the embodiment is defined as a full-connection undirected graph GP＝(NP,LP)，NPIs a set of underlying physical nodes, LPIs the set of underlying physical links. The invention considers two resource use conditions, namely computing resources and storage resources. Let CnRepresenting the computational resources owned by node n, MnRepresenting the storage resources owned by node N, where N, m ∈ Np，lnmRepresents a physical link between physical nodes n and m, and lnm∈LP。
A network slice layer is defined as a set of SFCs, denoted as F ═ Fi1,2, I, the ith SFC chain is abstracted as a directed graph
For all VNFs on the ith SFC,
is the set of all virtual links on the ith SFC. The computational resource requirement of the jth VNF on the ith SFC is
The storage resource needs are
Virtual link between jth VNF and kth VNF on SFC i
The bandwidth requirement is
Is provided with
A binary variable indicating whether VNF j in the ith SFC maps to physical node n,
meaning that VNF j maps onto physical node n,
denotes the jth on SFC iVirtual link between VNF and kth VNF
Whether or not to map to physical link/nmIn the above-mentioned manner,
to represent
Mapping to lnmIn the above-mentioned manner,
in this embodiment, when predicting VNF resource demand, each physical node acquires CPU, storage, and bandwidth resource characteristics of a VNF thereon, and for a VNF j on an SFC i, the network resource demand characteristics may be expressed as
Respectively, the CPU, storage, and bandwidth resource characteristics of VNF j.
Referring to fig. 1, fig. 1 is a flowchart of a VNF resource demand prediction algorithm of the present invention, including the following steps:
step 1): parameter aggregator SDN controller issuing initial parameter w0To each VNF on SFC i.
Step 2): the VNF on the SFC collects network resource usage history data, and for the VNF j on the SFC i, a series of data feature sample sets o ═ o of the VNF j can be obtained by collecting receipts1,o2,...,os,., inputting the feature sample set into a local Bi-GRU model of VNF j in combination with an initial parameter w0And (5) training.
Step 3): each VNF on SFC i will train the resulting parameter w1,w2,...,wjUploading to an SDN controller for federal averaging to obtain new global parameters
Step 4): SDN controller to be newGlobal parameters
And issuing all VNFs on the SFC i, and continuing Bi-GRU model training by the local VNFs until the training times are reached.
In the embodiment, the energy consumption of the physical node running state and the energy consumption of state switching are mainly considered when the energy consumption of the network system is calculated. The operation state energy consumption comprises the basic energy consumption of the operation of the physical node and the load energy consumption of the node, the load energy consumption is in direct proportion to the load of the CPU, and the operation energy consumption of the physical node at the moment t is represented as follows:
wherein Z isn(t) a binary variable indicating whether physical node n is on, where physical node n remains on as long as VNF is mapped to physical node n, and Z is at this timen(t) 1, otherwise the physical node enters the sleep state, Zn(t) is 0. Is provided with
For the fundamental energy consumption of the operation of the physical node n,
the energy consumption that the CPU resource is full is the physical node.
The CPU resource utilization rate of the physical node n at time t is expressed as:
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) may be expressed as follows:
the switching energy consumption generated by the change of the working state of the node n at the time t can be expressed as
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system can be expressed as:
load balancing model:
the closer the average network resource variance usage is, the stronger the network load balancing capability of the physical node is, assuming that at time t, the VNF j on the SFC i triggers the migration condition, then the VNF j may cause load change of the physical node m after being migrated from the physical node n to the physical node m, and the load of the physical node m at time t may be represented as:
wherein,
and
the loads of the CPU and the storage resource of the physical node m at the time t are respectively represented, the network resource variance of a single physical node cannot measure the load balance of the whole network system, and the load balance of the whole network system can ensure good network performance. Therefore, the average value of the CPU and the storage load of the whole network system at the time t is expressed as follows：
Wherein, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m, the CPU and storage resource variance of the whole network system can be represented as:
the resource variance of the entire network system can be expressed as:
referring to fig. 2, fig. 2 is a VNF migration algorithm for distributed near-end policy optimization according to this embodiment, and the steps are as follows:
step 1): taking an SFC as an agent, placing each agent in different threads to independently and synchronously train a PPO algorithm to obtain different PPO network parameters, wherein for an agent n, the PPO network updating mode is as follows:
the criticic network process of agent n is a loss function for minimizing TD error, and the loss function is expressed as follows:
the Actor network loss function of the n-local PPO network of the agent is as follows:
wherein s isn(T) represents the operating state of agent n at iteration step T, V(s)n(T +1)) means that the agent n is in the working state snFunction of state value of (T +1), an(t) represents the VNF mapping action taken by agent n at time t, Et[·]Represents the expectation function at time t, sigma represents the hyperparameter of the PPO network,
an Actor network parameter representing agent n;
for instant awards, gammav∈[0，1]For the discounting factor, T is the iteration step size in a round of training, A(s)n(t),an(t)) is a merit function;
for the ratio of old and new mapping strategies, σ is set to 0.2 for the over-parameter.
Step 2): loss function of agent n
Critic network gradient for updating global PPO network, global Critic network gradient delta thetacAnd a parameter thetacThe update mode is as follows:
θc＝θc+εcΔθc
wherein epsiloncIs the learning rate of the Critic network.
Gradient delta theta of global Actor networkaAnd a parameter thetaaThe update method (2) is as follows:
θa＝θa+εaΔθa
wherein epsilonaIs the learning rate of the Actor network.
Step 3): when the gradient is accumulated to the updating frequency times of the global network, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
step 4): and pushing the parameters obtained by the global network to each intelligent agent, wherein the intelligent agent interactively updates the gradient between the obtained parameters and the slice network system, pushes the parameters to the global network to update the global network parameters, and repeats the steps until the energy consumption and the load of the network system are balanced.
Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.
Claims (10)
1. A VNF migration method based on bidirectional GRU resource demand forecasting of federal learning is characterized by comprising the following steps:
s1: in a network slice scene, a VNF migration problem caused by time-varying network flow and a VNF migration delay problem caused by the fact that the resource demand of the VNF is lack of prediction are considered, and the resource demand of the VNF is predicted by adopting a FedBi-GRU algorithm; the FedBi-GRU represents a bidirectional gating cycle unit network based on federal learning, and the VNF represents a virtual network function;
s2: calculating the resource utilization rate of the physical node according to the resource demand prediction result obtained in the step S1, determining the physical node with overloaded resource use or underloaded resource use in the network system, and migrating through the VNF, thereby realizing system energy consumption optimization and load balancing while ensuring network performance;
s3: and obtaining the optimal decision of VNF migration by adopting a deep reinforcement learning method of distributed near-end strategy optimization.
2. The VNF migration method according to claim 1, wherein in step S1, the network slice scenario includes a three-tier architecture: the physical infrastructure layer provides physical resources of the system and consists of a plurality of physical nodes; the virtualization layer abstracts physical resources into virtual resources and provides resource support for the slice layer; the network slice layer is used for processing user network services and is processed by SFC formed by arranging a series of VNFs according to a set sequence; wherein SFC represents a service function chain;
the VNF migration problem is selecting a remapped physical node for a VNF;
the resource requirements of the VNF are CPU, storage and bandwidth resources required by the VNF at a future time.
3. The VNF migration method of claim 1, wherein in step S1, the FedBi-GRU algorithm performs local Bi-GRU resource demand prediction model training on each deployed physical node for VNFs on SFCs, and sends the prediction model parameters to each VNF to continue model training after performing aggregation averaging at an SDN controller, where Bi-GRU represents a bidirectional gating cycle unit, and SDN represents a software defined network; the method specifically comprises the following steps:
s11: issuing initial global parameters by the SDN controller;
s12: each VNF on the SFC receives the initial parameters to carry out local Bi-GRU resource demand prediction model training;
s13: uploading parameters of a local Bi-GRU resource demand prediction model by each VNF on the SFC;
s14: the SDN controller performs federal averaging on the collected parameters to obtain new global parameters;
s15: the VNF on the SFC receives the new global parameters and continues to train the Bi-GRU resource demand prediction model;
s16: and repeating the steps S11-S15 until the Bi-GRU resource demand forecasting models of all VNFs on the SFC converge.
4. The VNF migration method according to claim 1, wherein in step S2, the resource utilization of the physical node is a ratio of a sum of CPU resource requirements of all VNFs on the physical node to a CPU capacity of the physical node;
the resource utilization rate of the physical node with the resource use overload or the resource use underload is higher than the highest CPU resource use threshold value of the physical node or lower than the lowest CPU resource use threshold value;
the network performance is to satisfy the network service quality of the user.
5. The VNF migration method according to claim 1, wherein in step S2, the system energy consumption is optimized to minimize network system energy consumption after VNF migration, where the network system energy consumption includes an operation state energy consumption and a state switching energy consumption of an underlying physical node; the operation state energy consumption comprises basic energy consumption of physical nodes capable of operating and load energy consumption of the nodes, the load energy consumption is in direct proportion to the utilization rate of CPU resources, and the operation energy consumption of the physical nodes at the moment t is represented as:
wherein Z isn(t) a binary variable indicating whether physical node n is on, where physical node n remains on as long as VNF is mapped to physical node n, and Z is at this timen(t) 1, otherwise the physical node enters the sleep state, Zn(t) ═ 0; is provided with
For the fundamental energy consumption of the operation of the physical node n,
the energy consumption is the energy consumption that the CPU resource of the physical node is occupied;
the CPU resource utilization rate of the physical node n at time t is expressed as:
wherein,
indicating whether VNF j in the ith SFC at time instant tth maps to a binary variable of physical node n,
indicating that VNF j maps onto physical node n; f is a set of SFCs,
all VNFs on the ith SFC are collected; cnRepresenting the computing resources owned by node n;
let Sn(t) represents a binary variable of which the operating state of the physical node n changes at the time t, if SnWhen (t) '1' indicates that the operating state of the physical node n has changed, Sn(t) is represented as follows:
the switching energy consumption generated by the change of the working state of the node n at the moment t is expressed as
Therefore, when the VNF migrates from physical node n to physical node m, the energy consumption of the entire network system is expressed as:
wherein N isPIs a set of underlying physical nodes.
6. The VNF migration method according to claim 1, wherein in step S2, the load balancing is to minimize a network resource variance, where the network resource variance is a variance of utilization rates of all physical nodes CPU and storage resources of the network system;
assuming that at time t, VNF j on SFC i triggers a migration condition, after VNF j is migrated from physical node n to physical node m, load change of physical node m may be caused, and the load of physical node m at time t is represented as:
wherein,
and
respectively representing the CPU and storage resource loads of the physical node m at the moment t;
a binary variable indicating whether VNF j in the ith SFC maps to physical node n,
indicating that VNF j maps onto physical node n;
respectively representing the computing resource requirement and the storage resource requirement of the jth VNF on the ith SFC at time instant tth,
all VNFs on the ith SFC are collected; CPU mean value of the entire network system at time t
Mean value of storage load
Is represented as follows:
wherein N isPIs a set of underlying physical nodes, CmAnd MmRepresenting the CPU and storage resources owned by the physical node m;
the CPU and storage resource variance of the entire network system is represented as:
the resource variance of the entire network system is expressed as:
7. the VNF migration method according to claim 1, wherein in step S3, the distributed near-end policy optimization deep reinforcement learning method specifically includes the following steps:
s31: taking an SFC as an agent, and independently and synchronously carrying out PPO algorithm training on each agent in different threads to obtain different PPO network parameters; wherein PPO represents near-end policy optimization;
s32: each agent regularly pushes a loss function gradient to the global PPO network;
s33: when the gradient of each agent is accumulated to the updating frequency times of the global network by collecting the gradient of each agent, the global network updates the global network parameters by using a gradient descent mode, wherein the updating mode of the global network parameters is asynchronous updating;
s34: pushing parameters obtained by the global network to each agent, enabling the agents to interactively update the gradient with the slice network system, pushing the parameters to the global network to update global network parameters, and repeating the steps until the DPPO algorithm is converged; where DPPO represents distributed near-end policy optimization.
8. The VNF migration method of claim 1, wherein in step S3, the optimal decision for VNF migration is to find a VNF mapping policy that achieves approximately minimum system energy consumption and network resource variance by using a DPPO algorithm.
9. The VNF migration method according to claim 7, wherein in step S31, for agent n, the PPO network update method is as follows:
the loss function for agent n is:
wherein r isn(t) is the instant prize, γv∈[0，1]For the discounting factor, T is the iteration step size in a round of training,sn(T) represents the operating state of agent n at iteration step T, V(s)n(T +1)) means that the agent n is in the working state snA state value function of (T + 1);
the Actor network loss function of the n-local PPO network of the agent is as follows:
wherein,
for the ratio of old and new mapping strategies, A(s)n(t),an(t)) is the merit function, an(t) represents the VNF mapping action taken by agent n at time t, Et[·]Represents the expectation function at time t, sigma represents the hyperparameter of the PPO network,
representing the Actor network parameter of agent n.
10. The VNF migration method of claim 9, wherein in step S32, a loss function of agent n is used
Updating the Critic network gradient of the global PPO network, wherein the global Critic network gradient delta thetacAnd a parameter thetacThe updating method comprises the following steps:
θc＝θc+εcΔθc
wherein epsiloncLearning rate of Critic network;
gradient delta theta of global Actor networkaAnd a parameter thetaaIn an update manner of：
θa＝θaa+εaΔθa
Wherein epsilonaIs the learning rate of the Actor network.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919085.3A CN113490254B (en) | 2021-08-11 | 2021-08-11 | VNF migration method based on bidirectional GRU resource demand prediction in federal learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110919085.3A CN113490254B (en) | 2021-08-11 | 2021-08-11 | VNF migration method based on bidirectional GRU resource demand prediction in federal learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113490254A true CN113490254A (en) | 2021-10-08 |
| CN113490254B CN113490254B (en) | 2022-03-29 |
Family
ID=77946234
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110919085.3A Active CN113490254B (en) | 2021-08-11 | 2021-08-11 | VNF migration method based on bidirectional GRU resource demand prediction in federal learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113490254B (en) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114065951A (en) * | 2021-11-17 | 2022-02-18 | 黑龙江大学 | Semi-supervised federal learning method based on non-IID data |
| CN114374605A (en) * | 2022-01-12 | 2022-04-19 | 重庆邮电大学 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN114401192A (en) * | 2021-12-20 | 2022-04-26 | 广西壮族自治区公众信息产业有限公司 | Multi-SDN controller collaborative training method |
| CN114726743A (en) * | 2022-03-04 | 2022-07-08 | 重庆邮电大学 | Service function chain deployment method based on federal reinforcement learning |
| CN114827284A (en) * | 2022-04-21 | 2022-07-29 | 中国电子技术标准化研究院 | Service function chain arrangement method and device in industrial Internet of things and federal learning system |
| CN114943345A (en) * | 2022-06-10 | 2022-08-26 | 西安电子科技大学 | Federal learning global model training method based on active learning and model compression |
| CN115130683A (en) * | 2022-07-18 | 2022-09-30 | 山东大学 | Asynchronous federal learning method and system based on multi-agent model |
| CN115361288A (en) * | 2022-08-19 | 2022-11-18 | 南京工业大学 | Dynamic reconfiguration method for service function chain in air-space-ground integrated scene |
| CN116137593A (en) * | 2023-02-20 | 2023-05-19 | 重庆邮电大学 | Virtual network function migration method for digital twin auxiliary dynamic resource demand prediction |
| CN117540215A (en) * | 2024-01-05 | 2024-02-09 | 中国民航大学 | Flight delay prediction model training method, electronic equipment and storage medium |
| WO2024118108A1 (en) * | 2022-11-29 | 2024-06-06 | Rakuten Symphony India Private Limited | System and method for optimizing the scheduling of o-cloud nodes in a telecommunications network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190140913A1 (en) * | 2018-12-28 | 2019-05-09 | Intel Corporation | Techniques for artificial intelligence capabilities at a network switch |
| CN110505099A (en) * | 2019-08-28 | 2019-11-26 | 重庆邮电大学 | A kind of service function chain dispositions method based on migration A-C study |
| CN111130904A (en) * | 2019-12-30 | 2020-05-08 | 重庆邮电大学 | Virtual network function migration optimization algorithm based on deep certainty strategy gradient |
| CN111314235A (en) * | 2020-02-19 | 2020-06-19 | 广东技术师范大学 | Network delay optimization method based on virtual network function resource demand prediction |
| CN111526070A (en) * | 2020-04-29 | 2020-08-11 | 重庆邮电大学 | Service function chain fault detection method based on prediction |
| CN112631717A (en) * | 2020-12-21 | 2021-04-09 | 重庆大学 | Network service function chain dynamic deployment system and method based on asynchronous reinforcement learning |
-
2021
- 2021-08-11 CN CN202110919085.3A patent/CN113490254B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190140913A1 (en) * | 2018-12-28 | 2019-05-09 | Intel Corporation | Techniques for artificial intelligence capabilities at a network switch |
| CN110505099A (en) * | 2019-08-28 | 2019-11-26 | 重庆邮电大学 | A kind of service function chain dispositions method based on migration A-C study |
| CN111130904A (en) * | 2019-12-30 | 2020-05-08 | 重庆邮电大学 | Virtual network function migration optimization algorithm based on deep certainty strategy gradient |
| CN111314235A (en) * | 2020-02-19 | 2020-06-19 | 广东技术师范大学 | Network delay optimization method based on virtual network function resource demand prediction |
| CN111526070A (en) * | 2020-04-29 | 2020-08-11 | 重庆邮电大学 | Service function chain fault detection method based on prediction |
| CN112631717A (en) * | 2020-12-21 | 2021-04-09 | 重庆大学 | Network service function chain dynamic deployment system and method based on asynchronous reinforcement learning |
Non-Patent Citations (2)
| Title |
|---|
| SUMAN PANDEY等: "GRU and EdgeQ-Learning based Traffic Prediction and Scaling of SFC", 《 2021 IEEE 7TH INTERNATIONAL CONFERENCE ON NETWORK SOFTWARIZATION (NETSOFT)》 * |
| 唐伦等: "基于深度确定性策略梯度的虚拟网络功能迁移优化算法", 《电子与信息学报》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114065951A (en) * | 2021-11-17 | 2022-02-18 | 黑龙江大学 | Semi-supervised federal learning method based on non-IID data |
| CN114401192A (en) * | 2021-12-20 | 2022-04-26 | 广西壮族自治区公众信息产业有限公司 | Multi-SDN controller collaborative training method |
| CN114401192B (en) * | 2021-12-20 | 2024-02-06 | 广西壮族自治区公众信息产业有限公司 | Multi-SDN controller cooperative training method |
| CN114374605B (en) * | 2022-01-12 | 2024-01-05 | 西安盈科思泰网络技术有限公司 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN114374605A (en) * | 2022-01-12 | 2022-04-19 | 重庆邮电大学 | Dynamic adjustment and migration method for service function chain in network slice scene |
| CN114726743A (en) * | 2022-03-04 | 2022-07-08 | 重庆邮电大学 | Service function chain deployment method based on federal reinforcement learning |
| CN114827284A (en) * | 2022-04-21 | 2022-07-29 | 中国电子技术标准化研究院 | Service function chain arrangement method and device in industrial Internet of things and federal learning system |
| CN114827284B (en) * | 2022-04-21 | 2023-10-03 | 中国电子技术标准化研究院 | Service function chain arrangement method and device in industrial Internet of things and federal learning system |
| CN114943345A (en) * | 2022-06-10 | 2022-08-26 | 西安电子科技大学 | Federal learning global model training method based on active learning and model compression |
| CN114943345B (en) * | 2022-06-10 | 2024-06-11 | 西安电子科技大学 | Active learning and model compression-based federal learning global model training method |
| CN115130683A (en) * | 2022-07-18 | 2022-09-30 | 山东大学 | Asynchronous federal learning method and system based on multi-agent model |
| CN115361288A (en) * | 2022-08-19 | 2022-11-18 | 南京工业大学 | Dynamic reconfiguration method for service function chain in air-space-ground integrated scene |
| CN115361288B (en) * | 2022-08-19 | 2023-06-23 | 南京工业大学 | Service function chain dynamic reconstruction method under space-air-ground integrated scene |
| WO2024118108A1 (en) * | 2022-11-29 | 2024-06-06 | Rakuten Symphony India Private Limited | System and method for optimizing the scheduling of o-cloud nodes in a telecommunications network |
| CN116137593A (en) * | 2023-02-20 | 2023-05-19 | 重庆邮电大学 | Virtual network function migration method for digital twin auxiliary dynamic resource demand prediction |
| CN117540215A (en) * | 2024-01-05 | 2024-02-09 | 中国民航大学 | Flight delay prediction model training method, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113490254B (en) | 2022-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113490254B (en) | VNF migration method based on bidirectional GRU resource demand prediction in federal learning | |
| CN108260169B (en) | QoS guarantee-based dynamic service function chain deployment method | |
| CN111953758B (en) | Edge network computing unloading and task migration method and device | |
| CN106411770B (en) | A kind of data center network energy-saving routing algorithm based on SDN framework | |
| CN111130904B (en) | Virtual network function migration optimization algorithm based on deep certainty strategy gradient | |
| CN105610715B (en) | A kind of cloud data center multi-dummy machine migration scheduling method of planning based on SDN | |
| CN108965014B (en) | QoS-aware service chain backup method and system | |
| CN108900358A (en) | Virtual network function dynamic migration method based on deepness belief network resource requirement prediction | |
| CN112087509B (en) | Task migration method in edge computing platform | |
| WO2023039965A1 (en) | Cloud-edge computing network computational resource balancing and scheduling method for traffic grooming, and system | |
| CN111445111A (en) | Electric power Internet of things task allocation method based on edge cooperation | |
| CN107748693A (en) | Group's virtual machine scheduling policy under cloud computing environment | |
| WO2020134133A1 (en) | Resource allocation method, substation, and computer-readable storage medium | |
| CN111836284B (en) | Energy consumption optimization calculation and unloading method and system based on mobile edge calculation | |
| CN114374605A (en) | Dynamic adjustment and migration method for service function chain in network slice scene | |
| CN112162789A (en) | Edge calculation random unloading decision method and system based on software definition | |
| Bouzidi et al. | Dynamic clustering of software defined network switches and controller placement using deep reinforcement learning | |
| CN106656555A (en) | Dynamic adjustment method of service resources of cloud computing system | |
| CN110597598B (en) | Control method for virtual machine migration in cloud environment | |
| CN111510319B (en) | Network slice resource management method based on state perception | |
| CN109976879B (en) | Cloud computing virtual machine placement method based on resource usage curve complementation | |
| CN105704180B (en) | The configuration method and its system of data center network | |
| CN104298539B (en) | Scheduling virtual machine and dispatching method again based on network aware | |
| CN108810109A (en) | A kind of virtual data center laying method based on energy optimization | |
| CN105577834B (en) | Two layers of bandwidth allocation methods of cloud data center with Predicable performance and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |