CN110275758A - A kind of virtual network function intelligence moving method - Google Patents
A kind of virtual network function intelligence moving method Download PDFInfo
- Publication number
- CN110275758A CN110275758A CN201910382250.9A CN201910382250A CN110275758A CN 110275758 A CN110275758 A CN 110275758A CN 201910382250 A CN201910382250 A CN 201910382250A CN 110275758 A CN110275758 A CN 110275758A
- Authority
- CN
- China
- Prior art keywords
- vnf
- network
- node
- migration
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The present invention relates to a kind of virtual network function intelligence moving methods, belong to mobile communication technology field.In the method, on each discrete time slots, under the premise of guaranteeing that highest average delay constraint, the limitation of nodal cache resource consumption and the link bandwidth capacity of each slice limit, according to each virtual network function (Virtualized Network Function, VNF) queue state information of example, node status information and link-state information, operation energy consumption is averaged as target to minimize generic server, is formulated optimal VNF migration strategy for slice and is dynamically adjusted the allocation strategy of network node cpu resource.This method can not only make full use of cpu resource, but also can satisfy the requirement for being sliced average end-to-end time delay；The shared of VNF example is realized, the stability of system is effectively kept while saving system average energy consumption.
Description
Technical field
The invention belongs to mobile communication technology fields, are related to a kind of virtual network function (Virtualized Network
Function, VNF) intelligence moving method.
Background technique
5G network slice and NFV technology be virtual resource centralized dispatching and VNF flexible layout bring it is huge
Advantage.Under NFV framework, conventional network elements are abstracted in the form of software and are independent functional module, and is deployed in logical
With on server platform；Meanwhile by the network resource integration of various dimensions in unified resource pool, according to user to different slices
Demand, the resource in resource pool is dynamically dispatched and is distributed according to need.This mode makes SFC in infrastructure
Mobile management becomes possibility, with the variation of link and node state in network load and infrastructure, dynamically adjusts
The deployment way of SFC, i.e., migrate VNF and carry out resource and reconfigure, and can improve while guaranteeing network service performance
The resource utilization of generic server in infrastructure.Meanwhile the flexible layout of VNF also provides advantageous item for the saving of energy consumption
Part migrates the VNF on the lower server of resource utilization under the mechanism that VNF shares, and closes corresponding service
Device is to achieve the purpose that reduce energy consumption.
Although the migration of VNF helps to optimize the operation energy consumption of network and the utilization rate of resource, actual
In application scenarios, the selection on migration opportunity, VNF to be migrated and migration target is improper, is all likely to result in the increasing of migration overhead
Greatly, the problems such as QoS declines.Therefore, it is sliced scene for the 5G network of business demand dynamic change, how dynamic is carried out to VNF
Migration, while guaranteeing each slice service feature, takes into account energy optimization brought by migration operation, is in 5G network slice
One of resource management and scheduling mechanism critical issue to be solved.
In the pertinent literature of research VNF migration at present, most of work, which rests under fixed network environment, instantaneously to be referred to
On the problem of mark optimization, namely single time slot optimization, and for the Dynamic Slicing business scenario for having certain life cycle, research benefit
The work for migrating the average behavior index under long time scale with online resource management techniques optimization VNF is few.In addition, needle
To VNF example share state under, i.e., single VNF example can simultaneously by multiple slice traffic schedulings when, if some chip property
It is unable to satisfy business demand, current research method can not formulate the migration that effective strategy realizes service function chain VNF.
Summary of the invention
In view of this, this method can the purpose of the present invention is to provide a kind of virtual network function intelligence moving method
While effectively meeting each slice QoS demand and keeping system queue stability, the average energy consumption of infrastructure is reduced.
In order to achieve the above objectives, the invention provides the following technical scheme:
A kind of virtual network function intelligence moving method on each discrete time slots, is guaranteeing each to cut in the method
Under the premise of highest average delay constraint, the limitation of nodal cache resource consumption and the link bandwidth capacity of piece limit, according to each
The queue state information of a virtual network function (Virtualized Network Function, VNF) example, node state letter
Breath and link-state information are averaged operation energy consumption as target to minimize generic server, formulate optimal VNF for slice and move
It moves strategy and dynamically adjusts the allocation strategy of network node cpu resource；Specifically includes the following steps:
S1: in the case where 5G network is sliced scene, for the feature of service traffics dynamic change, the more queues of VNF example are established
Time Delay Model and system energy consumption model, wherein when the VNF scheduling overall delay of network slice i is by processing delay and link transmission
Prolong two parts composition, and the allocation strategy of cpu resource influences the processing delay of VNF；The energy consumption model mainly includes two parts
Content: the basic energy consumption generated when node is in the open state and the operation energy consumption generated with the variation of VNF example load；
S2: the migration of VNF and the distribution of cpu resource are created as based on limited Markovian decision process
The Stochastic Optimization Model of (Constrained Markov Decision Process, CMDP), the model is to minimize general clothes
Business device is averaged operation energy consumption as target, while being limited to each slice average delay constraint and average cache, bandwidth resource consumption
Constraint；
S3: unrestricted Markovian decision process MDP problem is converted for CMDP problem by Lagrangian theory, i.e., will
Optimization aim is converted into the optimal Q function of searching from optimal policy is found；
S4: establishing the VNF based on intensified learning frame and migrate on-line study method intelligently come approximate behavior value function, thus
Seek optimal VNF migration strategy according to current system conditions in each discrete time slot for each network slice and CPU is provided
Source allocation plan.
Further, in step s 2, the Stochastic Optimization Model based on CMDP is expressed as minimizing expectation accumulation discount
Return:
Wherein: Ψ (t) is the movement vector of the VNF migration in time slot t, and Z (t) is the cpu resource of each VNF in time slot t
Set of actions is distributed, r (t) is state vector of the system in time slot t,It is averaged operation energy consumption for generic server；
The slice delay constraint are as follows: all slice end-to-end time delay require to meet
WhereinFor the average end-to-end time delay for being sliced i, τiTo be sliced SiEnd-to-end time delay constraint；
Cache resources consumption constraint are as follows: all newly arrived data volumes of nodal cache should meet
WhereinThe consumption of discount cache resources, χ are accumulated for expectationhThe cache resources total amount rented for node h；
The bandwidth resource consumption constraint are as follows: the data volume of all link transmissions should meet
WhereinDiscount bandwidth resource consumption, Δ are accumulated for expectationh,lHold for the link bandwidth of node h to node l
Amount.
In each discrete time slots migration VNF and virtual resource is dynamically distributed, needs to guarantee the team of all VNF examples in system
Column will not propulsion at any time and continue to increase, and then be sliced end-to-end time delay and persistently increase；In system in all nodal caches
Data volume, the data volume of link transmission will not propulsion at any time and continue to increase.
Further, in step s3, optimization aim is converted into the optimal policy π of acquisition state r*,β, and meetThe wherein optimal policy π*,βDistribution side including VNF migration strategy and cpu resource
Case, the Q*,β(r a) is optimal action value function, and a is movement vector, and the r is system mode vector.
Further, in step s 4, depth Q study (DQN) training function f is utilizedapCome the distribution of approximate Q value, this method
Using state r as input, then corresponding after neural network analysis to export each Q value acted, main process is in Q
Increase a target Q network (fixedQ-targets) on the basis of network to calculate target Q value, the two network structures are identical
But parameter is different, optimizes weight w, Lai Shixian neural network forecast by minimizing the loss function between Q network and target Q network
The promotion of performance.The update mode of the network are as follows: target Q network just will be updated once whithin a period of time, and Q network is every
It can all be updated after secondary iterative process.
Further, the approximate specific steps of cost function are as follows:
1) experience replay pond, Q network, target Q network, Lagrange multiplier are initialized；
2) using ε-greedy strategy generating action at, that is, a Probability p is randomly choosed, if p >=ε, calculates VNF migration
And cpu resource allocation strategyOtherwise a random action is selected
3) action a is executedt, Lagrange return is obtained, and observe subsequent time state, experience sample deposit experience is returned
Put pond；
4) one group of experience sample is randomly selected from experience replay pond, using target Q network query function target Q value, and is utilized
Gradient descent method is updated weight；
5) every time span TqMore fresh target Q network；
6) Lagrange multiplier described in random subgradient algorithm online updating is utilized；
7) after iteration for several times, judge whether to meet the condition of convergence；
8) if presently described VNF migration and cpu resource allocation strategy meet the condition of convergence, the VNF is migrated
It notifies with cpu resource allocation strategy to the virtual network scheduler and resource management entity；
9) according to each VNF migration and cpu resource allocation strategy, each node VNF example is according to the queue
Update mode updates the buffer queue size, and waits next time slot scheduling.
Further, in each time slot scheduling, the quene state of each node VNF example, the node of system are given
State and link state calculate the optimal VNF migration and Resource Allocation Formula, specific steps are as follows:
1) global state under current time slots t, the input by current state r (t) as Q network are monitored；
If 2) node or link fail in time slot t, in order to improve the reliability of network, system to corresponding VNF into
On the basis of row migration, optimal VNF migration strategy is selectedEnergy consumption and time delay are optimized；
3) optimal migration strategy is otherwise directly selected
4) next time slot scheduling t+1 is waited.
The beneficial effects of the present invention are: the present invention is in each discrete time slot scheduling according to current system mode pair
VNF is migrated and is adjusted the method for salary distribution of cpu resource, and cpu resource can be not only made full use of, but also can satisfy the average end of slice
To the requirement of terminal delay time.In addition, the VNF deployment process simulation of different types SFC is realized VNF at M/M/1 queuing process
Example is shared, and the Stochastic Optimization Model established based on CMDP theory is averaged operation energy consumption as mesh to minimize generic server
Mark, effectively keeps the stability of system while saving system average energy consumption.
Other advantages, target and feature of the invention will be illustrated in the following description to a certain extent, and
And to a certain extent, based on will be apparent to those skilled in the art to investigating hereafter, Huo Zheke
To be instructed from the practice of the present invention.Target of the invention and other advantages can be realized by following specification and
It obtains.
Detailed description of the invention
To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention is made below in conjunction with attached drawing excellent
The detailed description of choosing, in which:
Fig. 1 is the system scenarios schematic diagram in the present invention；
Fig. 2 is that the virtual network function in the present invention migrates flow chart；
Fig. 3 is that the virtual network function in the present invention based on DQN intelligently migrates on-line study architecture diagram；
Fig. 4 is that the virtual network function migration strategy in the present invention selects flow chart.
Specific embodiment
In a kind of virtual network function intelligence moving method provided by the invention on each discrete time slots, guarantee it is each
Under the premise of highest average delay constraint, the limitation of nodal cache resource consumption and the link bandwidth capacity of slice limit, according to
Queue state information, node status information and the link-state information of each VNF example, it is average to minimize generic server
Operation energy consumption is target, formulates optimal VNF migration strategy for slice and dynamically adjusts the distribution of network node cpu resource
Strategy；
Specifically, the VNF intelligence moving method the following steps are included:
S1: in the case where 5G network is sliced scene, for the feature of service traffics dynamic change, the more queues of VNF example are established
Time Delay Model and system energy consumption model；
S2: the migration of VNF and the distribution of cpu resource are created as based on limited Markovian decision process
The Stochastic Optimization Model of (Constrained Markov Decision Process, CMDP), the model is to minimize general clothes
Business device is averaged operation energy consumption as target, while being limited to each slice average delay constraint and average cache, bandwidth resource consumption
Constraint；
S3: converting CMDP problem to unrestricted MDP problem by Lagrangian theoretical, i.e., by optimization aim from finding
Optimal policy, which is converted into, finds optimal Q function；
S4: establish a kind of VNF based on intensified learning frame intelligently migration on-line study method come approximate behavior value function,
To in each discrete time slot according to current system conditions be each network slice seek optimal VNF migration strategy and
Cpu resource allocation plan.
The Time Delay Model of the more queues of VNF example:
The VNF scheduling overall delay of the network slice i is made of processing delay and chain circuit transmission time delay two parts, and wherein CPU is provided
The allocation strategy in source influences the processing delay of VNF, according to functional expression:
Data flow is calculated in VNFfijGeneration processing delay, wherein describedFor the VNF example of node hData packet reach
The mean value of process, the Qh,jIndicate the VNF example of node hCurrent queue length size, it is describedFor data package size
Mean value, it is describedForService speed, B (h, the fij) indicate VNFfijWhether ∈ F is deployed in node h.
According to functional expression:Data flow is calculated to exist
The chain circuit transmission time delay generated in transmission process, wherein the σijI, which is sliced, for network dispatches VNFfjThe expectation average time of ∈ F
σij, the P (fp|fj, i) and indicate that user requests slice SiData flow in VNFfjVNFf is transmitted to after processedpRatio, institute
State the data transmission delay between two nodes of δ (h, l) expression, B (h, the fij) indicate VNFfijWhether ∈ F is deployed in node
h。
VNF Example queues information specifically:
According to functional expression:Each VNF example is calculated in each time slot scheduling
The buffer queue size of upper update, wherein the Qh,j(t+1) the VNF example for being node hStart in next time slot scheduling
When queue length size, the Qh,j(t) the VNF example for being node hQueue length when current scheduling time slot starts
Size, it is describedFor the VNF example of node hWhen current scheduling time slot starts, newly arrived total amount of data, describedFor the VNF example of node hService speed when current scheduling time slot starts.
The energy consumption model are as follows:
The energy consumption model of optimization mainly includes two parts content: the basic energy consumption that is generated when node is in the open state and with
VNF example load variation generate operation energy consumption.It can specifically be expressed as
It is wherein describedIt indicates to be in the state opened in moment t node h, is otherwise 0, the uh∈ (0,1) is section
Point h constant power dissipation percentage, the PhIndicate the maximum energy consumption of the occupied generation of cpu resource of node h, the ρhFor load
Density.
The Stochastic Optimization Model based on CMDP can indicate are as follows:
It is expected that accumulating discount returnIt can indicate are as follows:
Wherein the Ψ (t) is the movement vector of the VNF migration in time slot t, and the Z (t) is each VNF in time slot t
Cpu resource distributes set of actions, and the r (t) is state vector of the system in time slot t, and the γ is discount factor.
The slice delay constraint are as follows: all slice end-to-end time delay require to meet
Described in whereinFor the average end-to-end time delay for being sliced i, τiTo be sliced SiEnd-to-end time delay constraint；
The cache resources consume constraint are as follows: all newly arrived data volumes of nodal cache should meet
It is wherein describedThe consumption of discount cache resources, the χ are accumulated for expectationhThe cache resources rented for node h are total
Amount；
The bandwidth resource consumption constraint are as follows: the data volume of all link transmissions should meet
It is wherein describedDiscount bandwidth resource consumption, the Δ are accumulated for expectationh,lFor node h to the link of node l
Bandwidth capacity；
The optimal action value function can indicate are as follows:
Therefore optimization aim is converted into the optimal policy π of acquisition state r*,β, and meet
The wherein optimal movement π*,βAllocation plan including VNF migration strategy and cpu resource,
The wherein Q*,β(r a) is optimal action value function, the gβ(r is a) Lagrange return, describedFor state transition probability, the V*,βIt (r') is optimum state value function, the β is Lagrange multiplier, a
To act vector, the r is system mode vector, and the γ is discount factor, and the r' indicates the state of next time slot.
The approximate specific steps of the cost function based on intensified learning are as follows:
Initialization: experience replay pond, Q network, target Q network, Lagrange multiplier；
Use ε-greedy strategy generating action at；
Execute action at, Lagrange return is obtained, and observe subsequent time state, experience sample is stored in experience replay
Pond；
One group of experience sample is randomly selected from experience replay pond, using target Q network query function target Q value, and utilizes ladder
Degree descent method is updated weight；
Every time span TqMore fresh target Q network；
Utilize Lagrange multiplier described in random subgradient algorithm online updating；
After iteration for several times, judge whether to meet the condition of convergence；
If presently described VNF migration and cpu resource allocation strategy meet the condition of convergence, by VNF migration and
Cpu resource allocation strategy is notified to the virtual network scheduler and resource management entity；
According to each VNF migration and cpu resource allocation strategy, each node VNF example according to the queue more
New paragon updates the buffer queue size, and waits next time slot scheduling；
The update of the weight w, specifically:
Using the gradient of loss function and optimization is declined based on gradient, weight can be trained, renewal process
It can be indicated with functional expression are as follows:
It is wherein describedIndicate the gradient of loss function, the α is learning rate, and the G is loss function, and the W is
Weight.
It is described to calculate optimal VNF migration and Resource Allocation Formula, specific steps are as follows:
Monitor the global state under current time slots t, the input by current state r (t) as Q network；
If node or link fail in time slot t, in order to improve the reliability of network, system is carried out to corresponding VNF
On the basis of migration, optimal VNF migration strategy is selectedEnergy consumption and time delay are optimized；
Otherwise optimal migration strategy is directly selectedAnd wait next time slot scheduling.
Specific embodiments of the present invention are described in detail with reference to the accompanying drawing.
Referring to Fig. 1, Fig. 1 is the system scenarios schematic diagram in the present invention, and it includes three layers, it is each that application layer, which is mainly responsible for,
Slice request provides orderly VNF set to handle the data flow of arrival, and Cloud Server is provided comprising calculating in infrastructure layer
A plurality of types of virtual network resources such as resource, cache resources, bandwidth resources, virtualization layer is according to the business of virtual network user
State, QoS demand etc. realize the flexible allocation of the dynamic migration of VNF, virtual network resource.Wherein based on the performance evaluation of DQN
The assessment result of module is unable to satisfy the QoS demand of each slice if current VNF deployment way, and virtual network scheduler is then
The migration strategy of VNF is determined according to current queue state and bottom layer node state to achieve the purpose that performance optimizes；Resource management
Entity is the optimal virtual resource amount of each virtual network function module assignment after SFC completes deployment；Node link state prison
The effect for surveying entity is the real-time status of each node of observation and link.
Referring to fig. 2, Fig. 2 is that the virtual network function in the present invention migrates flow chart, and steps are as follows:
Step 201: establishing full connecting-type virtual network topology, generate different types of network slice and realize that this is cut
The service function chain of piece business forms；
Step 202: each parameter of initialized setting system, including Q network, target Q network, time factor TmaxDeng especially
Ground enables the time indicate t=0, and the queue of all VNF example cachings is 0 in system；
Step 203: collecting overall situation VNF Example queues status information, network node, link-state information；
Step 204: using global state information as the input of DQN network, according to optimal action value function formula:
Calculate optimal policy π*,β, and meetThe wherein optimal movement π*,βPacket
The allocation plan of VNF migration strategy and cpu resource is included,
The wherein Q*,β(r a) is optimal action value function, the gβ(r is a) Lagrange return, describedFor state transition probability, the V*,βIt (r') is optimum state value function, the β is Lagrange multiplier, a
To act vector, the r is system mode vector, and the γ is discount factor, and the r' indicates the state of next time slot；
Step 205: judging whether loss function exceeds thresholding, i.e., whether meet the condition of convergence, if not satisfied, then continuing to jump
Step 206 is gone to, it is no to then follow the steps 207；
Step 206: training DQN network utilizes formula:
The gradient of loss function and optimization is declined based on gradient, weight is trained, wherein describedIt indicates
The gradient of loss function, α are learning rate, and the G is loss function, and the W is weight；
Step 207: utilizing the value of Lagrange multiplier described in random subgradient algorithm online updating；
Step 208: judging whether the value of Lagrange multiplier is less than predetermined with the difference of the value of preceding an iteration after updating
Threshold value, if not satisfied, then continue to jump to step 203, it is no to then follow the steps 209；
Step 209: VNF migration and cpu resource Decision of Allocation are notified to virtual network scheduler and resource management entity,
The migration of VNF is executed based on optimal action, and carries out the distribution of cpu resource, and according to functional expression:
The buffer queue size that each VNF example updates in each time slot scheduling is calculated,
The wherein Qh,j(t+1) the VNF example for being node hQueue length when next time slot scheduling starts is big
It is small, the Qh,j(t) the VNF example for being node hQueue length size when current scheduling time slot starts, it is described
For the VNF example of node hWhen current scheduling time slot starts, newly arrived total amount of data, describedFor node h's
VNF exampleService speed when current scheduling time slot starts；
Step 210: judging whether time slot scheduling reaches maximum slot values Tmax, if not satisfied, step 203 is then jumped to,
Otherwise algorithm terminates.
Referring to Fig. 3, Fig. 3 intelligently migrates on-line study architecture diagram for the virtual network function based on DQN in the present invention, due to
In intensified learning, what obtained observation data were ordered into, in order to guarantee the independence of data, DQN network application experience replay
The data sample of each iteration is stored in pond (experience replay), and is taken out from experience replay pond using random manner
Update of a part of data for network parameter is taken, the association between data is broken with this.Meanwhile DQN is on the basis of Q network
A target Q network (fixed Q-targets) is increased to calculate target Q value, the two network structures are identical but parameter not
Together, in order to improve the performance of neural network forecast, weighting function needs study repeatedly and training to be fitted complex environment feature, i.e.,
By minimizing the loss function between Q network and target Q network optimize weight w, thus in each time slot according to currently
Global state formulates optimal VNF migration and cpu resource allocation plan.
Referring to fig. 4, Fig. 4 is that the virtual network function migration strategy in the present invention selects flow chart, and steps are as follows:
Step 401: each parameter of initialized setting system, including Q network, target Q network, time factor TmaxDeng especially
Ground enables the time indicate t=0, and the queue of all VNF example cachings is 0 in system；
Step 402: collect overall situation VNF Example queues status information, network node, link-state information, and by global shape
Input of the state information as DQN network；
Step 403: judge whether node or link fail, if so, continue to jump to step 404, it is no to then follow the steps
405；
Step 404: using be deployed in the node or by the link transmission data stream VNF as VNF to be migrated；
Step 405: pass through formula:
Optimal VNF migration strategy and cpu resource allocation strategy are calculated,
The wherein rtFor the system mode vector of time slot t, the w is the weight of DQN network；
Step 406: executing optimal action and update queue；
Step 407: judging whether time slot scheduling reaches maximum slot values Tmax, if not satisfied, step 402 is then jumped to,
Otherwise algorithm terminates.
Finally, it is stated that the above examples are only used to illustrate the technical scheme of the present invention and are not limiting, although referring to compared with
Good embodiment describes the invention in detail, those skilled in the art should understand that, it can be to skill of the invention
Art scheme is modified or replaced equivalently, and without departing from the objective and range of the technical program, should all be covered in the present invention
Scope of the claims in.
Claims (6)
1. a kind of virtual network function intelligence moving method, it is characterised in that: in the method, on each discrete time slots,
Guarantee the highest average delay constraint of each slice, the premise of the limitation of nodal cache resource consumption and link bandwidth capacity limitation
Under, according to the queue state information of each virtual network function (Virtualized Network Function, VNF) example,
Node status information and link-state information are averaged operation energy consumption as target to minimize generic server, formulate for slice
Optimal VNF migration strategy and the allocation strategy for dynamically adjusting network node cpu resource；Specifically includes the following steps:
S1: establishing the Time Delay Model and system energy consumption model of the more queues of VNF example, wherein the VNF scheduling of network slice i is total
Time delay is made of processing delay and chain circuit transmission time delay two parts, and the allocation strategy of cpu resource influences the processing delay of VNF；
The energy consumption model mainly includes two parts content: the basic energy consumption that generates when node is in the open state and with VNF example
The operation energy consumption that load variation generates；
S2: the migration of VNF and the distribution of cpu resource are created as based on limited Markovian decision process (Constrained
Markov Decision Process, CMDP) Stochastic Optimization Model, which is averagely run with minimizing generic server
Energy consumption is target, while being limited to each slice average delay constraint and average cache, bandwidth resource consumption constraint；
S3: converting unrestricted Markovian decision process MDP problem for CMDP problem by Lagrangian theory, i.e., will optimization
Target is converted into the optimal Q function of searching from optimal policy is found；
S4: establishing the VNF based on intensified learning frame, intelligently migration on-line study method is come approximate behavior value function, thus every
Seek optimal VNF migration strategy and cpu resource point according to current system conditions in a discrete time slot for each network slice
With scheme.
2. a kind of virtual network function intelligence moving method according to claim 1, it is characterised in that: in step s 2,
The Stochastic Optimization Model based on CMDP is expressed as minimizing expectation accumulation discount return:
Wherein: Ψ (t) is the movement vector of the VNF migration in time slot t, the cpu resource distribution that Z (t) is each VNF in time slot t
Set of actions, r (t) are state vector of the system in time slot t,It is averaged operation energy consumption for generic server；
The slice delay constraint are as follows: all slice end-to-end time delay require to meet
WhereinFor the average end-to-end time delay for being sliced i, τiTo be sliced SiEnd-to-end time delay constraint；
Cache resources consumption constraint are as follows: all newly arrived data volumes of nodal cache should meet
WhereinThe consumption of discount cache resources, χ are accumulated for expectationhThe cache resources total amount rented for node h；
The bandwidth resource consumption constraint are as follows: the data volume of all link transmissions should meet
WhereinDiscount bandwidth resource consumption, Δ are accumulated for expectationh,lFor node h to the link bandwidth capacity of node l.
3. a kind of virtual network function intelligence moving method according to claim 2, it is characterised in that: in step s3,
Optimization aim is converted into the optimal policy π of acquisition state r*,β, and meetIt is wherein described
Optimal policy π*,βAllocation plan including VNF migration strategy and cpu resource, the Q*,β(r a) is optimal action value function, institute
A is stated as movement vector, the r is system mode vector.
4. a kind of virtual network function intelligence moving method according to claim 3, it is characterised in that: in step s 4,
Utilize depth Q learning training function fapCarry out the distribution of approximate Q value, then this method passes through neural network using state r as input
The Q value of corresponding each movement of output after analysis, main process are to increase a target Q network on the basis of Q network to come
Target Q value is calculated, the two network structures are identical but parameter is different, by minimizing the loss between Q network and target Q network
Function optimizes weight w, the promotion of Lai Shixian neural network forecast performance.
5. a kind of virtual network function intelligence moving method according to claim 4, it is characterised in that: cost function is approximate
Specific steps are as follows:
1) experience replay pond, Q network, target Q network, Lagrange multiplier are initialized；
2) using ε-greedy strategy generating action at, that is, a Probability p is randomly choosed, if p >=ε, calculates VNF migration and CPU
Resource allocation policyOtherwise a random action is selected
3) action a is executedt, Lagrange return is obtained, and observe subsequent time state, experience sample is stored in experience replay pond；
4) one group of experience sample is randomly selected from experience replay pond, using target Q network query function target Q value, and utilizes gradient
Descent method is updated weight；
5) every time span TqMore fresh target Q network；
6) Lagrange multiplier described in random subgradient algorithm online updating is utilized；
7) after iteration for several times, judge whether to meet the condition of convergence；
If 8) presently described VNF migration and cpu resource allocation strategy meet the condition of convergence, the VNF is migrated and CPU
Resource allocation policy is notified to the virtual network scheduler and resource management entity；
9) it is updated according to each VNF migration and cpu resource allocation strategy, each node VNF example according to the queue
Mode updates the buffer queue size, and waits next time slot scheduling.
6. a kind of virtual network function intelligence moving method according to claim 5, it is characterised in that: in each scheduling
In gap, the quene state of each node VNF example is given, the node state and link state of system calculate described optimal
VNF migration and Resource Allocation Formula, specific steps are as follows:
1) global state under current time slots t, the input by current state r (t) as Q network are monitored；
2) if node or link fail in time slot t, in order to improve the reliability of network, system is moved to corresponding VNF
On the basis of shifting, optimal VNF migration strategy is selectedEnergy consumption and time delay are optimized；
3) optimal migration strategy is otherwise directly selected
4) next time slot scheduling t+1 is waited.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910382250.9A CN110275758B (en) | 2019-05-09 | 2019-05-09 | Intelligent migration method for virtual network function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910382250.9A CN110275758B (en) | 2019-05-09 | 2019-05-09 | Intelligent migration method for virtual network function |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110275758A true CN110275758A (en) | 2019-09-24 |
| CN110275758B CN110275758B (en) | 2022-09-30 |
Family
ID=67959680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910382250.9A Active CN110275758B (en) | 2019-05-09 | 2019-05-09 | Intelligent migration method for virtual network function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110275758B (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111130904A (en) * | 2019-12-30 | 2020-05-08 | 重庆邮电大学 | Virtual network function migration optimization algorithm based on deep certainty strategy gradient |
| CN111371742A (en) * | 2020-02-21 | 2020-07-03 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| CN111404815A (en) * | 2020-03-06 | 2020-07-10 | 武汉大学 | Constrained routing method based on deep learning |
| CN111432005A (en) * | 2020-03-30 | 2020-07-17 | 中科边缘智慧信息科技（苏州）有限公司 | Service migration method under narrow-band weak networking condition |
| CN111538587A (en) * | 2020-03-06 | 2020-08-14 | 重庆邮电大学 | Service function chain reconfiguration method based on load balancing |
| CN111538567A (en) * | 2020-04-26 | 2020-08-14 | 国网江苏省电力有限公司信息通信分公司 | Method and equipment for deploying virtual network function chain on edge equipment |
| CN112000459A (en) * | 2020-03-31 | 2020-11-27 | 华为技术有限公司 | Method for expanding and contracting service and related equipment |
| CN112311578A (en) * | 2019-07-31 | 2021-02-02 | 中国移动通信集团浙江有限公司 | VNF scheduling method and device based on deep reinforcement learning |
| CN112506658A (en) * | 2020-12-09 | 2021-03-16 | 华南理工大学 | Dynamic resource allocation and task scheduling method in service chain |
| CN112506648A (en) * | 2020-11-20 | 2021-03-16 | 鹏城实验室 | Traffic stateless migration method of virtual network function instance and electronic equipment |
| CN112737854A (en) * | 2020-12-30 | 2021-04-30 | 国网河南省电力公司信息通信公司 | Service chain migration method and device based on energy consumption and service quality |
| CN112887156A (en) * | 2021-02-23 | 2021-06-01 | 重庆邮电大学 | Dynamic virtual network function arrangement method based on deep reinforcement learning |
| CN113163018A (en) * | 2021-01-15 | 2021-07-23 | 郑州轻工业大学 | Virtual network online migration method and device with time delay, resource and energy consumption perception |
| CN113347016A (en) * | 2021-03-10 | 2021-09-03 | 福州大学 | Virtualization network function migration method based on resource occupation and time delay sensitivity |
| CN113411819A (en) * | 2021-05-30 | 2021-09-17 | 西安电子科技大学 | 5G NFV slice reconfiguration method, system and wireless communication system |
| CN113608830A (en) * | 2021-07-13 | 2021-11-05 | 全球能源互联网研究院有限公司 | VNF migration method and device based on fault prediction |
| CN114401194A (en) * | 2021-12-29 | 2022-04-26 | 山东省计算中心（国家超级计算济南中心） | Dynamic expansion method and platform supporting network function virtualization and computer |
| CN114938335A (en) * | 2022-04-27 | 2022-08-23 | 中国人民解放军战略支援部队信息工程大学 | Dynamic scheduling method and device for network slicing service function |
| CN115118526A (en) * | 2022-08-23 | 2022-09-27 | 北京润尼尔网络科技有限公司 | VR device data migration method |
| CN115118599A (en) * | 2021-03-18 | 2022-09-27 | 诺基亚通信公司 | Network management |
| CN115361288A (en) * | 2022-08-19 | 2022-11-18 | 南京工业大学 | Dynamic reconfiguration method for service function chain in air-space-ground integrated scene |
| CN115484304A (en) * | 2022-08-02 | 2022-12-16 | 重庆邮电大学 | Real-time service migration method based on lightweight learning |
| CN115765826A (en) * | 2022-09-19 | 2023-03-07 | 西安电子科技大学 | Unmanned aerial vehicle network topology reconstruction method for on-demand service |
| CN116436782A (en) * | 2023-06-09 | 2023-07-14 | 中国电信股份有限公司四川分公司 | Deterministic network transmission method for embodying business willingness |
| CN116528255A (en) * | 2023-06-29 | 2023-08-01 | 中国电信股份有限公司 | Network slice migration method, device, equipment and storage medium |
| CN117082009A (en) * | 2023-10-16 | 2023-11-17 | 泰雷兹数字科技(北京)有限公司 | Cloud resource management method and management system based on software definition security |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203255A (en) * | 2016-03-20 | 2017-09-26 | 田文洪 | Power-economizing method and device are migrated in a kind of network function virtualized environment |
| CN108063830A (en) * | 2018-01-26 | 2018-05-22 | 重庆邮电大学 | A kind of network section dynamic resource allocation method based on MDP |
| CN108900358A (en) * | 2018-08-01 | 2018-11-27 | 重庆邮电大学 | Virtual network function dynamic migration method based on deepness belief network resource requirement prediction |
| CN109062668A (en) * | 2018-08-01 | 2018-12-21 | 重庆邮电大学 | A kind of virtual network function moving method of the multipriority based on 5G access network |
| US20190089780A1 (en) * | 2017-09-15 | 2019-03-21 | Nec Europe Ltd. | Application function management using nfv mano system framework |
-
2019
- 2019-05-09 CN CN201910382250.9A patent/CN110275758B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107203255A (en) * | 2016-03-20 | 2017-09-26 | 田文洪 | Power-economizing method and device are migrated in a kind of network function virtualized environment |
| US20190089780A1 (en) * | 2017-09-15 | 2019-03-21 | Nec Europe Ltd. | Application function management using nfv mano system framework |
| WO2019052704A1 (en) * | 2017-09-15 | 2019-03-21 | NEC Laboratories Europe GmbH | Application function management using nfv mano system framework |
| CN108063830A (en) * | 2018-01-26 | 2018-05-22 | 重庆邮电大学 | A kind of network section dynamic resource allocation method based on MDP |
| CN108900358A (en) * | 2018-08-01 | 2018-11-27 | 重庆邮电大学 | Virtual network function dynamic migration method based on deepness belief network resource requirement prediction |
| CN109062668A (en) * | 2018-08-01 | 2018-12-21 | 重庆邮电大学 | A kind of virtual network function moving method of the multipriority based on 5G access network |
Non-Patent Citations (2)
| Title |
|---|
| KOJI SUGISONO等: ""Migration for VNF instances forming service chain"", 《2018 IEEE 7TH INTERNATIONAL CONFERENCE ON CLOUD NETWORKING》 * |
| 唐伦等: ""基于深度信念网络资源需求预测的虚拟网络功能动态迁移算法"", 《电子与信息学报》 * |
Cited By (45)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112311578A (en) * | 2019-07-31 | 2021-02-02 | 中国移动通信集团浙江有限公司 | VNF scheduling method and device based on deep reinforcement learning |
| CN112311578B (en) * | 2019-07-31 | 2023-04-07 | 中国移动通信集团浙江有限公司 | VNF scheduling method and device based on deep reinforcement learning |
| CN111130904B (en) * | 2019-12-30 | 2022-03-29 | 重庆邮电大学 | Virtual network function migration optimization algorithm based on deep certainty strategy gradient |
| CN111130904A (en) * | 2019-12-30 | 2020-05-08 | 重庆邮电大学 | Virtual network function migration optimization algorithm based on deep certainty strategy gradient |
| CN111371742A (en) * | 2020-02-21 | 2020-07-03 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| CN111371742B (en) * | 2020-02-21 | 2022-04-29 | 重庆邮电大学 | SVDD (singular value decomposition and direct data decomposition) -based network slice physical node anomaly detection method |
| CN111404815B (en) * | 2020-03-06 | 2021-03-16 | 武汉大学 | Constrained routing method based on deep learning |
| CN111538587B (en) * | 2020-03-06 | 2022-04-29 | 重庆邮电大学 | Service function chain reconfiguration method based on load balancing |
| CN111538587A (en) * | 2020-03-06 | 2020-08-14 | 重庆邮电大学 | Service function chain reconfiguration method based on load balancing |
| CN111404815A (en) * | 2020-03-06 | 2020-07-10 | 武汉大学 | Constrained routing method based on deep learning |
| CN111432005B (en) * | 2020-03-30 | 2022-07-29 | 中科边缘智慧信息科技（苏州）有限公司 | Service migration method under narrow-band weak networking condition |
| CN111432005A (en) * | 2020-03-30 | 2020-07-17 | 中科边缘智慧信息科技（苏州）有限公司 | Service migration method under narrow-band weak networking condition |
| CN112000459A (en) * | 2020-03-31 | 2020-11-27 | 华为技术有限公司 | Method for expanding and contracting service and related equipment |
| WO2021197364A1 (en) * | 2020-03-31 | 2021-10-07 | 华为技术有限公司 | Scaling method for a service, and related device |
| CN111538567A (en) * | 2020-04-26 | 2020-08-14 | 国网江苏省电力有限公司信息通信分公司 | Method and equipment for deploying virtual network function chain on edge equipment |
| CN111538567B (en) * | 2020-04-26 | 2023-06-09 | 国网江苏省电力有限公司信息通信分公司 | Deployment method and device for virtual network function chains on edge device |
| CN112506648A (en) * | 2020-11-20 | 2021-03-16 | 鹏城实验室 | Traffic stateless migration method of virtual network function instance and electronic equipment |
| CN112506658A (en) * | 2020-12-09 | 2021-03-16 | 华南理工大学 | Dynamic resource allocation and task scheduling method in service chain |
| CN112506658B (en) * | 2020-12-09 | 2024-04-26 | 华南理工大学 | Dynamic resource allocation and task scheduling method in service chain |
| CN112737854A (en) * | 2020-12-30 | 2021-04-30 | 国网河南省电力公司信息通信公司 | Service chain migration method and device based on energy consumption and service quality |
| CN113163018A (en) * | 2021-01-15 | 2021-07-23 | 郑州轻工业大学 | Virtual network online migration method and device with time delay, resource and energy consumption perception |
| CN112887156A (en) * | 2021-02-23 | 2021-06-01 | 重庆邮电大学 | Dynamic virtual network function arrangement method based on deep reinforcement learning |
| CN113347016A (en) * | 2021-03-10 | 2021-09-03 | 福州大学 | Virtualization network function migration method based on resource occupation and time delay sensitivity |
| CN115118599B (en) * | 2021-03-18 | 2024-05-28 | 诺基亚通信公司 | Network management |
| CN115118599A (en) * | 2021-03-18 | 2022-09-27 | 诺基亚通信公司 | Network management |
| CN113411819A (en) * | 2021-05-30 | 2021-09-17 | 西安电子科技大学 | 5G NFV slice reconfiguration method, system and wireless communication system |
| CN113608830A (en) * | 2021-07-13 | 2021-11-05 | 全球能源互联网研究院有限公司 | VNF migration method and device based on fault prediction |
| CN114401194B (en) * | 2021-12-29 | 2023-08-01 | 山东省计算中心（国家超级计算济南中心） | Dynamic expansion method, platform and computer supporting network function virtualization |
| CN114401194A (en) * | 2021-12-29 | 2022-04-26 | 山东省计算中心（国家超级计算济南中心） | Dynamic expansion method and platform supporting network function virtualization and computer |
| CN114938335A (en) * | 2022-04-27 | 2022-08-23 | 中国人民解放军战略支援部队信息工程大学 | Dynamic scheduling method and device for network slicing service function |
| CN114938335B (en) * | 2022-04-27 | 2024-01-26 | 中国人民解放军战略支援部队信息工程大学 | Dynamic scheduling method and device for network slice service function |
| CN115484304A (en) * | 2022-08-02 | 2022-12-16 | 重庆邮电大学 | Real-time service migration method based on lightweight learning |
| CN115484304B (en) * | 2022-08-02 | 2024-03-19 | 重庆邮电大学 | Lightweight learning-based live service migration method |
| CN115361288B (en) * | 2022-08-19 | 2023-06-23 | 南京工业大学 | Service function chain dynamic reconstruction method under space-air-ground integrated scene |
| CN115361288A (en) * | 2022-08-19 | 2022-11-18 | 南京工业大学 | Dynamic reconfiguration method for service function chain in air-space-ground integrated scene |
| CN115118526B (en) * | 2022-08-23 | 2022-11-25 | 北京润尼尔网络科技有限公司 | VR device data migration method |
| CN115118526A (en) * | 2022-08-23 | 2022-09-27 | 北京润尼尔网络科技有限公司 | VR device data migration method |
| CN115765826A (en) * | 2022-09-19 | 2023-03-07 | 西安电子科技大学 | Unmanned aerial vehicle network topology reconstruction method for on-demand service |
| CN115765826B (en) * | 2022-09-19 | 2024-05-31 | 西安电子科技大学 | Unmanned aerial vehicle network topology reconstruction method for on-demand service |
| CN116436782A (en) * | 2023-06-09 | 2023-07-14 | 中国电信股份有限公司四川分公司 | Deterministic network transmission method for embodying business willingness |
| CN116436782B (en) * | 2023-06-09 | 2023-09-29 | 中国电信股份有限公司四川分公司 | Deterministic network transmission method for embodying business willingness |
| CN116528255A (en) * | 2023-06-29 | 2023-08-01 | 中国电信股份有限公司 | Network slice migration method, device, equipment and storage medium |
| CN116528255B (en) * | 2023-06-29 | 2023-10-10 | 中国电信股份有限公司 | Network slice migration method, device, equipment and storage medium |
| CN117082009A (en) * | 2023-10-16 | 2023-11-17 | 泰雷兹数字科技(北京)有限公司 | Cloud resource management method and management system based on software definition security |
| CN117082009B (en) * | 2023-10-16 | 2024-02-27 | 天翼安全科技有限公司 | Cloud resource management method and management system based on software definition security |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110275758B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110275758A (en) | A kind of virtual network function intelligence moving method | |
| Yi et al. | Task optimization and scheduling of distributed cyber–physical system based on improved ant colony algorithm | |
| CN109818786B (en) | Method for optimally selecting distributed multi-resource combined path capable of sensing application of cloud data center | |
| CN104657221B (en) | The more queue flood peak staggered regulation models and method of task based access control classification in a kind of cloud computing | |
| CN105550033B (en) | Resource dispatching strategy method based on genetic tabu hybrid algorithm under privately owned cloud environment | |
| Huang et al. | Rlsk: A job scheduler for federated kubernetes clusters based on reinforcement learning | |
| CN108174394A (en) | A kind of Arrangement algorithm of 5G networks slice | |
| CN108512772A (en) | Quality-of-service based data center's traffic scheduling method | |
| CN104954278B (en) | Network traffics dispatching method based on bee colony optimization under multi-QoS Constraints | |
| CN102932422A (en) | Cloud environment task scheduling method based on improved ant colony algorithm | |
| CN107566535B (en) | Self-adaptive load balancing method based on concurrent access timing sequence rule of Web map service | |
| Rugwiro et al. | Task scheduling and resource allocation based on ant-colony optimization and deep reinforcement learning | |
| CN111010295A (en) | SDN-MEC-based power distribution and utilization communication network task migration method | |
| Liu | Intelligent routing based on deep reinforcement learning in software-defined data-center networks | |
| Monil et al. | QoS-aware virtual machine consolidation in cloud datacenter | |
| CN114938372B (en) | Federal learning-based micro-grid group request dynamic migration scheduling method and device | |
| CN113596868A (en) | 5G network slice resource management mechanism based on SDN and NFV | |
| CN109062682A (en) | A kind of resource regulating method and system of cloud computing platform | |
| CN107018018A (en) | A kind of server delta online upgrading method and system based on SDN | |
| CN108304253A (en) | Map method for scheduling task based on cache perception and data locality | |
| Ebrahim et al. | Resilience and load balancing in fog networks: A multi-criteria decision analysis approach | |
| Zheng et al. | Application of improved ant colony algorithm in load balancing of software-defined networks | |
| WO2024036909A1 (en) | Fair load unloading and migration method for edge service network | |
| Tang et al. | Optimization search strategy for task offloading from collaborative edge computing | |
| Mampage et al. | A deep reinforcement learning based algorithm for time and cost optimized scaling of serverless applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |