CN113015196A - Network slice fault healing method based on state perception - Google Patents

Abstract

The invention relates to a network slice fault healing method based on state perception, and belongs to the technical field of mobile communication. Firstly, a VNF healing strategy selection method based on state perception is provided, and the VNF healing strategy is selected according to the fault state of a VNF on a physical node and the node resource use condition so as to adapt to different fault scenes. Further, for VNFs that need to be migrated, a computational model based on minimizing the healing overhead is proposed. And finally, searching and obtaining the optimal migration strategy of the VNF through a deep reinforcement learning method DQN. The method provided by the invention can reduce the healing overhead and improve the reliability of the network while healing the network slice fault.

Description

Network slice fault healing method based on state perception

Technical Field

The invention belongs to the technical field of mobile communication, and relates to a network slice fault healing method based on state perception.

Background

Network slicing integrates technologies such as SDN and NFV, and realizes vision of network operators by simultaneously realizing multiple heterogeneous logic network slices on the same infrastructure platform. Each logical slice is a set of network resources tailored to meet specific customer QoS or resiliency requirements, and VNFs are programmatically combined into SFCs in a certain order. The network slice enables an operator to construct an agile and flexible network to adapt to various use cases in different industry vertical fields, meets the customization requirements of different use cases in the vertical industry through a single underlying physical network, and effectively reduces the CAPEX of the mobile communication network. The application of SDN, NFV and other technologies also provides technical support for the automatic management of network slices, and the deployment of a self-healing technology can bring higher reliability guarantee for a mobile communication network.

The reliability of the traditional network is ensured by special hardware, the network function realizes software and hardware decoupling under a network slicing scene, the expensive and hard-to-reconfigure special hardware is replaced by a VNF running on a general platform, the cost is reduced, the flexibility is improved, and meanwhile, a new challenge is provided for the reliability of the network. Because VNFs are susceptible to various problems, such as hardware problems, software crashes and overloads, etc., the underlying physical node failure has the most severe impact on network performance. The shared nature of the network slice results in service degradation and even disruption of all VNF-related SFCs deployed on the failed physical node. In order to avoid serious QoS degradation after a physical node failure occurs, and even to avoid a situation of violating SLA, a self-healing capability needs to be provided for a network slice, that is, the SFC automatically recovers from an abnormality, and the capability of ensuring reliable operation of a network service is ensured.

Disclosure of Invention

In view of the above, the present invention provides a method for healing a network slice fault based on state sensing.

In order to achieve the purpose, the invention provides the following technical scheme:

a network slice fault healing method based on state perception comprises the following steps:

s1: establishing a virtualized network function VNF healing strategy selection method based on state perception;

s2: establishing a migration overhead calculation model based on the minimized healing overhead;

s3: and establishing a VNF transfer learning method based on the reinforcement learning DQN.

Optionally, the S1 specifically includes:

s21: the VNF performance observation time sequence data is input into a CNN-GRU network, and whether each VNF on a physical node is in a fault state or not is detected;

s22: calculating the abnormal index theta of the physical node according to the fault state information of the VNF on the physical node, and judging whether the theta exceeds an abnormal threshold theta_ThrJudging whether the physical node is in an abnormal condition or not; theta > theta_ThrAbnormal, otherwise normal;

s23: determining a fault VNF healing strategy by jointly considering two factors of whether the physical node is abnormal and whether the physical node is overloaded: healing a fault VNF at a current node, migrating the fault VNF and migrating all VNFs of physical nodes;

s24: according to a healing strategy selection method, a VNF list to be migrated and a failed physical node are obtained, related information of the VNF list to be migrated is sent to a Network Function Virtualization (NFVMANO), a VNF migration algorithm is operated at the NFVMANO according to the latest real-time network topology and network state information, and a migration strategy of the VNF to be migrated is determined.

Optionally, the S2 specifically includes:

in the healing of the network slice, the healing overhead mainly comes from network reconfiguration overhead and VNF state migration overhead; the network reconfiguration overhead refers to that a controller selects a migration destination node and a link to complete the overhead generated by the link and node configuration; the VNF state migration overhead is the overhead of VNF state data migrating from an abnormal physical node to a destination node;

the healing overhead for migrating VNFi from physical node m to n is defined as:

C(i,m,n)＝k_i(T_T(i,m,n)+T_C)

wherein k is_iThe average amount of data received per second on VNFi; t is_T(i, m, n) is the migration implementation duration for migrating VNFi from the underlying physical node m to n, T_CThe duration is reconfigured for the network.

Optionally, the S3 specifically includes:

s41: monitoring a network state r (t) under the current time slot t, wherein the network state r (t) comprises a global node state zeta (t) and a global link state eta (t);

s42: taking the current state r (t) as the input of the Q network;

s43: calculating an optimal VNF migration strategy with the aim of minimizing healing overhead;

s44: the migration of the VNF is performed based on the optimal action.

The invention has the beneficial effects that: according to the invention, different types of fault healing strategies are selected based on VNFs, invalid VNF migration is avoided, a migration model based on minimized healing overhead is established, a deep reinforcement learning method DQN is introduced to determine the migration strategies, the reliability of the network is guaranteed, and the healing overhead is reduced.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objectives and other advantages of the invention may be realized and attained by the means of the instrumentalities and combinations particularly pointed out hereinafter.

Drawings

For the purposes of promoting a better understanding of the objects, aspects and advantages of the invention, reference will now be made to the following detailed description taken in conjunction with the accompanying drawings in which:

FIG. 1 is a schematic diagram of a failed VNF healing strategy selection;

FIG. 2 is a schematic diagram of a network slicing scenario;

fig. 3 is a diagram of a DQN network model.

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention in a schematic way, and the features in the following embodiments and examples may be combined with each other without conflict.

Wherein the showings are for the purpose of illustrating the invention only and not for the purpose of limiting the same, and in which there is shown by way of illustration only and not in the drawings in which there is no intention to limit the invention thereto; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.

The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", etc., based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not an indication or suggestion that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes, and are not to be construed as limiting the present invention, and the specific meaning of the terms may be understood by those skilled in the art according to specific situations.

Referring to fig. 1-3, the principle of the method of the present invention is shown.

Example one

The embodiment provides a VNF healing strategy selection method based on state awareness, which specifically includes:

firstly, whether the VNF is abnormal is detected by adopting a fault detection method based on a CNN-GRU network. The method takes preprocessed VNFs observation data as input, adopts a CNN network to extract features, and inputs the extracted features into a GRU to predict a future working state value of the network. Calculating an abnormal score by calculating a reconstruction error between the predicted value and the true value, and judging whether each VNF is in a fault state by judging whether the reconstruction error is greater than a certain threshold value;

second, according to the failure status of the VNF on the physical nodeThe state information calculates the abnormal index theta of the physical node and judges whether the theta exceeds an abnormal threshold theta_ThrAnd judging whether the physical node is in an abnormal condition or not. Theta > theta_ThrAbnormal, otherwise normal;

defining a physical node anomaly index theta and a threshold theta of the physical node anomaly index_Thr：

Wherein, | a | represents the number of failed VNFs on the physical node m, and | m | is the number of VNFs on the physical node m.

When theta is larger than theta_ThrWhen the physical device is considered to have a fault, the physical node is in a serious abnormal state which is unavailable, the reliability and the QoS of the network are considered to be guaranteed, the current physical node cannot continuously provide services, and all VNFs on the abnormal physical node need to be immediately migrated to realize the self-healing of the SFC in the network slice.

If theta is less than theta_ThrIn a performance degradation stage or a normal state on the physical node, at this time, the failure of the VNF may be caused by a software operation problem or insufficient resources required by the VNF. In order to accurately determine a VNF healing policy, avoid the overhead problem caused by invalid VNF migration, and further consider the influence of the physical node resource usage on VNF failure.

Defining the CPU resource requirement and the memory resource requirement of VNFi in SFC as

And

the CPU resource and the memory resource of the physical node m are respectively

And

the CPU load and the memory load of the physical node m are

And

wherein, the CPU resource requirement and the memory resource requirement of each VNFi in the SFC are respectively

And

θ_i,mbeing binary variables, theta_i,mWith 1 denotes VNFi deployed on the underlying physical node m, θ _i,m0 means that VNFi is not deployed on the underlying physical node m. Setting a threshold value

And

when in use

Or

Meanwhile, the physical node m is in a resource overload state.

(1) When the resource usage on the physical node is in an overload state, the failure of the VNF at this time may be caused by a problem of insufficient resources, and the VNF with the failure is selected for migration in consideration of load balancing of the network; (2) and when the physical node is in a normal load state, healing the VNF with the fault at the current physical node, and implementing measures such as restarting a virtual machine and redistributing resources.

And finally, obtaining a VNF list to be migrated and a failed physical node list according to a healing strategy selection method, sending related information to the NFV MANO, operating a VNF migration algorithm at the NFV MANO according to the latest real-time network topology structure and network state information, and determining a migration strategy of the VNF to be migrated.

Example two

The embodiment provides a cost model based on minimized healing overhead for calculating the cost generated by the migration strategy, which is as follows:

in order to ensure that data is not lost and transparent to users in the process of restoring the virtual network function to other physical nodes, a restoration mechanism must first store all data packets sent to the VNF to be restored, and sequentially forward the data packets to the target VNF after completing the internal state migration of the VNF. During the migration process, the network needs to cache all data packets that should be received by the VNF to be migrated. Assuming that the VNF to be migrated receives a packets per second and the whole migration operation takes b seconds, the network needs to buffer a × b packets during SFC healing. As the size of networks increases, the amount of data to be buffered increases rapidly.

In network slice healing, the healing overhead comes mainly from network reconfiguration overhead and VNF state migration overhead. The network reconfiguration overhead refers to that a controller selects a migration destination node and a link to complete the overhead generated by the link and node configuration; the VNF state migration overhead is the overhead of VNF state data migrating from the abnormal physical node to the destination node.

Giving the network reconfiguration duration T_CAnd a state transition duration T_TWherein the reconfiguration duration T_CDetermined by the size and computing power of the network. The data size of VNF state migration mainly comes from the migration of memory data, so the state migration duration of the VNF is defined as the ratio of the memory data of the VNF to be migrated to the available network bandwidth, and the migration implementation duration for migrating the VNFi from the bottom-layer physical node m to n is as follows:

wherein the content of the first and second substances,

is eta_pThe remaining available bandwidth, η, on the largest path_pEfficiency for path p, defined as:

where | p | is the number of physical nodes on path p, d_mlThe distance between the underlying physical nodes m and l is defined as the number of network hops. The further the distance between the source node and the destination node of information transmission in the network, the greater the amount of resources consumed to implement the information transmission. As can be seen from the above equation, the greater the number of physical nodes on the path p, the greater the path efficiency η_pThe smaller the value of (c).

Thus, the healing overhead of migrating VNFi from physical node m to n is defined as:

C(i,m,n)＝k_i(T_T(i,m,n)+T_C)

wherein k is_iFor the average amount of data received per second on VNFi, T_T(i, m, n) State transition duration, T_CThe network reconfiguration duration.

EXAMPLE III

The embodiment provides a DQN migration policy learning method, which specifically includes:

reinforcement learning is an enhanced online learning method based on a Markov decision process. In the learning process, the intelligent learner agent searches for the optimal decision-making action in a trial and error manner through continuous interaction with the environment. Each decision action causes the environment to transition from one state to another, thereby obtaining an instantaneous reward value. The purpose of an Agent is to learn the action that seeks to maximize the return value for the environment. The reinforcement learning strategy can be expressed as pi: s → A, where pi represents the learning strategy, s represents the current environment state, and A represents the selected execution action. Because the influence of action execution is many-sided, it not only can influence instantaneous reward punishment evaluation value to can influence next environmental state, and then influence the selection of next action, thereby can influence holistic final reward punishment evaluation value.

Q learning is a model-independent reinforcement learning method. Q learning is to maximize the expected value of the cumulative return of the conversion by evaluating the Q function of the state-action pairs. Evaluation function Q(s)_t,a_t) Representing slave states s_tStarting and executing action a_tThe instantaneous immediate reward obtained later plus the reduced cumulative reward value following the optimal strategy later is defined as:

wherein: r is_t+1Indicating the current state s_tLower execution action a_tThe latter instantaneous return value, γ representing the discount factor, δ(s)_t,a_t) Represents a state s_tLower execution action a_tThe resulting state of the last.

Is shown in state δ(s)_t,a_t) The maximum reduced cumulative return value that can be obtained following the optimal strategy. And in the state s_tThe following optimal strategy is an action strategy which can lead the conversion of the immediate return and the immediate subsequent state to the accumulated return value to be maximum:

at the same time

The Q-value evaluation function under the recursive definition can thus be expressed as:

the Q function update that converts it to iteration mode is expressed as:

wherein: α represents a learning rate, determines the influence of new information on existing information, and controls the convergence rate. In Q learning, agent only needs to compare Q(s) of each state-action pair_t,a_t) As a function, the optimal policy action can be determined.

The DQN network adopts the mechanisms of experience playback, a target network and an adaptive learning rate adjusting method. The experience playback means that the experience can be(s) in the process of interaction between the Agent and the environment_t,a_t,r_t+1,s_t+1) The form of the data is stored in a memory base D, and a batch of data is randomly sampled from D for training each training, so that the correlation among samples can be eliminated to a certain extent. The target network refers to the DQN which uses two networks, one is the current network, and interacts with the environment and is continuously updated. The other network is a target network, which does not interact with the environment and is not updated at each time step, but is updated at regular time steps, and each update directly assigns the parameters of the current network to the target network.

1. First, training of the DQN network is performed:

1) initializing a memory library D, wherein the capacity of the memory library D is N and the memory library D is used for storing training samples;

2) initializing a current value convolutional neural network N1 and randomly initializing a weight parameter theta;

3) the target value convolutional neural network N2 is initialized, and its structure and initialization weight parameter θ are the same as N1.

4) Selecting one execution action a for each state by adopting an E-greedy method_t；

5) Get the execution action a_tRear prize r_t+1And input s to the next network_t+1；

6) Order the state from s_tGo to s_t+1；

7) Storing the experience(s) in a memory library D_t,a_t,r_t+1,s_t+1)；

8) Randomly selecting a batch of data from the D;

9) the target value TargetQ and the current value Q (s, a; θ).

10) A loss function L (θ) is calculated, defined as the target value TargetQ and the current value Q (s, a; θ) mean square error between:

L(θ)＝E[(TargetQ-Q(s,a；θ))²]

11) updating the network parameter theta in a mode of SGD random gradient descent,

12) updating the parameter theta of the target value network after each C iterations^-For the current value of the parameter theta of the network, i.e. theta^-＝θ；

13) Repeating 4) -12) until convergence;

2. secondly, after the DQN network training is finished, monitoring the network state r (t) under the current time slot t, wherein the network state r (t) comprises a global node state zeta (t) and a global link state eta (t);

3. taking the current state r (t) as the input of the Q network;

4. calculating an optimal VNF migration strategy with the aim of minimizing healing overhead;

5. finally, the migration of the VNF is performed based on the optimal action.

Finally, the above embodiments are only intended to illustrate the technical solutions of the present invention and not to limit the present invention, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions, and all of them should be covered by the claims of the present invention.

Claims (4)

1. A network slice fault healing method based on state perception is characterized in that: the method comprises the following steps:

s1: establishing a virtualized network function VNF healing strategy selection method based on state perception;

s2: establishing a migration overhead calculation model based on the minimized healing overhead;

s3: and establishing a VNF transfer learning method based on the reinforcement learning DQN.

2. The method for network slice fault healing based on state perception according to claim 1, wherein: the S1 specifically includes:

s21: the VNF performance observation time sequence data is input into a CNN-GRU network, and whether each VNF on a physical node is in a fault state or not is detected;

s22: calculating the abnormal index theta of the physical node according to the fault state information of the VNF on the physical node, and judging whether the theta exceeds an abnormal threshold theta_ThrJudging whether the physical node is in an abnormal condition or not;

Θ＞Θ_Thrabnormal, otherwise normal;

s23: determining a fault VNF healing strategy by jointly considering two factors of whether the physical node is abnormal and whether the physical node is overloaded: healing a fault VNF at a current node, migrating the fault VNF and migrating all VNFs of physical nodes;

s24: according to a healing strategy selection method, a VNF list to be migrated and a failed physical node are obtained, related information of the VNF list to be migrated is sent to a Network Function Virtualization (NFVMANO), a VNF migration algorithm is operated at the NFVMANO according to the latest real-time network topology and network state information, and a migration strategy of the VNF to be migrated is determined.

3. The method for network slice fault healing based on state perception according to claim 2, wherein: the S2 specifically includes:

in the healing of the network slice, the healing overhead comes from network reconfiguration overhead and VNF state migration overhead;

the network reconfiguration overhead refers to that a controller selects a migration destination node and a link to complete the overhead generated by the link and node configuration;

the VNF state migration overhead is the overhead of VNF state data migrating from an abnormal physical node to a destination node;

the healing overhead for migrating VNFi from physical node m to n is defined as:

C(i,m,n)＝k_i(T_T(i,m,n)+T_C)

wherein k is_iThe average amount of data received per second on VNFi; t is_T(i, m, n) is the migration implementation duration for migrating VNFi from the underlying physical node m to n, T_CThe duration is reconfigured for the network.

4. The method for network slice fault healing based on state perception according to claim 3, wherein: the S3 specifically includes:

s41: monitoring a network state r (t) under the current time slot t, wherein the network state r (t) comprises a global node state zeta (t) and a global link state eta (t);

s42: taking the current state r (t) as the input of the Q network;

s43: calculating an optimal VNF migration strategy with the aim of minimizing healing overhead;

s44: the migration of the VNF is performed based on the optimal action.

Images