CN116760718A - SDN flow scheduling method based on machine learning classification prediction - Google Patents
SDN flow scheduling method based on machine learning classification prediction Download PDFInfo
- Publication number
- CN116760718A CN116760718A CN202310574479.9A CN202310574479A CN116760718A CN 116760718 A CN116760718 A CN 116760718A CN 202310574479 A CN202310574479 A CN 202310574479A CN 116760718 A CN116760718 A CN 116760718A
- Authority
- CN
- China
- Prior art keywords
- flow
- data
- classification
- module
- traffic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 238000010801 machine learning Methods 0.000 title claims abstract description 19
- 230000006870 function Effects 0.000 claims abstract description 23
- 230000009467 reduction Effects 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 35
- 230000005540 biological transmission Effects 0.000 claims description 26
- 208000037918 transfusion-transmitted disease Diseases 0.000 claims description 14
- 230000009471 action Effects 0.000 claims description 11
- 238000011176 pooling Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 8
- 230000009191 jumping Effects 0.000 claims description 6
- 210000002569 neuron Anatomy 0.000 claims description 6
- 230000000737 periodic effect Effects 0.000 claims description 6
- 230000004913 activation Effects 0.000 claims description 5
- 230000003044 adaptive effect Effects 0.000 claims description 5
- 238000007781 pre-processing Methods 0.000 claims description 5
- 238000013144 data compression Methods 0.000 claims description 3
- 101100346656 Drosophila melanogaster strat gene Proteins 0.000 claims description 2
- 238000010276 construction Methods 0.000 claims description 2
- 230000007613 environmental effect Effects 0.000 claims description 2
- 238000000605 extraction Methods 0.000 claims description 2
- 238000005070 sampling Methods 0.000 claims description 2
- 230000007704 transition Effects 0.000 claims description 2
- 238000013527 convolutional neural network Methods 0.000 abstract description 18
- 230000007787 long-term memory Effects 0.000 abstract description 2
- 238000013507 mapping Methods 0.000 abstract description 2
- 230000006403 short-term memory Effects 0.000 abstract description 2
- 230000000875 corresponding effect Effects 0.000 description 13
- 238000013461 design Methods 0.000 description 13
- 230000008859 change Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000011160 research Methods 0.000 description 6
- 238000005457 optimization Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 241001522296 Erithacus rubecula Species 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/12—Discovery or management of network topologies
- H04L41/122—Discovery or management of network topologies of virtualised topologies, e.g. software-defined networks [SDN] or network function virtualisation [NFV]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L43/00—Arrangements for monitoring or testing data switching networks
- H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
- H04L43/0876—Network utilisation, e.g. volume of load or congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Environmental & Geological Engineering (AREA)
- Medical Informatics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides an SDN flow scheduling method based on machine learning classification prediction, which eliminates the hysteresis of the traditional scheduling method. The user equipment UE classification module based on the convolutional neural network CNN is designed, the sliding window based on the CNN is designed to read network flow image characteristics after the low-resolution dimension reduction is carried out on the flow image, the mapping relation from the flow characteristics to the categories is established, and the UE classification is output. A classification flow prediction module based on long-term and short-term memory is designed, and the problems of large flow distribution difference and low prediction performance of users in a complex scene are solved. According to the result output by the last classification module, the module trains an independent LSTM prediction module aiming at each type of flow so as to improve the prediction accuracy of similar flows with close distribution. Aiming at the predicted flow, a scheduling module based on DRL and a reward function integrating user level and network level are designed, and self-learning dynamic flow scheduling is realized.
Description
Technical Field
The invention belongs to the technical field of communication.
Background
With the advancement of communication technology, the large number of deployed terminal devices and increasingly abundant network applications and services make network traffic and diversity thereof appear to increase explosively, and dynamic network states pose challenges for guaranteeing quality of service (Quality of Service, qoS) for various types of traffic tasks. The accurate grasp of the states of the network equipment and the flow has important significance for realizing intelligent management of the network and maximizing the utilization rate of network resources. In conventional network architecture, a single network device makes traffic decisions according to a configured routing table. When the transmission performance encounters a bottleneck, insufficient flexibility of the network architecture and excessive cost of hardware equipment are all limiting factors that the traditional network architecture cannot adapt to the current requirements. Software Defined Networking (SDN) is the core technology of the next generation network, an emerging architecture that is dynamic, manageable, low cost, and adaptable. These features make it an ideal choice for meeting the high bandwidth, dynamic characteristics of today's applications. SDN differs from traditional network architecture primarily in decoupling the control plane and the data plane. The strategies such as routing algorithm, resource scheduling and the like are deployed in a control plane, an optimal decision is made by utilizing the global view of the controller and sent to a data plane, and the SDN switch specifically executes data forwarding operation; the controller and the switch communicate through a southbound interface using protocols such as Openflow/P4. The controller manages the network device through software programming, and can flexibly adjust the data forwarding strategy according to the current network state and service requirements. The SDN architecture is shown in fig. 1.
The efficient and reasonable flow scheduling is a basic stone for ensuring the data transmission efficiency and the stable network performance. The conventional rule-based static scheduling method provides a basic traffic scheduling mechanism by pre-formulating scheduling rules for different network states and traffic requests. When a traffic request arrives, a scheduling flow is executed according to a predetermined rule. For example, a conventional Round-Robin (RR) method sequentially schedules requests to different devices in a Round manner. The method is simple in rule realization and easy to deploy, but the operation adopted for all data flows is consistent, the states of all devices and links are not considered, and load imbalance is easy to cause. On this basis, the improvement results in a weighted round robin scheduling method. But as the network environment becomes more and more complex, on one hand, the network topology becomes complicated, and on the other hand, as a large number of heterogeneous devices are deployed and the types of network services increase, the network traffic is characterized by rapid change. Under dynamic environments, the conventional method is more difficult to adapt to the requirements of the traffic scheduling task. Thus, utilizing machine learning algorithms is a reasonable solution to achieve flexible and intelligent traffic scheduling. For example, researchers have proposed adaptive resource scheduling methods based on neural networks and mobile traffic prediction. The method constructs a mobile traffic predictor based on LSTM, constructs network resources into resource blocks in a heat map mode, and the scheduler distributes the resource blocks to different transmission tasks according to the output result of the predictor so as to achieve the effects of improving the overall throughput and reducing the delay of data packets.
Traffic characteristics come from complex interactions of user behavior and various application layer protocols, making network traffic prediction challenging, especially for single user level traffic predictions over short periods of time. Traditional rule-based static scheduling methods require precise modeling of network state and specification of various rules for different traffic tasks. Such methods have poor adaptability to dynamic environments and are increasingly unable to adapt to current network environments. According to scientific data, the network traffic of the device can be simplified into 3 categories according to the characteristics, and the traffic characteristics change with time as shown in fig. 2.
With the deep research on SDN architecture and the development of machine learning algorithms, many researches on intelligent traffic scheduling by using the machine learning algorithms also appear. At present, the research focus in the field is mainly focused on constructing a flow prediction model by using a machine learning algorithm, and designing a heuristic or machine learning scheduling algorithm to realize intelligent scheduling of predicted flow. However, the current method has the following problems: 1. the research surface is single, the flow prediction is carried out by using a machine learning algorithm, or an intelligent scheduling system is designed by using the machine learning algorithm, and the periodic characteristic of the network flow is not fully utilized. And when the traffic peak arrives, the scheduling is performed again, which often causes the network performance to be reduced, and the scheduling has hysteresis. 2. The robustness to different usage environments and scenes is poor. That is, the designed system is only aimed at a special scene of research design, but in the actual situation, the current research method needs to be adjusted for scene change under the condition that the network topology and the access equipment types are different, so that the applicability is low. 3. To realize more accurate flow scheduling, user-level flow prediction is needed, and the traditional method has complex structure, more parameters and great training difficulty on a flow prediction model. On the basis, fine-granularity flow prediction is performed, and the system calculation overhead is high.
Therefore, we invented an SDN traffic scheduling method based on machine learning classification prediction. In the invention, a predictor and a scheduler are integrated together at a system level, and the prediction of incoming flow data is utilized to make proper scheduling decisions, so that the hysteresis of the traditional scheduling method is eliminated. A User Equipment (UE) classification module based on a convolutional neural network (Convolutional Neural Networks, CNN) is designed, after low-resolution dimension reduction is carried out on a flow image, a sliding window based on CNN is designed to read network flow image characteristics, and an expert is utilized to carry out model training on samples marked by different flow characteristic classification, so that a mapping relation from flow characteristics to categories is established by the model, and UE classification is output. The module can classify UEs with similar flow characteristics into one class with lower computational overhead and faster speed. And then, a classification flow prediction module based on long-term memory (Long Short Term Memory, LSTM) is designed, so that the problems of large flow distribution difference of users and low performance of the traditional prediction model in a complex scene are solved. According to the result output by the last classification module, the module trains an independent LSTM prediction module aiming at each type of flow so as to improve the prediction accuracy of similar flows with close distribution. Finally, aiming at the predicted flow, a scheduling module based on DRL and a reward function integrating the user level and the network level are designed, so that self-learning dynamic flow scheduling is realized.
Disclosure of Invention
The network traffic has the characteristics of periodicity, randomness and the like, but under different scenes, the traffic distribution difference among different UE is large due to different factors such as architecture, tasks and the like, so that the network performance fluctuates. In order to improve network stability and link load balancing, the invention designs a flow scheduling method based on machine learning classification prediction under SDN architecture, realizes low-calculation-overhead UE classification, more accurate flow prediction and flow scheduling comprehensively considering user and network performances, and specifically comprises the following functions:
1. under the SDN architecture, an SDN flow scheduling method based on machine learning classification prediction is designed and deployed on an application plane, and the method architecture diagram is shown in figure 3. The method comprises a CNN-based UE classification module, an LSTM-based classification traffic prediction module and a DRL-based traffic scheduling module, wherein the UE is classified by collecting network state data, traffic task data and UE traffic historical data information in real time and utilizing the relevance of historical traffic; training corresponding prediction models aiming at different types of UE to realize more accurate flow prediction performance; and scheduling the user traffic based on the predicted traffic to improve the network performance and reduce the influence of the peak traffic on the network stability.
2. The invention designs a CNN-based UE classification module. A process is described for generating a class for a UE using all historical traffic data. Along with the rapid increase of the number of network services, the functions of the devices are refined, and the flow generated among different devices gradually has the characteristic of differentiation. The traditional method utilizes a general model to predict the user flow with larger distribution difference, and has space for improving accuracy and robustness. The module firstly performs compression pretreatment on the historical flow data, defines a time transmission interval (Transmission Time Interval, TTI) on a time axis to be 5 minutes, and then the flow data of 24 hours is 288 TTIs on the time axis. On the traffic axis, define the maximum bandwidthb max The actual bandwidth at each instant is b i The compressed traffic in one TTI is
In the formula, square brackets are rounding functions. By this equation, the flow is compressed to a discrete value of [0,20 ]. Thus, each UE generates a traffic image size of (20,288) at 24 hours.
The invention designs a flow characteristic extraction model, which has the following structure:
the first layer is formed by sequentially sliding the flow images through 5 sliding windows with the sizes of (20, 20), and the output vector shape is (269,1,5);
the second layer is a pooling layer with pooling size (2, 1) being the maximum pooling layer for reducing the model size and parameter number, and the output vector shape is (134,1,5);
the third layer is a convolution layer with 10 convolution kernels of size (20, 1), the shape of the output vector being (115,1,10);
the fourth layer is the same as the second layer, and is a layer with a pooling size of (2, 1) as the largest pooling layer, and the shape of the output vector is (57,1,10)
The fifth layer is a flat layer, and after the flat operation of the flat layer is performed, the output shape is (570,1);
the sixth and seventh layers are two fully connected layers with 64 neurons and 3 neurons, respectively, and access the softmax function to output the predicted UE category. In the model, the activation functions of the convolution layers are Relu, and the model structure is shown in FIG. 4.
In the aspect of a data set, scientific data simplifies the network traffic of equipment into 3 categories according to the characteristics of the equipment, and the invention classifies the traffic of other characteristics into 1 category and 4 categories on the basis. Each type of flow creates 1000 samples, which together make up a 4000 sample dataset, with 65% of the samples used for training and 35% of the samples used for validation.
The UE classification module based on CNN reduces the subsequent calculation cost by compressing the flow data; and constructing a sliding window by utilizing a convolution network to extract flow characteristics, and classifying the UE according to the flow characteristics so that the UE classified into one class has close flow characteristics.
The CNN-based UE classification flow is shown in fig. 5.
(1) Initializing historical flow data in a system cache area;
(2) All flow data are preprocessed, and data are compressed from the time axis and the flow axis respectively. Discrete compressing 24-hour flow history data into 288 TTIs, wherein the compressed flow value is obtained by a formula (1), a low-resolution flow image is generated, the dimension reduction of the flow data is realized, and the compressed flow data is a matrix with the shape of (20, 288);
(3) The UE classification module based on CNN is utilized to sweep the flow data, and the class corresponding to the UE is output according to the flow characteristics;
(4) Classifying all the UEs according to the class result output in the step (3);
(5) Outputting a UE list L under different categories according to the classification result UE,k ;
(6) Outputting a flow data list L corresponding to different categories tra,k ;
3. An LSTM-based classification traffic prediction module is designed. Considering that the flow distribution among the UEs of different types and different tasks may be large, a flow predictor is independently trained for each type of UE, so that the influence of specific samples with large distribution difference on the accuracy of a prediction model is avoided. Each LSTM module obtains traffic data of a corresponding category, and trains and predicts traffic generated by the UE for 1 hour in the future using the data of the first 24 hours. The 24 hour duration is 288 TTIs and the 1 hour is 12 TTIs. Thus, the input of the model is an original traffic data matrix, the shape of the matrix is (1, 288), and each dimension comprises a TTI and a traffic value corresponding to the TTI; the number of LSTM units is set to 64, and the activation function is tanh; the output matrix shape of the module is (1, 12), i.e. the predicted flow value for the next 12 TTIs.
4. A DRL-based traffic scheduling module is designed for planning data based on predicted trafficAnd (3) a transmission path. Assume that the UE set in the network is u= { U 1 ,u 2 ,…,u n Link set l= { L } 1 ,l 2 ,…,l k The set of available link bandwidths is b= { B } 1 ,b 2 ,…,b k The set of available data transmission paths between UEs is p= { P 1 ,p 2 ,…,p m }. Assuming that the current scheduled task is from the initial node u ori To destination node u des Data is transmitted. u (u) ori And u des Represented by one-hot code, i.e. u ori And u des The corresponding encoded position value is 1 and the other position values are 0. The bandwidth required by the transmission data output by the prediction module is b pre . The state s of the current time observed by the DRL module t =[u ori ,u des ,B,b pre ]The vector has dimensions 2n+k+1. The module contains two hidden layers, each with 128 neurons, with an activation function of Relu. The output of the module is action a performed by the DRL t =π(s t )=[p 1 ,p 2 ,…,p m ]I.e. the optimal transmission path for the current task, the selected transmission path value is 1, the other path values are 0, and the vector dimension is m. Furthermore, we have devised a reward function that considers both user level and network level as:
wherein, alpha, beta and delta are weight parameters which are respectively 0.9, 0.5 and 0.6; sigma (sigma) n thr n Representing the sum of UE throughput, t strat And t end The first two parts are indexes for describing the user level;representing link load rate variance, +.>Representing the average value of available bandwidths of links, the smaller the variance value is, the more balanced the load among the links is. By utilizing the reward function, the DRL model is guided towards improving the throughput of the UE, reducing the transmission delay and balancing the link load, and the problem of single optimization target of the traditional algorithm is avoided.
5. A DRL model training process under an SDN scene is designed. The DRL model is described as observing system information and predicted traffic generation state vector s at each time step t t Generating action a according to self-strategy t In the execution of action a t Obtaining rewards r from the environment t At the same time transition to the next state s t+1 . Obtaining a four-element training sample<s t ,a t ,r t ,s t+1 >There is an experience pool. And after the flow task is completed, randomly extracting samples from the experience pool, and training the DRL model. The training process is shown in fig. 6.
(1) The DRL model obtains the source node, the destination node and the network state of the current task, namely u ori 、u des And the set of available bandwidths of each link b= { B 1 ,b 2 ,…,b k };
(2) The DRL model obtains the predicted value output by the LSTM module, namely b pre ;
(3) Forming an input vector s of the DRL model at the time t t =[u ori ,u des ,B,b pre ];
(4) Generating a target s t Action a of (2) t =π(s t ) I.e. the output best transmission link;
(5) Executing the action, obtaining the rewards r of the environmental feedback according to the formula (2) t Updating the available bandwidth set B of each link after data transmission and according to the source node u of the next task ori Destination node u des And predicted value b pre Together form the next state s t+1 。
(6) Construction of a four-tuple training sample<s t ,a t ,r t ,s t+1 >Saving to an experience pool;
(7) Judging whether the task is completed, if so, continuing to the next step, otherwise, returning to the step (1);
(8) DRL model training is performed by randomly sampling from an experience pool.
6. An adaptive classified flow scheduling process is designed. The invention is described that the flow data of the UE is utilized to classify and predict the flow, and the improvement of the performance indexes of the user level and the network level is realized through the flow scheduling. However, in the current environment, the number and types of UEs accessing the network may change, resulting in dynamic changes in network topology and traffic distribution. The invention can check the UE classification result, adjust the classification quantity and train a new LSTM flow predictor according to the changed environment or time interval, realize high-accuracy flow prediction, provide guarantee for flow scheduling and enhance the robustness of the system to the network environment with changed topology and equipment. The adaptive traffic scheduling flow is shown in fig. 7.
(1) starting a task, checking whether periodic UE classification checking is performed, if not, continuing the step (2), and if so, jumping to the step (3);
(2) Checking whether the network topology is changed, if so, continuing to step (3), and if not, jumping to step (9);
(3) Acquiring historical flow data of UE;
(4) Preprocessing data, including data compression of a time axis and a flow axis, and reducing data dimension so as to reduce calculation cost of classification steps;
(5) Extracting image features of flow data by using CNN;
(6) UE classification is carried out based on the image characteristics, and category number and a corresponding UE list are generated;
(7) Checking whether the classification result is updated or not, if so, continuing to step (8), and if not, jumping to step (9);
(8) Training LSTM prediction models of all categories according to the category number, the corresponding UE list and the historical flow data;
(9) According to the task information, obtaining predicted flow by using a current LSTM model;
(10) Acquiring network state information, i.e. u ori 、u des And a set of available bandwidths of each link B;
(11) Combining the predicted flow b pre Inputting a DRL scheduling module and outputting a traffic scheduling strategy;
(12) The task is completed.
According to the description, the SDN flow scheduling method based on machine learning classification prediction, which is designed by the invention, considers the problems that the traditional flow prediction model has low robustness and the flow scheduling method has hysteresis along with the change of the type and flow distribution of the UE, firstly designs a CNN-based UE classification module which performs dimension reduction treatment on historical flow data, reduces classification calculation cost, utilizes the flow image characteristics extracted by a sliding window and obtains a UE list of a corresponding type based on the flow characteristics; the LSTM-based classified flow prediction module trains the predictor by using the same type of flow data, and more accurate flow prediction can be realized because the same type of flow distribution is more similar; and then the predicted flow, the network state and the task information form a state vector, and a DRL scheduling module generates a scheduling strategy. Because the flow characteristics of the UE can be changed along with topology change and periodic inspection, even if new UE is added or flow distribution changes caused by UE task change, the UE with close flow distribution can be classified into the same class after reclassification, so that the prediction accuracy and the robustness of a prediction model are enhanced, and the scheduling module is supported to output a flow scheduling strategy which comprehensively considers user level and network level indexes. In addition, the invention designs a training flow of the DRL scheduling model and a self-adaptive classified flow scheduling flow with the change of network topology or the periodic checking of classification results.
1. The invention designs an SDN flow scheduling method based on machine learning classification prediction.
The method is deployed on an application plane of SDN, network state and UE flow data are collected by a controller, and the UE is classified according to flow characteristics, so that flow distribution among similar UEs is close, and the LSTM-based predictor can improve prediction accuracy; and the DRL scheduling module outputs a scheduling strategy considering user level and network level performance indexes according to the predicted flow, the network state and the task information, and achieves the effects of reducing transmission delay, improving throughput and balancing link load. The method is applicable to a new environment without any adjustment of the architecture even if the network topology changes, and has strong robustness.
2. The invention designs a CNN-based UE classification module. The module firstly carries out preprocessing on the historical flow data of the UE, including data compression on a time axis and a flow axis, reduces the calculation overhead and improves the classification efficiency. And extracting flow image features by utilizing the sliding window, and classifying the UE according to the image features. The category number and the UE list of each category obtained by the module are the pre-steps of the subsequent flow prediction.
3. The invention designs a classification flow prediction module based on LSTM. The module obtains a UE classification result, creates a prediction model for each category independently, and trains by using the historical flow data of the same type to obtain a predictor for a specific type. Because the flow distribution among the types is close through classified calculation, a more accurate flow prediction result can be realized to support subsequent flow scheduling according to prediction.
4. The invention designs a traffic scheduling module based on DRL. The module forms a state vector by utilizing a flow prediction result, a network state and task information, outputs a current flow transmission path according to the current state, and utilizes an SDN controller to issue a scheduling strategy to an SDN switch to execute data flow forwarding. The rewarding function of the module comprehensively considers indexes of the user level and the network level, and the rewarding function can guide the model to adjust the output strategy towards the direction of maximum rewarding, so that the effects of reducing transmission delay, improving throughput and balancing link load are achieved, and the problem that the optimization target of the traditional algorithm is single is solved.
5. The invention designs a training process of a DRL scheduling module. The flow describes the training process of the traffic scheduling module. The method comprises the steps of obtaining current task information, obtaining a network state from a data plane, forming a state vector by combining predicted traffic, and generating corresponding actions by using a DRL. The environment feeds back rewards according to the actions and shifts to the next state to finish a time step, and meanwhile, a training sample is generated and stored in the experience pool. After the task is completed, the module extracts samples from the experience pool to perform model training, so that the DRL scheduling module can learn autonomously to improve performance.
6. The invention designs a self-adaptive classified flow scheduling flow. The flow describes the process of the present invention for adaptively adjusting the UE classification results to support more accurate traffic prediction and more efficient traffic scheduling. Firstly judging whether periodic checking calculation is needed to be executed at present, if so, verifying whether the classification result is changed, if not, further judging whether the network topology is changed, if so, executing the classification calculation to retrain the LSTM prediction model, and if the topology is not changed, predicting the flow by using a current predictor. The flow ensures that under the conditions of topology change caused by the change of the number of the UE in the network or traffic distribution change caused by the change of the task of the UE, and the like, the invention triggers the corresponding mechanism to update the classification result of the UE, and ensures that the LSTM predictor can always predict the traffic of the same type with high accuracy so as to support the subsequent traffic scheduling operation.
Drawings
FIG. 1SDN architecture diagram
FIG. 2 is a schematic diagram of 3 traffic characteristics in a general network
FIG. 3 is a schematic diagram of SDN traffic scheduling method based on machine learning classification prediction
Fig. 4 is a schematic diagram of a UE classification module based on CNN
Fig. 5 CNN-based UE classification flow chart
FIG. 6DRL model training flow diagram
FIG. 7 is a flow chart for adaptive classified traffic scheduling
Detailed Description
With the rapid growth of the number of UE devices and network services, dynamic network environments present challenges for guaranteeing QoS for various types of traffic tasks. The cost of upgrading the network hardware equipment is high, and the performance improvement is limited. SDN is an open layered architecture that separates the data plane and the control plane, and flexible scheduling can be achieved through network programmable paths. The invention designs an SDN flow scheduling method based on machine learning classification prediction, which is deployed on an application plane and comprises a CNN-based UE classification module, an LSTM-based classification flow prediction module and a DRL-based flow scheduling module. The method comprises the steps of firstly judging whether periodical inspection is carried out and whether network topology is changed, firstly compressing a time axis and a flow axis of historical flow to 288 and 20-dimensional discrete values respectively by a UE classification module, constructing a flow graph with the shape of (20, 288), randomly taking 65% of training samples as training sets for model training in 4000 training samples in total, and taking 35% of samples as verification sets for evaluating classification model accuracy. And obtaining flow characteristics by utilizing the sliding window, and dividing the flow into 4 types according to the characteristics to obtain the category number and the UE list contained in each type. In experiments, the loss function is cross entropy, the learning rate is 0.001 by using an Adam optimization algorithm, the total iteration is 30000, and 64 samples are extracted from a training set for training by using one mini-batch for each iteration. When training was completed, the model loss was 0.736% and the classification accuracy was 97.89%. The classified traffic prediction module of LSTM creates separate prediction models for each type, model training using traffic in the same class. Since the flow distribution of the same type is close, each predictor can improve the prediction accuracy. In the invention, the flow is divided into 4 types, so that 4 LSTM models are constructed. Each LSTM model is trained using the corresponding type of first 24 hours of flow data, the shape of the input vector is (1, 288) to output the predicted value of the last 1 hour more conforming to the flow characteristics, and the shape of the output vector is (1, 12). In experiments, the LSTM model designed by the invention is trained by using an Adam optimization algorithm, the loss function is mean square error (Mean Squared Error, MSE), the learning rate is 0.001, and the total iteration is 5000 times. Upon completion of training, the model loss was 0.214% and the predicted mean absolute percentage error was 3.479%. And then, the DRL scheduling module acquires task information and network state data by utilizing a control plane, forms a state vector by combining the predicted flow, and outputs a corresponding action according to the current state vector, namely a transmission path aiming at the current task. The reward function of the module considers performance indexes of a user level and a network level, and the model adjusts parameters towards the direction of maximizing the reward function after training by utilizing a DRL algorithm autonomous learning mechanism, so that the task completion time is shortened as much as possible, the throughput is improved, and the network load balancing performance is ensured. In the experiment, the discount rate is set to 0.99, the iteration number is 10000, and the learning rate is 0.01 by using an Adam optimization algorithm. The scheduling strategy output by the DRL scheduling module has the performance improvement of improving throughput, reducing transmission delay and balancing link load, and achieves the effect expected to be achieved by the invention.
Claims (8)
1. An SDN flow scheduling method based on machine learning classification prediction is characterized in that:
under SDN architecture, an SDN flow scheduling method based on machine learning classification prediction is designed and deployed on an application plane, and the method comprises a CNN-based UE classification module, an LSTM-based classification flow prediction module and a DRL-based flow scheduling module, wherein the UE is classified by collecting network state data, flow task data and UE flow historical data information in real time and utilizing the relevance of historical flows; training corresponding prediction models for different types of UEs; user traffic is scheduled based on the predicted traffic.
2. The method according to claim 1, characterized in that a CNN-based UE classification module is designed;
firstly, compressing and preprocessing historical flow data, defining a Time Transmission Interval (TTI) on a time axis to be 5 minutes, and setting the flow data of 24 hours to be 288 TTIs on the time axis; on the traffic axis, define the maximum bandwidth b max The actual bandwidth at each instant is b i The compressed traffic in one TTI is
Square brackets in the middle are rounding functions; compressing the flow into discrete values of [0,20] by the formula; thus, each UE generates a traffic image size of (20,288) at 24 hours.
3. The method according to claim 1, characterized in that: a flow characteristic extraction model is designed, and the model structure is as follows:
the first layer is formed by sequentially sliding the flow images through 5 sliding windows with the sizes of (20, 20), and the output vector shape is (269,1,5);
the second layer is a pooling layer with pooling size (2, 1) being the maximum pooling layer for reducing the model size and parameter number, and the output vector shape is (134,1,5);
the third layer is a convolution layer with 10 convolution kernels of size (20, 1), the shape of the output vector being (115,1,10);
the fourth layer is the same as the second layer, and is a layer with a pooling size of (2, 1) as the largest pooling layer, and the shape of the output vector is (57,1,10)
The fifth layer is a flat layer, and after the flat operation of the flat layer is performed, the output shape is (570,1);
the sixth and seventh layers are two fully connected layers with 64 neurons and 3 neurons respectively, and access a softmax function to output a predicted UE class; in this model, the activation functions of the convolutional layers are all Relu.
4. The method of claim 1, wherein the CNN-based UE classification procedure is as follows:
(1) Initializing historical flow data in a system cache area;
(2) Preprocessing all flow data, and compressing the data from a time axis and a flow axis respectively; discrete compressing 24-hour flow history data into 288 TTIs, wherein the compressed flow value is obtained by a formula (1), a low-resolution flow image is generated, the dimension reduction of the flow data is realized, and the compressed flow data is a matrix with the shape of (20, 288);
(3) The UE classification module based on CNN is utilized to sweep the flow data, and the class corresponding to the UE is output according to the flow characteristics;
(4) Classifying all the UEs according to the class result output in the step (3);
(5) Outputting UE list under different categories according to the classification resultL UE,k ;
(6) Outputting a flow data list L corresponding to different categories tra,k 。
5. The method according to claim 1, wherein an LSTM based classification traffic prediction module is designed; considering that the flow distribution among the UEs of different types and different tasks can be greatly different, training a flow predictor for each class of UE independently;
each LSTM module acquires flow data of a corresponding category, and trains and predicts the flow generated by UE in the future 1 hour by utilizing the data of the previous 24 hours; the 24 hours duration is 288 TTIs, and the 1 hour is 12 TTIs; thus, the input of the model is an original traffic data matrix, the shape of the matrix is (1, 288), and each dimension comprises a TTI and a traffic value corresponding to the TTI; the number of LSTM units is set to 64, and the activation function is tanh; the output matrix shape of the module is (1, 12), i.e. the predicted flow value for the next 12 TTIs.
6. The method of claim 1 wherein a DRL-based traffic scheduling module is designed for planning a data transmission path based on the predicted traffic; assume that the UE set in the network is u= { U 1 ,u 2 ,…,u n Link set l= { L } 1 ,l 2 ,…,l k The set of available link bandwidths is b= { B } 1 ,b 2 ,…,b k The set of available data transmission paths between UEs is p= { P 1 ,p 2 ,…,p m -a }; assuming that the current scheduled task is from the initial node u ori To destination node u des Transmitting data; u (u) ori And u des Represented by one-hot code, i.e. u ori And u des The corresponding coding position value is 1, and the other position values are 0; the bandwidth required by the transmission data output by the prediction module is b pre The method comprises the steps of carrying out a first treatment on the surface of the The state s of the current time observed by the DRL module t =[u ori ,u des ,B,b pre ]The vector has dimensions 2n+k+1; the module comprises two hidden layers, each having 128 neurons, and excitingThe living function is Relu; the output of the module is action a performed by the DRL t =π(s t )=[p 1 ,p 2 ,…,p m ]The optimal transmission path of the current task is selected, the value of the selected transmission path is 1, the values of other paths are 0, and the vector dimension is m; the bonus function that comprehensively considers the user level and the network level is designed as follows:
wherein, alpha, beta and delta are weight parameters which are respectively 0.9, 0.5 and 0.6; sigma (sigma) n thr n Representing the sum of UE throughput, t strat And t end The first two parts are indexes for describing the user level;representing link load rate variance, +.>The smaller the variance value is, the more balanced the load among links is.
7. The method of claim 1, wherein a DRL model training procedure in an SDN scenario is designed; the DRL model is described as observing system information and predicted traffic generation state vector s at each time step t t Generating action a according to self-strategy t In the execution of action a t Obtaining rewards r from the environment t At the same time transition to the next state s t+1 The method comprises the steps of carrying out a first treatment on the surface of the Obtaining a four-element training sample<s t ,a t ,r t ,s t+1 >Exists in an experience pool; after completing the flow task, randomly extracting samples from the experience pool, and training a DRL model;
(1) The DRL model obtains the source node, the destination node and the network state of the current task, namely u ori 、u des And available bandwidth for each linkSet b= { B 1 ,b 2 ,…,b k };
(2) The DRL model obtains the predicted value output by the LSTM module, namely b pre ;
(3) Forming an input vector s of the DRL model at the time t t =[u ori ,u des ,B,b pre ];
(4) Generating a target s t Action a of (2) t =π(s t ) I.e. the output best transmission link;
(5) Executing the action, obtaining the rewards r of the environmental feedback according to the formula (2) t Updating the available bandwidth set B of each link after data transmission and according to the source node u of the next task ori Destination node u des And predicted value b pre Together form the next state s t+1 ;
(6) Construction of a four-tuple training sample<s t ,a t ,r t ,s t+1 >Saving to an experience pool;
(7) Judging whether the task is completed, if so, continuing to the next step, otherwise, returning to the step (1);
(8) DRL model training is performed by randomly sampling from an experience pool.
8. The method according to claim 1, wherein an adaptive classified traffic scheduling procedure is designed:
(1) starting a task, checking whether periodic UE classification checking is performed, if not, continuing the step (2), and if so, jumping to the step (3);
(2) Checking whether the network topology is changed, if so, continuing to step (3), and if not, jumping to step (9);
(3) Acquiring historical flow data of UE;
(4) Preprocessing data, including data compression of a time axis and a flow axis, and reducing data dimension so as to reduce calculation cost of classification steps;
(5) Extracting image features of flow data by using CNN;
(6) UE classification is carried out based on the image characteristics, and category number and a corresponding UE list are generated;
(7) Checking whether the classification result is updated or not, if so, continuing to step (8), and if not, jumping to step (9);
(8) Training LSTM prediction models of all categories according to the category number, the corresponding UE list and the historical flow data;
(9) According to the task information, obtaining predicted flow by using a current LSTM model;
(10) Acquiring network state information, i.e. u ori 、u des And a set of available bandwidths of each link B;
(11) Combining the predicted flow b pre Inputting a DRL scheduling module and outputting a traffic scheduling strategy;
(12) The task is completed.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310574479.9A CN116760718A (en) | 2023-05-21 | 2023-05-21 | SDN flow scheduling method based on machine learning classification prediction |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310574479.9A CN116760718A (en) | 2023-05-21 | 2023-05-21 | SDN flow scheduling method based on machine learning classification prediction |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116760718A true CN116760718A (en) | 2023-09-15 |
Family
ID=87954216
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310574479.9A Pending CN116760718A (en) | 2023-05-21 | 2023-05-21 | SDN flow scheduling method based on machine learning classification prediction |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116760718A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117609814A (en) * | 2024-01-24 | 2024-02-27 | 广东奥飞数据科技股份有限公司 | SD-WAN intelligent flow scheduling optimization method and system |
-
2023
- 2023-05-21 CN CN202310574479.9A patent/CN116760718A/en active Pending
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117609814A (en) * | 2024-01-24 | 2024-02-27 | 广东奥飞数据科技股份有限公司 | SD-WAN intelligent flow scheduling optimization method and system |
CN117609814B (en) * | 2024-01-24 | 2024-05-07 | 广东奥飞数据科技股份有限公司 | SD-WAN intelligent flow scheduling optimization method and system |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Qi et al. | Knowledge-driven service offloading decision for vehicular edge computing: A deep reinforcement learning approach | |
Van Huynh et al. | Optimal and fast real-time resource slicing with deep dueling neural networks | |
CN113158543B (en) | Intelligent prediction method for software defined network performance | |
Shi et al. | Machine learning for large-scale optimization in 6g wireless networks | |
CN111369042A (en) | Wireless service flow prediction method based on weighted federal learning | |
CN111628855B (en) | Industrial 5G dynamic multi-priority multi-access method based on deep reinforcement learning | |
CN111245718B (en) | Routing optimization method based on SDN context awareness | |
CN112202672A (en) | Network route forwarding method and system based on service quality requirement | |
CN114757352B (en) | Intelligent agent training method, cross-domain heterogeneous environment task scheduling method and related device | |
CN116760718A (en) | SDN flow scheduling method based on machine learning classification prediction | |
CN114710439B (en) | Network energy consumption and throughput joint optimization routing method based on deep reinforcement learning | |
CN116743635A (en) | Network prediction and regulation method and network regulation system | |
Xu et al. | Living with artificial intelligence: A paradigm shift toward future network traffic control | |
Sharara et al. | A recurrent neural network based approach for coordinating radio and computing resources allocation in cloud-ran | |
Sivakumar et al. | Prediction of traffic load in wireless network using time series model | |
Ergun et al. | A survey on how network simulators serve reinforcement learning in wireless networks | |
Bhattacharyya et al. | QFlow: A learning approach to high QoE video streaming at the wireless edge | |
Liu et al. | Scalable deep reinforcement learning-based online routing for multi-type service requirements | |
Wang et al. | Towards adaptive packet scheduler with deep-q reinforcement learning | |
Shaio et al. | A reinforcement learning approach to congestion control of high-speed multimedia networks | |
CN115935563A (en) | Network bandwidth prediction method and device based on graph neural network | |
Mohammadi et al. | Automatic delay-sensitive applications quality of service improvement with deep flows discrimination in software defined networks | |
CN115334002A (en) | AOS intelligent frame generation method combined with improved queue management algorithm under flow prediction | |
Rao et al. | A deep learning-based constrained intelligent routing method | |
Yan et al. | A Review of Routing Optimization Techniques for Quality of Service Assurance in Software-Defined Networks |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |