CN110730099A

CN110730099A - Flow prediction method based on historical flow data time sequence

Info

Publication number: CN110730099A
Application number: CN201910999685.8A
Authority: CN
Inventors: 周琨; 汪文勇; 唐勇; 黄鹂声; 张骏; 张文; 刘宝阳
Original assignee: University of Electronic Science and Technology of China
Current assignee: University of Electronic Science and Technology of China
Priority date: 2019-10-21
Filing date: 2019-10-21
Publication date: 2020-01-24
Anticipated expiration: 2039-10-21
Also published as: CN110730099B

Abstract

The invention belongs to the technical field of network traffic data monitoring, and discloses a traffic prediction method based on a historical traffic data time sequence.

Description

Flow prediction method based on historical flow data time sequence

Technical Field

The invention belongs to the technical field of network traffic data monitoring, and particularly relates to a traffic prediction method based on a historical traffic data time sequence.

Background

The continuous development of computer network technology has promoted the massive application of various network services, which has higher requirements on the performance and the service quality of the network. Network traffic prediction has attracted a great deal of researchers as one of important ways for network management and regulation, and it is also an important direction to predict future traffic based on time-series data of historical traffic.

The time series (or called dynamic number series) refers to a number series formed by arranging the numerical values of the same statistical index according to the occurrence time sequence. The main purpose of time series analysis is to predict the future based on existing historical data. Most of the economic data is given in time series. The time in the time series may be year, quarter, month or any other form of time depending on the time of observation.

A time series is a set of random variables ordered in time, which is typically the result of observing some potential process at a given sampling rate over equally spaced time periods. The time series data essentially reflects the trend of some random variable or random variables changing with time, and the core of the time series prediction method is to extract the law from the data and use the law to estimate the future data.

Time-series constituent elements: long-term trends, seasonal variations, cyclical variations, irregular variations, wherein:

1) the long-term tendency (T) phenomenon is an overall tendency to change over a long period of time due to a fundamental factor.

2) The phenomenon of seasonal variation (S) occurs regularly and periodically with the change of seasons over the course of a year.

3) The cyclic variation (C) phenomenon shows regular variations in the undulation pattern of the waves over a period of several years.

4) Irregular variation (I) is an irregularly following variation, and includes both types of strictly random variations and variations in which irregular bursts have a large effect.

Time series modeling and prediction can be roughly divided into traditional and deep learning-based methods, and traditional modeling such as a random process, an autoregressive process AR, a moving average process MA or an autoregressive moving average process ARMA is relatively mature in research. Although the existing deep learning technology such as the LSTM technology obtains better accuracy in the field of sequence prediction than the traditional model, the complexity of the existing deep learning technology is generally higher than that of the traditional model, a deep learning neural network needs to be designed, some simple parameters of the traditional model can be calculated and obtained even in excel office software, and the required computing resources are small; and deep learning models generally lack interpretability, such as input layer and hidden layer node number design, hyper-parameter design and the like.

Disclosure of Invention

The invention provides a solution for flow prediction, which designs a method for flow prediction according to a computer network historical flow time sequence by analyzing network flow characteristics and comprehensively considering the requirements of prediction accuracy, complexity, optimized maintenance and the like.

The invention discloses a flow prediction method based on a historical flow data time sequence, which is characterized by comprising the following steps of:

acquiring and processing historical flow data, namely dividing the time length to be predicted of flow to be predicted into a plurality of continuous prediction time periods as a specified time scale; obtaining network historical traffic data, dividing the time length corresponding to the network historical traffic data into a plurality of continuous historical time periods according to a specified time scale, sorting and correspondingly collecting the network historical traffic data into the corresponding historical time periods according to respective time, and calculating to obtain a network historical traffic time sequence of the network historical traffic data under the specified time scale;

a real-time traffic data acquisition and processing step, namely acquiring real-time network traffic data through an SNMP technology, preprocessing the real-time network traffic data, and collecting the real-time network traffic data according to the acquired real-time network traffic data time to form a real-time network traffic data time sequence under the specified time scale in the historical traffic data acquisition and processing step;

and analyzing and modeling, namely performing characteristic analysis on the network historical traffic time sequence in the historical traffic data acquisition and processing step by adopting a deep learning method to establish a model base containing time-interval traffic data, modeling by using an ARIMA (autoregressive integrated moving average) model to obtain a time sequence prediction model, and predicting to obtain a predicted traffic result according to the characteristics of the real-time network traffic data time sequence data in the real-time traffic data acquisition and processing step.

The simple network management protocol SNMP is a network management standard that includes an application layer protocol (application layer protocol), a database schema (database schema), and data objects. The method is used for collecting and organizing various information such as the state and performance of the managed equipment (routers, switches, servers and the like) of the IP network, and is widely applied to network management and monitoring. The basic components include Network-management systems (Network-management systems), managed devices (managed devices), and agents (agents), which are Network management software modules in the managed devices. The managed device collects and stores management information through a Management Information Base (MIB), and the network management system acquires the information through an agent.

According to the technical scheme, flow index data of a network related interface needs to be acquired, data on the designated interface of the router is selected, and data such as bit number Bps or byte number Bps or packet number pps is transmitted to a network management system at regular time (for example, every 5 minutes) when the data is transmitted and received every second. The router and the network management system are configured to enable the SNMP protocol, and set related parameters: the managed device is configured with network management system IP address, protocol port UDP161, 162, etc. for the community character string required by SNMPv2 version.

The ARIMA model comprehensively considers autoregressive AR, a moving average MA process and differential processing, and is more suitable for time series modeling and prediction. Usually, the sequence has non-stationarity (the stationarity needs to meet the condition, and the general white gaussian noise meets the stationarity requirement), but the non-stationarity characteristic can be removed after the difference processing. The AR part indicates that the variable is at time T_nAnd the first i times T_n-iAutoregressive (ARMA model, left of equation 1, etc.), MA part shows that the regression error is a linear combination of the previous errors (right of equation 1, etc.).

Specifically, the differential processing of the ARMA model:

X_t-iis replaced by it1 order difference X_t-i-X_t-i-1The 2-step difference is: (X)_t-i-X_t-i-1)-(X_t-i-1-X_t-i-2)＝X_t-i-2X_t-i-1+X_t-i-2

The n-order difference is analogized in the same way;

X_t-iis replaced by its 1 st order difference X_t-i-X_t-i-1The 2-step difference is: (X)_t-i-X_t-i-1)-(X_t-i-1-X_t-i-2)＝X_t-i-2X_t-i-1+X_t-i-2

) I.e. an ARIMA model, as shown in the following formula 2, the time difference operator L: l isⁱX_t＝X_t-i。

X_t-α₁X_t-1-…α_pX_t-p＝ε_t+θ₁ε_t-1…+θ_qε_t-q(1)

ε_tObeying a normal distribution of epsilon_t～N(0，σ²) And is denoted as ARIMA (p, d, q). Training data is 'calculated', and the (p, d, q) with the minimum error loss under a certain standard (AIC, BIC and the like) is selected. The model has various variations to model the characteristics of the data more accurately, and equation 2 models non-seasonal data. Whether seasonality exists in the data or not is determined by analyzing specific data: if the sales income is different from season to season, but there are similarities between the 4 seasons of the previous year and the current year (the 1 st quarter of the previous year and the 1 st quarter of the current year), it indicates that there may be seasonality, and the seasonality does not necessarily refer to four seasons of the year. The patent only gives a modeling process without seasonal data, and the seasonal data modeling can be referred to the method. The model ARIMA (1, 0, 0) p is 1, d is 0, and q is 0, that is: AR (1), ARIMA (0, 0, 1) in the same manner, is MA (1). The research on the model is mature, parameters (p, d and q) in the ARIMA model are determined according to selection criteria, and a Chichi information content criterion (AIC) and a Bayesian Information Criterion (BIC) are selected for evaluating modeling results.

In the step of collecting and processing the historical flow data, the historical flow data of the network is obtained by a flow monitoring function module of commercial and/or open source network management software through an SNMP technology. The business software or the open source software can acquire the flow data of the specified network interface and display the flow data by the flow graph, so that the network flow rule can be conveniently found. Preferably, the network traffic monitoring graphical analysis tool is developed based on PHP, MySQL, SNMP and RRDTool, such as Cacti software.

The sorting of the network historical traffic data comprises observing traffic similarity, periodicity, traffic trend and stability of the network historical traffic data. These rules are used for subsequent modeling.

The real-time network flow data is obtained by the SNMP technology, and SNMP is configured on network equipment and/or a network management server to collect flow data of a specified interface and is sent to the network management server at regular time. The flow data can be processed into time series by network management software, for example, if the flow data is every 5 minutes and the predicted time scale is 24 hours, then 12 data of 5 minutes need to be preprocessed, and the processing method can be simple to sum and take an average value, or take a representative value to represent data of 1 hour, etc.

And in the step of acquiring and processing the real-time flow data, forming a real-time network flow data time sequence and storing the real-time network flow data time sequence as a csv format file for storage. The csv format file facilitates the use of python processing.

The method of deep learning in the analyzing and modeling step includes plotting the network historical traffic time series to determine if the time series is steady, has a trend of rising or falling, and its periodicity. The steady-state process generally can use a random process model, a sequence with a certain trend does not generally belong to the steady-state model, but can be a stable process through differential processing, and the characteristics of the periodicity, the trend and the like of the sequence need to be designed during modeling.

The method of deep learning in the analyzing and modeling step comprises

Step 1, analyzing network flow data characteristics, finding a flow peak and periodicity, and carrying out d-order differential operation on the similarity and the trend of two continuous periodic data before and after analysis to form a stable time sequence;

and 2, solving an Autocorrelation Coefficient (ACF) and a Partial Autocorrelation Coefficient (PACF) of the stationary time sequence, and drawing an ACF and PACF graph of the d-order difference.

The trend includes non-zero mean, slope, deterministic trend, random trend, and combinations thereof.

For example: the flow between 9-23 points is obviously larger than other time periods, flow peaks are formed near 10-11 points and 14-16 points to form periodic characteristics, the data in the current week are similar to the data in the previous week, the trend is not obvious, and the characteristics of a stable time sequence are not provided, so that d-order difference operation can be performed firstly to form the stable time sequence;

and drawing a time series scatter diagram by taking the flow data in the morning of a certain day as an example, and observing the stability. There is an insignificant trend, considering the 1 st order difference;

the parameter d is the difference order, and in order to make the variance steady, the general experience sets d to 1, 2 can draw a difference graph from which the steady state of the sequence is observed.

If d is set to be 1, the sequence is stable after 1-step differentiation; solving an Autocorrelation Coefficient (ACF) and a Partial Autocorrelation Coefficient (PACF) of the stationary time sequence, and drawing 1-order difference ACF and PACF graphs;

generally the PACF of the stationary sequence is truncated and the ACF is trailing, applicable to the AR model; if the PACF of the stable sequence is trailing, and the ACF is truncated, the MA model is suitable; if both the PACF and ACF of the plateau sequence are tailing, the ARMA model is suitable. All the function values are 0 after the lag period k is greater than q, and the result is called truncation; as k increases, the function value exhibits an exponential or oscillatory decay, tending to 0, called smearing. The above figure can be fit to the ARMA model with the difference d equal to 1, i.e.: ARIMA (p, 1, q).

ACF shows that there are two steps of hysteresis beyond the confidence boundary (the first line is the starting point, not within the hysteresis range); PACF shows that lag 1 to 2 exceeds the confidence bound, and from the reduction of the partial autocorrelation values to 0 after lag 2, p is initially set to 2 and q is set to 1, i.e., ARIMA (2, 1, 1).

Compared with the prior art, the invention has the following advantages:

the method clearly describes the steps of analyzing network traffic characteristics, comprehensively considering requirements, designing a traffic model and related indexes and the like, and explains the modeling process by taking campus network traffic modeling prediction as an example. Other flow model designs can refer to the thought of the patent, and have strong practicability.

The deep learning technology represented by the LSTM is superior to the traditional time series model in the aspect of time series prediction, but the ARIMA model is designed and used by comprehensively considering factors such as network structure, flow characteristics, accuracy, complexity and the like. Compared with the deep learning model LSTM and the like, the method has the advantages of low time complexity, less required data than the deep learning model, simpler maintenance and debugging of the linear model, less time required for training and better interpretability.

Drawings

The foregoing and following detailed description of the invention will be apparent when read in conjunction with the following drawings, in which:

FIG. 1 is a schematic view of a CaCt flow monitoring;

FIG. 2 is a schematic of a time series scatter plot;

FIG. 3 is a graph of autocorrelation and partial correlation in time series;

FIG. 4 is a time series prediction diagram.

Detailed Description

The technical solutions for achieving the objects of the present invention are further illustrated by the following specific examples, and it should be noted that the technical solutions claimed in the present invention include, but are not limited to, the following examples.

Example 1

As a most basic implementation of the present invention, the method for predicting flow based on the time series of the historical flow data disclosed in this embodiment includes the following steps:

acquiring and processing historical flow data, namely dividing the time length to be predicted of flow to be predicted into a plurality of continuous prediction time periods as a specified time scale; obtaining network historical traffic data, dividing the time length corresponding to the network historical traffic data into a plurality of continuous historical time periods according to a specified time scale, sorting and correspondingly collecting the network historical traffic data into the corresponding historical time periods according to respective time, and calculating to obtain a network historical traffic time sequence of the network historical traffic data under the specified time scale; the network traffic characteristic analysis and design are used for preliminary qualitative analysis of historical network traffic data, the rules of the historical network traffic data are found, and the time scale of the traffic to be predicted is combined, wherein the specific time scale depends on related applications, for example, some applications are concerned about the traffic of 5 minutes, 1 hour or 1 day, 1 week and the like in the future. The time scales of the acquired flow data are different according to different requirements. Predicting 1 hour in the future, and taking the time scale of historical data every 5 minutes or even 1 minute; half an hour or 1 hour can be taken for 1 day in the future. However, the scheme behind the time scale is not good in landing, so that the flow of 24 hours in the future is supposed to be predicted subsequently, and the time scale is taken to be 1 hour for guiding the subsequent model design.

A real-time traffic data acquisition and processing step, namely acquiring real-time network traffic data through an SNMP technology, preprocessing the real-time network traffic data, and collecting the real-time network traffic data according to the acquired real-time network traffic data time to form a real-time network traffic data time sequence under the specified time scale in the historical traffic data acquisition and processing step; and acquiring and preprocessing flow data, acquiring the flow data by adopting an SNMP technology according to an analysis result of the historical flow data acquisition and processing step, preprocessing the data, and performing subsequent preprocessing mainly aiming at forming a network historical flow time sequence under a specified time scale. SNMP is configured on managed network equipment and a network management server to collect flow data of a specified interface and send the flow data to the network management server at regular time. The flow data can be processed into a time sequence by network management software, for example, the flow data is every 5 minutes, the predicted time scale is 24 hours, and 12 pieces of 5-minute data need to be preprocessed: the average can be simply summed or a representative value can be taken to represent 1 hour of data.

And analyzing and modeling, namely performing characteristic analysis on the network historical traffic time sequence in the historical traffic data acquisition and processing step by adopting a deep learning method to establish a model base containing time-interval traffic data, modeling by using an ARIMA (autoregressive integrated moving average) model to obtain a time sequence prediction model, and predicting to obtain a predicted traffic result according to the characteristics of the real-time network traffic data time sequence data in the real-time traffic data acquisition and processing step. The accuracy and the complexity are comprehensively considered, the method of the neural network such as the deep learning at present, particularly the LSTM introduced in the background technology obtains good results in the time series prediction field and exceeds the classic time series prediction method in the accuracy aspect.

The method comprises the steps of determining a flow model and important parameters thereof by using data acquired in the real-time flow data acquisition and processing step, evaluating the prediction accuracy of the model, and optimizing model parameters according to indexes (since the traditional time series models used for various types such as AR, MA, ARMA, ARIMA, SARIMA and the like have various types, which models need to be adopted according to specific network flow characteristics, after specific network flow characteristics are analyzed, the ARIMA model is proposed to be used in the scheme, so that the method is particularly designed for parameters (p, d, q) of the ARIMA model and judgment indexes (also has various types of judgment indexes), and the optimal parameter combination is selected to minimize the judgment indexes AIC and BIC.step 3 is to determine to use the ARIMA model according to the time series characteristics, and step 4 is to further determine the important parameters p, d, q of the model).

Due to different network structures, scales and the like, network traffic data may be very different and determine the design of a network traffic model. For the purpose of describing the design process, given a campus network traffic prediction as an example, the method may be referred to for the design of other network traffic models.

Example 2

As a preferred implementation of the present invention, on the basis of example 1, in the step of collecting and processing historical traffic data, the historical traffic data of the network is obtained by a traffic monitoring function module of business and/or open source network management software through SNMP technology. The business software or the open source software can acquire the flow data of the specified network interface and display the flow data by the flow graph, so that the network flow rule can be conveniently found. Preferably Cacti software, a set of network traffic monitoring graphical analysis tools developed based on PHP, MySQL, SNMP and RRDTool, as shown in fig. 1.

The method of deep learning in the analyzing and modeling step comprises

Example 3

In another preferred embodiment of the present invention, the analysis and design of network traffic characteristics refers to finding out the rules thereof through analyzing the historical data of network traffic, and combining the time scale of the traffic to be predicted to guide the subsequent model design. The flow rule comprises the characteristics of flow similarity, periodicity, trend, stability and the like. And installing open source software cacti to monitor the network flow. The network flow rule of the enterprise network/campus network is found by observing historical data: monday to Friday are similar, weekend and holiday are less than weekday flow, this week is similar to the last week, etc. The flow time scale is to predict the flow at a certain time point in a day, a given day in a week, a week and a month, and influences the design of the subsequent data acquisition frequency. For example, the collection can be carried out once every 10 minutes for a certain time point in the day; one point-in-time data may be obtained hourly for a given day of the week. These rules and time scale requirements, among others, determine the next step of data acquisition and preprocessing. The scheme is designed on the premise of outlet flow prediction of a given day in a campus/enterprise network week.

The flow data acquisition and preprocessing refers to designing how to acquire and acquire what flow data according to the data analysis result of the previous step, and includes steps of designing, configuring and preprocessing, wherein the designing is for data acquisition and preprocessing and does not include model designing. The purpose is to explain how and what traffic data is acquired for modeling, i.e., method and content. The method for acquiring the flow is to collect the flow data of the specified interface every 5 minutes and send the flow data to the network management server at regular time through SNMP; the data content is the traffic of the interface every 5 minutes. The purpose of the pre-processing is to form a satisfactory time series as model input.

The method comprises the following specific design: acquiring 24-hour network aggregation node flow data from a network switch or a router at regular time (every 5 minutes) by using an SNMP (simple network management protocol) technology, wherein a data acquisition point is arranged at a corresponding convergent point interface; the data for each 5 minutes were averaged over the hour period to obtain hourly flow data, 24 data points a day.

Configuration: the router and the network management system are configured with an SNMP protocol, and relevant parameters are set: for example, the community character string required by the SNMPv2 version, the managed device is configured with the IP address of the network management system, the protocol ports UDP161, 162, and the like.

Pretreatment: converting the data points into time series data X_tThe storage as the csv file format is considered for generality.

And then, designing a flow model, namely further analyzing the acquired data, wherein the data is the data preprocessed by the flow data acquisition and preprocessing steps. Further analysis of the data is critical to modeling because it determines what model to use (e.g., assuming the sequence is steady-state, the parameter d can be set to 0, i.e., no difference can be made). The features of the data are modeled and designed by plotting the data and empirical rules as described below. The scheme is designed by modeling when the data falls to the campus network, and the modeling is carried out through the following specific steps 1) -4), the data can be plotted, and the characteristics of the time sequence are judged: such as whether the sequence is steady, has some tendency (up, down), and periodic (this week is similar to the previous week). The steady-state process generally can use a random process model, a sequence with a certain trend does not generally belong to the steady-state model, but can be a stable process through differential processing, and the characteristics of the periodicity, the trend and the like of the sequence need to be designed during modeling. The method specifically comprises the following steps:

taking campus network traffic data as an example:

1) summarizing characteristics of network traffic data of the campus network: the flow between 9-23 points is obviously larger than other time periods, flow peaks are formed near 10-11 points and 14-16 points to form periodic characteristics, the data in the current week is similar to the data in the previous week, the trend is not obvious, the characteristics of a stable time sequence are not provided, d-order difference operation can be firstly carried out, and the data are converted into the stable time sequence

Whether the general body is in an ascending or descending trend is observed through a sequence diagram. Sequence smoothingMeaning that its random characteristics do not change over time. Mathematically: random sequence { X_nN is equal to or greater than 0, is stationary, the joint distribution function of its random variables is F (X)₁，X₂，…，X_m)＝F(X_1+t，X_2+t，…，X_m+t) (ii) a (m is more than or equal to 2), the distribution of the sequence at the time of translation t is kept unchanged, which shows that the joint distribution function does not change along with the time. Certain subjective judgment and experience rules can be adopted in judging the sequence stability, and the ACF and PACF graphs are combined from the sequence graph for judgment: if the visual observation shows that the visual observation has a certain ascending or descending trend, the visual observation is not stable. If the difference is not stable, performing difference processing (see the following steps): and judging whether the first-order difference is stable or not, and judging whether the second-order difference is stable or not until the second-order difference is stable. But generally exceeding the second order and still not being stable, the sequence should be considered to be processed, such as taking the logarithm and then differentiating, etc. The standard methods for judging stability include unit root test statistical methods such as ADF test, PP test, KPSS test and the like, and the R/Python language has a related software library to implement, which is not described herein.

The trend has a non-zero mean, slope, deterministic trend, random trend, and combinations thereof.

2) And drawing a time series scatter diagram by taking the flow data in the morning of a certain day as an example, and observing the stability. There is an insignificant trend, considering the 1 st order difference.

3) The parameter d is the difference order in order to make the variance steady. General experience sets d to 1, 2 can draw difference maps from which the stability of the sequence is observed. The scheme sets d to 1, namely the sequence is stable after 1-step difference.

4) And solving an Autocorrelation Coefficient (ACF) and a Partial Autocorrelation Coefficient (PACF) of the stationary time sequence, and analyzing and judging the model to be suitable for the AR, MA or ARMA model by drawing 1-order difference ACF and PACF graphs.

Generally, the PACF of a stationary sequence is truncated, the ACF is trailing, and an AR model is suitable (in an ARIMA model, a parameter p is not equal to 0, and a parameter d is q is 0, namely an Autoregressive (AR) model, and the right side of a medium number in formula 1 is 0. the model can be used for flow modeling prediction, whether the model is selected to be used depends on the characteristics of the sequence, and if the PACF is truncated and the ACF is trailing, the model can be considered to be used; if the PACF of the stationary sequence is trailing and the ACF is truncated, the MA model is applied (in the ARIMA model, the parameter q ≠ 0, and d ═ p ═ 0 is the Moving Average (MA) model, and the left side α 1Xt-1-. α pXt-p in equation 1 is 0. the model can also be used for traffic modeling prediction, whether to choose to use the model depends on the characteristics of the sequence, and if the PACF trailing and ACF truncation can consider using the model, the ARIMA model is finally used in the present scheme); if both the PACF and ACF of the plateau sequence are tailing, the ARMA model is suitable. All the function values are 0 after the lag period k is greater than q, and the result is called truncation; as k increases, the function value exhibits an exponential or oscillatory decay, tending to 0, called smearing. The above figure can be fit to the ARMA model with the difference d equal to 1, i.e.: ARIMA (p, 1, q).

The AR model and the MA model are special forms of ARMA models, the ARMA can be regarded as the synthesis of AR and MA, and the ARMA model is ARIMA according to d-order difference.

Then modeling, determining a flow model, for example, analyzing the characteristics of the flow data time series, corresponding to the steps 1) to 4) above, and finally determining and selecting an ARIMA model, and important parameters (p, d, q are important parameters of the ARIMA, and the determination of the values: and selecting p, d and q corresponding to the minimum evaluation index AIC and the BIC value. And 4, determining parameters p, d and q of the ARIMA model, wherein a scheme original text has a comparative example), evaluating the prediction accuracy of the model, and optimizing the model parameters according to indexes. And selecting the model with the minimum AIC and BIC evaluation indexes.

P, d, q are preliminarily determined through step 3, and a combination of several values can be selected to finally determine p, d, q according to the minimum value of AIC/BIC. And (3) selecting two groups of values (2, 1, 1) and (2, 1, 2) to carry out AIC and BIC index comparison, and finally determining p, d and q. At present, some software can automatically determine parameter values according to indexes, and the scheme only utilizes the self-carrying function of excel office software to carry out calculation.

Example 4

and step S1, analyzing and designing network flow characteristics, carrying out preliminary qualitative analysis on network flow historical data, finding out rules of the historical data, and combining the time scale of the flow to be predicted to guide subsequent model design.

And step S2, acquiring and preprocessing flow data, acquiring the flow data by adopting the SNMP technology according to the analysis result in the step 1, and preprocessing the data.

And S3, designing a flow model, comprehensively considering the requirements of accuracy, complexity, maintainability, optimization and the like, and designing a proper flow model and evaluation indexes.

And step S4, modeling and predicting, namely determining a flow model and important parameters thereof by using the data acquired in the step 2, evaluating the prediction accuracy of the model, and optimizing the model parameters aiming at indexes.

Step S1 includes the following steps:

and step S11, obtaining network historical flow data by using a flow monitoring function module of network management software. The business software or the open source software can acquire the flow data of the specified network interface and display the flow data by the flow graph, so that the network flow rule can be conveniently found. In the scheme, cacti software (a set of network traffic monitoring graphical analysis tools developed based on PHP, MySQL, SNMP and RRDTool) is used for obtaining the network traffic monitoring graphical analysis tool. The schematic diagram is shown in figure 1.

And step S12, observing historical network flow rules, such as flow similarity, periodicity, trend, stability and other characteristics, and using the historical network flow rules for subsequent modeling. And (3) designing the flow collection frequency in the step (2) in combination with the scale of the data to be predicted.

Step S2 includes the following steps:

step S21, data acquisition, configuring and starting SNMP protocol on the router and the network management system, and setting related parameters: for example, the community character string required by the SNMPv2 version, the managed device is configured with the IP address of the network management system, the protocol ports UDP161, 162, and the like. And checking that the network management server can receive the traffic data sent by the router.

Step S22, preprocessing, converting the data points into time series X_t(see table below), data taken every 5 minutes, 12 data averaged over 1 hour, or processed according to specific needs, taking into account the data scale. Consider the commonality save as the.csv file format.

Step S3 includes the following steps:

step S31, summarizing characteristics of the campus network traffic data: the flow between points 9-23 is obviously larger than other time periods, flow peaks are formed near points 10-11 and points 14-16 to form periodic characteristics, the data in the current week are similar to the data in the previous week, the trend is not obvious, the characteristics of a stable time sequence are not provided, and d-order difference operation can be firstly carried out to be converted into the stable time sequence. The trend has a non-zero mean, slope, deterministic/random trend, and combinations thereof.

Step S32, a time series scatter diagram (see fig. 2) is drawn by taking the flow data at morning of a certain day as an example, and the stability is observed. There is an insignificant trend, considering the 1 st order difference.

In step S33, the parameter d is the difference order in order to stabilize the variance. General experience sets d to 1, 2 can draw difference maps from which the stability of the sequence is observed. The scheme sets d to 1, namely the sequence is stable after 1-step difference.

Step S34, the stationary time series is evaluated for its Autocorrelation Coefficient (ACF) and Partial Autocorrelation Coefficient (PACF), and the ACF and PACF maps (see fig. 3) with 1-order difference are drawn to analyze and determine that the model is suitable for the AR, MA, or ARMA model.

Generally the PACF of the stationary sequence is truncated and the ACF is trailing, applicable to the AR model; if the PACF of the stable sequence is trailing, and the ACF is truncated, the MA model is suitable; if both the PACF and ACF of the plateau sequence are tailing, the ARMA model is suitable. The function values are all 0 after the lag period k is more than q, and the result is called truncation; as k increases, the function value exhibits an exponential or oscillatory decay, tending to 0, called smearing. The above figure can be fit to the ARMA model with the difference d equal to 1, i.e.: ARIMA (p, 1, q).

Step S4 includes the following steps:

and step S41, selecting a combination of several values, and selecting the minimum AIC and BIC values according to the evaluation criteria to finally determine p, d and q. And (3) selecting two groups of values (2, 1, 1) and (2, 1, 2) to carry out AIC and BIC index comparison, and finally determining p, d and q. It can be seen from the table that the parameters (2, 1, 1) are superior to (2, 1, 2).

Key index comparison table

At present, some software can automatically determine parameter values according to indexes, and according to the scheme, SSE, sqrt mse (rmse), AIC and BIC calculation is carried out by utilizing the self-carrying function of excel office software.

The ith value prediction error:

AIC＝n*LN(SSE/n)+2*(p+q+2)

BIC＝n*LN(SSE/n)+LN(n)*(p+q+2)

wherein:

is the predicted value, the ith value prediction error: e.g. of the type_iN is the sequence length

After the parameters are determined in step S42, prediction is performed using the model. The scheme predicts the flow data of the next 5 points according to the historical flow data. In FIG. 4, the solid curve is historical data and the back dotted curve is plotted against the predicted data.

Model ARIMA (2, 1, 1): equation 2 can be calculated using the python language statmodels libraryAnd theta_iAnd the calculation result is as follows:

θ₁＝-1.1438(1-0.445822L¹-0.56124L²)(1-L)X_t＝(1-1.1438L¹)ε_tε_t～N(0，σ²)。

Claims

1. the flow prediction method based on the historical flow data time sequence is characterized by comprising the following steps of:

2. The method of traffic prediction based on a time series of historical traffic data according to claim 1, characterized by: in the step of collecting and processing the historical flow data, the historical flow data of the network is obtained by a flow monitoring function module of commercial and/or open source network management software through an SNMP technology.

3. A method of traffic prediction based on a time series of historical traffic data according to claim 1 or 2, characterized in that: the sorting of the network historical traffic data comprises observing traffic similarity, periodicity, traffic trend and stability of the network historical traffic data.

4. The method of traffic prediction based on a time series of historical traffic data according to claim 1, characterized by: the real-time network flow data is obtained by the SNMP technology, and SNMP is configured on network equipment and/or a network management server to collect flow data of a specified interface and is sent to the network management server at regular time.

5. The method of traffic prediction based on a time series of historical traffic data according to claim 1, characterized by: and in the step of acquiring and processing the real-time flow data, forming a real-time network flow data time sequence and storing the real-time network flow data time sequence as a csv format file for storage.

6. The method of traffic prediction based on a time series of historical traffic data according to claim 1, characterized by: the method of deep learning in the analyzing and modeling step includes plotting the network historical traffic time series to determine if the time series is steady, has a trend of rising or falling, and its periodicity.

7. The method of traffic prediction based on a time series of historical traffic data according to claim 1, characterized by: the method of deep learning in the analyzing and modeling step comprises

8. The method of historical flow data time series based flow prediction as claimed in claim 7 wherein: the trend includes non-zero mean, slope, deterministic trend, random trend, and combinations thereof.