LT5869B - Analysis method of computer network traffic in real time - Google Patents

Abstract

The invention relates to the digital information systems for monitoring and testing of computer networks using packet data transmission and network traffic analysis for real time. Analysis method of computer network traffic in real-time allows to select the network traffic aggregation method and parameters, and is characterized by fault – tolerance and high speed computing. The invention is characterized that at real time is fixed network traffic, are formed time series, which are aggregated using moving average smoothing or method of sum of transfer dates flow by time interval. Line, which is stored in a temporary memory, constantly supplemented with new members. Singularity of line introduces by recuperating method using robastic method of empirical quintile regression, IR statistic, R/S statistic. For evaluation of singularity network traffic is proposed to calculate ? stability index, IR estimation and Hurst coefficient.

Description

BACKGROUND OF THE INVENTION The present invention relates to systems for monitoring and testing digital information when computer networks use packet data transmission for real-time network traffic analysis.

When transmitting information over computer networks, it is important that the data packets sent by the sender reach the recipient successfully. Each user on a computer network is provided with the service immediately upon request, which often results in unexpected network congestion in individual network segments resulting in data loss on the network. In order to ensure the reliability of data transmission, it is recommended to forecast congestion of network segments in the computer network and to take steps to reduce the loss of transmitted information.

There is a known real-time analysis of data flow creep analysis (see US 5,343,465), which captures the time of data packet entry to form time series for which descriptive statistics parameters are calculated in real time: coefficient of variation, correlation coefficient, dispersion coefficient. The dispersion coefficient characterizes the flow creep. This technical solution does not provide estimates of the specificity of the flow.

In another patent (see patent WO1999040703), a real-time network waveform analysis is used for real-time network traffic analysis. Disadvantage - The present invention employs only one flow analysis and one aggregation method.

There is a known patent (see EP 1983687) which proposes to measure the degree of self-property by dynamically calculating the flow Hurst coefficient. The first coefficient is calculated from the first time series formed in the computation module from the real data sequence, while the others are obtained by adding a new member to the current one and calculating the new Hurst coefficient from the newly obtained time series starting with the second member. The calculations use R / S statistics, methods for analysis of time variance, residual dispersion, time absolute moments, peak vvhittles, signal wave estimates, embedded branching, immature fluctuation. The present invention does not perform string aggregation.

Also known is a network traffic analyzer (see EP1780955), which generates a 1000-member time series for network traffic analysis, which is then supplemented with new members while maintaining the same line length. The row is formed in a buffer that calculates R / S statistics, time dispersion, residual dispersion, time absolute moments, maximum likelihood of narrowing, signal waves, embedded branching and immature fluctuation analysis methods. Results are classified and stored in the output buffer.

A disadvantage of the present invention is that several different aggregation methods cannot be used for the measurements, and the Hurst coefficients are not stored for further analysis.

The object of the present invention is to assist in the assessment of the possibility of repetition of network congestion and to make a decision on the limitation of minor traffic components or the replacement of hardware. The proposed method allows choosing the method and parameters of network traffic aggregation, it is characterized by error resistance and high computing speed.

This objective is achieved by:

real-time aggregation of network traffic using the smoothing of moving averages method or the sum of transmitted traffic over time;

caches the formed string in cache memory;

recursively calculates dynamically formed string self-response parameters using robust empirical quantile regression, IR statistics and R / S statistics.

storing the calculation results in a cache memory and / or in a read only memory for further analysis; it is suggested to calculate a stability index, IR estimate and Hurst coefficient.

The essence of the invention is explained in the drawings:

FIG. 1 is a flowchart for realizing a proposed real-time analysis of computer network traffic;

FIG. 2 - realization algorithm for real-time analysis of computer network traffic.

A block diagram of a device according to which a real-time analysis of computer network traffic 25 can be implemented is shown in FIG. It consists of sequentially interconnected network traffic aggregation, analysis criteria and options entry module 1, packet scanning module 2, network packet header processing module 3, time series aggregation and real time module 4, time series cache 5, time series identity analysis module 6, Computation Output Module 7, and Computation Output Repository 8.

The calculation result generation module 7 is additionally connected to the calculation result viewer interface 9.

At the beginning of network traffic analysis, in the network traffic aggregation and input criteria and options module 1, select one of the data traffic aggregation methods, select flow analysis methods, specify the length of the line to be formed, select the time series aggregation interval and indicate whether the results should be stored. This module relates to packet scanning module 2, which reads data packets from the physical medium of a computer network. The network packet header processing module 3 separates the information used to form an aggregate string from the packet header. According to the method of aggregating using moving averages selected in the Input Module 1 or the sum of the transmitted data stream over the time interval and the aggregation interval in the real time aggregation and shaping module 4, the time series is formed. The formed time series is stored in the cache 5. After the first row is formed, each subsequent row in this repository will be recursively formed. The aggregated time series is then transmitted to the self-analysis module (6), which calculates the stability index, IR estimate and Hurst coefficient according to the chosen analysis methods (robust empirical quantile regression, IR statistics and / or R / S statistics). The obtained results are transmitted to the computation result generation module 7, where they are redirected to the computation result viewer interface 9. The results can be presented in numerical or graphical form.

If the user has chosen to store the calculation results, they can be stored in the calculation results repository 8.

Mathematical description of the proposed approach.

During the analysis of a computer network, a time series is formed in real time, which corresponds to measurements made at intervals / ,, / ₂ , where n is the number of members of the line. The packet arrival times t, where ze [l, «] are non-uniform i _t - * t _M -t _t , and the data packets at appropriate times x ₁ , x ₂ , ..., x _n determine the channel occupancy to analyze such a time series it is necessary to aggregate it, ie to calculate data flows at equal intervals Δ / = Z. - ί._ _λ = t _M -1 .. Data aggregation can be done in two ways:

The first is the smoothing of moving averages over a selected time interval

Calculate the average flow for the ΔΖ data line:

where ΔΖ = Z _y −Ζ _Λ , ye [1, / 1] where n - integers, ke [l, «] where n - integers, / and k satisfy the inequality j> k.

The resulting time series describe the average changes in data traffic at each time point ΔΖ. In the second way, by calculating the amount of data traffic over time interval ΔΖ:

*, ^Σ = Σ *. « ⁽²⁾ / -y where ΔΖ = tj - t _k , where n is an integer, ke [l, n] where n is an integer, j and k satisfy the inequality j> k .

Using the smoothing method of moving averages, the aggregate row will slightly reduce network traffic spikes based on the aggregation interval selected by the user. Applying a method of aggregating the amount of data traffic transmitted over a time interval will give more prominence to network traffic spikes based on the time interval you choose.

By forming time series for analysis, discrete aggregation intervals can be selected, which can be varied and compared to evaluate the fractality of the flow. When choosing ΔΖ, it is recommended to take into account the measured network traffic intensity, i. y. at higher flow rates Δζ is selected lower, otherwise higher.

Calculation of estimates of stable random variables.

The peculiar symmetric process described by the formulas and characterized by infinite dispersion is known as the α-stable process [Samorodnitsky G, Taqqu MS. Stable Non-Gaussian Processes: Stochastic Models with Infinite Variance // Chapman and Hali: New York, 1994. - P. 636.], if for each random process Y (t) can be described the network traffic bursts according to the formula:

P (| F (Z) |> x) ~ cx ~ ^a (3) where:

P is the probability of a network traffic surge, t is a variable, also called a time component,

Y (t) is a function of time, x is a number and x -> oo, c is a constant and c> 0, a is a stability index.

It is proved that when 1 <a <2 the mean of the above formula is finite and when 0 <a <1 is infinite [G. Samorodnitsky, Long Range Dependence. Foundations and Trends in Stochastic Systems. Vol. 1, No. 3, 163-257, 2006].

The stable random variable 5 _α (β, σ, μ) is characterized by four stability parameters:

a is the stability index ae (0,2), also called the tail index, which describes the process creepiness, β is the asymmetry index β e [-1,1], which describes the process offset to zero, σ is the scaling index, σ> 0 and describes the process the size of the elements, μ is the position index μ e R.

The stability index is sufficient to estimate the self-sufficiency of a time series. The calculation of the remaining three stability parameters only allows estimation of the properties of the string, so their formulas are not given. Robust estimation methods with high error tolerance and high computational speed were chosen for the evaluation of the stable random stability index a. More on this at work [L. Kaklauskas, L. Sakalauskas (2011). Study of on-line measurement of traffic self-similarity. Central European Journal of Operations Research, DOI: 10.1007 / sl0100011-0216-5].

In our proposed approach, a robust empirical quantile regression method is applied. This method for estimating a -stable magnitude parameters is proposed in [I. A. Koutrouvel (1981). An iterative procedure for estimating parameters of stable law. Communications in Statistics - Simulation and Computation 10, 17-28, 1981]. Applications of this method are described in more detail in [I. Belov, A.Kabashinskas, L. Sakalauskas, (2005). Investigation of Stock Market Stable Models. Proceedings of the Conference on Information Technology 2005, Technology,

Kaunas, p. 439-462].

The robust empirical quantile regression method calculates the stability index using the formula:

(4)

S,

K where: K _Si = J * for K = 10, * = iy _t = iog (-> og (M „Ū *) l ² .

ⁿ / = 1 n is the number of elements of the time series formed, if z is the z'th time element, t is the selected time series aggregation interval,

> * = log IG I »

K ^s 3 = ^ yk>

A = 1

K $ 3 = 2 ^ '* = 1

K _Si = iny _k , teiK = 10.

A = 1

If the obtained value of a is 1 <a <2, then the string is characterized by self-identity, and the closer the value of a to 2, the longer the memory of self-identity. As mentioned above, the calculation of the other three string stability parameters allows estimating the properties of the string, but provides no additional information about its specificity.

The second calculation method is IR Statistics (Increment Ratio Statistics). More specifically, such a calculation is described in [D. Surgailis, G. Teyssiere, M. Vaičiulis. The increment ratio statistics. Journal of Multivariate Analysis Volume 99, Issue 3, March 2008, pages 510-541], This method evaluates whether the analyzed network traffic has a feature with long-term memory. According to the IR statistical method, the IR estimate is calculated using the formula:

k + m k + 2m

IR.-1 n-3m n-3m-1

Σ

A = 0

Σ < ^χ »- ~ ^χ . '> ⁺ Σ <* i = k + l k + m

Σ <*, ...- * 3 = A + 1 issk + m + 1 k + 2m

Z + m (5)

Σ <* »i'sft + m + 1 where: A is the variable used to calculate the IR estimate, n is the number of members of the formed string, m = [m ^{1 73} ], and only the integer portion of the m value is taken to calculate the IR estimate,

X to - The z-th member of the formed string.

With a confidence level of 0.95, it is possible to say that an aggregate string has a self-sufficiency with long-term memory if inequality is satisfied:

77? -Λ (0)> ζ _α σ (0) -— (6)

V n-3m where: A (0) = 0.5881, σ (0) = 0.2080, ^ 0.95 ⁼ 1 »64,

The Hurst coefficient H is calculated according to a known R / S statistic, e.g., the method described in [G. Samorodnitsky. Long Range Dependence. Foundations and Trends in Stochastic Systems 2006b; 1 (3): 163-257].

It is recommended that the robust empirical quantile regression method be used when more accurate estimation of network traffic specificity and its long-term memory is required by computing an index. Classical R / S statistics should be used as a control for the resulting a-index, which essentially confirms the property of network traffic specificity. It is recommended to use IR statistics when evaluating trends in computer network traffic.

An algorithm that evaluates the eigenvalue of computer network traffic according to the presented method is shown in FIG. 2. First Step - Saving User Options 10 - Entering information about the selected aggregation interval, the aggregation method, the time series length being formed, the methods of estimating self-sufficiency and storing the results. Step Two - Retrieve Network Packets from Network Device 11. Step Three - Retrieve Information from Packet Headers 12. Step Four - Time Line Aggregation 13. If moving averaging smoothing method 14 is selected, this row is formed using this aggregation method, otherwise the row is formed using data 15. The fifth step is storing the aggregated time series 16 (usually in temporary or operative memory) until an aggregated time series is formed according to the user-specified time series length.

Each successive line will be formed recursively, i.e. y. deletes the first member of the formed string and adds a new aggregated member.

Step Six - Calculation of Stability Parameters: Robust Empirical Quantum Regression Method 17, Stability Index Calculation 17a, IR Statistical Method 18,

IR estimation 18a, R / S statistical method selected 19,

Calculation of the Hurst Factor 19a.

If all three methods are selected, all parameters are calculated.

Step Seven - Delivering Results to the User 20. Step Eight - Storing Results

21. If the results have been selected to be stored, the resulting estimates are stored in read-only memory 21a.

The proposed method has been tested using computer simulation method and calculations with real network data packets. To simulate random flows, a module was created that generates time series, described by α-stable values S „(P, 1, 0), when a f - 1. Characteristic rows were generated with the following parameters: a = 1.8 (H = 0.56) ), β = 0, σ = 1, μ = 0, the number of generated rows is 1000, and the number of members of each row is n = 3000. The results of the calculation were consistent with the selected values, i.e. y. 1.73 <a <1.86, and 0.538 <H <0.578.

In order to implement the proposed method, a library of self-similarity estimators (SSEs) has been created. The proposed method was used for real-time measurements of the network unit of Šiauliai University Distance Learning Center. The values of the 309 aggregate time series a stability index and Hurst coefficient were calculated. Calculation of the IR estimate showed that 87.24% of the time series exhibit long-term memory (1 <a <2). Consequently, the packet data traffic over aggregated lines is a continuous, long-term memory process. By changing the aggregation interval, the results of the calculation are similar, meaning that time series have fractal properties.

The standard deviation of the results obtained by the SSE program showed that the methods for estimating the stability index 5, the IR estimate and the Hurst coefficient are suitable for real-time determination of the self-stability parameters.

The library of software modules can also be used to measure load specificity on a network node and on a client computer.

Measurement results and the applied network model allow forecasting traffic surges at 10 network nodes and taking preventive measures to reduce them.

DEFINITION OF INVENTION

Claims (3)

DEFINITION OF INVENTION 1. A real-time computer network traffic analysis method, which recursively forms a string, computes its statistical parameters, and stores the results in a cache memory that: 5 aggregate network traffic in real time using the smoothing method of moving averages or the sum of transmitted traffic over time; caches the formed string in cache memory; recursively calculates the self-parameters of a dynamically formed string; the results of the calculation shall be stored in a temporary or / and permanent form for further analysis 10 in memory; 2. The method of claim 1, wherein the eigenvalue parameters are calculated using a robust empirical quantile regression method, IR statistics and R / S statistics. 3. A method according to claim 2, characterized in that the self-evaluation 15 calculates the a stability index, IR estimate, and Hurst coefficient.