CN109756379A - A network performance anomaly detection and localization method based on matrix differential decomposition - Google Patents

Abstract

The invention discloses a kind of network performance abnormality detection decomposed based on matrix difference and localization methods.Using the abnormality detection and localization method decomposed based on matrix difference, quantity and the position of Multiple outliers quickly and effectively can be detected and positioned.The present invention is based on the models for by round-trip delay (RTT) matrix decomposition being the sum of benchmark matrix and difference matrix, propose a kind of matrix difference decomposition method being simple and efficient, can detect in time and position the abnormal point of multiple network performances.The present invention has easily deployment, quickly detection and the multiple fault points of positioning network, the advantages such as accuracy rate is high, computational complexity is low compared to traditional method for detecting abnormality.

Description

A kind of network performance abnormality detection and localization method based on the decomposition of matrix difference

Technical field

The invention belongs to network communication field, specifically a kind of network performance decomposed based on matrix difference is examined extremely Survey and localization method, this method efficiently can detect and position the Multiple outliers in network, for management service network (including IP network, network function virtualization (NFV) network or IP and NFV hybrid network) effective operation offer new method.

Background technique

It is increasing as network size constantly expands with network application, so that network flow is increasingly sophisticated, network malice Behavior also increases increasingly, these cause Network Abnormal to happen occasionally.Once there is abnormal performance in network, can generate many bad shadows It rings, if network service quality (QoS) decline, the even network communication of user experience (QoE) variation are interrupted, causes huge economic damage It becomes estranged undesirable social influence.Network anomalous behaviors refer to occur in the course of network operation from different row under normal conditions For these Network anomalous behaviors mostly unusually or significantly increase along with network flow, thus cause network congestion, past Return phenomena such as time delay (RTT) time delay increases, packet loss increases.The reason of network performance exception is extremely complex, including network Equipment fault, network attack and flow peak gush.Although the probability occurred extremely is relatively low, it can generate sternly whole network The influence of weight.Accurate detection and quickly processing Network Abnormal are for guaranteeing that catenet stable and high effective operation plays a significant role.

In view of the factors such as technology, equipment actual deployment and existing background traffic are realized in network measure, quickly detect It is that ISP (ISP) pays close attention to and the hot spot studied and a pole all the time extremely with positioning network performance Has the task of challenge.Technological challenge therein includes: first, usually passes through monitoring, the change of analysis single link flow at present Change to detect Network Abnormal.However, this method may omit the exception in a region on other multilinks, therefore need It will be using the more comprehensively measurement method that can obtain exception information, such as traffic matrix.If but directly measuring and obtaining flow Matrix, it is extremely difficult in realization, and if estimated flow matrix, it is total between the traffic matrix and true traffic matrix estimated It is that there are deviations.Second, existing detection method is usually only capable of judging in network with the presence or absence of exception, but can not be carried out correct Differentiation, which has several abnormal points also, can not determine the position of abnormal point.Some detection methods are because measurement and processing complexity are too high, no It is easily put to practical.Third, in order to carry out verification experimental verification for the various Network anomaly detections and Location Theory that are proposed and method, Need a set of novel, similar with the real network environment efficient test method of research and development.

The present invention gives one according to based on the RTT performance matrix being made of between the multiple ports of network RTT measurement data The efficient matrix difference decomposition method (MADEL) of kind, this method can detect simultaneously in a network and positioning Multiple outliers, nothing It is IP network or NFV network, the network (hereinafter referred to as network) of even IP and NFV mixing by the network.Meaning of the invention Justice is that it can utilize RTT data with importance, efficiently and accurately determines quantity and the position of Multiple outliers, the party Method advantages such as low, easy deployment with computational complexity.

Summary of the invention

[goal of the invention]: the present invention propose it is a kind of based on matrix difference decompose network performance abnormality detection and positioning side Method accurately and efficiently can detect and position multiple fault points present in network, with solve can not be same existing for conventional method When detect multiple fault points, the position that can not determine fault point and computational complexity are high or the cost disposed in a network is big etc. Disadvantage.

[technical solution]: technical solution of the present invention will include the following contents:

1, a kind of system that can support actively to measure and form RTT performance matrix, it is characterised in that the following steps are included:

(1) for the network area of progress network performance abnormality detection and positioning, need to be arranged an Analysis server, and And the interface connecting at the n in the region with external network is all arranged with the RTT middleboxes for carrying out actively measuring function.

(2) each RTT middleboxes actively measure RTT numerical value to other RTT middleboxes, and the RTT data measured are sent out It is sent to Analysis server.

(3) temporally the period summarizes the RTT data that all RTT middleboxes are sent to Analysis server, in each period The measured value that measurement obtains is arranged in the vector of a 1 × P, then i.e. one T of composition of all RTT values obtained in T period The RTT matrix A of × P_T×P.Wherein P=n × (n-1) indicates the number of RTT measured value between each port measured in each period (do not include itself and arrive itself)；T indicates the periodicity of measurement；I-th row indicates all end-to-end RTT values within i-th of period, the J column indicate that the RTT time series measured between j-th of edge device pair, the matrix A of composition are known as RTT matrix (referring to Fig. 1).

2, a kind of abnormality detection and localization method based on matrix difference decomposition method, it is characterised in that including following step It is rapid:

According to the RTT Criterion-matrix collected under proper network, matrix difference point will be utilized containing abnormal low-rank RTT matrix Solution method quickly detects polyisocyanate constant amount and realizes abnormal positioning.MADEL algorithm mainly comprises the steps that

(1) abnormal matrix A will first difference processing: be contained₁∈R^T×NWith Criterion-matrix A₀Difference processing is carried out, difference is obtained Different matrix A_e=A₁-A₀；Again to A_eElementJudged, if(c₀For small constant value) it is then marked as normally It (is denoted as), otherwise isolate the dataObtain difference matrix A_e。

(2) abnormal data extracts.According to the property of the nonnegative matrix of matrix theory and matrix in block form, looked for from difference submatrix Non- negative anomaly submatrix outWherein α is A_eThe all rows chosen Label set, β A_eThe all column label set chosen, obtained abnormal submatrix are the nonnegative matrix containing abnormal data.

(3) abnormal quantity detects.ForConsider the column label chosen in its β to element, each element β_i, 0≤i≤| β | Comprising abnormal edge device label to information, by relevant warping apparatus label to being added in column label set P.It counts and counts Label is to (P in calculation label set P_k1, P_k2), 0≤k≤| P | the frequency of appearance, and define (P_k1, P_k2)=(P_k2, P_k1), if super Frequency threshold f out₀, then by label to being added in abnormal point set APS (Anomalies Points Set), APS gathers big The small abnormal number detected, label can be used for positioning exception to being related suspicious border router pair.

(4) abnormal positioning.To the suspicious border router pair of each of APS concentration, pass through network measurement tools Traceroute is detected back and forth obtains suspicious off path information, and all hop counts and IP address collected on detective path are denoted as IPS, while marking and the response time is averaged the abnormal subpath of one-hop delay significantly more than IPS on statistical path.It will be abnormal sub Path finally navigates to the abnormal subpath with highest frequency of occurrence, i.e., jumps extremely according to the number descending sort of appearance The corresponding network equipment of source and destination, output positioning abnormal results collection LAS (Localized Anomalies Set).

(5) the MADEL algorithm key step used includes:

[beneficial effect]: the method for the present invention accurately and efficiently can detect and position multiple fault points present in network, It compared with traditional method for detecting abnormality, have the advantages that be easy to dispose, computation complexity is low, can be accurate simultaneously Multiple performance fault points present in ground detection and positioning network etc..

Detailed description of the invention

Fig. 1 network RTT matrix is established and matrix difference is decomposed

The detection of Fig. 2 small scale network and positioning case

Fig. 3 large scale network

Fig. 4 Multi net voting abnormal performance test result

Network performance abnormal test results when Fig. 5 intensity difference

Fig. 6 heterogeneous networks scale and structural complexity test result

Specific embodiment

The mode that the present invention uses specifically is introduced below in conjunction with drawings and concrete examples.

1, RTT performance matrix is established

According to the requirement of different network test environment, network, and configuration of IP, link are built using different network technologies The critical networks parameter such as bandwidth, Routing Protocol；The support RTT program for actively measuring function is deployed to the border router of network In.

In the NET of experimental network region, if it has n border router port, the RTT process of measurement of deployment is utilized, (period) measures the end-to-end RTT performance between other ports at a certain time interval.Measurement obtains in each period Measured value be arranged in the vector of a 1 × P, then all RTT values obtained are the RTT for constituting a T × P in T period Matrix A_T×P.Wherein P=n × (n-1) indicates that the number of RTT measured value between each port measured in each period (does not include Itself arrives itself)；T indicates the periodicity of measurement；I-th row indicates all end-to-end RTT values within i-th of period, and jth column indicate The RTT time series measured between j-th of edge device pair, the matrix A of composition are known as RTT matrix.

2, detect and position the case study on implementation of network performance exception

The deployment of 2.1 experimental enviroments and the foundation of RTT performance matrix

In the network shown in fig. 1,15 routers are shared；Wherein R1, R2 and R3 are border routers, other 12 sections Point (n1~n12) is internal router.The setting of its parameter are as follows: distribute IP address for each router port；Each of the links Bandwidth is set as 10Mbps；Routing Protocol is used as using ospf (OSPF)；Every five seconds injects one to the network A intensity is the UDP Poisson flow of 1Mbps as background stream etc..To measure and collecting convenient for RTT, by RTT process of measurement middleboxes M1~M3 is deployed in respectively in border router R1~R3.

After starting network and process of measurement, the RTT value building RTT performance matrix of corresponding time cycle is acquired.In the net In network, there are three border routers, therefore can obtain 6 RTT values；Our continuous samplings within T period, the then NET RTT performance can use the matrix A of T × 6_T×6It indicates.Wherein matrix elementIt indicates in t-th of period inner boundary equipment i to boundary The RTT measured value of equipment j, the matrix thus constituted are exactly RTT matrix.Specific process has: from the 1st period (period is 8 seconds long) Start to the 20th period, unimplanted any abnormal flow, the RTT data in 16 periods under the proper network of collection, and constructs Criterion-matrix A₀；Out of the 21st the period to the 40th period, internally router n9 injects abnormal flow, we collect at this stage The data in 16 periods, and construct containing abnormal matrix A₁。

The process of 2.2 network performance abnormality detections and positioning

Fig. 2 gives the main processes that positioning is carried out abnormality detection using MADEL algorithm.

(1) abnormal matrix A difference processing: will be contained first₁With Criterion-matrix A₀, obtained using matrix difference decomposition method To difference matrix difference matrix A_e, as shown in the Step (1) of Fig. 2.C is set₀=3, by the RTT data in normal range (NR)It is set as -1,Abnormal data be set asRealize the isolation of abnormal data.

(2) abnormal data extracts: by sparse abnormal data and extracting, and finds out non-negative anomaly submatrixIts The β of middle extraction={ 2,5 }, as abnormal data part is isolated in Fig. 2 right middle.

(3) abnormal quantity detects: in conjunction with actual network topological information shown in FIG. 1, counting suspicious border router pair It is (R1, R3) to get ASP={ (R1, R3) } is arrived, and can determine that we are tested with | ASP |=1 exception, and it is abnormal related Edge device be R1 and R3.

(4) abnormal positioning: as the lower right of Fig. 2 uses traceroute to suspicious border router to (R1, R3) Detection of the R1 and R3 back and forth on two paths is carried out, all hop counts and IP address collected on detective path are denoted as IPS, same to markers Remember that the response time is significantly more than the abnormal subpath of one-hop delay average in IPS on simultaneously statistical path.In conjunction with network topological information Statistical analysis, obtains the facility information of abnormal subpath, counts and number count_ that its abnormal subpath that sorts occurs Times, finally navigates to the abnormal subpath { (n9, n10) } of highest frequency of occurrence, that is, the abnormal results positioned are LAS= { (n9, n10) }.

3, large scale network case study on implementation

3.1 network environments and evaluation index

In network as shown in Figure 3,34 routers are shared；Wherein R1~R10 is the boundary road of autonomous system (AS) By device, other 24 nodes (n1~n24) are internal routers.The setting of its parameter are as follows: distribute IP for each router port Address；The bandwidth of each of the links is set as 10Mbps；Routing Protocol is used as using ospf (OSPF)；Every 5 Second UDP Poisson flow that intensity is 1Mbps is injected as background stream to the network.It measures and collects for the ease of RTT, we 10 RTT middleboxes M1~M10 measured is deployed in respectively in border router R1~R10.

We assess the performance of abnormality detection and localization method using following index:

● true positive rate (TPR): correctly detect abnormal ratio；

● true negative rate (TNR): correctly detect non-abnormal ratio；

● false negative rate (FNR): undetected unnatural proportions；

● rate of false alarm (FPR): it is mistakenly detected as abnormal non-abnormal ratio；

● correct localization (CLR): has and be properly positioned abnormal ratio.

Higher TPR (TNR, CLR) and lesser FPR (FNR) mean better abnormality detection performance.

The abnormal collection of injection is denoted as IAS (Injected Anomalies Set), abnormal quantity is | IAS |=N_IAS； Non- anomalous routes device set is denoted as NAS (Non-Anomalies Set), and non-abnormal quantity is | NAS |.Make after injecting exception With MADEL algorithm, the exception boundary router detected is APS to information, and the abnormal results of positioning are LAS, and what is detected is different Constant amount is | APS |.According to defined above, it is known that

The situation of 3.2 Multi net voting abnormal performances

Preferably to assess MADEL performance, considering more abnormal situations, by anomaly ratio rate is defined as:Wherein anomalies_num indicates abnormal quantity, and n is the quantity of border router in network, it is assumed that anomalies_num≤n.It is as shown in Figure 4 using the test result of MADEL algorithm under Fig. 3 network: with the increasing of abnormal ratio Add, TPR, TNR and CLR of MADEL algorithm are down to 0.82,0.86 and 0.82 from 1 respectively, this is because the increase meeting of abnormal quantity Increase the overlapping in Network Abnormal path, leads to detection and the decline of positioning accuracy.However, TPR, TNR of MADEL algorithm and CLR is generally larger than 0.8, this shows that the detection normal to polyisocyanate of MADEL algorithm and positioning have good effect.

The situation of network performance exception when 3.3 intensity difference

The abnormal scene of three types is constructed in network shown in Fig. 3, it is right for the degree difference of investigating Network Abnormal The influence of MADEL algorithm.Three scenes are respectively set are as follows: the abnormal Poisson flow for being 5Mbps in the 10th period injection Mean Speed, Abnormal lasting is 10 periods；In the 30th period, it is 20Mbps's that Mean Speed is injected into identical internal router Abnormal Poisson flow, Abnormal lasting are also 10 periods；In the 50th period, it is to identical position injection Mean Speed The abnormal Poisson flow of 50Mbps continues 10 periods.As a result as shown in Figure 5.

Fig. 5's (a) the result shows that, with the increase of abnormal intensity of flow, abnormal flow increases suddenly will lead to serious net Network congestion, abnormal data value has apparent increase in difference matrix.Fig. 5 (b)-(d) shows to make under these three abnormal scenes Increased with TPR, TNR and CLR of MADEL algorithm with the increase of the size of injection exception stream, i.e., when the abnormal flow of injection When rate is larger, the performance of MADEL algorithm is more preferable.

The situation of 3.4 heterogeneous networks scales and structural complexity

It establishes one and meets power law type, larger, the higher mesh network topologies of Connected degree, and keep other conditions not Become, the border router in the network is 30, and internal router is 100 (being denoted as Prototype2), by Prototype2 It is compared with network shown in Fig. 3 (being denoted as Prototype1), the examination of more abnormality detections and positioning is carried out using MADEL algorithm It is as shown in Figure 6 to test result.

From fig. 6 it can be seen that when abnormal quantity is less than network boundary router quantity, and abnormal quantity only accounts for internal road By device quantity sub-fraction when, the performance of MADEL algorithm is more preferable；Correlation between abnormal quantity increase or multiple exceptions When very strong, the accuracy of detection and positioning can be reduced slightly；For different network size and structural complexity, algorithm There is no significant difference between TPR, TNR and CLR.

Claims (3)

1. a network performance anomaly detection and positioning method based on matrix differential decomposition, is characterized in that comprising the following steps: (1) Differentially process the round-trip delay (RTT) performance matrix and extract abnormal performance data: perform differential processing between the RTT matrix A ₁ ∈ R ^T×N containing anomalies and the reference matrix A ₀ to obtain the difference matrix A _e , and then compare A The abnormal data of _e is isolated and extracted, and the non-negative abnormal sub-matrix is found from the difference sub-matrix. (2) Detection and location of abnormal numbers: Statistical analysis is performed on the extracted non-negative abnormal sub-matrix, combined with the real network topology, to determine the information of suspicious border router pairs, and the number of abnormalities detected; for each suspicious border router pair , obtain the suspicious abnormal path information through the back and forth detection of the measurement tool traceroute, and locate the network device corresponding to the abnormal sub-path. 2. the network performance anomaly detection and positioning method based on matrix differential decomposition according to claim 1, is characterized in that in described step (1) processing, it mainly comprises following 2 steps: (1) Differential processing: first perform differential processing between the abnormal matrix A ₁ ∈ R ^T×N and the reference matrix A ₀ to obtain the difference matrix A _e =A ₁ -A _{0 ;} judge, if (c ₀ is a small constant value), then mark it as normal (denoted as ), otherwise isolate the data The difference matrix A _e is obtained. (2) Abnormal data extraction. According to the properties of non-negative matrices and block matrices in matrix theory, find out non-negative outlier submatrixes from difference matrices α _i ={1,...,T}, d≤n ² , where α is the set of row labels selected by A _e , and β is the set of column labels selected by A _e , and the obtained abnormal submatrix is a non-negative matrix containing abnormal data The main steps of the algorithm used include: 1: Obtain the tested RTT matrix A ₁ ; the reference RTT matrix A ₀ ; the number of border routers N; the abnormal data threshold c ₀ 2: Differential processing A _e =A ₁ -A ₀ 3: for each in A _e 4: 5: add j to β 6: 7: end for 8: output . 3. the network anomaly detection and localization method based on matrix differential decomposition according to claim 1, is characterized in that in described step (2) processing, it mainly comprises following 2 steps: (1) Abnormal quantity detection. for Considering the column label pair elements selected in β, each element β _i , 0≤i≤|β| contains abnormal boundary device label pair information, and the related abnormal device label pair is added to the column label set P. Count and calculate the frequency of occurrence of label pairs (P _k1 , P _k2 ) in the label set P, 0≤k≤|P|, and define (P _k1 , P _k2 )=(P _k2 , P _k1 ), if it exceeds the frequency threshold f ₀ , the label pair is added to the APS, the size of the APS set is the number of detected anomalies, and the label pair is the relevant suspicious border router pair, which can be used to locate the anomaly. (2) Abnormal positioning. For each suspicious border router pair in the APS set, obtain suspicious abnormal path information through traceroute back-and-forth detection, collect all hops and IP addresses on the detected path and record it as IPS, and mark and count the response time on the path that significantly exceeds the average single hop of IPS. Delayed exception subpath. The abnormal sub-paths are sorted in descending order of occurrence times, and finally the abnormal sub-path with the highest occurrence frequency is located, that is, the network device corresponding to the source and destination of the abnormal hop. The main steps of the algorithm used include: 1: Get anomaly submatrix Select column label set β; abnormal frequency threshold f ₀ 2: for each β _i in β 3: do Add (P _k1 , P _k2 ) to P //P _k1 and P _k2 are Source and destination addresses in column β _i 4: 5: then add (P _k1 , P _k2 ) to the APS 6: end for 7: |APS| is the number of detected anomalies 8: for each(P _k1 , P _k2 ) in APS 9: IPS←Use traceroute to detect the hop count and IP address on the two paths from P _k1 to P _k2 and from P _k2 to P _k1 10: count_times←Count the number of abnormal sub-paths whose response time significantly exceeds the IPS average single-hop delay on the path 11: Sort count_times and locate the first abnormal subpath (x1, x2), add (x1, x2) to LAS 12: end for 13: Output APS and LAS.