CN102655465B - Method for quickly detecting time-frequency domains of abnormal network traffic - Google Patents

Abstract

The invention discloses a method for quickly detecting time-frequency domains of abnormal network traffic, which belongs to the field of network abnormality detection in a large-scale network environment. The method can be used for extracting abnormal characteristics by virtue of a processing way of related coefficients, thereby improving the detection accuracy; and simultaneously, the time-frequency domain analysis is performed on the abnormal characteristics by virtue of short-time Fourier transform, thereby detecting the abnormality relatively accurately and judging the occurrence time of the abnormality. The method can be used for analyzing data on the time-frequency domains, and in theory, the more layers are analyzed, the more accurate frequency division is, and finally, the obtained effect is relatively good.

Description

A kind of time-frequency domain rapid detection method of exception flow of network

Technical field

The present invention belongs to large-scale network environment lower network abnormity detecting field, particularly a kind of time-frequency domain rapid detection method of exception flow of network.

Background technology

Along with the development of network, Network anomalous behaviors comprises network failure, user misoperation, network attack and Network Virus Propagation etc., and these abnormal behaviours usually cause in network that on wall scroll or multilink, network traffics depart from normal phenomenon.Although abnormal flow does not show obvious off-note on wall scroll or multilink, in network, the abnormal flow summation of multilink is larger.Thereby Traffic Anomaly has great harmfulness and destructiveness to network and the various device that runs on network thereof.Because Traffic Anomaly disguise is extremely strong, outbreak suddenly, unknown characteristic, can bring great harm (for example by specific network attack program or network worm virus, to break out caused burst flow and cause network paralysis in the short time at the utmost point operational outfit on network or network, communication disruption), therefore accurately and rapidly detect exception of network traffic behavior, judgement causes the reason of Traffic Anomaly and takes rapidly correct responsive measures is the important prerequisite that guarantees the efficient operation of network security, and become the Front Scientific Problems of the common concern of academia and industrial quarters both at home and abroad at present.

Along with the development of information technology, exception of network traffic disguise is more and more stronger.From the relative normal discharge very greatly and in continuous variation, very little abnormal flow relatively detected, as " looking for a needle in a haystack ", must adopt new detection techniques, method and mechanism.

The difficulty of Traffic anomaly detection is mainly reflected on the Time precision of abnormity detecting.Only have and adopt Time-frequency method to analyze to abnormal signal, give prominence to the feature of abnormity point, could detect exactly the moment of abnormal appearance.

Chen-Mou Cheng etc. has proposed a kind of frequency spectrum analysis method and has identified normal TCP flow, they found to RTT(round-trip time) normal TCP stream is bound to demonstrate periodically in relevant bag transmission, and attack stream does not have this characteristic conventionally; Calendar year 2001 V.Alarcon-Aquino etc. has proposed a kind of based on detecting and locating the faint change in variance and frequency of given time series, but the yardstick of this algorithm picks is limited and algorithm idea is complicated; P.Barford etc. have proposed a kind ofly network traffics to be carried out to multiple dimensioned dyadic wavelet decomposes and reconstruct is comprehensive becomes high, medium and low three frequency ranges respectively with departing from the method that mark detects, this algorithm can be good at detecting the Flash Crowds that occurs in the past period and Short-time Anomalies etc., but the self adaptation that there is no well to solve a Wavelet Detection yardstick is chosen the problem with Real-Time Monitoring.Therefore present stage, exception of network traffic method for detecting still faces the problems such as adaptive problem, detecting real-time problem and detecting precision of method for detecting.

Summary of the invention

The deficiency existing for existing method, the present invention proposes a kind of time-frequency domain rapid detection method of exception flow of network, to reach detecting real-time, to improve the object of detecting precision.

Technical scheme of the present invention is achieved in that a kind of time-frequency domain rapid detection method of exception flow of network, comprises the following steps:

Step 1: in network topology structure, catch and contain abnormal data on flows, as time-domain signal to be analyzed;

Step 2: extract the data on flows on the OD stream in time-domain signal with identical destination node, carry out correlation analysis, calculate each OD stream and there is the coefficient correlation between the OD stream of identical destination node with him, calculated some coefficient correlations are averaged as the average correlation coefficient of this OD stream; Accumulation using the average correlation coefficient sum of all OD streams as coefficient correlation;

Step 3: the Accumulation of step 2 is carried out to joint time frequency analysis: adopt Short Time Fourier Transform, determine the time-frequency characteristics parameter of time-domain signal under certain time window; Change time window size, calculate the time-frequency characteristics parameter under time windows, by the above-mentioned time-frequency characteristics parameter stack calculating, obtain the accumulation time-frequency characteristics parameter of outstanding abnormal signal feature;

Step 4: set decision threshold, if accumulation time-frequency characteristics parameter surpasses this decision threshold, determine that this place exists abnormal flow.

Wherein, the decision threshold described in step 4, the method for determining is: calculate average and the variance of accumulation time-frequency characteristics parameter, get variance and be multiplied by a coefficient, then be added and obtain threshold value with average.

Described coefficient is a natural number within the scope of 1-10.

Advantage of the present invention: the time-frequency domain rapid detection method of exception flow of network of the present invention, the method that proposes the processing of employing coefficient correlation is extracted off-note, has improved the accuracy of detecting; Utilize Short Time Fourier Transform to carry out Time-Frequency Analysis to off-note simultaneously, detect more accurately abnormal and realize abnormal generation judgement constantly.The inventive method is to data analysis on time-frequency domain, and the number of plies of decomposing is in theory more, and more accurate to the division of frequency, the effect finally obtaining will be better.

Accompanying drawing explanation

Fig. 1 is the execution mode general flow chart that provides the time-frequency domain rapid detection method of exception flow of network of the present invention;

Fig. 2 is that one embodiment of the present invention destination node is 6 OD stream data from the sample survey bag schematic diagram, wherein, (a) represents that source node is 1; (b) represent that source node is 2; (c) represent that source node is 3; (d) represent that source node is 4; (e) represent that source node is 5; (f) represent that source node is 6; (g) represent that source node is 7; (h) represent that source node is 8; (i) represent that source node is 9; (j) represent that source node is 10; (k) represent that source node is 11; (l) represent that source node is 12;

Fig. 3 is that one embodiment of the present invention destination node is that in 6 OD stream, each OD stream, with the average correlation coefficient schematic diagram of other OD streams, wherein, (a) represents that source node is 1; (b) represent that source node is 2; (c) represent that source node is 3; (d) represent that source node is 4; (e) represent that source node is 5; (f) represent that source node is 6; (g) represent that source node is 7; (h) represent that source node is 8; (i) represent that source node is 9; (j) represent that source node is 10; (k) represent that source node is 11; (l) represent that source node is 12;

Fig. 4 is the Accumulation schematic diagram of each OD stream average correlation coefficient of one embodiment of the present invention;

Fig. 5 is the extraction schematic flow sheet of one embodiment of the present invention time-frequency characteristics parameter;

Fig. 6 is the time-frequency characteristics parameter distribution schematic diagram that the window of the different sizes of one embodiment of the present invention obtains;

Fig. 7 is the Accumulation schematic diagram of the time-frequency characteristics parameter of the different big or small windows acquisitions of one embodiment of the present invention;

Fig. 8 is the Energy distribution schematic diagram of one embodiment of the present invention characteristic parameter Accumulation;

Fig. 9 is one embodiment of the present invention Accumulation abnormity detecting result schematic diagram;

Figure 10 is that one embodiment of the present invention adopts and to depart from fraction method and each OD stream is carried out to the result schematic diagram of abnormity detecting, wherein, (a) represents the OD stream that source node is 1, destination node is 6; (b) represent the OD stream that source node is 2, destination node is 6; (c) represent the OD stream that source node is 3, destination node is 6; (d) represent the OD stream that source node is 4, destination node is 6; (e) represent the OD stream that source node is 5, destination node is 6; (f) represent the OD stream that source node is 6, destination node is 6; (g) represent the OD stream that source node is 7, destination node is 6; (h) represent the OD stream that source node is 8, destination node is 6; (i) represent the OD stream that source node is 9, destination node is 6; (j) represent the OD stream that source node is 10, destination node is 6; (k) represent the OD stream that source node is 11, destination node is 6; (l) represent the OD stream that source node is 12, destination node is 6.

Embodiment

Below in conjunction with accompanying drawing, embodiments of the present invention are described in further detail.

Fig. 1 is the general flow chart of execution mode of the time-frequency domain rapid detection method of exception flow of network.This flow process starts from step 101.In step 102, in network topology structure, catch and contain abnormal data on flows, as time-domain signal to be analyzed.In normal situation, the data on flows of network is more stably, and when there is abnormal flow, somewhere, flow value sometime depart from normal value, such as some flow value of constantly locating increases suddenly or reduces suddenly, can tentatively judge that this place may occur extremely.Present embodiment adopts the method for calculating similitude to give prominence to off-note, the party's ratio juris is: proper network data have certain similitude, can estimate data below with one piece of data above, if but in data below, had extremely, utilizing so certain segment data would above be to be unable to estimate out data below.The method of calculating by similitude is given prominence to off-note, is convenient to utilize Short Time Fourier Transform to process the similitude of accumulation.

In step 103, first adopt the method for structure OD stream and source, destination node relational matrix, extract the data on flows on the OD stream in time-domain signal with identical destination node, method is:

If total p node in network topology structure, exists p*p bar OD stream, establishing selected destination node is q, structure source, destination node relational matrix F _(OD), formula is as follows:

$F_{\left(\mathrm{OD}\right)}=\left[\begin{array}{cccccc}{f}_1^1& f_2^1& \·\·\·& f_q^1& \·\·\·& f_p^1\\ f_1^2& f_2^2& \·\·\·& f_q^2& \·\·\·& f_p^2\\ \·& & \·& & \·& \·\\ \·& & \·& & \·& \·\\ \·& & \·& & \·& \·\\ f_1^p& f_2^p& \·\·\·& f_q^p& \·\·\·& f_p^p\end{array}\right]---\left(1\right)$

In formula, represent that source node is that j destination node is the OD stream of i, wherein, 1≤i≤p, 1≤j≤p, for example, in the matrix shown in formula (1), represent that source node is that 1 destination node is 1 OD stream, represent that source node is that 2 destination nodes are 1 OD stream, the like, until represent that source node is that p destination node is the OD stream of p.

By formula (1), obtain the OD stream that all destination nodes are q.Extract all p bar OD stream, if the length of every OD stream is K, obtain traffic matrix F _q, formula is:

$\left\{\begin{array}{c}{F}_q=(f_q^1\left(t\right),f_q^2\left(t\right),...,f_q^i\left(t\right),...,f_q^p\left(t\right))\\ 1\≤t\≤K\\ 1\≤i\≤p\end{array}\right.---\left(2\right)$

In formula, F _qbe the traffic matrix of a p*K, wherein, be illustrated in the t OD stream that constantly source node is 1, destination node is q, the like, being illustrated in t moment source node is the OD stream that p, destination node are q.

Secondly, carry out correlation analysis, calculate each OD stream and have the coefficient correlation between the OD stream of identical destination node with him, calculated some coefficient correlations are averaged as the average correlation coefficient of this OD stream, method is:

Ask traffic matrix F _qin any two OD stream coefficient correlation.First the covariance matrix of calculated flow rate matrix formula is as follows:

$\mathrm{cov}(f_q^i,f_q^j)=E\left[(f_q^i-{\mathrm{\μ}}_i)(f_q^j-{\mathrm{\μ}}_j)\right]---\left(3\right)$

In formula, e represents to ask mathematic expectaion, and has j ≠ i.According to formula (3), determine OD stream between coefficient correlation formula is:

$\mathrm{corrf}(f_q^i,f_q^j)=\frac{\mathrm{cov}(f_q^i,f_q^j)}{\sqrt{\mathrm{cov}(f_q^i,f_q^i)\mathrm{cov}(f_q^i,f_q^j)}}---\left(4\right)$

In conjunction with formula (4) calculated flow rate matrix F _qin the average correlation coefficient of each OD stream between flowing with other OD, formula is as follows:

In formula, represent that k bar OD stream is with the average correlation coefficient of other p-1 bar OD stream.

Finally, the coefficient correlation sum between all OD stream is as the Accumulation of coefficient correlation: the Accumulation C that calculates the coefficient correlation between all OD stream _∑, formula is as follows:

$C_{\mathrm{\Σ}}=\underset{j=1}{\overset{p}{\mathrm{\Σ}}}{C_{\mathrm{OD}}}^j---\left(6\right)$

Accumulation C _∑in fact can portray well the minutia of original anomaly flow signal, increase below and utilized Short Time Fourier Transform to carry out the accuracy of abnormity detecting.

In present embodiment, in network topology structure, there are 12 nodes, totally 144 OD stream, selected destination node is 6, according to can obtain destination node according to the relational matrix of formula (1) structure, is that 6 OD stream is respectively totally 12, the figure of corresponding every OD stream, as shown in Figure 2.By formula (2), (3), solving every OD stream is averaged with coefficient correlation summation between other 11 OD streams, obtain result as described in Figure 3, the stack of 12 OD stream is obtained to the Accumulation of coefficient correlation, as shown in Figure 4, this Accumulation can reflect the minutia of abnormal data well, therefore can use Short Time Fourier Transform to analyze judgement to this minutia.

In step 104, the Accumulation of step 103 is carried out to joint time frequency analysis: adopt Short Time Fourier Transform, determine the time-frequency characteristics parameter of time-domain signal under certain time window; Change time window size, calculate the time-frequency characteristics parameter under time windows, by the above-mentioned time-frequency characteristics parameter stack calculating, obtain the accumulative total time-frequency characteristics parameter of outstanding abnormal signal feature;

Short Time Fourier Transform is defined as:

${\mathrm{STFT}}_z(t,f)={\∫}_{-\∞}^{+\∞}z\left(t^{\′}\right){\mathrm{\η}}^{*}(t^{\′}-t)e^{-j2\mathrm{\πf}{t}^{\′}}{\mathrm{dt}}^{\′}---\left(7\right)$

In formula, STFT _z(t, f) represents time-frequency characteristics parameter, and z (t') represents pending signal, refers in the present embodiment Accumulation C _∑; η (t) is a window function that time width is very short, η (t '-t) represents a window function centered by t, η * (t '-t) represents the conjugation of η (t '-t), wherein, * represent conjugate operation symbol, (t '-t) represents the time delay of window function, and, centered by t, length is the time period of (t '-t); F represents the frequency values for the signal at a t preset time place, and z (t') η * (t '-t) represents near the section of signal putting t analysis time.Sliced section is being carried out after Fourier transform, and all signal properties in corresponding its width of window centered by time t all can be by time-frequency characteristics parameter S TFT _z(t, f) shows.In sum, the selection that can find out window function has a significant impact the performance of Short Time Fourier Transform.In the present embodiment, choice for use Hanning window η (t), formula is as follows:

$\mathrm{\η}\left(t\right)=0.5(1-\cos \left(2\mathrm{\π}\frac{n}{N-1}\right)),0\≤n\≤N---\left(8\right)$

In formula, N represents window size.

According to Heisenberg uncertainty principle, temporal resolution Δ t and frequency resolution Δ f can not be simultaneously arbitrarily small, and their product meets therefore will improve temporal resolution will reduce frequency resolution, and vice versa.In the present embodiment, we select different window sizes to carry out Short Time Fourier Transform, and finally in order to obtain result more accurately, the transformation results summation that can obtain different windows size is processed.Calculate time-frequency characteristics parameter S TFT _z(t, f), Hanning window is time, obtain following formula:

${\mathrm{STFT}}_z(t,f)={\∫}_{-\∞}^{+\∞}z\left(t^{\′}\right)e^{0.5(1-\cos \left(2\mathrm{\π}\frac{n}{N-1}\right))}{e}^{-j2\mathrm{\πf}{t}^{\′}}{\mathrm{dt}}^{\′}---\left(9\right)$

Can obtain as calculated the time-frequency characteristics parameter S TFT of signal z (t') _z(t, f).

One group of time-frequency characteristics parameter S TFT of the z (t') that each window correspondence obtains _z,i(t, f), executes all windows and can obtain many stack features parameter, will after parameter delivery, superpose, and extracts the accumulation time-frequency characteristics parameter s um_STFT of outstanding abnormal signal feature for abnormal detecting, has improved the accuracy of result.

In present embodiment, as shown in Figure 5, wherein signal is input as coefficient correlation in the extraction of time-frequency characteristics parameter, and signal is output as the characteristic ginseng value sum that different size windows are carried out the different frequency that Short Time Fourier Transform correspondence obtains, as shown in Figure 6.In figure, different shades represents different frequency value, to after the parameter delivery of synchronization different frequency, superpose, extract the parameter s um_STFT of outstanding abnormal signal feature for abnormal detecting, improved the accuracy of detecting result, the figure of sum_STFT as shown in Figure 7.

Coefficient correlation, after having passed through Short Time Fourier Transform, obtains a series of time-frequency characteristics parameter vector.Abnormal detection method is actually and operates in time-frequency characteristics parameter, in step 104, extract the parameter s um_STFT of outstanding abnormal signal feature, due to the energy at abnormal signal place large (Energy distribution of coefficient correlation as seen from Figure 8), in figure, more to deeply feel the energy at bright this place larger for color.

In step 105, in present embodiment, adopt traditional self-adaptive decision threshold method to obtain threshold value threshold: to calculate average and the variance of Accumulation parameter, get variance and be multiplied by a coefficient, then be added and obtain threshold value threshold with average.Described coefficient is a natural number within the scope of 1-10.With characteristic parameter sum_STFT comparison, if surpassing the moment of decision threshold, sum_STFT intermediate value judges that this exists abnormal flow signal constantly again.As shown in Figure 9.

In actual emulation process, for the advantage of this algorithm is described better, in present embodiment, use simultaneously and depart from fraction method and carry out abnormity detecting and compare.In present embodiment, selecting detection window size is 20, sizes of history window is 30, the coefficient correlation obtaining in calculation procedure 103 depart from mark, it is 0.5 that alarm threshold is set adaptively, coefficient of deviation by more all 12 OD streams all has many places higher than alarm threshold, therefore judges that each OD stream all has abnormal.In order to detect abnormal generation constantly, it is 0.9 that abnormal threshold value is set in embodiment of the present invention adaptively, if phase relation numerical value is higher than abnormal threshold value, this phase relation numerical value is made as to 1.4, find in coefficient correlation and be 1.4 initial time continuously and stop constantly, be abnormal starting point and terminal, and by all be that 1.4 value resets 1 continuously, other zero setting, as Figure 10, wherein white dashed line represents abnormal thresholding, black dotted lines shows abnormity detecting result, from Figure 10, can find significantly, depart from fraction method can only detect whether have abnormal, cannot detect abnormal generation constantly.

Although more than described the specific embodiment of the present invention, one skilled in the art should be appreciated that these only illustrate, and can make various changes or modifications to these execution modes, and not deviate from principle of the present invention and essence.Scope of the present invention is only limited by appended claims.

Claims (3)

1. a time-frequency domain rapid detection method for exception flow of network, is characterized in that: comprise the following steps:

Step 1: in network topology structure, catch and contain abnormal data on flows, as time-domain signal to be analyzed;

Step 2: extract the data on flows on the OD stream in time-domain signal with identical destination node, carry out correlation analysis, calculate each OD stream and there is the coefficient correlation between the OD stream of identical destination node with him, calculated some coefficient correlations are averaged as the average correlation coefficient of this OD stream; Accumulation using the average correlation coefficient sum of all OD streams as coefficient correlation;

Step 3: the Accumulation of step 2 is carried out to joint time frequency analysis: adopt Short Time Fourier Transform, determine the time-frequency characteristics parameter of time-domain signal under certain time window; Change time window size, calculate the time-frequency characteristics parameter under time windows, by the above-mentioned time-frequency characteristics parameter stack calculating, obtain the accumulation time-frequency characteristics parameter of outstanding abnormal signal feature, extract the parameter s um_STFT of outstanding abnormal signal feature for abnormal detecting;

Step 4: set decision threshold, if accumulation time-frequency characteristics parameter surpasses this decision threshold, determine that this place exists abnormal flow, then with characteristic parameter sum_STFT comparison, if sum_STFT intermediate value surpasses the moment of decision threshold, judge that this exists abnormal flow signal constantly;

Step 5: it is 0.9 that abnormal threshold value is set adaptively, if phase relation numerical value, higher than abnormal threshold value, is made as 1.4 by this phase relation numerical value, finds in coefficient correlation to be 1.4 initial time continuously and to stop constantly, i.e. abnormal starting point and terminal.

2. the time-frequency domain rapid detection method of exception flow of network according to claim 1, it is characterized in that: the decision threshold described in step 4, the method of determining is: calculate average and the variance of accumulation time-frequency characteristics parameter, get variance and be multiplied by a coefficient, then be added and obtain threshold value with average.

3. the time-frequency domain rapid detection method of exception flow of network according to claim 2, is characterized in that: described coefficient, span is a natural number in 1-10.