CN103618651A - Network abnormality detection method and system based on information entropy and sliding window - Google Patents

Abstract

The invention discloses network abnormality detection method and system based on information entropy and a sliding window. The method comprises the following steps: defining the size of a time window and a sliding distance of the time window; progressively calculating the entropy and the entropy ratio of each time window orderly according to the sliding window; judging that network abnormality occurs when data mutation is generated in the time window or does not accord with the previous law if the entropy of the time window is smaller than a given threshold or the entropy ratio is greater than the given threshold. By adopting the method and the system, an information entropy model and a sliding window technology are led in, so as to find out the problem of network abnormality; the network abnormity can be quickly found out, the model is simplified to a certain extent, and the network abnormality can be quickly found out.

Description

A kind of network anomaly detection method and system based on comentropy and sliding window

Technical field

The present invention relates to a kind of network anomaly detection method and system, particularly relate to a kind of network anomaly detection method and system based on comentropy and sliding window.

Background technology

The method of Network anomaly detection is mainly the method based on statistics at present, wherein mainly comprises following five kinds: 1) threshold detection technique.For example, detect the number of times of password mistake at short notice.2) average and standard deviation modelling technique.By average and the standard deviation of calculating parameter, set confidential interval, when measured value surpasses the scope of confidential interval, show to have abnormal.3) set up multivariate model.Its detection is to note abnormalities based on two or more parameters are carried out to correlation analysis.4) Markov model.Each of audit event is dissimilar as a state variable, use a state-transition matrix to describe state variation, it may be extremely to produce a little that the state matrix that probability is less shifts.5) time series models.Consider that order, the time of advent and value that a series of observations occur note abnormalities.

Yet above-mentioned network anomaly detection method all respectively has following shortcoming: the model of first method is comparatively simple, yet it cannot detect more abnormal behaviour type; For second method, because confidential interval need to artificially arrange by experience, therefore need failure and the experience of more number of times to generate believable confidential interval; The third method model is complicated, and result can along with parameter difference, tool has a greater change; Send out method for the 4th kind and be applicable to the situation that variable is continuous parameter, for being sampled as centrifugal pump, obtaining situation and cannot obtain effective result; The result of Lung biopsy depends on the size that time window arranges.

Summary of the invention

The deficiency existing for overcoming above-mentioned prior art, the present invention's object is to provide a kind of network anomaly detection method and system based on comentropy and sliding window, by comentropy model and sliding window technique introducing Network Abnormal are pinpointed the problems, discovering network is abnormal quickly, and also discovering network is abnormal fast to have simplified to a certain extent model.

For reaching above-mentioned and other object, the present invention proposes a kind of network anomaly detection method based on comentropy and sliding window, comprises the steps:

Step 1, the sliding distance of definition time window size and time window;

Step 2, goes forward one by one successively and calculates entropy and the entropy ratio of each time window according to sliding window setting;

Step 3, if calculate, the entropy of the time window obtaining is less than given threshold value or entropy ratio is greater than given threshold value, judges in this time window and has data sudden change or the situation of rule before that do not meet to occur, and Network Abnormal occurs.

Further, step 2 also comprises the steps:

Step 2.1, the bit number x of interior each time point of window computing time _iwith normalized value z _i;

Step 2.2, according to bit number x _iwith normalized value z _icalculate z on each time point _iprobability p (z _i);

Step 2.3, according to z on each time point _iprobability p (z _i) computing time window at z _ion entropy and entropy ratio.

Further, in step 2.1, according to the bit number x of each time point in following formula window computing time _iwith normalized value z _i:

x _i＝b _i-b _i-1；

$z_i=\frac{x_i}{\stackrel{\‾}{x}},$ When $x_i\≤\stackrel{\‾}{x}$ Time;

$z_i=\frac{2\stackrel{\‾}{x}-x_i}{\stackrel{\‾}{x}},$ When $x_i>\stackrel{\‾}{x}$ Time,

B wherein _ifor time point i (i=k, k+1 ..., the desired value of k+n) locating,

Further, in step 2.2, according to following formula, calculate z on each time point _iprobability p (z _i):

$p\left(z_i\right)=\frac{1}{\mathrm{\σ}\sqrt{2\mathrm{\π}}}{e}^{-\frac{{(z_i-\stackrel{\‾}{z})}^2}{{2\mathrm{\σ}}^2}}$

Wherein $\stackrel{\‾}{z}=\frac{1}{n}\underset{i=k}{\overset{k+n}{\mathrm{\Σ}}}{z}_i$ For average, ${\mathrm{\σ}}^2=\frac{1}{n}\underset{i=k}{\overset{k+n}{\mathrm{\Σ}}}{(z_i-\stackrel{\‾}{z})}^2$ For variance.

Further, in step 2.3, according to following formula window computing time TW _kat z _ion entropy E (TW _k):

$E\left({\mathrm{TW}}_k\right)=-\underset{i=k}{\overset{k+n}{\mathrm{\Σ}}}p\left(z_i\right)\log \left(p\left(z_i\right)\right),$

Further, in step 2.3, the entropy ratio of i time window is that the mean value of entropy of a front s window is divided by the entropy of i time window.

Further, this desired value is chosen the interfaces class in router administration information bank.

For achieving the above object, the present invention also provides a kind of Network anomaly detection system based on comentropy and sliding window, at least comprises:

Time window arranges module, for the sliding distance p of definition time window size n and time window;

Entropy and entropy ratio calculation module, go forward one by one successively and calculate entropy and the entropy ratio of each time window according to sliding window setting;

Judge module, according to calculating the entropy of time window or the comparative result of entropy ratio and given threshold value obtaining, judges whether to occur Network Abnormal.

Further, if calculate, the entropy of the time window obtaining is less than given threshold value or entropy ratio is greater than given threshold value, and this judge module judges in this time window has data sudden change or the situation of rule before that do not meet to occur, and Network Abnormal occurs.

Further, this entropy and the entropy ratio calculation module bit number x of each time point in computing time window first _iwith normalized value z _i, then according to the bit number x of each time point obtaining _iwith normalized value z _icalculate z on each time point _iprobability p (z _i), finally according to z _iprobability p (z _i) computing time window at z _ion entropy and entropy ratio.

Compared with prior art, a kind of network anomaly detection method based on comentropy and sliding window of the present invention is by pinpointing the problems comentropy model and sliding window technique introducing Network Abnormal, discovering network is abnormal quickly, and also discovering network is abnormal fast to have simplified to a certain extent model.

Accompanying drawing explanation

Fig. 1 is that the MIB interface class of certain catenet supply equipment business in preferred embodiment of the present invention records sectional drawing;

Fig. 2 sets gradually equal-sized time window schematic diagram in preferred embodiment of the present invention;

Fig. 3 be in preferred embodiment of the present invention sliding time window schematic diagram is set;

Fig. 4 is the flow chart of steps of a kind of network anomaly detection method based on comentropy and sliding window of the present invention;

Fig. 5 is the system architecture diagram of a kind of Network anomaly detection system based on comentropy and sliding window of the present invention

Fig. 6 carries out the entropy schematic diagram of 9 windows of abnormality detection with ifInOctets variable in the present invention's experiment 1;

Fig. 7 is IfInOctets and the IfInDiscards index ASSOCIATE STATISTICS schematic diagram of router gw2 mouth in the present invention's experiment 2;

Fig. 8 is entropy and the entropy ratio schematic diagram of each time window in 4320-4560 minute in the present invention's experiment 2.

Embodiment

Below, by specific instantiation accompanying drawings embodiments of the present invention, those skilled in the art can understand other advantage of the present invention and effect easily by content disclosed in the present specification.The present invention also can be implemented or be applied by other different instantiation, and the every details in this specification also can be based on different viewpoints and application, carries out various modifications and change not deviating under spirit of the present invention.

Before introducing the present invention, first data source and theory basis selected for the present invention and that gather are done an introduction:

(1) data source choosing and gathering

Router administration information bank (Management Information Base, abbreviation MIB) there are 11 class object data, comprise system essential information as system class or with protocol-dependent information as IP class and TCP class etc., due to the nonumeric type of these data and too strong with network application correlation, be not suitable as the abnormality detection of generality.

In the present invention, the interfaces class of choosing in router administration information bank MIB is index set, what the type identified is the information of network interface, as the quantity of the packet by interface etc., with concrete protocol-independent, therefore this class data target has the generality irrelevant with application, is applicable to the abnormality detection as the router of generality.Interfaces class mainly comprises 12 kinds of numeric type variable indexs, as shown in table 1 below:

Interfaces class leading indicator list in table 1.MIB

Object indications	ASN.1 coding	Data type	Object factory
				ifInOctets	1.3.6.1.2.1.2.2.1.10	Counter32	Total bit number that interface is received
ifInUcastPkts	1.3.6.1.2.1.2.2.1.11	Counter32	The unicast packet number that interface is received
				ifInNUcastPkts	1.3.6.1.2.1.2.2.1.12	Counter32	The non-unicast bag number that interface is received
ifInDiscards	1.3.6.1.2.1.2.2.1.13	Counter32	The bag number that interface is received and abandoned
				ifInErrors	1.3.6.1.2.1.2.2.1.14	Counter32	The bag number of makeing mistakes that interface is received
ifInUnknownProtos	1.3.6.1.2.1.2.2.1.15	Counter32	The unknown protocol bag number that interface is received
				ifOutOctcts	1.3.6.1.2.1.2.2.1.16	Counter32	Total bit number that interface sends out
ifOutUcastPkts	1.3.6.1.2.1.2.2.1.17	Counter32	The unicast packet number that interface sends out
				ifOutNUcastPkts	1.3.6.1.2.1.2.2.1.18	Counter32	The non-unicast bag number that interface sends out
ifOutDiscards	1.3.6.1.2.1.2.2.1.19	Counter32	The bag number that the need that interface abandons transmit
				ifOutErrors	1.3.6.1.2.1.2.2.1.20	Counter32	The bag number of makeing mistakes that interface cannot transmit
ifOutQLcn	1.3.6.1.2.1.2.2.1.21	Unsigned32	Transmit the length that bag is lined up

In preferred embodiment of the present invention, data source is picked up from the real-time MIB data record that certain catenet equipment supplier provides, and within router every 2 minutes, all can upgrade MIB.For example, Fig. 1 is that in MIB, interfaces class records sectional drawing, as shown in Figure 1, total bit number (ifInOctets index) that 11: 44 Monday of August 4, interface was received is 828590480, and interface is received and the bag number (ifInDiscards index) that abandons is 0.

(2) technology path

In the ifInOctets index of take in MIB, carry out Network anomaly detection as example, establish time window TW _krepresentative from time point k to time point k+n(wherein window size be n, time point unit is minute) time period, time point i (i=k, k+1 ..., the ifInOctets desired value of k+n) locating (being total bit number that time point i place interface is received) is b _i, the bit number that interface receives at time point i is

x _i＝b _i-b _i-1. (1)

For weighing time window TW _kinterior reception bit x _iuncertainty, can embody by its comentropy.Normalization x _i, order

$z_i=\frac{x_i}{\stackrel{\‾}{x}},$ When $x_i\≤\stackrel{\‾}{x}---\left(2\right)$

Or

$z_i=\frac{2\stackrel{\‾}{x}-x_i}{\stackrel{\‾}{x}},$ When $x_i>\stackrel{\‾}{x}---\left(3\right)$

Wherein can suppose z _iapproximate certain probability distribution of obeying, for example normal distribution, calculates z _idistribution probability

$p\left(z_i\right)=\frac{1}{\mathrm{\σ}\sqrt{2\mathrm{\π}}}{e}^{-\frac{{(z_i-\stackrel{\‾}{z})}^2}{{2\mathrm{\σ}}^2}},---\left(4\right)$

Wherein for average,

for variance.Time window TW _kat variable z _ion entropy be calculated as

$E\left({\mathrm{TW}}_k\right)=-\underset{i=k}{\overset{k+n}{\mathrm{\Σ}}}p\left(z_i\right)\log \left(p\left(z_i\right)\right),---\left(5\right)$

Due to p (z _i) interval be (0,1], so log (p (z _i))≤0 and with p (z _i) monotonic increase, E (TW _k) also with p (z _i) monotonic increase.Therefore, abnormal if the bit number that window interior receives occurs, z _ithe probability that meets normal distribution reduces, and entropy diminishes.Vice versa.

In order to reflect better the variation of network reception bit in nearly a period of time, can for example, by calculating for the previous period the mean entropy of (s time window before) and the ratio of current window entropy, reflect the variation of current entropy, the entropy ratio that defines i window is that the mean value of entropy of a front s window is divided by the entropy of i window, that is:

$\mathrm{ER}=\left({\mathrm{TW}}_i\right)=\frac{\frac{1}{s}\underset{j=i-s-l}{\overset{i-1}{\mathrm{\Σ}}}E\left({\mathrm{TW}}_j\right)}{E\left({\mathrm{TW}}_i\right)},---\left(6\right)$

If entropy ratio ER is (TW _i) exceed given threshold value, in this time window, there is Network Abnormal.

Arranging of time window, there are two kinds of modes: the first is to choose successively equal-sized window (TW ₁, TW ₂... TW _m), calculate entropy or the entropy ratio of each window, as Fig. 2; Another mode, adopts sliding window technique, and to be previous time window form (being first kind of way when the p=n) to the individual time point of front slide p (p < n) to a rear time window.As Fig. 3.

Fig. 4 is the flow chart of steps of a kind of network anomaly detection method based on comentropy and sliding window of the present invention.As shown in Figure 4, a kind of network anomaly detection method based on comentropy and sliding window of the present invention, comprises the steps:

Step 401, the sliding distance p of definition time window size n and time window.

Step 402, goes forward one by one successively and calculates entropy and the entropy ratio of each time window according to sliding window setting.In preferred embodiment of the present invention, step 402 further comprises the steps:

(1) according to aforementioned formula (1), the bit number x of each time point in (2) and (3) window computing time _iwith normalized value z _i;

(2) according to aforementioned formula (4), calculate z on each time point _iprobability p (z _i);

(3) according to aforementioned formula (5), (6) window computing time at z _ion entropy and entropy ratio.

Step 403, judgement is abnormal: if computing time window entropy be less than given threshold value or entropy ratio is greater than given threshold value, judge in this time window, have data sudden change or do not meet before the situation of rule occur, may occur abnormal.

Fig. 5 is the system architecture diagram of a kind of Network anomaly detection system based on comentropy and sliding window of the present invention.As shown in Figure 5, a kind of Network anomaly detection system based on comentropy and sliding window of the present invention, at least comprises that time window arranges module 501, entropy and entropy ratio calculation module 502 and judge module 503.

Wherein time window arranges module 501 for the sliding distance p of definition time window size n and time window; Entropy and entropy ratio calculation module 502 are gone forward one by one successively and are calculated entropy and the entropy ratio of each time window according to sliding window setting, and specifically, entropy and entropy ratio calculation module 502 be the bit number x of interior each time point of window computing time first _iwith normalized value z _i(according to formula (1), (2), (3)), then according to the bit number x of each time point obtaining _iwith normalized value z _icalculate z on each time point _iprobability p (z _i) (according to formula (4), finally according to z _iprobability p (z _i) computing time window at z _ion entropy and entropy ratio (according to formula (5), (6)); 503 entropy of time window or comparative results of entropy ratio and given threshold value that obtain according to calculating of judge module, judge whether to occur Network Abnormal, if calculate, the entropy of the time window obtaining is less than given threshold value or entropy ratio is greater than given threshold value, judge in this time window and have data sudden change or the situation of rule before that do not meet to occur, may occur abnormal.

Below will to the present invention's beneficial effect, carry out corresponding checking by several experiments.

Experiment 1.

With the ifInOctets index in MIB database, test, according to above-mentioned algorithm, tested the entropy in 9 time windows, wherein abscissa is time (unit for minute), ordinate is that z (t) represents ifInOctets desired value, and each window entropy calculates as Fig. 6.

As can be seen from Figure 6, the place that curve is milder, entropy is less, abnormal more likely generation, vice versa.

Experiment 2.

Y (t) expression for IfInDiscards(of integrated survey index IfInOctets (representing with x (t)) and index), total bit number that wherein IfInOctets indication interface is received, the bag number that IfInDiscards indication interface is received and abandoned, if z (t)=x (t)+α y (t), wherein α=3000 are weight.Fig. 7 is the statistical Butut to these data at router gw2 interface.

For Fig. 7 (c), establishing window size is 10, and sliding distance is 10, utilizes algorithm to calculate 4320-4560 minute each window entropy and entropy ratio is shown in Fig. 8.

In Fig. 8, the first row data are entropy of each time window, the entropy ratio of second each time window of behavior, and we adopt current window entropy and the ratio of the mean entropy of 12 windows (2 hours) before here.Can find out, time window [4410,4420] entropy and entropy ratio are respectively 2.7659e-008 and 6.2034, time window [4430,4440] entropy and entropy ratio are respectively 1.4009e-008 and 9.0558, time window [4440,4450] entropy and entropy ratio are respectively 1.7876e-008 and 5.7915, all far beyond the threshold value of setting in program.Therefore system is judged: 4410 minutes to 4450 minutes (with the window of circles mark), Network Abnormal detected in these 40 minutes.Clearly, this data exception also can obtain from Fig. 7 (c) in manual observation.

In sum, a kind of network anomaly detection method based on comentropy and sliding window of the present invention is by pinpointing the problems comentropy model and sliding window technique introducing Network Abnormal, discovering network is abnormal quickly, and also discovering network is abnormal fast to have simplified to a certain extent model.

Above-described embodiment is illustrative principle of the present invention and effect thereof only, but not for limiting the present invention.Any those skilled in the art all can, under spirit of the present invention and category, modify and change above-described embodiment.Therefore, the scope of the present invention, should be as listed in claims.

Claims (10)

1. the network anomaly detection method based on comentropy and sliding window, comprises the steps:

Step 1, the sliding distance of definition time window size and time window;

Step 2, goes forward one by one successively and calculates entropy and the entropy ratio of each time window according to sliding window setting;

Step 3, if calculate, the entropy of the time window obtaining is less than given threshold value or entropy ratio is greater than given threshold value, judges in this time window and has data sudden change or the situation of rule before that do not meet to occur, and Network Abnormal occurs.

2. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 1, is characterized in that, step 2 also comprises the steps:

Step 2.1, the bit number x of interior each time point of window computing time _iwith normalized value z _i;

Step 2.2, according to bit number x _iwith normalized value z _icalculate z on each time point _iprobability p (z _i);

Step 2.3, according to z on each time point _iprobability p (z _i) computing time window at z _ion entropy and entropy ratio.

3. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 2, is characterized in that, in step 2.1, according to the bit number x of each time point in following formula window computing time _iwith normalized value z _i:

x _i＝b _i-b _i-1；

when

time;

when

time,

B wherein _ifor time point i (i=k, k+1 ..., the desired value of k+n) locating,

4. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 3, is characterized in that, in step 2.2, according to following formula, calculates z on each time point _iprobability p (z _i):

Wherein

for average, for variance.

5. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 4, is characterized in that, in step 2.3, according to following formula window computing time TW _kat z _ion entropy E (TW _k):

。

6. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 5, is characterized in that, in step 2.3, the entropy ratio of i time window is that the mean value of entropy of a front s window is divided by the entropy of i time window.

7. a kind of network anomaly detection method based on comentropy and sliding window as claimed in claim 6, is characterized in that: this desired value is chosen the interfaces class in router administration information bank.

8. the Network anomaly detection system based on comentropy and sliding window, at least comprises:

Time window arranges module, for the sliding distance p of definition time window size n and time window;

Entropy and entropy ratio calculation module, go forward one by one successively and calculate entropy and the entropy ratio of each time window according to sliding window setting;

Judge module, according to calculating the entropy of time window or the comparative result of entropy ratio and given threshold value obtaining, judges whether to occur Network Abnormal.

9. a kind of Network anomaly detection system based on comentropy and sliding window as claimed in claim 8, it is characterized in that: if calculate, the entropy of the time window obtaining is less than given threshold value or entropy ratio is greater than given threshold value, this judge module judges in this time window has data sudden change or the situation of rule before that do not meet to occur, and Network Abnormal occurs.

10. a kind of Network anomaly detection system based on comentropy and sliding window as claimed in claim 8, is characterized in that: this entropy and entropy ratio calculation module be the bit number x of interior each time point of window computing time first _iwith normalized value z _i, then according to the bit number x of each time point obtaining _iwith normalized value z _icalculate z on each time point _iprobability p (z _i), finally according to z _iprobability p (z _i) computing time window at z _ion entropy and entropy ratio.

Images