CN101060444A - Bayesian statistical model based network anomaly detection method - Google Patents

Abstract

The related network abnormal detection method based Bayes statistical model comprises: 1. grasping TCP/IP flow data package by bypass interception way; 2. decomposing attributes to form a data matrix; 3. mining data to build normal training matrix, known-abnormal training matrix, and unknown-abnormal training matrix; 4. continual grasping the TCP/IP package to detect them with Bayes evaluator; and 5. alarming the abnormal condition and filling into the known-abnormal training matrix by self-learning way; or else, back to step 4. This invention overcomes defects in prior art.

Description

Network anomaly detection method based on Bayesian statistical model

Technical field:

The present invention relates to exception flow of network and detect and the Intrusion Detection Technique field, be specifically related to a kind of based on the unusual detection method of the network of Bayesian statistical model.

Background technology:

Be accompanied by the normal use flow of network, various abnormal flows are also following on the network, have influence on the normal operation of network, threatening the safety and the use of subscriber's main station.Network is often caused by reasons such as network attack, worm-type virus, net abuses unusually, for example: diverse network scanning, ddos attack, network worm virus, malice downloads, all can cause network performance to descend to the improper use of Internet resources etc., can influence normal network when serious uses, cause network congestion, even cause the inefficacy of network interruption, the network equipment.Therefore, network traffics are monitored in real time and managed, the network of finding the known type that exists in the network and UNKNOWN TYPE is unusual, and having become needs the matter of utmost importance that solves in the network security management, and it has great significance to the reliabilty and availability that improves network.

Traditional exception flow of network detection is analysis, the study by long network operation flow information; set up the performance parameter reference range that network normally uses pattern; when network operation state and normal baseline have obvious deviation, then there be unusual the generation in the decision network.This method can find that basic network is unusual, and still, it exists the parameter benchmark scope to be difficult to determine, to lack defectives such as flexibility and rate of false alarm height.

Summary of the invention:

Main purpose of the present invention provides a kind of method of the network abnormality detection based on Bayesian statistical model, is difficult to determine, lack flexibility and the high problem of rate of false alarm to overcome the parameter benchmark scope that prior art exists.

For overcoming the problem that prior art exists, of the present inventionly realize by following step:

Step 1: intercept mode with bypass and grasp TCP/IP data on flows bag on the network;

Step 2: carry out the attribute decomposition for grabbing the packet that comes,, and form data matrix for the preliminary treatment of data is carried out in next step operation;

Step 3: the data matrix that preliminary treatment is obtained carries out data mining, makes up the training data matrix of normal condition, known exception state and unknown abnormality;

Step 4: continue to grasp in real time the TCP/IP data on flows bag on the network, it is detected by Bayes's evaluator;

Step 5: if note abnormalities, then report to the police and unusual kind is packed into the known exception slip condition database in the mode of self study, otherwise execution in step (four).

The attribute of above-mentioned steps two described packets decomposes and is meant that the network packet that will grab decomposes classification according to the attribute item.That is, produce the attribute record that each TCP/IP connects by grasping the form of network packet, the form of these records is as follows:

R(T，Src.IP，Src.Port，Dst.IP，Dst.Port，FLAG)

Wherein, the T representative connects the time of beginning; Src.IP represents source IP; Src.Port represents source port; Dst.IP represents purpose IP; Dst.Port represents destination interface; FLAG represents the state that TCP/IP connects.By above attribute item, system will be an attribute record collection of each TCP/IP linkage record R.

The above-mentioned steps three described data matrixes that preliminary treatment is obtained carry out data mining and are meant that continuing conclusion for training data matrix given, that include normal condition, known exception state and unknown abnormality handles, form a probability tables, in this table, the attributive character of a kind of stateful example of each row representative, a kind of stateful example of each row representative.This tableau format is as shown in the table:

	X 0	…	X j	…
					A 0	T 000，…，T 00g	…	T 0j0，…，T 0jg	…
_	_	_	_	_
					A i
_	_	_	_	_
					A I-1	T (I-1)00，…，T (I-1)0g	…	T (I-1)j0，…，T (I-1)jg	…
A I	T I00，…，T I0g	…	T Ij0，…，T Ijg	…

A represents the title of stateful example in the last table, and these states comprise three types of normal condition, known exception state and unknown abnormalities; The property parameters of every kind of state of X representative.

The algorithm of above-mentioned steps four described Bayes's evaluators is as follows:

Make incident X=(X ₁, X ₂..., X _t), parameter $N={\mathrm{\Σ}}_{i=1}^t{X}_i$ And p=(p ₁..., p _t)

Observe one of them example x=(x ₁, x ₂..., x _t), the probability function of its multinomial distribution is as can be known:

$l\left(p\right|x)=l(p_1,p_2,\mathrm{\Λ},p_t|x_1,x_2,\mathrm{\Λ},x_t)=\underset{i=1}{\overset{1}{\mathrm{\Π}}}{p}_i^{x_i}$

This distribution can be deformed into Dirichlet

$f\left(p\right|\mathrm{\β})=\mathrm{\Γ}\left(\underset{i=1}{\overset{t}{\mathrm{\Σ}{\mathrm{\β}}_i}}\right)\underset{i=1}{\overset{t}{\mathrm{\Π}}}\frac{p_i^{\mathrm{\β}-1}}{\mathrm{\Γ}\left(\mathrm{\βi}\right)}$

Here there is β for all i _i＞0, and $\mathrm{\Γ}\left(y\right)={\∫}_0^{\∞}{e}^{-z}{z}^{y-1}\mathrm{dz}$

Make parameter $K=\underset{i-1}{\overset{t}{\mathrm{\Σ}}}{\mathrm{\β}}_i$ With ${\mathrm{\λ}}_i=\frac{{\mathrm{\β}}_i}{K}$

Can draw its prior probability mathematic expectaion thus is:

E(p _i|K，λ)＝λ _i

Its posterior probability mathematic expectaion is:

$E\left(p_i\right|K,\mathrm{\λ},x)=\frac{x_i+K{\mathrm{\λ}}_i}{N+K}$

Thus, through calculating that we can draw following formula and calculate unusual estimated value:

$\hat{K}=(N^2-\mathrm{\Σ}{t}_{\mathrm{ijk}}^2)/{\mathrm{\Σ}}_{i,j}{(t_{\mathrm{ijk}}-N{\mathrm{\λ}}_{\mathrm{ijk}})}^2$

${\hat{t}}_{\mathrm{ijk}}=N(t_{\mathrm{ijk}}+\hat{K}{\mathrm{\λ}}_{\mathrm{ijk}})/N+K$

Obtain thus

$P\left(x_j\right|A_i)=\frac{{\hat{t}}_{\mathrm{ijk}}}{{\mathrm{\Σ}}_{k=0}^g{\hat{t}}_{\mathrm{ijk}}}$

Utilize Bayes to obtain:

$P(C=A_i|X=x)=\frac{\left({\mathrm{\Π}}_{j=0}^{J-1}P\left(x_{\mathrm{mj}}\right)P(X=A_i)\right)}{P(X=x)}---\left(1\right)$

The x here _MjBe meant the mj row in the table, and $P(C=A_i)=\frac{1}{J}{\mathrm{\Σ}}_{j=0}^{J-1}({\mathrm{\Σ}}_{K=0}^g{\hat{t}}_{\mathrm{ijk}}/N)$

Can calculate current network conditions by above step and meet the sort of state.

The self-learning function of above-mentioned steps five is meant, when system is first find one new when unusual, system joins this in tranining database unusually, when finding that once more this is unusual, then is known exception.

Compared with prior art, advantage of the present invention is:

Network anomaly detection method based on Bayesian statistical model is the learning functionality that adopts Bayes' theorem to disclose, find the relation between a large amount of variablees, data are predicted, classified, set up unusual intrusion detection Bayesian network, come phase-split network unusual by this network then, judged result.This method by Bayesian statistical model find, unusual in the decision network, have the advantage of flexible, the intelligent degree height of method, accuracy of judgement.

Description of drawings:

Accompanying drawing is the network anomaly detection method flow chart that the present invention is based on Bayesian statistical model.

Embodiment:

Bayesian statistical analysis combines prior information with sample information, be used among the statistical inference.Comprehensive with Bayesian formula prior information and sample information, obtain posterior information.And the posterior information that obtains can be used as the priori that a new round is calculated, and is comprehensive with the sample information of further acquisition, the next posterior information of asking.Along with this process continues, posterior information is more and more to approach true value really.That is to say that the study mechanism of bayes method is existence really and effective.The process of this study is actually the process of an iteration.

Step of the present invention is:

(1) intercept mode with bypass and catch packet on the network:

(2) packet is carried out the decomposition of attribute with set form,

The attribute of packet decomposes and is meant that the network packet that will grab decomposes classification according to the attribute item.That is, produce the attribute record that each TCP/IP connects by grasping the form of network packet, the form of these records is as follows:

R(T，Src.IP，Src.Port，Dst.IP，Dst.Port，FLAG)

Wherein, the T representative connects the time of beginning; Src.IP represents source IP; Src.Port represents source port; Dst.IP represents purpose IP; Dst.Port represents destination interface; FLAG represents the state that TCP/IP connects.By above attribute item, system will be an attribute record collection of each TCP/IP linkage record R.

(3) data matrix that preliminary treatment is obtained carries out data mining, makes up the training data matrix of normal condition, known exception state and unknown abnormality,

Decompose classification according to the attribute item, form is as follows:

Connect	T	Src.IP	Src.Port	Dst.IP	Dst.Port	FLAG
							L1	T1	Src.IP1	Src.Port1	Dst.IP1	Dst.Port1	FLAG1
L2	T2	Src.IP2	Src.Port2	Dst.IP2	Dst.Port2	FLAG2
							L3	T3	Src.IP3	Src.Port3	Dst.IP3	Dst.Port3	FLAG3
_	_	_	_	_	_	_
							Ln	Tn	Src.IPn	Src.Portn	Dst.IPn	Dst.Portn	FLAGn

(4) continue the real-time TCP/IP data on flows bag that grasps on the network, and carry out attribute and decompose, according to the formula of front

$P(C=A_i|X=x)=\frac{\left({\mathrm{\Π}}_{J=0}^{J-1}P\left(x_{\mathrm{mj}}\right)P(C=A_i)\right)}{P(X=x)}---\left(1\right)$

The x here _MjBe meant the mj row in the table, and $P(C=A_i)=\frac{1}{J}{\mathrm{\Σ}}_{j=0}^{J-1}({\mathrm{\Σ}}_{k=0}^g{\hat{t}}_{\mathrm{ijk}}/N)$

Calculate the Bayesian Estimation value of the packet of current period, and judge its state thus;

(5), then report to the police if note abnormalities.And unusual kind is packed into the known exception slip condition database in the mode of self study, otherwise directly carry out next step;

(6) forward (four) to.

It should be noted last that: above execution mode is the unrestricted technical scheme of the present invention in order to explanation only, although the present invention is had been described in detail with reference to above-mentioned execution mode, those of ordinary skill in the art is to be understood that: still can make amendment or be equal to replacement the present invention, and any modification that does not break away from the spirit and scope of the present invention is replaced with local, and it all should be encompassed in the claim scope of the present invention.

Claims (4)

1, based on the network anomaly detection method of Bayesian statistical model, comprises the steps successively

Step 1: intercept mode with bypass and grasp TCP/IP data on flows bag on the network;

Step 2: carry out the attribute decomposition for grabbing the packet that comes,, and form data matrix for the preliminary treatment of data is carried out in next step operation;

Step 3: the data matrix that preliminary treatment is obtained carries out data mining, makes up the training data matrix of normal condition, known exception state and unknown abnormality;

Step 4: continue to grasp in real time the TCP/IP data on flows bag on the network, it is detected by Bayes's evaluator;

Step 5: if note abnormalities, then report to the police and unusual kind is packed into the known exception slip condition database in the mode of self study, otherwise execution in step (four).

2, the network anomaly detection method based on Bayesian statistical model as claimed in claim 1, it is characterized in that: the attribute of the described packet of described step 2 decomposes and is meant that the network packet that will grab decomposes classification according to the attribute item, that is by grasping the form of network packet, produce the attribute record that each TCP/IP connects, the form of these records is as follows:

R(T，Src.IP，Src.Port，Dst.IP，Dst.Port，FLAG)

Wherein, the T representative connects the time of beginning; Src.IP represents source IP; Src.Port represents source port; Dst.IP represents purpose IP; Dst.Port represents destination interface; FLAG represents the state that TCP/IP connects.By above attribute item, system will be an attribute record collection of each TCP/IP linkage record R.

3, the network anomaly detection method based on Bayesian statistical model as claimed in claim 1 or 2, it is characterized in that: the described data matrix that preliminary treatment is obtained of described step 3 carries out data mining and is meant that continuing conclusion for training data matrix given, that include normal condition, known exception state and unknown abnormality handles, form a probability tables, in this table, the attributive character of a kind of stateful example of each row representative, a kind of stateful example of each row representative, this tableau format is as shown in the table:

X ₀ … X _j … A ₀ T ₀₀₀，…，T _00g … T _0j0，…，T _0jg … _ _ _ _ _ A _i _ _ _ _ _ A _I-1 T _(I-1)00，…，T _(I-1)0g … T _(I-1)j0，…，T _(I-1)jg … A _I T _I00，…，T _I0g … T _Ij0，…，T _Ijg …

A represents the title of stateful example in the last table, and these states comprise three types of normal condition, known exception state and unknown abnormalities; The property parameters of every kind of state of X representative.

4, the network anomaly detection method based on Bayesian statistical model as claimed in claim 3 is characterized in that: the algorithm of the described Bayes's evaluator of described step 4 is as follows

$P(C=A_i|X=x)=\frac{\left({\mathrm{\Π}}_{j=0}^{J-1}P\left(x_{\mathrm{mj}}\right)P(C=A_i)\right)}{P(X=x)}---\left(1\right)$

The x here _MjBe meant the mj row in the table, and $P(C=A_i)=\frac{1}{J}{\mathrm{\Σ}}_{j=0}^{J-1}({\mathrm{\Σ}}_{K=0}^g{\hat{t}}_{\mathrm{ijk}}/N).$

Images