CN101582813B - Distributed migration network learning-based intrusion detection system and method thereof - Google Patents

Abstract

The invention discloses a distributed migration network learning-based intrusion detection system and a method thereof, and mainly solves the problems that the prior method has low efficiency in detection of some attack types and is difficult to search data again. The whole system comprises a network behavior record preprocessing module, an abnormality detection module and an abnormal behavior analyzing module. The network behavior record preprocessing module completes the quantification and normalization processing of a network behavior record; the abnormality detection module uses an abnormality detection learning machine to completes the classification and identification for an input record, determines whether the record is a normal behavior, and completes the detection if the record isa normal behavior or transmits the record to the abnormal behavior analyzing module if the record is an abnormal behavior; and the abnormal behavior analyzing module uses an abnormal behavior analyzi ng learning machine to carry out the classification and identification of the input records and outputs the attach type of the record. The system and the method have the advantages of using other existing resources to improve the detection rate for the prior attach types with low detection rate and avoiding searching the data again and can be used for network intrusion detection.

Description

Intruding detection system and method thereof based on distributed migration network learning

Technical field

The invention belongs to network safety filed, be specifically related to a kind of intruding detection system, can be used for the intrusion detection of information security aspect.

Background technology

Along with being extensive use of of Internet, the increasing rogue attacks of computer network has been caused threat to the safety of information system, based on the network safety system of passive managements such as fire compartment wall back door for application layer, the attack that the unauthorized operation of internal user etc. causes or steal, destruction information has been powerless, and fire compartment wall itself is under attack easily, often feels simply helpless for the safety problem that internal network occurs.As a kind of important additional measure of network safety prevention instrument " fire compartment wall ", intruding detection system IDS demonstrates the importance that increases day by day.IDS can resist the attack from internal network, and can stop hacker's invasion and prevent spreading of virus, be the safe backing of fire compartment wall.Utilize record of the audit, IDS can identify any undesirable activity, thereby limits these activities, the safety of protection system.1998, MIT Lincoln laboratory and DARPA cooperation development the intruding detection system assessment, one of task of this plan provides the data set that is used for intrusion detection that comprises host log and network traffics, 9 all tcpdump data that 99 couples of DARPA of KDD CUP ' provide have been carried out suitable processing and feature extraction, as the intrusion detection data set of standard.

In recent years, development along with machine learning, new Intrusion Detection Technique also constantly occurs, as unit intrusion detection method based on neural net, Bayesian network, SVMs etc., but in the face of new environment and network security problem under the new situation, there are problems such as rate of false alarm height, adaptivity are poor, not high, the intelligent degree of the extent of reaction is not high automatically in above-mentioned Intrusion Detection Technique, so distributed algorithm becomes the detection speed that improves IDS and detects the essential of accuracy.2006, people such as Wang Shijun introduce the Boosting algorithm in the grader network, with the grader network and grader is integrated and use, and extend to distributed environment, propose the integrated algorithm DNB of distributed network, obtained having the classifier system of stronger generalization ability by communication between each node classifier and cooperation.Owing to, seldom the time, can not train sorter model preferably, and require training data and test data independent same distribution with existing other traditional machine learning methods based on the intrusion detection method of DNB, therefore have following shortcoming as label data:

1, there is energy imbalance in the network behavior verification and measurement ratio to different attack types, and is very low to the network behavior verification and measurement ratio of some attack type;

2 if improve verification and measurement ratio, needs user's gather data and learn this task expense costliness and spended time again;

3, can not utilize existing other available resources to improve the network behavior verification and measurement ratio of some attack type.

Summary of the invention

The objective of the invention is to overcome the shortcoming that above-mentioned intrusion detection method exists, transfer learning is introduced among the DNB, proposition is based on the intruding detection system and the method thereof of distributed migration network learning, utilizing existing other data to instruct the study of the lower network behavior of verification and measurement ratio, thereby improve its verification and measurement ratio.

For achieving the above object, detection system of the present invention comprises:

The network behavior record preprocessing module is used for quantification and normalization preliminary treatment finished in the network behavior record of collecting, and gives the abnormality detection module with pretreated result transmission;

The abnormality detection module is used for adopting the abnormality detection learning machine to carry out Classification and Identification to the record of input, determines whether this record belongs to normal behaviour, if this record belongs to normal behaviour and does not then deal with, detection of end, otherwise, this record is reached the abnormal behaviour analysis module;

The abnormal behaviour analysis module is used for adopting abnormal behaviour analytic learning machine to carry out Classification and Identification to the exception record of input, exports the affiliated attack type of this record.

Described network behavior record preprocessing module comprises:

Existing record preprocessing submodule is used for having label network behavior record collection to finish to quantize and normalized to existing, and will quantize with normalized after parameter import new record preliminary treatment submodule into;

New record preliminary treatment submodule, the parameter of utilizing existing record preprocessing submodule to import into quantizes and normalized new network behavior record.

Described abnormality detection module comprises:

Abnormality detection study submodule, existingly there is label network behavior record collection to be divided into normal and unusual two classes with pretreated, therefrom randomly draw the part sample respectively, adopt distributed network integrated study algorithm to learn, generate the abnormality detection learning machine, and this learning machine is transferred to abnormality detection test submodule;

Abnormality detection test submodule adopts the abnormality detection learning machine that the pretreated new record of input is carried out Classification and Identification, if the output result is normal, then do not deal with, and detection of end, otherwise, import this record into the abnormal behaviour analysis module.

Described abnormal behaviour analysis module comprises:

The migration sample is chosen submodule in advance, there are the network behavior record setting source territory sample and the aiming field of label that exemplar is arranged with existing, wait to instruct sample that territory, source sample is finished in advance according to aiming field and choose, with the source domain migration sample input abnormal behaviour analytic learning submodule of selecting;

Abnormal behaviour analytic learning submodule has exemplar together as training sample with source domain migration sample and the aiming field of importing, and adopts the distributed network integrated study algorithm of introducing transfer learning to learn, and generates abnormal behaviour analytic learning machine;

Abnormal behaviour analytical test submodule adopts abnormal behaviour analytic learning machine to carry out Classification and Identification to the abnormal behaviour of importing, and exports its attack type.

For achieving the above object, detection method of the present invention comprises the steps:

(1) imports the existing label network behavior record collection X that has, this data set is quantized and the normalization preliminary treatment, obtain pretreated X ' as a result;

(2) there is the pretreated X ' as a result of label network behavior record collection to be divided into normal and unusual two classes with existing, comprise M class attack type in unusual, from normal and exceptional sample, randomly draw a part of sample respectively, adopt distributed network integrated study algorithm containing K ₁Carry out T on the network topology structure of individual node ₁The wheel training, the grader network system of generation abnormality detection learning machine;

(3) set the sample of the middle normal type of X ' as territory, source sample set X ^S, sample number is m, the sample of Exception Type is as aiming field sample set X ^T, X ^TThe sample of the Exception Type that middle verification and measurement ratio is lower waits to instruct sample set X as aiming field ^T1, sample number is n ₁, and with X ^SDivide equally and be m/n ₁Part, be expressed as: $X^S=X_1^S\∪X_2^S\∪...\∪X_{[m/n_1]}^S,$ Wherein [] is rounding operation, with X _i ^SWith X ^T1Be combined as training set T _i ¹(i=1,2 ..., [m/n ₁]), adopt the method for adjusting sample weights in the AdaBoost algorithm training process to adjust sample weights, select bigger territory, the source sample subset X of weight _Sub ^S

(4) with territory, source sample subset X _Sub ^SWith other type sample of aiming field as training sample, reuse in the AdaBoost algorithm training process method of adjusting sample weights and adjust sample weights, from X _Sub ^SMiddle bigger territory, the source sample of weight of removing is with X _Sub ^SMiddle samples remaining is formed source domain migration sample set TR _D

(5) from aiming field sample set X ^TIn randomly draw a part of sample and form aiming field sample subclass TR _S, with the source domain migration sample set TR that selects _DTogether as training sample, with TR _DGive with aiming field and wait the label that instructs sample identical, layout contains K ₂The network topology structure of individual node, input sampling rate ρ ₂, exercise wheel counts T ₂, with TR _SAnd TR _DBe distributed on each node, generate the training sample set S on each node _k, k=1,2 ..., K ₂, generate abnormal behaviour analytic learning machine as follows according to this training sample set:

5a) each node training sample set S of initialization _kThe weight of middle sample;

5b) to each node training sample set S _kThe weight sampling of putting back to is arranged, obtain the training subclass of each node, in the learning algorithm of each node, train, obtain the basic grader C of each node _{K, t} ², with the basic grader of each node to this S _kClassify, wherein t is current exercise wheel number;

5c) basis is to S _kClassification results calculate the weighting error rate ε of aiming field sample on each node _{K, t}, and according to ε _{K, t}, calculate the weight of each basic grader _{K, t} ²:

5d) the weight of renewal source domain migration sample and aiming field sample is as t＜T ₂The time, change step 5b, work as t=T ₂The time, finish training, obtain by all basic grader C _{K, t} ²(k=1,2 ..., K ₂, t=1,2 ..., T ₂) the grader network system of the abnormal behaviour analytic learning machine formed;

(6) the new network behavior record x of input ", it is quantized and the normalization preliminary treatment, obtain pretreated network behavior record x as a result " ';

(7) with x " ' be input in the grader network system of the abnormality detection learning machine that step 2 generates and classify, obtain classification results:

$H_1\left(x^{\&prime;\&prime;\&prime;}\right)=\mathrm{sign}\left(\underset{k=1}{\overset{{\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="H2009100230731E00011.png,H2009100230731E00031.png"\; state="\{\{state\}\}">K</figure-callout>}}_1}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_1}{\mathrm{\&Sigma;}}}({\mathrm{\&alpha;}}_{k,t}^1{h}_{k,t}^1\left(x^{\&prime;\&prime;\&prime;}\right)+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^1{h}_{p,t}^1\left(x^{\&prime;\&prime;\&prime;}\right))\right)$

Wherein, h _{K, t} ¹(x " ') be in the abnormality detection learning machine each basic grader to x " ' classification results, α _{K, t} ¹Be the weight of each basic grader, p is the neighboring node label of node k, works as H ₁(x " ') be 1 o'clock, expression belongs to normal type, does not do any processing, the detection of end process; Work as H ₁(x " ') be-1 o'clock, expression belongs to Exception Type, then changes step (8) over to;

(8) with x " ' be input in the grader network system of the abnormal behaviour analytic learning machine that step 5 generates and classify, obtain classification results:

$H_2\left(x^{\&prime;\&prime;\&prime;}\right)=\underset{y\&Element;Y}{\arg \max }\left(\underset{k=1}{\overset{K_2}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_2}{\mathrm{\&Sigma;}}}({\mathrm{\&alpha;}}_{k,t}^{2}I[h_{k,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y]+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^{2}I[h_{p,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y])\right)$

Wherein, h _{K, t} ²(x " ') be in the abnormal behaviour analytic learning machine each basic grader to x " ' classification results, and $h_{k,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)\&Element;Y,$ Y={1 wherein, 2 ..., M}, 1,2 ..., M is respectively the call number of M kind attack type, I[] be indicator function, its value is 0 or 1, H ₂(x " ') ∈ Y;

(9) with H ₂(x " ') as call number, search the attack type of this call number correspondence, this attack type is exported as final testing result.

The present invention has the following advantages compared with prior art:

1) the present invention can utilize existing other to have label data to instruct the study of the lower attack type of verification and measurement ratio owing to introduce transfer learning, need not gather data again;

2) the present invention can select the migration sample that the lower attack type of verification and measurement ratio is had directive significance owing to adopt the AdaBoost algorithm to adjust the method for sample weights;

3) the present invention is owing to adopt the distributed network integrated study algorithm of introducing transfer learning, and the abnormal behaviour analytic learning machine of generation has the higher detection rate to the lower attack type of former verification and measurement ratio;

4) the present invention is owing to adopt distributed network integrated study algorithm, and the abnormality detection learning machine of generation has higher abnormality detection precision;

The present invention is based on the intruding detection system of network, can be used in the various complex network environments.Simulation result shows, to the large scale network intrusion detection data set KDD CUP ' 99 of standard, can improve about 87.3% before adopting the intrusion detection method based on distributed migration network learning of the present invention that the verification and measurement ratio of R2L attack type is introduced transfer learning.

Description of drawings

Fig. 1 is the intruding detection system schematic diagram based on distributed migration network learning of the present invention;

Fig. 2 is the intrusion detection method flow chart based on distributed migration network learning of the present invention;

Fig. 3 chooses flow chart in advance for source domain migration sample among the present invention;

Fig. 4 is for generating the flow chart of abnormal behaviour analytic learning machine among the present invention.

Embodiment

The present invention is described in detail below in conjunction with the drawings and specific embodiments.

With reference to Fig. 1, the intruding detection system based on distributed migration network learning of the present invention mainly comprises: network behavior record preprocessing module, abnormality detection module and abnormal behaviour analysis module.Wherein:

The network behavior record preprocessing module comprises: existing record preprocessing submodule and new record preliminary treatment submodule.Existing record preprocessing submodule is used for having label network behavior record collection to finish to quantize and normalized to existing, and will quantize with normalized after parameter import new record preliminary treatment submodule into; The parameter that the existing record preprocessing submodule of new record preliminary treatment submodule utilization imports into is finished new record and is quantized and normalized.

The abnormality detection module comprises: abnormality detection study submodule and abnormality detection test submodule.Existingly there is label network behavior record collection to be divided into normal and unusual two classes with pretreated, therefrom randomly draw the part sample respectively, adopt distributed network integrated study algorithm to learn, generate the abnormality detection learning machine, and this learning machine is transferred to abnormality detection test submodule; Abnormality detection test submodule adopts the abnormality detection learning machine that the pretreated new record of input is carried out Classification and Identification, if the output result is normal, then do not deal with, and detection of end, otherwise, import this record into the abnormal behaviour analysis module.

The abnormal behaviour analysis module comprises: the migration sample is chosen submodule, abnormal behaviour analytic learning submodule and abnormal behaviour analytical test submodule in advance.The migration sample is chosen submodule in advance has the network behavior record setting source territory sample and the aiming field of label that exemplar is arranged with existing, wait to instruct sample that territory, source sample is finished in advance according to aiming field and choose, with the source domain migration sample input abnormal behaviour analytic learning submodule of selecting; Abnormal behaviour analytic learning submodule uses the source domain migration sample and the aiming field of input exemplar to be arranged together as training sample, adopts the distributed network integrated study algorithm of introducing transfer learning to learn, and generates abnormal behaviour analytic learning machine; Abnormal behaviour analytical test submodule adopts abnormal behaviour analytic learning machine to carry out Classification and Identification to the abnormal behaviour of input, exports its attack type.

Intruding detection system of the present invention quantizes and the normalization preliminary treatment existing network behavior record, and the parameter after the record preprocessing; Pretreated existing record is divided into normal and unusual two classes, and each extracting part divided data adopts distributed network integrated study algorithm to learn as training sample from two classes respectively, obtains the abnormality detection learning machine; Again with normal recordings as territory, source sample, other exception records are as the aiming field sample, adjust sample weights by the AdaBoost algorithm, according to sample weights source domain migration sample are chosen in advance; From four class abnormal datas of pretreated existing record, randomly draw a part of sample respectively, the source domain migration sample of obtaining with preliminary election is together as training sample, adopt the integrated algorithm of distributed network of introducing transfer learning to learn, generate abnormal behaviour analytic learning machine; When new network behavior record is imported, the parameter that obtains according to existing network behavior record preprocessing, it is quantized and the normalization preliminary treatment, then pretreated result being input to the abnormality detection learning machine tests, if detect normally, then do not deal with detection of end, test otherwise be input to abnormal behaviour analytic learning machine, finally export its affiliated attack type.

With reference to Fig. 2, intrusion detection method of the present invention comprises the steps:

Step 1: import the existing label network behavior record collection X that has, this data set is quantized and the normalization preliminary treatment, obtain pretreated X ' as a result.

Pretreated detailed process is as follows:

Be the attribute of character string 1a), add up its type and quantize, obtain quantized result X property value among the X ₁

1b) to X ₁Attribute in the span of property value of source address byte number and destination address byte number be defined as [0,1.3 * 10 ⁹], two property values are done log10 () conversion, its range conversion to [0.0,9.14], is obtained the X as a result after the conversion ₂

1c) to the X as a result after the conversion ₂Carry out following normalization:

Suppose X ₂Comprise n sample, each sample has the d dimensional feature, and the d dimensional feature vector of n sample is expressed as X ₂=[F ₁, F ₂..., F _d], and the i dimensional feature vector is expressed as F _i=[f _I1, f _I2..., f _In], all features are done following normalization conversion:

f′ _ij＝f _ij/max(F _i)，i＝1，2，...，d，j＝1，2，...，n

f′ _i＝[f′ _i1，f′ _i2，...，f′ _in]，i＝1，2，...，d

X′＝[F′ ₁，F′ ₂，...，F′ _d]

Obtain pretreated X ' as a result.

Step 2: have the pretreated X ' as a result of label network behavior record collection to be divided into normal and unusual two classes with existing, comprise M class attack type in unusual, from normal and exceptional sample, randomly draw a part of sample respectively, adopt distributed network integrated study algorithm containing K as training sample ₁Carry out T on the network topology structure of individual node ₁The wheel training, the grader network system of generation abnormality detection learning machine.

Step 3: existing other type has exemplar as territory, source sample set X among the setting X ' ^S, sample number is m, the sample of Exception Type is as aiming field sample set X ^T, X ^TThe sample of the Exception Type that middle verification and measurement ratio is lower waits to instruct sample set X as aiming field ^T1, sample number is n ₁, according to X ^SAnd X ^T1, adopt the method for adjusting sample weights in the AdaBoost algorithm training process, select bigger territory, the source sample subset X of sample weights _Sub ^S

Above-mentioned bigger territory, the source sample subset X of sample weights of selecting _Sub ^SChoose flow process shown in Fig. 3 (a), its concrete steps are as follows:

3a) input source territory sample set X ^S, aiming field sample set X ^T, what wherein aiming field waited to instruct type has an exemplar collection X ^T1, sample weights threshold value W ₁

3b) because n ₁＜＜m sets X ^SLabel be+1, X ^T1Label be-1, for making two class sample balances, with X ^SDivide equally and be m/n ₁Part, promptly $X^S=X_1^S\&cup;X_2^S\&cup;...\&cup;X_{[m/n_1]}^S,$ Wherein [] is rounding operation, forms training set ${\mathrm{<figure-callout\; id="1"\; label="T\; i"\; filenames="H2009100230731E00011.png,H2009100230731E00031.png"\; state="\{\{state\}\}">T</figure-callout>}}_i^{}=X_i^S\&cup;X^{T1}(i=\mathrm{1,2},...,[m/n_1]);$

3c) with T _i ¹(i=1,2 ..., [m/n ₁]) import in the AdaBoost algorithm and train, after too much training in rotation is practiced, respectively from T _i ¹(i=1,2 ..., [m/n ₁]) in select sample weights greater than threshold value W ₁And belong to X _i ^S(i=1,2 ..., [m/n ₁]) sample, form territory, source sample subset X _Sub ^S

Step 4: with territory, source sample subset X _Sub ^SWith other type sample of aiming field as training sample, reuse in the AdaBoost algorithm training process method of adjusting sample weights and adjust sample weights, from X _Sub ^SMiddle bigger territory, the source sample of weight of removing is with X _Sub ^SSample in the residue is formed source domain migration sample set TR _D

Above-mentioned from X _Sub ^SThe middle flow process of removing bigger territory, the source sample of weight is shown in Fig. 3 (b), and its concrete steps are as follows:

4a) with aiming field sample set X ^TIn remove the sample set X wait to instruct type ^T1Outside, the sample set of other Exception Types is expressed as X ^T2, set X _Sub ^SLabel be+1, X ^T2Label be-1, input sample weights threshold value W ₂

4b) with X _Sub ^SWith X ^T2Form training set T ², be input in the AdaBoost algorithm and train, make sample weights obtain adjustment, after many wheel training finished, sample weights was greater than threshold value W in the removal training set ₂And belong to X _Sub ^SSample, with X _Sub ^SMiddle samples remaining is formed source domain migration sample set TR _D

Step 5: from aiming field sample X ^TIn randomly draw a part of sample and form aiming field sample subclass TR _S, with the source domain migration sample set TR that selects _DTogether as training sample, with TR _DGive with aiming field and wait the label that instructs sample identical, adopt the distributed network integrated study algorithm of introducing transfer learning to train, generate abnormal behaviour analytic learning machine, detailed process as shown in Figure 4:

5a) input contains K ₂The network topology structure of individual node, sample rate ρ, exercise wheel is counted T ₂, and with aiming field sample set TR _SWith source domain migration sample set TR _DBe distributed on each node, generate training sample set S on each node _k, k=1,2 ..., K ₂

5b) the weight D of last i the sample of each node k of initialization _{K, 1}(x _i)=1/l _k, l wherein _kFor node k goes up training set S _kIn contained number of samples, i=1,2 ..., l _k, k=1,2 ..., K ₂

5c) employing has the weight sampling of putting back to, from the training set S of each node _kThe middle training subclass of sampling and obtaining each node, wherein sample number is l _kρ, k=1,2 ..., K ₂, t is current exercise wheel number;

5d) according to each node training subclass, the basic grader C at training node k place _{K, t} ², and use basic grader C _{K, t} ²To the training set S on this node _kClassification;

5e) basis is to S _kClassification results, calculate the weighting error rate ε of aiming field sample _{K, t}:

${\mathrm{\&epsiv;}}_{k,t}=\underset{x_i\&Element;S_k\&cap;{\mathrm{TR}}_S}{\mathrm{\&Sigma;}}{D}_{k,t}\left(x_i\right)I[y\left(x_i\right)\&NotEqual;h_{k,t}^2\left(x_i\right)],k=\mathrm{1,2},...,K_2;$

H wherein _{K, t} ²(x _i) be basic grader C _{K, t} ²To sample x _iClassification results, $h_{k,t}^2\left(x_i\right)\&Element;Y,$ Y={1,2 ..., M}, 1,2 ..., M is respectively the call number of M kind attack type, y (x _i) be sample x _iKnown label;

5f) calculate basic grader C _{K, t} ²Weight _{K, t} ²:

${\mathrm{\&alpha;}}_{k,t}^2=0.5\&times;\log \left(\frac{1-{\mathrm{\&epsiv;}}_{k,t}}{{\mathrm{\&epsiv;}}_{k,t}}\right),k=\mathrm{1,2},...,K_2;$

5g) the weight undated parameter of calculating aiming field sample is ${\mathrm{\&beta;}}_{k,t}=\frac{{\mathrm{\&epsiv;}}_{k,t}}{1-{\mathrm{\&epsiv;}}_{k,t}}$ Weight undated parameter with source domain migration sample ${\mathrm{\&gamma;}}_k=\frac{1}{1+\sqrt{2\ln (m_k/T_2)}},$ M wherein _kBe source, node k place domain migration sample number;

5h) the sample x of new node k place more _iWeight D _{K, t}(x _i), obtain upgrading back weight D _{K, t+1}(x _i):

$D_{k,t+1}\left(x_i\right)=\left\{\begin{array}{cc}\frac{D_{k,t}\left(x_i\right)\&CenterDot;{{\mathrm{\&beta;}}_{k,t}}^{{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}}{Z_{k,t}},& x_i\&Element;S_k\&cap;{\mathrm{TR}}_S\\ \frac{D_{k,t}\left(x_i\right)\&CenterDot;{{\mathrm{\&gamma;}}_k}^{-{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}}{Z_{k,t}},& x_i\&Element;S_k\&cap;{\mathrm{TR}}_D\end{array}\right.$

Wherein, x _i∈ S _k∩ TR _SExpression x _iBelong to S _kIn the aiming field sample, x _i∈ S _k∩ TR _DExpression x _iBelong to S _kIn source domain migration sample, and

$Z_{k,t}=\underset{x_i\&Element;S_k\&cap;{\mathrm{TR}}_S}{\mathrm{\&Sigma;}}{D}_{k,t}\left(i\right)\&CenterDot;{{\mathrm{\&beta;}}_{k,t}}^{{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}+\underset{x_i\&Element;S_k\&cap;{\mathrm{TR}}_D}{\mathrm{\&Sigma;}}{D}_{k,t}\left(i\right)\&CenterDot;{{\mathrm{\&gamma;}}_k}^{-{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}$

${\mathrm{\&lambda;}}_{k,t}\left(x_i\right)=-2{\mathrm{\&alpha;}}_{k,t}^2(I(y\left(x_i\right)\&NotEqual;h_{k,t}^2\left(x_i\right))-1/2)-2\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^2(I(y\left(x_i\right)\&NotEqual;h_{p,t}^2\left(x_i\right))-1/2)$

Wherein, p is the neighboring node label of node k;

5i) as t＜T ₂The time, change step 5c, work as t=T ₂The time, finish training, obtain by all basic grader C _{K, t} ², k=1,2 ..., K ₂, t=1,2 ..., T ₂The grader network system of the abnormal behaviour analytic learning machine of forming.

Step 6: import new network behavior record x ", it is carried out preliminary treatment, obtain pretreated result and be x " '.

Be the attribute of character string to property value 6a), according to step 1a) method it is quantized, quantizing the back result is x " ₁

6b) source address byte number and two property values of destination address byte number are done log10 () conversion respectively, obtain after the conversion x as a result " ₂

6c) will write down x " ₂The d dimensional feature be expressed as x " ₂=f " ₁, f " ₂..., f " _d, and to x " ₂Carry out following normalization, obtain pretreated x as a result " ':

$f_i^{\&prime;\&prime;\&prime;}=\begin{array}{}\end{array}\left\{\begin{array}{cc}{f}_i^{\&prime;\&prime;}/\max \left(F_i\right)& f_i^{\&prime;\&prime;}\&le;\max \left(F_i\right)\\ 1,& f_i^{\&prime;\&prime;}>\max \left(F_i\right)\end{array}\right.,i=\mathrm{1,2},...,d$

x″′＝{f″′ ₁，f″′ ₂，...，f″′ _d}。

Step 7: with x " ' be input in the grader network system of the abnormality detection learning machine that step 2 generates and classify, obtain classification results:

$H_1\left(x^{\&prime;\&prime;\&prime;}\right)=\mathrm{sign}\left(\underset{k=1}{\overset{{\mathrm{<figure-callout\; id="1"\; label="K"\; filenames="H2009100230731E00011.png,H2009100230731E00031.png"\; state="\{\{state\}\}">K</figure-callout>}}_1}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_1}{\mathrm{\&Sigma;}}}({\mathrm{\&alpha;}}_{k,t}^1{h}_{k,t}^1\left(x^{\&prime;\&prime;\&prime;}\right)+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^1{h}_{p,t}^1\left(x^{\&prime;\&prime;\&prime;}\right))\right)$

Wherein, h _{K, t} ¹(x " ') be in the abnormality detection learning machine each basic grader to x " ' classification results, α _{K, t} ¹For the weight of each basic grader, work as H ₁(x " ') be 1 o'clock, expression belongs to normal type, does not do any processing, and the detection of end process is worked as H ₁(x " ') be-1 o'clock, expression belongs to Exception Type, then changes step (8) over to.

Step 8: with x " ' be input in the grader network system of the abnormal behaviour analytic learning machine that step 5 generates and classify, obtain classification results:

$H_2\left(x^{\&prime;\&prime;\&prime;}\right)=\underset{y\&Element;Y}{\arg \max }\left(\underset{k=1}{\overset{K_2}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_2}{\mathrm{\&Sigma;}}}({\mathrm{\&alpha;}}_{k,t}^{2}I[h_{k,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y]+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^{2}I[h_{p,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y])\right)$

Wherein, h _{K, t} ²(x " ') be in the abnormal behaviour analytic learning machine each basic grader to x " ' classification results, and $h_{k,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)\&Element;Y,$ H ₂(x″′)∈Y。

Step 9: with H ₂(x " ') as call number, search the attack type of this call number correspondence, this attack type is exported as final testing result.

Effect of the present invention can further specify the extensive standard intrusion detection of standard emulated data by following:

1, simulated conditions

Emulation of the present invention is at Windows XP, and SPI, CPU Pentium (R) 4, fundamental frequency 2.4GHZ, software platform are the VC++6.0 operation.The original intrusion detection that emulation is selected for use is data from common data sets KDD CUP ' 99, and this data centralization comprises a kind of normal type Normal and four kinds of invasions type: DOS, Probe, R2L and U2R, and every network behavior record comprises 41 features.Choose wherein two sub-data sets " kddcup99.data.10_percent " and " corrected ", " kddcup99.data.10_percent " is as training dataset, " corrected " as test data set, its sample distribution situation is as shown in table 1.

Table 1 data set sample distribution

2, simulation result

Specific implementation process to emulation intrusion detection of the present invention is:

(1) with the existing capable preliminary treatment of network behavior record preprocessing submodule of data set " kddeup99.data.10_percent " input;

(2) data in the pretreated data set " kddcup99.data.10_percent " are divided into two classes, Normal is a normal class, label is 1, DOS, Probe, R2L, U2R are a unusual class, label is-1, from normal and abnormal data, extract 5000 and 10000 samples respectively, with this sample that extracts as training sample, adopt distributed network integrated study algorithm to learn, obtain the abnormality detection learning machine, using the node number in the emulation is that 20 BA does not have the scale network, and sample rate is 0.7, the weight undated parameter is 0.8, and exercise wheel is counted T ₁=10, basic grader is nuclear matching tracing learning machine KMPLM;

(3) with Normal class sample as territory, source sample, sample number is 97278, R2L class sample is as aiming field sample to be instructed, DOS, Probe, U2R class sample are other type sample of aiming field, Normal class sample is chosen in advance, obtain sample number and be 1874 source domain migration sample set, the basic grader of AdaBoost algorithm is nuclear matching tracing learning machine KMPLM in the emulation;

(4) randomly drawing sample by a certain percentage in the unusual four class data from pretreated data set " Kddcup99.data.10_percent ", all kinds of ratios is: DOS 2.5%, Probe 75%, R2L 100%, U2R 100%, with source domain migration sample set together as training sample, wherein give the label identical with the R2L type with source domain migration sample, adopt the distributed network integrated study algorithm of introducing transfer learning to train, obtain abnormal behaviour analytic learning machine, selecting the node number in the emulation for use is that 20 BA does not have the scale network, and sample rate is 0.6, and exercise wheel is counted T ₂=10, basic grader is nuclear matching tracing learning machine KMPLM;

(5) data set " corrected " is carried out preliminary treatment;

(6) pretreated data set " corrected " is input to the abnormality detection learning machine and tests, simulation result is as shown in table 2;

Table 2 abnormality detection accuracy

(7) abnormal data of " corrected " data set of data set after the preliminary treatment is input in the abnormal behaviour analytic learning machine tests, its simulation result is as shown in table 3, and wherein DTNL represents to introduce the distributed network integrated study algorithm of transfer learning.

Table 3 abnormal behaviour analyzing and testing accuracy

As seen from Table 2, this emulation adopts the abnormality detection learning machine of distributed network integrated study generation that normal type and Exception Type are all had the higher detection rate.

As seen from Table 3, this emulation is adopted distributed network integrated study algorithm DNB respectively and is introduced the distributed network integrated study algorithm DTNL generation abnormal behaviour analytic learning machine of transfer learning, the distributed network integrated study algorithm of introducing transfer learning has improved about 87.3% than distributed network integrated study algorithm to the verification and measurement ratio of R2L, and the verification and measurement ratio of other abnormal behaviours does not obviously reduce.

Above-mentioned whole intrusion detection process all realizes its function by computer program, finishes the detection to network behavior.

This embodiment is being to implement under the prerequisite with the technical solution of the present invention, provided detailed execution mode and concrete operating process, but protection scope of the present invention is not limited to the foregoing description.

Claims (5)

1. intruding detection system based on distributed migration network learning comprises:

The network behavior record preprocessing module comprises existing record preprocessing submodule and new record preliminary treatment submodule; Should existing record preprocessing submodule, be used for having label network behavior record collection to finish to quantize and normalized to existing, and will quantize with normalized after parameter import new record preliminary treatment submodule into; This new record preliminary treatment submodule, the parameter of utilizing existing record preprocessing submodule to import into quantizes and normalized new network behavior record, and will quantize and normalized after result transmission to the abnormality detection module;

The abnormality detection module comprises abnormality detection study submodule and abnormality detection test submodule; This abnormality detection study submodule, existingly there is label network behavior record collection to be divided into normal and unusual two classes with pretreated, therefrom randomly draw the part sample respectively, adopt distributed network integrated study algorithm to learn, generate the abnormality detection learning machine, and this learning machine is transferred to abnormality detection test submodule; This abnormality detection test submodule adopts the abnormality detection learning machine that the pretreated new network behavior record of input is carried out Classification and Identification, if the output result is normal, then do not deal with, and detection of end, otherwise, import this record into the abnormal behaviour analysis module;

The abnormal behaviour analysis module comprises that moving sample chooses submodule, abnormal behaviour analytic learning submodule and abnormal behaviour analytical test submodule in advance; This migration sample is chosen submodule in advance, there are the network behavior record setting source territory sample and the aiming field of label that exemplar is arranged with existing, wait to instruct sample that territory, source sample is finished in advance according to aiming field and choose, with the source domain migration sample input abnormal behaviour analytic learning submodule of selecting; This abnormal behaviour analytic learning submodule has exemplar together as training sample with source domain migration sample and the aiming field of importing, and adopts the distributed network integrated study algorithm of introducing transfer learning to learn, and generates abnormal behaviour analytic learning machine; This abnormal behaviour analytical test submodule adopts abnormal behaviour analytic learning machine to carry out Classification and Identification to the exception record of importing, and exports its attack type.

2. the intrusion detection method based on distributed migration network learning comprises the steps;

(1) input is existing a label network behavior record collection X, existingly has label network behavior record collection to quantize and the normalization preliminary treatment to this, obtains pretreated X ' as a result;

(2) there is the pretreated X ' as a result of label network behavior record collection to be divided into normal and unusual two classes with existing, comprise M class attack type in unusual, from normal and exceptional sample, randomly draw a part of sample respectively, adopt distributed network integrated study algorithm containing K ₁Carry out T on the network topology structure of individual node ₁The wheel training, the grader network system of generation abnormality detection learning machine;

(3) set the sample of the middle normal type of X ' as territory, source sample set X ^S, sample number is m, the sample of Exception Type is as aiming field sample set X ^T, X ^TThe sample of the Exception Type that middle verification and measurement ratio is lower waits to instruct sample set X as aiming field ^T1, sample number is n ₁, and with X ^SDivide equally and be m/n ₁Part, be expressed as:

Wherein [] is rounding operation, will With X ^T1Be combined as training set Adopt the method for adjusting sample weights in the AdaBoost algorithm training process to adjust sample weights, select bigger territory, the source sample subclass of weight

(4) with territory, source sample subclass

With other type sample of aiming field as training sample, reuse in the AdaBoost algorithm training process method of adjusting sample weights and adjust sample weights, from

Middle bigger territory, the source sample of weight of removing will

Middle samples remaining is formed source domain migration sample set TR _D

(5) from aiming field sample set X ^TIn randomly draw a part of sample and form aiming field sample subclass TR _S, with the source domain migration sample set TR that selects _DTogether as training sample, with TR _DGive with aiming field and wait the label that instructs sample identical, layout contains K ₂The network topology structure of individual node, input sampling rate ρ ₂, exercise wheel counts T ₂, with TR _SAnd TR _DBe distributed on each node, generate the training sample set S on each node _k, k=1,2 ..., K ₂, generate abnormal behaviour analytic learning machine as follows according to this training sample set:

5a) each node training sample set S of initialization _kThe weight of middle sample;

5b) to each node training sample set S _kThe weight sampling of putting back to is arranged, obtain the training subclass of each node, in the learning algorithm of each node, train, obtain the basic grader of each node With the basic grader of each node to this S _kClassify, wherein t is current exercise wheel number;

5c) basis is to S _kClassification results calculate the weighting error rate ε of aiming field sample on each node _{K, t}, and according to ε _{K, t}, calculate the weight of each basic grader

5d) the weight of renewal source domain migration sample and aiming field sample is as t＜T ₂The time, change step 5b, work as t=T ₂The time, finish training, obtain by all basic graders (k=1,2 ..., K ₂, t=1,2 ..., T ₂) the grader network system of the abnormal behaviour analytic learning machine formed;

(6) existing network behavior record is quantized and normalized, and the parameter after the record preprocessing: as new network behavior record x " during input; according to having the parameter that the network behavior record preprocessing obtains; it is quantized and the normalization preliminary treatment, obtains pretreated network behavior record x ' as a result ";

(7) with x ' " be input in the grader network system of the abnormality detection learning machine that step 2 generates and classify, obtain classification results:

$H_1\left(x^{\&prime;\&prime;\&prime;}\right)=\mathrm{sign}\left(\underset{k=1}{\overset{K_1}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_1}{\mathrm{\&Sigma;}}}({\mathrm{\&alpha;}}_{k,t}^1{h}_{k,t}^1\left(x^{\&prime;\&prime;\&prime;}\right)+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^1{h}_{p,t}^1\left(x^{\&prime;\&prime;\&prime;}\right))\right)$

Wherein, For each basic grader in the abnormality detection learning machine to x ' " classification results,

Be the weight of each basic grader, p is the neighboring node label of node k, works as H ₁(x ' ") be 1 o'clock, expression belongs to normal type, does not do any processing, the detection of end process; Work as H ₁(x ' ") be-1 o'clock, expression belongs to Exception Type, then changes step (8) over to;

(8) with x ' " be input in the grader network system of the abnormal behaviour analytic learning machine that step 5 generates and classify, obtain classification results:

$H_2\left(x^{\&prime;\&prime;\&prime;}\right)=\underset{y\&Element;Y}{\arg \max }\left(\underset{k=1}{\overset{K_2}{\mathrm{\&Sigma;}}}\underset{t=1}{\overset{T_2}{\mathrm{\&Sigma;}}}\left({\mathrm{\&alpha;}}_{k,t}^{2}I\right[h_{k,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y]+\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^{2}I[h_{p,t}^2\left(x^{\&prime;\&prime;\&prime;}\right)=y\left]\right)\right)$

Wherein, For each basic grader in the abnormal behaviour analytic learning machine to x ' " classification results, and

Y={1 wherein, 2 ..., M}, 1,2 ..., M is respectively the call number of M kind attack type, I[] and be indicator function, its value is 0 or 1, H ₂(x ' ") ∈ Y;

(9) with H ₂(x ' ") as call number, search the attack type of this call number correspondence, this attack type is exported as final testing result.

3. method according to claim 2, wherein described bigger territory, the source sample subclass of sample weights of selecting of step 3

Choose as follows:

3a) set

Label be+1, X ^T1Label be-1, input sample weights threshold value W ₁

3b) will Import respectively in the AdaBoost algorithm and train, make sample weights obtain adjusting, after many wheel training finish, respectively from In select sample weights greater than threshold value W ₁And belong to

Sample, form territory, source sample subclass

4. method according to claim 2, wherein step 4 described from

Middle bigger territory, the source sample of weight of removing, remove as follows:

4a) with aiming field sample X ^TIn remove the sample X wait to instruct type ^T1Outside, other Exception Types be expressed as X ^T2, set Label be+1, X ^T2Label be-1, input sample weights threshold value W ₂

4b) will With X ^T2Form training set T ², be input in the AdaBoost algorithm and train, make sample weights obtain adjusting, after many wheel training finish, remove T ²Middle sample weights is greater than threshold value W ₂And belong to

Sample, will

Middle samples remaining is formed source domain migration sample set TR _D

5. method according to claim 2, the wherein weight of the described renewal of step 5d source domain migration sample and aiming field sample, renewal as follows:

5a) calculate the weight undated parameter of aiming field sample respectively

Weight with source domain migration sample

Undated parameter

M wherein _kBe the node k S of place _kIn contained source domain migration sample number;

5b) the sample x of new node k place more _iWeight D _{K, t}(x _i), obtain upgrading back weight D _{K, t+1}(x _i):

$D_{k,t+1}\left(x_i\right)=\left\{\begin{array}{cc}\frac{D_{k,t}\left(x_i\right)\&CenterDot;{{\mathrm{\&beta;}}_{k,t}}^{{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}}{Z_{k,t}},& x_i\&Element;S_k\&cap;{\mathrm{TR}}_S\\ \frac{D_{k,t}\left(x_i\right)\&CenterDot;{{\mathrm{\&gamma;}}_k}^{-{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}}{Z_{k,t}},& x_i\&Element;S_k\&cap;{\mathrm{TR}}_D\end{array}\right.$

Wherein, x _i∈ S _k∩ TR _SExpression x _iBelong to S _kIn the aiming field sample, x _i∈ S _k∩ TR _DExpression x _iBelong to S _kIn source domain migration sample, and

$Z_{k,t}=\underset{x_i\&Element;S_k\&cap;{\mathrm{TR}}_S}{\mathrm{\&Sigma;}}{D}_{k,t}\left(i\right)\&CenterDot;{{\mathrm{\&beta;}}_{k,t}}^{{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}+\underset{x_i\&Element;S_k\&cap;{\mathrm{TR}}_D}{\mathrm{\&Sigma;}}{D}_{k,t}\left(i\right)\&CenterDot;{{\mathrm{\&gamma;}}_k}^{-{\mathrm{\&lambda;}}_{k,t}\left(x_i\right)}$

${\mathrm{\&lambda;}}_{k,t}\left(x_i\right)=-2{\mathrm{\&alpha;}}_{k,t}^2(I(y\left(x_i\right)\&NotEqual;h_{k,t}^2\left(x_i\right))-1/2)-2\underset{p}{\mathrm{\&Sigma;}}{\mathrm{\&alpha;}}_{p,t}^2(I(y\left(x_i\right)\&NotEqual;h_{p,t}^2\left(x_i\right))-1/2)$

Wherein, y (x _i) be sample x _iKnown label,

For

To sample x _iClassification results, and $h_{k,t}^2\left(x_i\right)\&Element;Y.$

Images