CN101582813B

CN101582813B - Distributed migration network learning-based intrusion detection system and method thereof

Info

Publication number: CN101582813B
Application number: CN2009100230731A
Authority: CN
Inventors: 缑水平; 焦李成; 王宇琴; 田小林; 王爽; 马文萍; 吴建设; 慕彩红; 冯静
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2009-06-26
Filing date: 2009-06-26
Publication date: 2011-07-20
Anticipated expiration: 2029-06-26
Also published as: CN101582813A

Abstract

The invention discloses a distributed migration network learning-based intrusion detection system and a method thereof, and mainly solves the problems that the prior method has low efficiency in detection of some attack types and is difficult to search data again. The whole system comprises a network behavior record preprocessing module, an abnormality detection module and an abnormal behavior analyzing module. The network behavior record preprocessing module completes the quantification and normalization processing of a network behavior record; the abnormality detection module uses an abnormality detection learning machine to completes the classification and identification for an input record, determines whether the record is a normal behavior, and completes the detection if the record isa normal behavior or transmits the record to the abnormal behavior analyzing module if the record is an abnormal behavior; and the abnormal behavior analyzing module uses an abnormal behavior analyzi ng learning machine to carry out the classification and identification of the input records and outputs the attach type of the record. The system and the method have the advantages of using other existing resources to improve the detection rate for the prior attach types with low detection rate and avoiding searching the data again and can be used for network intrusion detection.

Description

Intrusion detection system and method based on distributed migration network learning

Technical Field

The invention belongs to the field of network security, and particularly relates to an intrusion detection system which can be used for intrusion detection in the aspect of information security.

Background

With the wide use of the Internet, more and more illegal attacks on computer networks threaten the security of information systems, and network security systems mainly managed by passive management, such as firewall, cannot attack, steal or destroy information caused by unauthorized operation of internal users and the like on backdoors of application layers, and the firewall is easy to be attacked, so that security problems occurring in the internal networks are often restrained. Intrusion detection systems IDS show increasing importance as an important complement to the network security protection tool "firewall". IDS is a security backdoor of firewalls that can protect against attacks from internal networks, deter hackers from intrusions, and prevent the spread of viruses. With audit records, the IDS is able to identify any unwanted activities, thereby limiting those activities and protecting the security of the system. In 1998, the MIT Lincoln laboratory developed intrusion detection system evaluation in cooperation with DARPA, and one of the tasks of the project was to provide data sets for intrusion detection including host logs and network traffic, and KDD CUP' 99 performed appropriate processing and feature extraction on the 9-week tcpdump data provided by DARPA as a standard intrusion detection data set.

In recent years, with the development of machine learning, new intrusion detection technologies, such as single-machine intrusion detection methods based on neural networks, bayesian networks, support vector machines, etc., have appeared continuously, but in the face of new environments and network security problems in new situations, the above intrusion detection technologies have the problems of high false alarm rate, poor adaptability, low degree of automatic response, low degree of intelligence, etc., so that the distributed algorithm becomes necessary to improve the detection speed and detection accuracy of the IDS. In 2006, the king army et al introduced the Boosting algorithm into a classifier network, integrated and used the classifier network with classifiers, and extended to a distributed environment, proposed a distributed network integration algorithm DNB, and obtained a classifier system with a stronger generalization ability through communication and cooperation between node classifiers. Because the DNB-based intrusion detection method, like other conventional machine learning methods, cannot train a better classifier model when there is little label data, and requires that training data and test data are independently and identically distributed, there are the following disadvantages:

1. the detection rate of the network behaviors of different attack types is unbalanced, and the detection rate of the network behaviors of certain attack types is low;

2. if the detection rate is to be improved, the user needs to collect data again and learn, and the task is expensive and takes time;

3. it is not possible to take advantage of the other available resources that exist to improve the network behavior detection rate for certain attack types.

Disclosure of Invention

The invention aims to overcome the defects of the intrusion detection method, introduces the migration learning into DNB, and provides an intrusion detection system based on distributed migration network learning and a method thereof, so as to guide the learning of network behaviors with lower detection rate by using other existing data, thereby improving the detection rate.

To achieve the above object, the detection system of the present invention comprises:

the network behavior record preprocessing module is used for completing quantization and normalization preprocessing on the collected network behavior records and transmitting the preprocessed result to the abnormality detection module;

the abnormal detection module is used for classifying and identifying the input record by adopting an abnormal detection learning machine, determining whether the record belongs to a normal behavior, if the record belongs to the normal behavior, not processing the record, and ending the detection, otherwise, transmitting the record to the abnormal behavior analysis module;

and the abnormal behavior analysis module is used for classifying and identifying the input abnormal records by adopting an abnormal behavior analysis learning machine and outputting the attack types of the records.

The network behavior record preprocessing module comprises:

the existing record preprocessing submodule is used for completing quantization and normalization processing on an existing labeled network behavior record set and transmitting parameters after quantization and normalization processing into the new record preprocessing submodule;

and the new record preprocessing submodule is used for quantizing and normalizing the new network behavior record by utilizing the parameters transmitted by the existing record preprocessing submodule.

The abnormality detection module includes:

the abnormity detection learning submodule divides the preprocessed existing labeled network behavior record set into a normal type and an abnormal type, respectively and randomly extracts partial samples from the abnormal and abnormal labeled network behavior record set, adopts a distributed network integration learning algorithm to learn, generates an abnormity detection learning machine, and transmits the learning machine to the abnormity detection testing submodule;

and the anomaly detection testing sub-module is used for classifying and identifying the input preprocessed new record by adopting an anomaly detection learning machine, if the output result is normal, the input new record is not processed, the detection is finished, and otherwise, the input new record is transmitted to the anomaly behavior analysis module.

The abnormal behavior analysis module comprises:

a migration sample pre-selection submodule, which sets a source domain sample and a target domain labeled sample for the existing labeled network behavior record, completes pre-selection on the source domain sample according to the target domain to-be-guided sample, and inputs the selected source domain migration sample into an abnormal behavior analysis learning submodule;

the abnormal behavior analysis learning submodule is used for taking the input source domain migration sample and the target domain labeled sample as training samples together, learning by adopting a distributed network integrated learning algorithm introduced with migration learning, and generating an abnormal behavior analysis learning machine;

and the abnormal behavior analysis and test submodule is used for classifying and identifying the input abnormal behavior by adopting an abnormal behavior analysis learning machine and outputting the attack type of the input abnormal behavior.

In order to achieve the above object, the detection method of the present invention comprises the following steps:

(1) inputting an existing labeled network behavior record set X, and carrying out quantization and normalization pretreatment on the data set to obtain a pretreated result X';

(2) dividing the result X' after the preprocessing of the existing labeled network behavior record set into normal and abnormal types, wherein the abnormality comprises M types of attack types, randomly extracting a part of samples from the normal and abnormal samples respectively, and adopting a distributed network ensemble learning algorithm to obtain a result containing K₁Performing T on network topology of individual nodes₁Performing round training to generate a classifier network system of the anomaly detection learning machine;

(3) setting the normal type sample in X' as the source domain sample set X^SThe number of samples is m, and the samples of abnormal types are taken as a target domain sample set X^T，X^TTaking samples of abnormal types with lower detection rate as target domain sample set X to be guided^T1The number of samples is n₁And X is^SThe average is m/n₁Parts, expressed as:

wherein[·]For rounding operation, X is_i ^SAnd X^T1Combined as a training set T_i ¹(i＝1，2，...，[m/n₁]) Adjusting sample weight by adopting a method for adjusting sample weight in the training process of AdaBoost algorithm, and selecting a source domain sample subset X with larger weight_sub ^S；

(4) Subset of source domain samples X_sub ^SUsing samples of other types of the target domain as training samples, adjusting the sample weight by using the method for adjusting the sample weight in the training process of the AdaBoost algorithm again, and adjusting the sample weight from X_sub ^SRemoving the source domain samples with larger weight from the sample list, and dividing X_sub ^SThe remaining samples constitute a source domain migration sample set TR_D；

(5) From a target domain sample set X^TIn the random sampling of a part of samples to form a target domain sample subset TR_SWith the selected source domain migration sample set TR_DTaking TR as training sample_DAssigning the same label as the target domain to be guided sample, the layout containing K₂Network topology of individual nodes, input sampling rate ρ₂Training round number T₂Will TR_SAnd TR_DDistributed on each node to generate a training sample set S on each node_k，k＝1，2，...，K₂And generating an abnormal behavior analysis learning machine according to the training sample set by the following steps:

5a) initializing each node training sample set S_kThe weight of the middle sample;

5b) training sample set S for each node_kCarrying out weighting sampling with the return to obtain a training subset of each node, carrying out training in a learning algorithm of each node to obtain a base classifier C of each node_k，t ²The base classifier of each node is used to pair S_kClassifying, wherein t is the number of current training rounds;

5c) according to the pair S_kCalculating the weighted error rate epsilon of the target domain samples on each node according to the classification result_k，tAnd according to epsilon_k，tCalculating the weight alpha of each base classifier_k，t ²：

5d) Updating the weights of the source domain migration sample and the target domain sample when T < T₂When T is T, go to step 5b₂Then finish training and get the result of all base classifiers C_k，t ²(k＝1，2，...，K₂，t＝1，2，...，T₂) A classifier network system of the abnormal behavior analysis learning machine is formed;

(6) inputting a new network behavior record x ', and carrying out quantization and normalization preprocessing on the new network behavior record x ' to obtain a preprocessed network behavior record result x ';

(7) inputting x' into the classifier network system of the anomaly detection learning machine generated in the step 2 for classification to obtain a classification result:

wherein h is_k，t ¹(x '") is the result of the classification of x'" by each base classifier in the anomaly detection learning machine, α_k，t ¹For the weight of each base classifier, p is the neighbor node index of node k, when H₁(x' ") is 1, it indicates that it is of a normal type, and the detection process is ended without any processing; when H is present₁If (x') is-1, indicating that the type is abnormal, the procedure goes to step (8);

(8) inputting x' into the classifier network system of the abnormal behavior analysis learning machine generated in the step 5 for classification, and obtaining a classification result:

<math><mrow><msub><mi>H</mi><mn>2</mn></msub><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><munder><mrow><mi>arg</mi><mi>max</mi></mrow><mrow><mi>y</mi><mo>&Element;</mo><mi>Y</mi></mrow></munder><mrow><mo>(</mo><munderover><mi>Σ</mi><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>K</mi><mn>2</mn></msub></munderover><munderover><mi>Σ</mi><mrow><mi>t</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>T</mi><mn>2</mn></msub></munderover><mrow><mo>(</mo><msubsup><mi>α</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mi>I</mi><mrow><mo>[</mo><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><mi>y</mi><mo>]</mo></mrow><mo>+</mo><munder><mi>Σ</mi><mi>p</mi></munder><msubsup><mi>α</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mi>I</mi><mrow><mo>[</mo><msubsup><mi>h</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><mi>y</mi><mo>]</mo></mrow><mo>)</mo></mrow><mo>)</mo></mrow></mrow></math>

wherein h is_k，t ²(x '") is the result of classifying x'" by each base classifier in the abnormal behavior analysis learning machine, and

<math><mrow><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>&Element;</mo><mi>Y</mi><mo>,</mo></mrow></math>

where Y ═ 1, 2, ·, M, 1, 2,. and M are the index numbers of M attack types, I [ ·, respectively]For indicating the function, its value is 0 or 1, H₂(x″′)∈Y；

(9) H is to be₂(x') as an index number, searching the attack type corresponding to the index number, and outputting the attack type as a final detection result.

Compared with the prior art, the invention has the following advantages:

1) because the invention introduces the transfer learning, the existing other labeled data can be utilized to guide the learning of the attack type with lower detection rate, and the data does not need to be collected again;

2) according to the invention, as a method of adjusting the sample weight by adopting an AdaBoost algorithm is adopted, a migration sample with guiding significance on an attack type with lower detection rate can be selected;

3) according to the invention, as a distributed network integrated learning algorithm introducing transfer learning is adopted, the generated abnormal behavior analysis learning machine has higher detection rate to the attack type with lower original detection rate;

4) the invention adopts the distributed network integrated learning algorithm, and the generated anomaly detection learning machine has higher anomaly detection precision;

the invention is a network-based intrusion detection system, which can be used in various complex network environments. Simulation results show that for a standard large-scale network intrusion detection data set KDD CUP' 99, the detection rate of the intrusion detection method based on distributed migration network learning to the R2L attack type can be improved by about 87.3% compared with the intrusion detection method based on distributed migration network learning.

Drawings

FIG. 1 is a schematic diagram of an intrusion detection system based on distributed migration network learning according to the present invention;

FIG. 2 is a flowchart of an intrusion detection method based on distributed migration network learning according to the present invention;

FIG. 3 is a flow chart of pre-selection of source domain migration samples in the present invention;

fig. 4 is a flowchart of the learning machine for generating abnormal behavior analysis according to the present invention.

Detailed Description

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

Referring to fig. 1, the intrusion detection system based on distributed migration network learning of the present invention mainly includes: the system comprises a network behavior record preprocessing module, an abnormality detection module and an abnormal behavior analysis module. Wherein:

the network behavior record preprocessing module comprises: an existing record preprocessing submodule and a new record preprocessing submodule. The existing record preprocessing submodule is used for completing quantization and normalization processing on an existing labeled network behavior record set and transmitting parameters after quantization and normalization processing into the new record preprocessing submodule; the new record preprocessing submodule completes quantization and normalization processing on the new record by using the parameters transmitted by the existing record preprocessing submodule.

The abnormality detection module includes: an anomaly detection learning sub-module and an anomaly detection testing sub-module. Dividing the pre-processed existing labeled network behavior record set into a normal type and an abnormal type, respectively and randomly extracting partial samples from the pre-processed existing labeled network behavior record set, learning by adopting a distributed network integration learning algorithm to generate an abnormal detection learning machine, and transmitting the learning machine to an abnormal detection testing submodule; and the anomaly detection testing sub-module adopts an anomaly detection learning machine to classify and identify the input preprocessed new record, if the output result is normal, the input new record is not processed, the detection is finished, and if the output result is normal, the record is transmitted to the anomaly behavior analysis module.

The abnormal behavior analysis module comprises: a migration sample pre-selection submodule, an abnormal behavior analysis learning submodule and an abnormal behavior analysis testing submodule. The migration sample pre-selection submodule sets a source domain sample and a target domain labeled sample for the existing labeled network behavior record, performs pre-selection on the source domain sample according to the target domain to-be-guided sample, and inputs the selected source domain migration sample into the abnormal behavior analysis learning submodule; the abnormal behavior analysis learning submodule uses the input source domain migration sample and the target domain labeled sample as training samples together, and learns by adopting a distributed network integrated learning algorithm introduced with migration learning to generate an abnormal behavior analysis learning machine; and the abnormal behavior analysis and test sub-module classifies and identifies the input abnormal behavior by adopting an abnormal behavior analysis learning machine and outputs the attack type of the input abnormal behavior.

The intrusion detection system of the invention carries out quantization and normalization preprocessing on the existing network behavior record and records the preprocessed parameters; dividing the pre-processed existing records into a normal record and an abnormal record, respectively extracting partial data from the normal record and the abnormal record to be used as training samples, and learning by adopting a distributed network integration learning algorithm to obtain an abnormal detection learning machine; taking the normal record as a source domain sample, taking other abnormal records as a target domain sample, adjusting the sample weight through an AdaBoost algorithm, and pre-selecting the source domain migration sample according to the sample weight; respectively randomly extracting a part of samples from the preprocessed recorded four types of abnormal data, taking the part of samples and the pre-selected source domain migration samples as training samples, learning by adopting a distributed network integration algorithm introduced with migration learning, and generating an abnormal behavior analysis learning machine; when a new network behavior record is input, the network behavior record is subjected to quantization and normalization preprocessing according to parameters obtained by preprocessing the existing network behavior record, then the preprocessed result is input to an abnormal detection learning machine for testing, if the detection is normal, the preprocessing is not performed, the detection is finished, otherwise, the preprocessed result is input to an abnormal behavior analysis learning machine for testing, and finally the attack type of the abnormal behavior record is output.

Referring to fig. 2, the intrusion detection method of the present invention includes the following steps:

step 1: inputting the existing labeled network behavior record set X, and carrying out quantization and normalization pretreatment on the data set to obtain a pretreated result X'.

The specific process of pretreatment is as follows:

1a) for the attribute with the attribute value in X as the character string, counting the type and quantizing to obtain the quantized result X₁；

1b) To X₁The value range of the attribute value of the source address byte number and the target address byte number in the attribute is determined as 0, 1.3 multiplied by 10⁹]Two attribute values are log10 (-) transformed to range to [0.0, 9.14 ]]Obtaining a transformed result X₂；

1c) For the transformed result X₂The following normalization was performed:

suppose X₂Comprising n samples, each sample having d-dimensional features, representing d-dimensional feature vectors of the n samples as X₂＝[F₁，F₂，...，F_d]And expressing the ith dimension feature vector as F_i＝[f_i1，f_i2，...，f_in]All features are normalized as follows:

f′_ij＝f_ij/max(F_i)，i＝1，2，...，d，j＝1，2，...，n

f′_i＝[f′_i1，f′_i2，...，f′_in]，i＝1，2，...，d

X′＝[F′₁，F′₂，...，F′_d]

the result X' after pretreatment was obtained.

Step 2: dividing the result X' after the preprocessing of the existing labeled network behavior record set into normal and abnormal types, wherein the abnormality comprises M types of attack types, randomly extracting a part of samples from the normal and abnormal samples respectively as training samples, and adopting a distributed network ensemble learning algorithm to perform K-contained test on the samples containing K₁Performing T on network topology of individual nodes₁And performing round training to generate a classifier network system of the anomaly detection learning machine.

And step 3: setting other types of labeled samples existing in X' as the source domain sample set X^SThe number of samples is m, and the samples of abnormal types are taken as a target domain sample set X^T，X^TTaking samples of abnormal types with lower detection rate as target domain sample set X to be guided^T1The number of samples is n₁According to X^SAnd X^T1Selecting a source domain sample subset X with a larger sample weight by adopting a method for adjusting the sample weight in the training process of the AdaBoost algorithm_sub ^S。

Selecting the subset X of the source domain samples with the larger sample weight_sub ^SThe selection process is shown in fig. 3(a), and the specific steps are as follows:

3a) input source domain sample set X^SSample set X of target domain^TLabeled sample set X of the type in which the target domain is to be guided^T1Sample weight threshold W₁；

3b) Since n is₁M, set to X^SIs marked with a labelIs +1, X^T1Is-1, and to balance the two types of samples, X is^SThe average is m/n₁Is prepared from

Wherein [. ]]For rounding operations, a training set is formed

3c) Will T_i ¹(i＝1，2，...，[m/n₁]) Inputting into AdaBoost algorithm for training, and respectively training from T after multiple rounds of training_i ¹(i＝1，2，...，[m/n₁]) The weight of the selected sample is larger than the threshold value W₁And belong to X_i ^S(i＝1，2，...，[m/n₁]) Of (2) constituting a source domain sample subset X_sub ^S。

And 4, step 4: subset of source domain samples X_sub ^SUsing samples of other types of the target domain as training samples, adjusting the sample weight by using the method for adjusting the sample weight in the training process of the AdaBoost algorithm again, and adjusting the sample weight from X_sub ^SRemoving the source domain samples with larger weight from the sample list, and dividing X_sub ^SThe samples in the remainder constitute the source domain migration sample set TR_D。

From X above_sub ^SRemoving the source domain samples with larger weightThe flow of (a) is shown in fig. 3(b), and the specific steps are as follows:

4a) sample set X of target domain^TDividing sample set X of type to be guided^T1In addition, the sample set for other exception types is denoted X^T2Setting X_sub ^SIs +1, X^T2Is labeled-1, input sample weight threshold W₂；

4b) Mixing X_sub ^SAnd X^T2Form a training set T²Inputting the training data into an AdaBoost algorithm for training to adjust the sample weight, and removing the sample weight in a training set from being larger than a threshold value W after the multi-round training is finished₂And belong to X_sub ^SA sample of (2), A1_sub ^SThe remaining samples constitute a source domain migration sample set TR_D。

And 5: from the target domain sample X^TIn the random sampling of a part of samples to form a target domain sample subset TR_SWith the selected source domain migration sample set TR_DTaking TR as training sample_DGiving a label the same as a target domain sample to be guided, training by adopting a distributed network integrated learning algorithm introduced with transfer learning, and generating an abnormal behavior analysis learning machine, wherein the specific process is as shown in fig. 4:

5a) the input contains K₂Network topology structure of each node, sampling rate rho and training round number T₂And combining the target domain sample set TR_SAnd source domain migration sample set TR_DDistributed on each node to generate a training sample set S on each node_k，k＝1，2，...，K₂；

5b) Initialize the weight D of the ith sample on each node k_k，1(x_i)＝1/l_kWherein l is_kFor training set S on node k_kThe number of samples contained in (i) 1, 2_k，k＝1，2，...，K₂；

5c) From the training set S of nodes using weighted sampling with putting back_kMiddle samplingAnd obtaining a training subset of nodes, wherein the number of samples is l_kρ，k＝1，2，...，K₂T is the current number of training rounds;

5d) training a base classifier C at node k according to each node training subset_k，t ²And using a base classifier C_k，t ²For training set S on the node_kClassifying;

5e) according to the pair S_kCalculating a weighted error rate epsilon of the target domain samples_k，t：

<math><mrow><msub><mi>ϵ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><munder><mi>Σ</mi><mrow><msub><mi>x</mi><mi>i</mi></msub><mo>&Element;</mo><msub><mi>S</mi><mi>k</mi></msub><mo>∩</mo><msub><mi>TR</mi><mi>S</mi></msub></mrow></munder><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mi>I</mi><mrow><mo>[</mo><mi>y</mi><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>&NotEqual;</mo><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>]</mo></mrow><mo>,</mo><mi>k</mi><mo>=</mo><mn>1,2</mn><mo>,</mo><mo>.</mo><mo>.</mo><mo>.</mo><mo>,</mo><msub><mi>K</mi><mn>2</mn></msub><mo>;</mo></mrow></math>

Wherein h is_k，t ²(x_i) Is the base classifier C_k，t ²For sample x_iAs a result of the classification of (a),

<math><mrow><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>&Element;</mo><mi>Y</mi><mo>,</mo></mrow></math>

y ═ 1, 2,. multidot.m, M is, respectivelyIndex number of M attack types, y (x)_i) Is a sample x_iA known tag of (a);

5f) compute basis classifier C_k，t ²Weight of alpha_k，t ²：

5g) Calculating a weight update parameter of the target domain samples as

Weight update parameters for source domain migration samples

Wherein m is_kMigrating the number of samples for the source domain at node k;

5h) updating sample x at node k_iWeight D of_k，t(x_i) Get the updated weight D_k，t+1(x_i)：

<math><mrow><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi><mo>+</mo><mn>1</mn></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><mfenced open='{' close=''><mtable><mtr><mtd><mfrac><mrow><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>·</mo><msup><msub><mi>β</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><msub><mi>λ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow></mrow></msup></mrow><msub><mi>Z</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub></mfrac><mo>,</mo></mtd><mtd><msub><mi>x</mi><mi>i</mi></msub><mo>&Element;</mo><msub><mi>S</mi><mi>k</mi></msub><mo>∩</mo><msub><mi>TR</mi><mi>S</mi></msub></mtd></mtr><mtr><mtd><mfrac><mrow><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>·</mo><msup><msub><mi>γ</mi><mi>k</mi></msub><mrow><mo>-</mo><msub><mi>λ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow></mrow></msup></mrow><msub><mi>Z</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub></mfrac><mo>,</mo></mtd><mtd><msub><mi>x</mi><mi>i</mi></msub><mo>&Element;</mo><msub><mi>S</mi><mi>k</mi></msub><mo>∩</mo><msub><mi>TR</mi><mi>D</mi></msub></mtd></mtr></mtable></mfenced></mrow></math>

Wherein x is_i∈S_k∩TR_SDenotes x_iBelong to S_kTarget domain samples of (1), x_i∈S_k∩TR_DDenotes x_iBelong to S_kThe source domain in (1) migrates the sample, and

<math><mrow><msub><mi>Z</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mo>=</mo><munder><mi>Σ</mi><mrow><msub><mi>x</mi><mi>i</mi></msub><mo>&Element;</mo><msub><mi>S</mi><mi>k</mi></msub><mo>∩</mo><msub><mi>TR</mi><mi>S</mi></msub></mrow></munder><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>·</mo><msup><msub><mi>β</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><msub><mi>λ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow></mrow></msup><mo>+</mo><munder><mi>Σ</mi><mrow><msub><mi>x</mi><mi>i</mi></msub><mo>&Element;</mo><msub><mi>S</mi><mi>k</mi></msub><mo>∩</mo><msub><mi>TR</mi><mi>D</mi></msub></mrow></munder><msub><mi>D</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>·</mo><msup><msub><mi>γ</mi><mi>k</mi></msub><mrow><mo>-</mo><msub><mi>λ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow></mrow></msup></mrow></math>

<math><mrow><msub><mi>λ</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow></msub><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>=</mo><mo>-</mo><mn>2</mn><msubsup><mi>α</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><mi>I</mi><mrow><mo>(</mo><mi>y</mi><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>&NotEqual;</mo><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>)</mo></mrow><mo>-</mo><mn>1</mn><mo>/</mo><mn>2</mn><mo>)</mo></mrow><mo>-</mo><mn>2</mn><munder><mi>Σ</mi><mi>p</mi></munder><msubsup><mi>α</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><mi>I</mi><mrow><mo>(</mo><mi>y</mi><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>&NotEqual;</mo><msubsup><mi>h</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>)</mo></mrow><mo>-</mo><mn>1</mn><mo>/</mo><mn>2</mn><mo>)</mo></mrow></mrow></math>

wherein p is the label of the neighbor node of the node k;

5i) when T < T₂When T is equal to T, go to step 5c₂Then finish training and get the result of all base classifiers C_k，t ²，k＝1，2，...，K₂，t＝1，2，...，T₂And forming a classifier network system of the abnormal behavior analysis learning machine.

Step 6: a new network behavior record x "is entered and preprocessed to obtain a preprocessed result x'".

6a) Quantizing the attribute value which is the attribute of the character string according to the method in the step 1a), wherein the result after quantization is x ″₁；

6b) Respectively carrying out log10 (-) transformation on the attribute values of source address byte number and destination address byte number to obtain a transformed result x ″₂；

6c) Will record x ″)₂D-dimensional feature of (a) is denoted as x ″)₂＝{f″₁，f″₂，...，f″_dAnd for x ″)₂The results x' "after pretreatment were obtained by performing the following normalization:

x″′＝{f″′₁，f″′₂，...，f″′_d}。

and 7: inputting x' into the classifier network system of the anomaly detection learning machine generated in the step 2 for classification to obtain a classification result:

wherein h is_k，t ¹(x '") is the result of the classification of x'" by each base classifier in the anomaly detection learning machine, α_k，t ¹For each base classifier weight, when H₁(x' ") is 1, indicating that it is of the normal type, no processing is performed, and the detection process is ended, when H is₁If (x' ") is-1, it indicates that the type is abnormal, the process proceeds to step (8).

And 8: inputting x' into the classifier network system of the abnormal behavior analysis learning machine generated in the step 5 for classification, and obtaining a classification result:

H₂(x″′)∈Y。

and step 9: h is to be₂(x') as an index number, searching the attack type corresponding to the index number, and outputting the attack type as a final detection result.

The effect of the invention can be further illustrated by the following standard large-scale standard intrusion detection simulation data:

1. simulation conditions

The simulation of the invention runs in Windows XP, SPI, CPU Pentium (R)4, basic frequency 2.4GHZ, and software platform VC + + 6.0. The raw intrusion detection data selected for simulation are derived from a common data set KDD CUP' 99, which comprises a Normal type Normal and four intrusion types: DOS, Probe, R2L, and U2R, each network behavior record containing 41 features. Two sub-data sets of 'kdcccup 99.data.10_ percent' and 'corrected', 'kdcccup 99.data.10_ percent' are selected as training data sets and 'corrected' is selected as a test data set, and the distribution of samples is shown in table 1.

Table 1 data set sample distribution

2. Simulation result

The specific implementation process of the simulation intrusion detection of the invention is as follows:

(1) inputting a data set 'kddeup 99.data.10_ percent' into an existing network behavior record preprocessing submodule for preprocessing;

(2) dividing data in a preprocessed data set 'kddcup 99.data.10_ percent' into two types, wherein Normal is a Normal type, labels are 1, DOS, Probe, R2L and U2R are abnormal types, labels are-1, respectively extracting 5000 and 10000 samples from Normal and abnormal data, taking the extracted samples as training samples, learning by adopting a distributed network ensemble learning algorithm to obtain an abnormal detection learning machine, using a BA scale-free network with the node number of 20 in simulation, the sampling rate of 0.7, the weight updating parameter of 0.8 and the training round number T of 0₁10, the base classifier is a kernel matching pursuit learning machine KMPLM;

(3) taking a Normal sample as a source domain sample, wherein the number of the samples is 97278, taking an R2L sample as a target domain sample to be guided, taking DOS, Probe and U2R samples as other types of samples of the target domain, pre-selecting the Normal sample to obtain a source domain migration sample set with the number of the samples 1874, and taking a base classifier of an AdaBoost algorithm in simulation as a kernel matching pursuit learning machine KMPLM;

(4) randomly extracting samples from four abnormal types of data in a preprocessed data set Kddcup99.data.10_ percent according to a certain proportion, wherein the proportion of each type is as follows: DOS 2.5%, Probe 75%, R2L 100%U2R 100% and the source domain migration sample set are used as training samples, wherein the source domain migration samples are endowed with labels with the same type as R2L, a distributed network integrated learning algorithm introducing migration learning is adopted for training to obtain an abnormal behavior analysis learning machine, a BA scale-free network with 20 nodes is selected in simulation, the sampling rate is 0.6, and the number of training rounds T is used as a training sample₂10, the base classifier is a kernel matching pursuit learning machine KMPLM;

(5) preprocessing the data set 'corrected';

(6) inputting the preprocessed data set 'corrected' into an anomaly detection learning machine for testing, wherein the simulation result is shown in table 2;

TABLE 2 anomaly detection accuracy

(7) Abnormal data of the 'corrected' data set of the preprocessed data set is input into an abnormal behavior analysis learning machine for testing, and a simulation result is shown in table 3, wherein DTNL represents a distributed network integrated learning algorithm for introducing migration learning.

TABLE 3 abnormal behavior analysis detection accuracy

As can be seen from Table 2, the simulation has a high detection rate for both normal types and abnormal types by using an abnormal detection learning machine generated by distributed network ensemble learning.

As can be seen from table 3, the abnormal behavior analysis learning machine is generated by the simulation respectively using the distributed network ensemble learning algorithm DNB and the distributed network ensemble learning algorithm DTNL introducing the migration learning, and the detection rate of the distributed network ensemble learning algorithm introducing the migration learning to R2L is improved by about 87.3% compared with the distributed network ensemble learning algorithm, and the detection rates of other abnormal behaviors are not significantly reduced.

The whole intrusion detection process realizes the functions thereof through a computer program to finish the detection of network behaviors.

The embodiment is implemented on the premise of the technical scheme of the invention, and a detailed implementation mode and a specific operation process are given, but the protection scope of the invention is not limited to the embodiment.

Claims

1. An intrusion detection system based on distributed migration network learning, comprising:

the network behavior record preprocessing module comprises an existing record preprocessing submodule and a new record preprocessing submodule; the existing record preprocessing submodule is used for completing quantization and normalization processing on an existing labeled network behavior record set and transmitting parameters after quantization and normalization processing into a new record preprocessing submodule; the new record preprocessing submodule carries out quantization and normalization processing on the new network behavior record by utilizing parameters transmitted by the existing record preprocessing submodule, and transmits the result after quantization and normalization processing to the abnormality detection module;

the abnormality detection module comprises an abnormality detection learning submodule and an abnormality detection testing submodule; the abnormity detection learning submodule divides the preprocessed existing labeled network behavior record set into a normal type and an abnormal type, respectively and randomly extracts partial samples from the normal type and the abnormal type, adopts a distributed network integration learning algorithm to learn, generates an abnormity detection learning machine, and transmits the learning machine to an abnormity detection testing submodule; the anomaly detection testing sub-module adopts an anomaly detection learning machine to classify and identify the input preprocessed new network behavior record, if the output result is normal, the input result is not processed, the detection is finished, and if the output result is not normal, the record is transmitted to an anomaly behavior analysis module;

the abnormal behavior analysis module comprises a migration sample pre-selection sub-module, an abnormal behavior analysis learning sub-module and an abnormal behavior analysis testing sub-module; the migration sample pre-selection submodule sets a source domain sample and a target domain labeled sample for the existing labeled network behavior record, completes pre-selection on the source domain sample according to the target domain to-be-guided sample, and inputs the selected source domain migration sample into an abnormal behavior analysis learning submodule; the abnormal behavior analysis learning submodule takes an input source domain migration sample and a target domain labeled sample as training samples together, and adopts a distributed network integrated learning algorithm introduced with migration learning to learn so as to generate an abnormal behavior analysis learning machine; the abnormal behavior analysis testing submodule classifies and identifies the input abnormal records by adopting an abnormal behavior analysis learning machine and outputs the attack types of the input abnormal records.

2. An intrusion detection method based on distributed migration network learning comprises the following steps;

(1) inputting an existing labeled network behavior record set X, and carrying out quantization and normalization pretreatment on the existing labeled network behavior record set to obtain a pretreated result X';

(2) dividing the result X' after the pre-processing of the existing labeled network behavior record set into normal and normalTwo types of abnormity, wherein the abnormity comprises M types of attack types, a part of samples are respectively and randomly extracted from normal samples and abnormal samples, and a distributed network integrated learning algorithm is adopted to process the samples containing K₁Performing T on network topology of individual nodes₁Performing round training to generate a classifier network system of the anomaly detection learning machine;

wherein [. ]]For rounding operation, willAnd X^T1Combined into a training setSample weight is adjusted by adopting a method for adjusting sample weight in the training process of AdaBoost algorithm, and a source domain sample subset with larger weight is selected

(4) Subset of source domain samples

And taking samples of other types of the target domain as training samples, and adjusting the sample weight by using the method for adjusting the sample weight in the training process of the AdaBoost algorithm again, so as to obtain the target domain sample weight

Removing the source domain samples with larger weight

The remaining samples constitute a source domain migration sample set TR_D；

(5) From a target domain sample set X^TIn the random sampling of a part of samples to form a target domain sample subset TR_SWith the selected source domain migration sample set TR_DTaking TR as training sample_DAssigning the same label as the target domain to be guided sample, the layout containing K₂Network topology of individual nodes, input sampling rate ρ₂Training round number T₂Will TR_SAnd TR_DDistributed on each node to generate a training sample set S on each node_k，k＝1，2，…，K₂And generating an abnormal behavior analysis learning machine according to the training sample set by the following steps:

5b) training sample set S for each node_kPerforming weighted sampling with the returns to obtain a training subset of each node, training in the learning algorithm of each node to obtain a base classifier of each nodePair S with base classifier of each node_kClassifying, wherein t is the number of current training rounds;

5c) according to the pair S_kCalculating the weighted error rate epsilon of the target domain samples on each node according to the classification result_k，tAnd according to epsilon_k，tCalculating the weight of each base classifier

5d) Updating the weights of the source domain migration sample and the target domain sample when T < T₂When T is T, go to step 5b₂Then finish training and obtain the classifier formed by all the base classifiers(k＝1，2，…，K₂，t＝1，2，…，T₂) A classifier network system of the abnormal behavior analysis learning machine is formed;

(6) carrying out quantization and normalization processing on the existing network behavior record, and recording the parameters after preprocessing: when a new network behavior record x 'is input, carrying out quantization and normalization pretreatment on the parameters obtained by pretreatment according to the existing network behavior record to obtain a pretreated network behavior record result x';

(7) inputting the x' ″ into a classifier network system of the anomaly detection learning machine generated in the step 2 for classification, and obtaining a classification result:

wherein,for detecting anomaliesThe classification result of each base classifier pair x' ″ in the learning machine,

for the weight of each base classifier, p is the neighbor node index of node k, when H₁When (x') is 1, indicating that the sample belongs to a normal type, carrying out no treatment and ending the detection process; when H is present₁If (x' ″) is-1, it indicates that the type is abnormal, then the procedure goes to step (8);

(8) inputting the x' ″ into a classifier network system of the abnormal behavior analysis learning machine generated in the step 5 for classification, and obtaining a classification result:

<math><mrow><msub><mi>H</mi><mn>2</mn></msub><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><munder><mrow><mi>arg</mi><mi> </mi><mi>max</mi></mrow><mrow><mi>y</mi><mo>&Element;</mo><mi>Y</mi></mrow></munder><mrow><mo>(</mo><munderover><mi>Σ</mi><mrow><mi>k</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>K</mi><mn>2</mn></msub></munderover><munderover><mi>Σ</mi><mrow><mi>t</mi><mo>=</mo><mn>1</mn></mrow><msub><mi>T</mi><mn>2</mn></msub></munderover><mrow><mo>(</mo><msubsup><mi>α</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mi>I</mi><mo>[</mo><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><mi>y</mi><mo>]</mo><mo>+</mo><munder><mi>Σ</mi><mi>p</mi></munder><msubsup><mi>α</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mi>I</mi><mo>[</mo><msubsup><mi>h</mi><mrow><mi>p</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msup><mi>x</mi><mrow><mo>′</mo><mo>′</mo><mo>′</mo></mrow></msup><mo>)</mo></mrow><mo>=</mo><mi>y</mi><mo>]</mo><mo>)</mo></mrow><mo>)</mo></mrow></mrow></math>

wherein,the classification result of each base classifier pair x' ″ in the learning machine is analyzed for abnormal behavior, and

where Y ═ 1, 2, …, M, 1, 2, …, M are the index numbers of M attack types, I [ · respectively]For indicating the function, its value is 0 or 1, H₂(x′″)∈Y；

(9) H is to be₂(x') is used as an index number, the attack type corresponding to the index number is searched, and the attack type is output as a final detection result.

3. The method of claim 2, wherein the step 3 of selecting the subset of source domain samples with larger sample weight

The method comprises the following steps:

3a) setting up

Is +1, X^T1Is labeled-1, input sample weight threshold W₁；

3b) Will be provided withRespectively inputting into AdaBoost algorithm for training to adjust sample weight, and respectively training after multiple rounds of trainingThe weight of the selected sample is larger than the threshold value W₁And belong to

Constitute a source domain sample subset

4. The method of claim 2, wherein the slave of step 4

Removing the source domain samples with larger weight according to the following steps:

4a) sampling the target domain X^TIn the division of samples X of the type to be guided^T1In addition, other exception types are denoted X^T2Setting upIs +1, X^T2Is labeled-1, input sample weight threshold W₂；

4b) Will be provided withAnd X^T2Form a training set T²Inputting the sample into AdaBoost algorithm for training to adjust sample weight, and removing T after multi-round training²The weight of the middle sample is greater than the threshold value W₂And belong to

A sample of (2) A

The remaining samples constitute a source domain migration sample set TR_D。

5. The method according to claim 2, wherein the updating of the weights of the source domain migration samples and the target domain samples in step 5d is performed by the following steps:

5a) separately calculating target field samplesWeight update parameter of book

And weight of source domain migration samples

Updating parameters

Wherein m is_kIs S at node k_kThe number of source domain migration samples contained in (a);

5b) updating sample x at node k_iWeight D of_k，t(x_i) Get the updated weight D_k，t+1(x_i)：

wherein, y (x)_i) Is a sample x_iIs known as a label for a tag to be used,

is composed of

For sample x_iIs classified into

<math><mrow><msubsup><mi>h</mi><mrow><mi>k</mi><mo>,</mo><mi>t</mi></mrow><mn>2</mn></msubsup><mrow><mo>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo>)</mo></mrow><mo>&Element;</mo><mi>Y</mi><mo>.</mo></mrow></math>