CN102158486A - Method for rapidly detecting network invasion - Google Patents

Abstract

The invention discloses a network intrusion rapid detection method in the technical field of information security, which is used to solve the problems of slow calculation speed and lack of real-time detection in the currently used network intrusion detection method. The method includes: preprocessing the feature attributes of each sample in the training set; selecting the feature attributes of the samples in the sample set to achieve feature dimensionality reduction; composing the feature attributes after dimensionality reduction into a first feature vector, and using the first feature The vector is used as the input of the ball vector machine learning algorithm to train the classifier; the feature attributes of each sample in the test set are preprocessed; the feature attributes of the samples in the test set are selected to achieve feature dimensionality reduction; the feature attributes after dimensionality reduction A second feature vector is formed, and the classifier is tested by using the second feature vector as an input of the trained classifier; and it is determined whether the network is invaded according to the test result. The invention reduces the computational complexity of network intrusion detection and improves the real-time performance and accuracy of detection.

Description

A Fast Detection Method of Network Intrusion

technical field

The invention belongs to the technical field of information security, in particular to a method for quickly detecting network intrusion.

Background technique

With the popularization and development of information technology, human society has entered the era of networking. However, the Internet is an open network for the public. The current Internet protocol does not fully consider the confidentiality of information and system security, and it is difficult to maintain security issues such as illegal intrusion, hacker attacks, and confidential information leakage. Computer networks are constantly being invaded illegally, and important intelligence data are being stolen constantly, which makes network intrusion detection work face a huge challenge. Existing network intrusion detection methods usually use machine learning-based modeling. The process is to first capture raw data from the network, and quantify and normalize the data; secondly, use machine learning algorithms to process Good data is used as its input to train the classifier.

In network intrusion detection, a network connection is regarded as a sample, and each sample data usually contains dozens of features, and all features are quantified and normalized during modeling, that is, each original sample data is expressed as a vector form, so that the set of all vectors is used as the training set of the classifier; then the machine learning algorithm is used, and the training set is used as its input to train the classifier. The input test sample is processed by the trained classifier, and the predicted output value can be obtained to determine whether it belongs to a normal connection or an abnormal connection.

The above process is a typical network intrusion detection method at the present stage. The main disadvantage of this detection technology is: the original data of intrusion detection usually contains dozens of features, and these data are used in some classification algorithms, which will make the classification speed very slow; In addition, not all of the dozens of features contained in the original data have a positive impact on the detection results, and some features may even have a negative impact. Furthermore, the above methods take into account the problem of classification speed that the training data usually used is small in scale, which leads to low detection accuracy and high false positive and false negative rates.

Aiming at the deficiencies in the detection technology mentioned above, the present invention proposes a BVM (Ball Vector Machine, Ball Vector Machine) network intrusion based on LLE (locally linear embedding algorithm in the manifold learning algorithm, Locally Linear Embedding) feature extraction. Detection method. Take the sample set containing both normal connections and abnormal connections as the training set, quantify and normalize each sample in the training set, and extract the key features of each sample to reduce the dimensionality of the intrusion detection data, and then use The classification algorithm trains the classifier, and finally uses the trained classifier to classify the unknown connection to determine whether it is a normal connection or an abnormal connection. In the process of dimensionality reduction of sample data, the manifold learning algorithm is introduced to analyze and process the feature attributes of a large number of samples to find meaningful low-dimensional structures hidden in high-dimensional data, so as to achieve the reduction of high-dimensional feature attributes. The purpose of dimension processing. The classification learning algorithm introduces the spherical vector machine algorithm, which introduces the concept of core set, and transforms the problem of solving the support vector in the support vector machine into the problem of solving the smallest sphere (MEB) containing the training sample set. Because the size of the core set is much smaller than the original training sample set, the cost of solving the optimization problem is greatly reduced. The BVM algorithm can effectively reduce the training time of samples while ensuring the detection rate, thereby effectively improving the real-time performance of detection.

Contents of the invention

The purpose of the present invention is to propose a fast detection method for network intrusion, which is used to solve the problems of slow operation speed caused by high calculation dimension and lack of real-time and accuracy in the currently used network intrusion detection method.

In order to achieve the above object, the technical solution provided by the present invention is a method for quickly detecting network intrusion, which is characterized in that the method includes:

Step 1: Take the sample set containing both normal connections and abnormal connections as the training set, and preprocess the feature attributes of each sample in the training set;

Step 2: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the sample set to achieve feature dimensionality reduction;

Step 3: Composing the feature attributes after step 2 dimension reduction into the first feature vector, using the first feature vector as the input of the ball vector machine learning algorithm to train the classifier;

Step 4: Preprocessing the feature attributes of each sample in the test set;

Step 5: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the test set to achieve feature dimensionality reduction;

Step 6: Composing the feature attributes after step 5 dimensionality reduction into a second feature vector, using the second feature vector as the input of the classifier trained in step 3, and testing the classifier;

Step 7: Determine whether the network has been invaded according to the test results.

The preprocessing of the feature attributes of each sample in the training set specifically includes:

Step 11: Find out the character features in each sample;

Step 12: replace character features with corresponding digital feature values according to the preset keyword list;

Step 13: Use the digital feature value in each sample as the feature attribute of the sample, and perform normalization processing on the feature attribute in each sample.

进行归一化The normalization process uses the maximum value normalization method, specifically using the formula

normalize

processing; among them xi is the characteristic attribute of the sample, and N is the number of samples.

Said step 2 includes:

Step 21: use the K nearest neighbor method to find K neighbor points for each sample, where K is a given value;

Step 22: Construct a local reconstruction weight matrix for each sample using the K neighbor points obtained in step 21;

Step 23: Calculate its low-dimensional output value from the local reconstruction weight matrix of each sample and its K neighbor points.

The local reconstruction weight matrix utilizes the error function to construct, where

且

其中I是m×m阶的单位矩阵。The low-dimensional output value y _i satisfies the mapping condition:

and

where I is an identity matrix of order m×m.

Said step 3 includes:

Step 31: given the radius r and l=0, and choose any point z in the training set as the initial core set, set the initial core set S ₀ ={z}, and calculate the initial center c ₀ of the sphere according to S ₀ ;

Step 32: Perform an iterative operation. In the first iteration, if the core set S _l contains all the samples in the training set, that is, all samples fall within the sphere B(c _l , (1+ε)r), Then the iteration ends; otherwise, go to step 33; where ε is a set value, and ε>0;

Step 33: In the kernel feature space, find any sample point φ(x) outside the sphere B(c _l , (1+ε)r), and generate a core set S _l+1 = S _l ∪{x}; Among them, φ(*) is the kernel mapping function;

Step 34: Solve the center c _l+ 1 of S _l+1 from the core set S _l+1 ; where, the update formula of c _l+1 is: c _l+1 =φ(x)+β _l (c _l - φ(x)), β _l = r/∥c _l -φ(x)∥.

Step 35: set l=l+1, return to step 32.

The invention reduces the computational complexity of network intrusion detection and improves the real-time performance and accuracy of detection.

Description of drawings

Fig. 1 is the flow chart of network intrusion rapid detection method;

Fig. 2 is a schematic diagram of feature dimensionality reduction for samples in a sample set using a local linear embedding algorithm in a manifold learning algorithm;

Fig. 3 is a flow chart of training a classifier with the feature vector as the input of the ball vector machine learning algorithm.

Detailed ways

The preferred embodiments will be described in detail below in conjunction with the accompanying drawings. It should be emphasized that the following description is only exemplary and not intended to limit the scope of the invention and its application.

The rapid network intrusion detection method provided by the present invention is a BVM rapid network intrusion detection method based on LLE feature extraction. The problem of poor performance and low detection rate. For this reason, the solution of the present invention is: use the sample set containing both normal connections and abnormal connections as the training set, quantize and normalize each sample in the training set, and use the LLE in the manifold learning algorithm The algorithm extracts the key features of each sample to reduce the dimensionality of the intrusion detection data, then uses the ball vector machine algorithm to train the classifier, and finally uses the trained classifier to classify the unknown connection to determine whether it is a normal connection or an abnormal connection.

Fig. 1 is a flow chart of a rapid network intrusion detection method. Among Fig. 1, a kind of network intrusion rapid detection method provided by the present invention comprises the following steps:

Step 1: Take the sample set containing both normal connections and abnormal connections as the training set, and preprocess the feature attributes of each sample in the training set.

The training set can be downloaded directly from the web. There is a data set dedicated to network intrusion detection and evaluation on the Internet, called the KDDCUP'99 data set. The samples in the training set are network connections. In order to make the trained classifier detect the test set more accurately, the samples in the training set should contain both normal connections and abnormal connections.

Preprocessing the attributes of each sample in the training set specifically includes:

Step 11: Find out the character features in each sample. Each sample, that is, each connection, contains digital features and/or character features, and a digital feature (actually a number) can be directly used as a digital feature value.

Step 12: According to the preset keyword table, the character features are replaced with corresponding digital feature values.

The keyword table includes at least two fields, which are character features and corresponding digital feature values. In this way, the character feature (actually character) of each sample can be converted into a numerical value, that is, a digital feature value.

Step 12 is actually a quantization process.

Step 13: Use the digital feature value in each sample as the feature attribute of the sample, and perform normalization processing on the feature attribute in each sample.

进行归一化处理；其中

x_i为样本的特征属性，N为样本个数。The normalization process uses the maximum value normalization method, specifically using the formula

Perform normalization processing; where

x _i is the characteristic attribute of the sample, and N is the number of samples.

Step 2: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the sample set to achieve feature dimensionality reduction.

Figure 2 is a schematic diagram of using the local linear embedding algorithm in the manifold learning algorithm to perform feature dimensionality reduction on the samples in the sample set. In Figure 2, step 2 specifically includes:

Step 21: Use the K nearest neighbor method to find K neighbor points for each sample, where K is a given value.

The K sample points closest to the sample point to be obtained are defined as the K neighbor points of the sample point to be obtained, wherein K is a predetermined value. The calculation of the distance can adopt the Euclidean distance calculation method, and the Euclidean distance algorithm is as follows: Let x, y∈R ^N , then the Euclidean distance of x, y can be obtained by the following formula:

${<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}^{\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Step 22: Use the K neighbor points obtained in Step 21 to construct a local reconstruction weight matrix for each sample.

Local reconstruction weight matrix W=(w _ij )∈M _{n, n} is such a weight matrix, if x _i and x _j are not adjacent, then w _ij =0, set x _i and x _jk (k=1, 2,...,K) are adjacent, then there is a constraint

Using XW to represent X approximately, there will be certain errors. Here, the Frobenius norm of the matrix is defined as follows: A=(a _{i, j} )∈M _{m, n} , then

即

其中，x_jk，i代表x_i的k个近邻点。这相当于求一系列最小二乘问题的解。如对x_i而言，由下面的方程组可以获得Find W by the following constraints:

Right now

Among them, x _{jk, i} represent the k nearest neighbor points of x _i . This is equivalent to finding the solution to a series of least squares problems. As for x _i , the following equations can be obtained

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; jk</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; :</span>\left\{\begin{array}{c}\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; jk</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ {\mathrm{<span\; class="notranslate">\; wxya</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}\right.$

Step 23: Calculate its low-dimensional output value from the local reconstruction weight matrix of each sample and its K neighbor points.

其中y_i是x_i的输出向量，y_jk，i(k＝1，2，...，K)是y_i的近邻点，并且要满足两个条件：与其中I是m×m的单位矩阵。由此，损失函数可重写为：

其中M是N×N的对称矩阵：M＝(I-W)^T(I-W)。Through the weight matrix W, a suitable y _i can be found in a low-dimensional space. This is done with the following constraints:

Where y _i is the output vector of x _i , y _{jk, i} (k=1, 2, ..., K) is the neighbor point of y _i , and two conditions must be met: and where I is the identity matrix of m×m. Thus, the loss function can be rewritten as:

where M is an N×N symmetric matrix: M=(IW) ^T (IW).

To minimize the value of the loss function, Y is the eigenvector corresponding to the smallest m non-zero eigenvalues of M. During the processing, the eigenvalues of M are arranged from small to large, and the first eigenvalue is almost close to zero, then the first eigenvalue is discarded. Usually, the eigenvector corresponding to the eigenvalue between 2nd and m+1 is taken as the output result.

Step 3: Composing the dimensionality-reduced feature attributes in step 2 into a first feature vector, and using the first feature vector as an input of the ball vector machine learning algorithm to train a classifier.

Fig. 3 is a flow chart of training a classifier with the feature vector as the input of the ball vector machine learning algorithm.

In Fig. 3, step 3 specifically includes:

Step 31: Given a radius r, select any point z∈S in the training set S as the initial core set S ₀ ={z} and calculate the initial center c ₀ of the sphere according to S ₀ . z is actually a sample.

Step 32: Perform iterative operation. In the first iteration, if the core set S _l contains all the samples in the training set, that is, all samples fall within the sphere B(c _l , (1+ε)r) ( ε is a set value, and ε>0), then the iteration ends; otherwise, go to step 33.

Step 33: In the kernel feature space, find any sample point φ(x) outside the sphere B(c _l , (1+ε)r), and generate a core set S _l+1 = S _l ∪{x}; Among them, φ(*) is the kernel mapping function.

Step 34: Find the center c _l+1 of S _l+1 from the core set S l ₊₁ . Wherein, the update formula of c _l+1 is: c _l+1 =φ(x)+β _l (c _l -φ(x)), β _l =r/‖c _l -φ(x)‖.

Step 35: set l=l+1, return to step 32.

Step 4: Preprocess the feature attributes of each sample in the test set.

The test set can be downloaded from the network, or can be obtained by simulating the real network environment, capturing packets of network connection data in the simulated network, and then analyzing them.

Step 5: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the test set to achieve feature dimensionality reduction.

The specific execution process of step 5 is the same as step 2, except that the object for feature dimensionality reduction is changed from the training set to the test set.

Step 6: Composing the feature attributes after dimension reduction in step 5 into a second feature vector, using the second feature vector as the input of the classifier trained in step 3, and testing the classifier.

The specific process of this step is similar to step 3, except that the input is the second feature vector, and the output result is normal connection or abnormal connection.

Step 7: Determine whether the network has been invaded according to the test results.

According to the output result of step 6, if the connection is normal, the network is not intruded; if there is no abnormal connection, it can be determined that the network is intruded.

The manifold learning algorithm used in the present invention has strong dimension reduction ability and can find useful features of data; the ball vector machine learning algorithm obtains the core set of samples by solving the minimum closed sphere, because the scale of the core set is far smaller than the original training sample set, so the time-consuming calculation and the memory space occupied are greatly reduced, and at the same time, the solution time of the support vector and the cost of solving the optimization problem are also reduced, which not only realizes the detection of network intrusion behavior, but also ensures the accuracy of detection. accuracy and timeliness.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present invention can easily think of changes or Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (7)

1. A method for fast detection of network intrusion, characterized in that said method comprises: Step 1: Take the sample set containing both normal connections and abnormal connections as the training set, and preprocess the feature attributes of each sample in the training set; Step 2: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the sample set to achieve feature dimensionality reduction; Step 3: Composing the feature attributes after step 2 dimension reduction into the first feature vector, using the first feature vector as the input of the ball vector machine learning algorithm to train the classifier; Step 4: Preprocessing the feature attributes of each sample in the test set; Step 5: Use the local linear embedding algorithm in the manifold learning algorithm to select the feature attributes of the samples in the test set to achieve feature dimensionality reduction; Step 6: Composing the feature attributes after step 5 dimensionality reduction into a second feature vector, using the second feature vector as the input of the classifier trained in step 3, and testing the classifier; Step 7: Determine whether the network has been invaded according to the test results. 2. A kind of network intrusion rapid detection method according to claim 1, it is characterized in that described characteristic attribute of each sample in training set is carried out preprocessing specifically comprises: Step 11: Find out the character features in each sample; Step 12: replace character features with corresponding digital feature values according to the preset keyword list; Step 13: Use the digital feature value in each sample as the feature attribute of the sample, and perform normalization processing on the feature attribute in each sample. 3. A kind of network intrusion quick detection method according to claim 2, it is characterized in that described normalization process uses the maximum value normalization method, specifically adopts the formula Perform normalization processing; where xi is the characteristic attribute of the sample, and N is the number of samples. 4. A kind of network intrusion quick detection method according to claim 1, it is characterized in that described step 2 comprises: Step 21: use the K nearest neighbor method to find K neighbor points for each sample, where K is a given value; Step 22: use the K neighbor points obtained in step 21 to construct a local reconstruction weight matrix for each sample; Step 23: Calculate its low-dimensional output value from the local reconstruction weight matrix of each sample and its K neighbor points. 5. A kind of network intrusion rapid detection method according to claim 4, it is characterized in that described local reconstruction weight matrix utilizes error function

to construct, where 6. a kind of network intrusion rapid detection method according to claim 4 is characterized in that described low-dimensional output value yi satisfies mapping condition:

and

where I is an identity matrix of order m×m. 7. A kind of network intrusion quick detection method according to claim 1, it is characterized in that described step 3 comprises: Step 31: given the radius r and l=0, and choose any point z in the training set as the initial core set, set the initial core set S ₀ ={z}, and calculate the initial center c ₀ of the sphere according to S ₀ ; Step 32: Perform an iterative operation. In the first iteration, if the core set S _l contains all the samples in the training set, that is, all samples fall within the sphere B(c _l , (1+ε)r), Then the iteration ends; otherwise, go to step 33; where ε is a set value, and ε>0; Step 33: In the kernel feature space, find any sample point φ(x) outside the sphere B(c _l , (1+ε)r), and generate a core set S _l+1 = S _l ∪{x}; Among them, φ(*) is the kernel mapping function; Step 34: Solve the center c _l+ 1 of S _l+1 from the core set S _l+1 ; where, the update formula of c _l+1 is: c _l+1 =φ(x)+β _l (c _l - φ(x)), β _l = r/∥c _l -φ(x)∥. Step 35: set l=l+1, return to step 32.

Images