CN113162926B - KNN-based network attack detection attribute weight analysis method - Google Patents

Abstract

The invention discloses a KNN-based network attack detection attribute weight analysis method, which comprises the following steps: step 1, downloading a DDoS data set and recording as a sample A; step 2, processing the sample A obtained in the step 1, converting the sample A into a file with a suffix name of csv format, and naming the file as a sample I; step 3, classifying the label columns in the sample I obtained in the step 2 by 0 and 1; step 4, preprocessing the sample II obtained in the step 3 to obtain a sample V; and 5, dividing the sample five obtained in the step 4 into a training set and a testing set, inputting the training set into the KNN model for training, adjusting adjustable parameters to obtain a trained KNN model, inputting the testing set into the trained KNN model for testing, and checking the accuracy. The method solves the problems of low data processing speed and small amount of obtained information in the existing method.

Description

An attribute weight analysis method for network attack detection based on KNN

technical field

The invention belongs to the technical field of network attack detection, and relates to a KNN-based network attack detection attribute weight analysis method.

Background technique

With the rapid development and widespread popularization of the Internet, there are more and more types and quantities of network intrusions, and network intrusion events appear more frequently. In the age of Internet information, the ability of computers to process information is accelerated, and more and more cyber attacks are aimed at the public's personal information. Such cyber attacks have caused social and economic losses and personal psychological panic. Take network security seriously. A common attack method in network attacks is a distributed denial of service attack, that is, a DDoS attack.

DDoS is one of the most important threats to the Internet today. DDoS attack means that the attacker sends a large number of continuous requests to the attack target by controlling multiple computers, so that the attack target cannot respond to legitimate users' requests for normal access to resources, which brings huge losses to the attack target. The main targets of DDoS attacks are websites and servers, by consuming server resources, including CPU, memory, and network bandwidth. In addition, DDoS can also attack the network infrastructure. Through huge attack traffic, including routers, switches, etc., the performance of the network where the attack target is located can be greatly reduced or even paralyzed.

The principle of a DDoS attack can be understood as the attacker hijacks and controls a large number of computers on the network by means of hackers, and launches an attack on the target. Therefore, this attack is also called a distributed attack. There are three common attack methods: the first is the SYN Flood attack, which uses the three-way handshake of the TCP protocol. Since the requested IP address is fake, the third handshake packet has not been confirmed, and the server has been in a semi-connected state until The waiting queue is full, and the server cannot provide normal services; the second is the UDP flood attack, which uses the connectionless nature of UDP to send a large number of UDP small packets to make the target unable to provide normal services; the third is the CC attack, which is generally used for Website attack, which makes the website unable to visit normally by sending data packets.

To better regulate network security, DDoS datasets need to be analyzed. By analyzing the effect of attributes in the dataset on network security, we can obtain whether DDoS attacks have occurred in the target network. Common DDoS datasets are CAIDADDoS Attack 2007, CIC-IDS2018, KDD and so on. Taking KDD Cup 99 as an example, it is a dataset used to detect abnormal connections from normal connections. There are 41 attributes and a label column in the dataset. The 41 attributes can be divided into basic characteristics of TCP connections; content characteristics of TCP connections; time-based network traffic statistics characteristics, using a time window of 2 seconds; host-based network traffic statistics characteristics, host characteristics, used to evaluate the duration in Attack for more than two seconds. Due to the class imbalance problem of KDD Cup99, NSL-KDD is a resampled version of the KDD Cup99 dataset. Researchers need to select appropriate attributes for preprocessing according to their actual needs and purposes, and then select appropriate algorithms for analysis. The datasets used by most researchers have the characteristics of small amount of data and few attributes. Selecting some attributes combined with traditional methods can meet the requirements. However, with the advancement of science and technology, the emergence of 5G technology and the gradual development of technologies such as the Internet of Things and artificial intelligence, the scale of data sets is getting larger and larger, and there are more and more attributes. Using traditional data processing methods, a series of data sets will appear. Problems include slow processing, inefficiency, and little information available.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a KNN-based network attack detection attribute weight analysis method, which solves the problems of slow data processing speed and small amount of information obtained by the existing method.

The technical solution adopted by the present invention is a KNN-based network attack detection attribute weight analysis method, which is specifically implemented according to the following steps:

Step 1, download the DDoS dataset, denoted as sample A;

Step 2, process the sample A obtained in step 1, convert it into a file with a suffix named .csv, and name it as sample 1;

Step 3, classify the label column in sample 1 obtained in step 2 by 0, 1;

Step 4, preprocess the sample 2 obtained in step 3 to obtain sample 5;

Step 5: Divide the sample 5 obtained in step 4 into a training set and a test set, input the training set into the KNN model for training, and adjust the adjustable parameters to obtain a trained KNN model, and input the test set into the trained KNN model for training. Test to see accuracy.

The present invention is also characterized in that,

The specific process of step 2 is: open the file in the form of read-only for sample A obtained in step 1, remove the spaces in each line of sample A, and then use the delimiter to slice the string, and convert it into a file with a suffix named .csv format. , named Sample One.

The specific process of step 3 is:

Step 3.1, select from the sample 1 obtained in step 2 that the protocol is tcp, the attack type is DoS and normal data, and the label column of normal data is set to 1, and the label column of DoS attack is set to 0, named as sample 2 ;

Step 3.2, count the number of normal flow and abnormal flow in the second sample obtained in step 3.1, and visualize the histogram with the help of matplotlib.pyplot.

The specific process of step 4 is:

In step 4.1, the second sample obtained in step 3 is read in through the pandas function, and the separator in step 2 is removed to obtain sample three;

Step 4.2, use the sample 3 obtained in step 4.1, check the number of rows by shape, and take the first 60% of the rows as sample 4;

Step 4.3, name each attribute of sample 4 obtained in step 4.2, and normalize it by column;

Step 4.4, for the sample 4 processed in step 4.3, select each attribute and label column as x and y in turn, convert x into a two-dimensional array and name it X, and convert y into a one-dimensional array and name it as Y, X and Y are spliced into sample five.

The specific process of step 5 is:

Step 5.1, clarify the values of the weights and leaf_size parameters in the KNN model;

Step 5.2: Divide the sample 5 obtained in step 4 into a training set and a test set, input the training set into the KNN model for training, and set n_neighbors according to the integer custom range in turn, and the one with the highest accuracy in the custom range is is the optimal KNN model;

Step 5.3, input the test set in step 5.2 into the optimal KNN model for testing to check the accuracy.

In step 5.2, the sample size ratio of training set and test set is 7:3.

The beneficial effect of the present invention is that the present invention is a KNN-based network attack detection attribute weight analysis method, which can detect attacks on DDoS data sets, select some attributes for analysis, and use the machine learning KNN model to analyze the accuracy of each attribute. Selecting some attributes with the highest accuracy rate can detect whether DDoS attacks occur in time, which has a strong reference, and the data processing speed is fast and the amount of information obtained is comprehensive.

Description of drawings

Fig. 1 is the algorithm flow chart in a kind of KNN-based network attack detection attribute weight analysis method of the present invention;

FIG. 2 is a histogram of normal traffic and abnormal traffic in the second statistical sample in a KNN-based network attack detection attribute weight analysis method of the present invention.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

The present invention provides a KNN-based network attack detection attribute weight analysis method, as shown in FIG. 1 , which is specifically implemented according to the following steps:

Step 1, download the DDoS dataset, denoted as sample A;

Step 2: Open the file in the form of read-only for the sample A obtained in step 1, remove the spaces in each line of the sample A, and then use the delimiter to slice the string, convert it into a .csv format file with a suffix, and name it as the sample one;

Step 3, classify the label column in sample 1 obtained in step 2 by 0, 1;

Step 3.1, select from the sample 1 obtained in step 2 that the protocol is tcp, the attack type is DoS and normal data, and the label column of normal data is set to 1, and the label column of DoS attack is set to 0, named as sample 2 ;

Step 3.2, count the number of normal flow and abnormal flow in sample 2 obtained in step 3.1, and visualize the histogram with the help of matplotlib.pyplot;

Step 4, preprocess the sample 2 obtained in step 3 to obtain sample 5;

In step 4.1, the second sample obtained in step 3 is read in through the pandas function, and the separator in step 2 is removed to obtain sample three;

Step 4.2, use the sample 3 obtained in step 4.1, check the number of rows by shape, and take the first 60% of the rows as sample 4;

Step 4.3: Name each attribute of sample 4 obtained in step 4.2 according to the official introduction of the DDoS dataset, and normalize it by column;

Step 4.4, for the sample 4 processed in step 4.3, select each attribute and label column as x and y in turn, convert x into a two-dimensional array and name it X, and convert y into a one-dimensional array and name it as Y, X Splicing with Y to form sample five;

Step 5: Divide the sample 5 obtained in step 4 into a training set and a test set. The ratio of the sample size of the training set and the test set is 7:3. Input the training set into the KNN model for training, and adjust the adjustable parameters to get the training set. Enter the KNN model of the test set into the trained KNN model for testing to check the accuracy;

Step 5.1, clarify the values of the weights and leaf_size parameters in the KNN model;

Step 5.2: Divide the sample 5 obtained in step 4 into a training set and a test set, input the training set into the KNN model for training, and set n_neighbors according to the integer custom range in turn, and the one with the highest accuracy in the custom range is is the optimal KNN model;

Step 5.3, input the test set in step 5.2 into the optimal KNN model for testing, check the accuracy rate, understand the prediction effect of the KNN model through the accuracy rate, and detect whether a DDoS attack occurs in time.

Example

Step 1. Download the DDoS dataset KDD99 (Data Mining and Knowledge Discovery Cup1999DataSet), denoted as sample A;

Step 2: Open the file in the form of read-only for the sample A obtained in step 1, remove the spaces in each line of the sample A, and then use the delimiter to slice the string, convert it into a .csv format file with a suffix, and name it as the sample one;

Step 3.1, from the sample 1 obtained in step 2, select the data whose protocal is tcp, the attack type is DoS and normal data, and set the label column of normal data to 1 and the label column of DoS attack to 0, named as sample two;

Step 3.2, count the number of normal flow and abnormal flow in sample 2 obtained in step 3.1, and use matplotlib.pyplot to visualize the bar chart, as shown in Figure 2. As can be seen from Figure 2, the abscissa display includes normal flow, Abnormal flow and total flow, the ordinate is the specific number of each flow, then the number of normal flow is 768670, the number of abnormal flow is 1074241, and the number of total flow is 1842911;

In step 4.1, the second sample obtained in step 3 is read in through the pandas function, and the separator in step 2 is removed to obtain sample three;

Step 4.2, use the sample 3 obtained in step 4.1, check the number of rows by shape, and take the first 60% of the rows as sample 4;

In step 4.3, the 41 samples obtained in step 4.2 are named according to the official introduction of the dataset, and are normalized by column. The 42nd column is named attack_type;

Step 4.4, select the four attributes of count, same_srv_rate, dst_host_serror_rate, dst_host_srv_serror_rate and the attack_type column for the sample four processed in step 4.3 and record them as x and y respectively, convert x into a two-dimensional array and name it X, and y into a dimensional array and named Y, X and Y are concatenated into sample five;

Step 5.1, clarify the values of the weights and leaf_size parameters in the KNN model. The weights values are uniform and distance. The prediction accuracy results of the two values are very close to the impact. After leaf_size is 29 and 30, the prediction accuracy trend tends to be flat. This implementation Example go weights='uniform', leaf_size=30;

Step 5.2: Divide the sample 5 obtained in step 4 into a training set and a test set. The ratio of the sample size of the training set to the test set is 7:3. Input the training set into the KNN model for training, n_neighbors=5, and get the trained KNN Model, check the accuracy rate, the accuracy rate is 0.9981128890282283, and the running time is 1340.890951 seconds;

Step 5.3, input the test set in step 5.2 into the trained KNN model for testing to check the accuracy.

The running time is expressed as: if the second sample obtained in step 3 has N samples, and the feature of each sample is a D-dimensional vector, in order to make predictions, all training samples need to be looped, and the time complexity is O(N). In addition, when we calculate the distance between two samples, the complexity depends on the feature dimension of the sample, so the time complexity is O(D); only one attribute is selected at a time, and the time complexity is O(1) , the process of looping samples is regarded as an outer loop, and the calculation of the distance between samples is regarded as an inner loop, so the time complexity is O(N*1), and D attributes are predicted, so the total time complexity is O (N*D).

Control example

Step 4.4, randomly select the four attributes of num_shells, num_root, dst_host_srv_diff_host_rate, srv_diff_host_rate and the attack_type column for the sample four processed in step 4.3 and record them as x and y respectively, convert x into a two-dimensional array and name it as X, and convert y into a dimensional array and named Y, X and Y are concatenated into sample six;

Step 5.2: Divide the sample 6 obtained in step 4 into a training set and a test set. The ratio of the sample size of the training set to the test set is 7:3. Input the training set into the KNN model for training, n_neighbors=5, and get the trained KNN Model, check the accuracy, the accuracy is 0.8638808165824601, and the running time is 3248.227169 seconds;

The rest of the steps are the same as in the embodiment.

From this, it can be seen that the accuracy of the embodiment of the present invention is increased by 15.528% compared to the control example by randomly selecting four attributes, and the running time ratio is increased by 58.719%. With the help of the machine learning KNN model to predict the accuracy of DDoS attack attributes, researchers can quickly select some attributes with the highest accuracy, and detect whether a DDoS attack occurs in time, which has a strong reference.

Claims (2)

1. a network attack detection attribute weight analysis method based on KNN, is characterized in that, is specifically implemented according to the following steps: Step 1, download the DDoS dataset, denoted as sample A; Step 2, process the sample A obtained in step 1, convert it into a file with a suffix named .csv, and name it as sample 1; The specific process is: open the file in the form of read-only for the sample A obtained in step 1, remove the spaces in each line of the sample A, and then use the delimiter to slice the string, and convert it into a .csv format file with a suffix named as sample one; Step 3, classify the label column in sample 1 obtained in step 2 by 0, 1; The specific process is: Step 3.1, select from the sample 1 obtained in step 2 that the protocol is tcp, the attack type is DoS and normal data, and the label column of normal traffic is set to 1, and the label column of DoS attack is set to 0, named as sample 2; Step 3.2, count the number of normal flow and abnormal flow in sample 2 obtained in step 3.1, and visualize the histogram with the help of matplotlib.pyplot; Step 4, preprocess the sample 2 obtained in step 3 to obtain sample 5; The specific process is: In step 4.1, the second sample obtained in step 3 is read in through the pandas function, and the separator in step 2 is removed to obtain sample three; Step 4.2, use the sample 3 obtained in step 4.1, check the number of rows by shape, and take the first 60% of the rows as sample 4; Step 4.3, name each attribute of sample 4 obtained in step 4.2, and normalize it by column; Step 4.4, for the sample 4 processed in step 4.3, select each attribute and label column as x and y in turn, convert x into a two-dimensional array and name it X, and convert y into a one-dimensional array and name it as Y, X Splicing with Y to form sample five; Step 5: Divide the sample 5 obtained in step 4 into a training set and a test set, input the training set into the KNN model for training, and adjust the adjustable parameters to obtain a trained KNN model, and input the test set into the trained KNN model for training. Test, check the accuracy; The specific process is: Step 5.1, clarify the values of the weights and leaf_size parameters in the KNN model; Step 5.2: Divide the sample 5 obtained in step 4 into a training set and a test set, input the training set into the KNN model for training, and set n_neighbors according to the integer custom range in turn, and the one with the highest accuracy in the custom range is is the optimal KNN model; Step 5.3, input the test set in step 5.2 into the optimal KNN model for testing to check the accuracy. 2. A KNN-based network attack detection attribute weight analysis method according to claim 1, characterized in that, in the step 5.2, the sample size ratio of the training set and the test set is 7:3.

Images