CN113114694A - DDoS attack detection method oriented to high-speed network packet sampling data acquisition scene - Google Patents

Abstract

The invention discloses a DDoS attack detection method facing a high-speed network packet sampling data acquisition scene, which comprises the steps of firstly sampling network packets by using an equidistant sampling technology, then respectively and rapidly extracting effective characteristics of UDP flow and TCP flow by using two improved Count-Min Sketch structures comprising a plurality of counters and hash tables, and respectively constructing UDP flow and SYN flow classifiers for UDP flow and TCP flow in an off-line manner by using a decision tree method of machine learning. By utilizing the well-constructed classifiers, feature vectors of sampling packet streams are recorded on line by using a Sketch structure, and the on-line detection of common DDoS attacks UDP flow and SYN flow in a high-speed network packet sampling data acquisition scene is realized. The method can realize the accurate identification of the DDoS attack in the high-speed network environment with lower time complexity and space complexity, wherein the UDP Flood detection method can well distinguish UDP Flood attack flow and DNS one-way request flow, thereby reducing the misjudgment rate and being used for the safety monitoring of the high-speed network.

Description

DDoS attack detection method oriented to high-speed network packet sampling data acquisition scene

Technical Field

The invention relates to a DDoS attack detection method for a high-speed network packet sampling data acquisition scene, and belongs to the technical field of network security.

Background

Distributed Denial of Service (DDoS) attacks refer to an attack mode in which an attacker sends a large number of invalid requests to a target host by controlling a large number of zombie hosts in a network, so that system or network resources of the target host are exhausted, and finally, the target host cannot provide valid services for legitimate users.

In recent years, DDoS attack events are layered endlessly, and various related reports are frequently found. In 2019, the CNCERT continuously analyzes attack resources for initiating DDoS every month, and the stability of the available resources is reduced. Compared with 2018, the number of the IP addresses of the domestic active control terminals which can be utilized per month is reduced by 15.0 percent proportionally, and the number of the active reflection servers is reduced by 34.0 percent proportionally. Meanwhile, sampling monitoring finds that the number of large-flow DDoS attack events with the peak value exceeding 10Gbps in China is 220 per day on average, and is increased by 40 percent on the same scale. As can be seen, DDoS attacks are one of the important threats of the internet today, and therefore, research on DDoS attack detection methods has been concerned.

In recent years, a series of Sketch-based methods have been proposed for anomaly detection in domestic and foreign research. Sketch is a hash-based data structure, and stores key value data with the same hash value into the same counter by setting a hash function, and is widely applied to a high-speed network environment for storing network traffic characteristics due to smaller occupied space resources. The Count-Min Sketch is a typical Sketch structure, and reduces hash collision by setting a plurality of hash functions, and taking the minimum value of a counter as a measurement result. In previous studies, the following two DDoS detection methods based on Count-Min Sketch were proposed, but both still have some problems and the effects are not particularly desirable.

(1) Detection method based on Count-Min Sketch and multichannel nonparametric CUSUM (MNP-CUSUM)

The CUSUM technique can accumulate small offsets in the process of amplifying the changing statistical features, thereby improving detection sensitivity. Although it can effectively detect flooding events, it has the following drawbacks, so that it is still insufficient to detect general distributed denial of service attacks.

First, this scheme features only the high frequency of packets in the stream as a characteristic of an exception event. However, this characteristic of the flow is not sufficient by itself to detect anomalies. For example, a high concurrency (Flash Crowd) caused by a large number of legitimate users accessing the same server at the same time may also cause traffic explosion, in which case the method may result in a high false positive rate. Furthermore, this approach has scalability problems because it must record each incoming destination IP address in order to find the IP address of the victim server at the time of attack, which makes it difficult to scale to high-speed network environments with large volumes of traffic due to huge memory consumption. Finally, it applies the MNP-CUSUM algorithm to each bucket in Sketch, resulting in a large computational overhead.

(2) Sketch-based two-stage DDoS detection method

The method uses two sketches to ensure the accuracy of the detection. The method adopts Modified Count-Min Sketch (MCS) to carry out rapid coarse grain detection; and (3) realizing fine granularity detection by adopting a Bidirectional Count Sketch (BCS) so as to achieve better detection precision. The method has the advantage that under the condition that all IP addresses in the flow are not required to be recorded, the identification of the damaged IP addresses is realized. The scheme can save more than 90% of IP address storage, which has important significance for detection in a high-speed environment. Nevertheless, this method still requires sufficient memory space and complex calculations.

Disclosure of Invention

In order to realize accurate detection on DDoS attacks in a high-speed network environment in limited memory and reasonable time, the invention firstly uses an equidistant sampling technology to sample network flow, then uses two improved Count-Min Sketch structures containing a plurality of counters and hash tables to respectively and quickly extract effective characteristics of UDP flow and TCP flow, and respectively constructs UDP flow and SYN flow classifiers for UDP flow and TCP flow in an off-line manner by a decision tree method of machine learning. The constructed classifiers can realize online detection of UDP flow and SYN flow of common DDoS attacks in a high-speed network environment by combining the previous sampling and Sketch structures.

In order to achieve the purpose, the technical scheme of the invention is as follows: a UDP Flood attack detection method oriented to a high-speed network packet sampling data acquisition scene comprises the following steps:

the method comprises the steps that (1) an open flow data set containing UDP flow is obtained, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of the flows is earlier than that of the second part of the flows, and the two parts of the flows contain common UDP flow and normal flow attacked by DDoS;

setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

step (3) extracting effective characteristics of the UDP packets by using the UDP Sketch shown in the attached figure 3; the UDP Sketch is composed of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket and is used for storing a plurality of characteristics of a data packet, each counting bucket comprises 2 counters and 2 hash tables, the counters respectively occupy 1 byte, the hash tables respectively occupy 2 bytes, and a UDP Sketch counting bucket structure for UDP Flood detection is listed in table 1; the UDP Sketch comprises three operations of updating, dereferencing and feature extraction shown in the table 2, wherein the updating operation is divided into three steps: extracting binary information (IP, protocol number) of a data packet as a key, calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, adding 1 to corresponding counters of the r counting barrels, and dividing the value taking operation into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, and taking out the minimum counting barrel value; the operation of extracting the features is divided into two steps: using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch;

table 1 UDP Sketch count bucket structure

Counter with a memory	Number of bytes occupied	Means of
			Rp	1 byte	Number of received UDP packets
Sp
		1 byte	Number of UDP packets to be sent
Hr				2 bytes	Hash table for recording IP address and port number distribution of receiving end
	Hs	2 bytes	Hash table for recording port number distribution of sending terminal

TABLE 2 Sketch operations

Operation of	Means of
		Update(key)	Adding 1 to the corresponding counter in the r counting buckets corresponding to the keys
GetMinBucket(key)	Taking out the minimum counting barrel value corresponding to the key
		GetFeature(key)	Taking out the characteristic value corresponding to the key

Step (4) constructing a training set for UDP flow classification according to a batch of samples containing the characteristic values obtained in the step (3), and judging whether the UDP flow in the samples is UDP flow attack flow according to the IP address to mark as the flow is from an open flow data set containing DDoS flow;

step 5, according to the training set constructed in the step 4, a UDP flow classifier is constructed in an off-line mode by using a decision tree method and used for detecting UDP flow attacks;

step (6) testing the constructed UDP Flood flow classifier by using the second part of flow;

and (7) directly applying the UDP Flood flow classifier tested according to the step (6) to a high-speed network environment to carry out UDP Flood attack detection.

In the step (3), the method for extracting the effective features by the UDP Sketch comprises the following steps:

(3.1) judging whether the sampling data packet uses a UDP protocol or not according to the IP protocol number, if so, continuing to process according to the step (3.2), and if not, not processing the data packet and waiting for the next sampling data packet to arrive;

(3.2) extracting binary information (source IP, protocol number) and (sink IP, protocol number) of the data packet as keys respectively, carrying out hash calculation on the keys by using a secure hash algorithm SHA-1 to obtain a 160-bit hash address, and dividing the hash address into r pieces

The hash address of the bit is used for executing updating operation after the subscript addresses of r counting buckets in the UDP Sketch of the key are obtained;

for a data packet, firstly mapping to a UDP Sketch structure by taking (source IP, protocol number) as a key, and updating sending counters in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to UDP Sketch structure once, and receiving counters in r counting buckets are updated;

(3.3) considering that when the Server is attacked by UDP Flood, the ports of the Server are accessed more intensively, and the ports of the Client are distributed widely, in order to record the characteristics, two hash tables shown in fig. 4 are used, each hash table occupies two bytes, the initial value of each position in each hash table is 0, each unit in each hash table is used for indicating whether a certain port is accessed, if the certain port is accessed, the initial value is set to 1, when a UDP data packet arrives, a (host IP address, host port number) is taken as a key to perform hash calculation by using SHA-1 to obtain the index address of the Hr of the corresponding hash table, and the value of the position is set to 1; then (source port number) is taken as a key to carry out Hash calculation by using SHA-1 to obtain a subscript address of Hs of a corresponding Hash table, the value of the position is set to be 1, when UDP Flood attacks occur, UDP data packets are almost sent to a Server end by a Client end, the (IP address and port number) of the Client end are constantly changed, and the port number of the Server end is fixed, so that the method is embodied in the Hash table as follows: 1 in the hash table Hr mapped by the Client is uniformly distributed, and 1 in the hash table Hs mapped by the Server is centrally distributed;

(3.4) considering that the process of providing domain name resolution service by DNS server and the process of UDP Flood attack suffered from server show the same characteristics: a plurality of clients send data packets to a Server, and due to the existence of asymmetric routes, a router may capture only one-way packets, and if only a DNS request packet can be captured but a DNS response packet cannot be captured, such a feature will interfere with detection of UDP Flood attacks, and therefore, a feature value needs to be added: the packet receiving rate Rp _ Spd is because the Server packet receiving rate in the DNS procedure is significantly smaller than the packet receiving rate in the DDoS attack procedure.

In addition, when the Server is attacked by UDP Flood, the packet receiving rate is much higher than the packet sending rate, so the characteristic value needs to be added: two characteristic values of packet transmission rates Sp _ Spd, Rp _ Spd and Sp _ SpdCan be calculated according to Rp and Sp counters in the counting bucket when UDP Sketch performs feature extraction operation, and the specific calculation formula refers to (1-1) and (1-2), t₂And t₁Respectively representing the current time when the feature extraction operation is successfully executed and the last time when the feature extraction operation is successfully executed, and samplingtratio represents the sampling ratio set in the step (2).

(3.5) performing feature extraction operation on the UDP Sketch, and when the sum of the two counters of Rp and Sp reaches a Threshold value Threshold, extracting 2 counter values and 2 hash table values in a UDP Sketch counting bucket and Rp _ Spd and Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

In the step (6), the method for testing the constructed UDP Flood traffic classifier by using the second part of traffic is as follows:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled UDP traffic by using UDP Sketch according to the step (3);

and (6.3) when the counter in the UDP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a UDP Flood flow classifier for detection, if the output is 1, indicating that the flow is UDP Flood attack flow, and if the output is 0, indicating that the flow is normal flow.

A SYN Flood attack detection method oriented to a high-speed network packet sampling data acquisition scene comprises the following steps:

the method comprises the steps that (1) an open flow data set containing SYN flow is obtained, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of the flows is earlier than that of the second part of the flows, and the two parts of the flows contain common SYN flow and normal flow attacked by DDoS;

setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

step (3) extracting effective characteristics of the TCP data packet by using the TCP Sketch shown in the figure 5; the TCP Sketch is composed of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket and is used for storing a plurality of characteristics of a data packet, each counting bucket comprises 6 counters which respectively occupy 1 byte, a TCP Sketch counting bucket structure for SYN Flood detection is listed in Table 3, the TCP Sketch comprises three operations of updating, valuing and extracting characteristics as shown in Table 2, and the updating operation is divided into three steps: extracting binary information (IP, protocol number) of the data packet as a key, calculating subscript addresses of r counting buckets corresponding to the key by using a hash function, and adding 1 to corresponding counters of the r counting buckets. The value taking operation is divided into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, and taking out the minimum counting barrel value. The operation of extracting the features is divided into two steps: and using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch.

TABLE 3 TCP Sketch count bucket architecture

And (4) constructing a training set of SYN flow classification by a batch of samples containing the characteristic values obtained in the step (3), and judging whether the TCP flow in the samples is SYN flow attack flow according to the IP address to mark as the flow comes from an open flow data set containing DDoS flow.

Step 5, according to the training set constructed in the step 4, a SYN Flood flow classifier is constructed in an off-line manner by using a decision tree method and is used for detecting the SYN Flood attack;

step (6) testing the constructed SYN Flood flow classifier by using the second part of flow;

and (7) the SYN Flood flow classifier tested according to the step (6) can be directly applied to a high-speed network environment for SYN Flood attack detection.

In the step (3), the method for extracting the effective characteristics of the TCP data packet by the TCP Sketch is as follows:

(3.1) judging whether the sampling data packet uses a TCP (transmission control protocol) protocol according to the IP protocol number, if so, continuing to process according to the step (3.2), otherwise, not processing the data packet, and waiting for the next sampling data packet to arrive;

(3.2) extracting binary information (source IP, protocol number) and (sink IP, protocol number) of the data packet as keys respectively, carrying out hash calculation on the keys by using a secure hash algorithm SHA-1 to obtain a 160-bit hash address, and dividing the hash address into r pieces

The hash address of the bit is used for executing updating operation after the subscript addresses of r counting buckets in TCP Sketch are obtained;

for a data packet, firstly mapping to a TCP Sketch structure by taking (source IP, protocol number) as a key, and updating a sending counter in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to TCP Sketch structure for the first time, and receiving counters in r counting buckets are updated;

(3.3) consider that the SYN Flood attack has such characteristics: when attacked by SYN Flood, the Server end shows that the packet receiving rate is much greater than the packet sending rate, so that 2 more characteristic values are required to be added: the two characteristic values can be calculated according to R0, Rd, S0 and Sd counters in a counting bucket when TCP Sketch performs characteristic extraction operation, and the specific calculation formula refers to (2-1) and (2-2), t₂And t₁Respectively representing the current successful execution and characteristic extraction time and the last successful executionTaking the characteristic operation time, samplingtrato represents the sampling ratio set in step (2).

(3.4) performing feature extraction operation on the TCP Sketch, and when the sum of the four counters R0, Rd, S0 and Sd reaches a Threshold, extracting the 6 counter values in the TCP Sketch counting bucket and the Rp _ Spd and the Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

In the step (6), the method for testing the constructed TCP flow classifier by using the second part of traffic is as follows:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled TCP flow by using TCP Sketch according to the step (3);

(6.3) when the counter in TCP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a SYN flow classifier for detection, if the output is 1, indicating that the flow is a SYN flow attack flow, and if the output is 0, indicating that the flow is a normal flow. Compared with the prior art, the technical scheme of the invention has the following beneficial technical effects.

(1) Compared with the existing Sketch structure, the improved Count-Min Sketch structure provided by the invention can better realize the storage of a plurality of characteristics of the same network flow in one Sketch structure and has stronger innovation.

(2) The invention can realize the rapid extraction of the high-speed and mass network flow characteristics by combining the equidistant sampling technology and the improved Count-Min Sketch structure comprising a plurality of counters and hash tables.

(3) The UDP flow and SYN flow detection method based on the improved Count-Min Sketch structural design can realize the accurate identification of DDoS attack in a high-speed network environment with lower time complexity and space complexity, and has stronger practicability. The UDP Flood detection method can well distinguish UDP Flood attack flow from DNS one-way request flow, so that misjudgment rate is reduced.

Drawings

FIG. 1 is a general flow block diagram of the present invention;

FIG. 2 is a schematic of flow sampling with the sampling ratio set to 256;

FIG. 3 is a structure diagram of UDP Sketch used for UDP Flood detection;

FIG. 4 is a diagram of two hash tables in UDP Sketch;

FIG. 5 is a diagram showing a structure of TCP Sketch for SYN Flood detection.

Detailed Description

The technical solutions provided by the present invention will be described in detail below with reference to specific examples, and it should be understood that the following specific embodiments are only illustrative of the present invention and are not intended to limit the scope of the present invention.

The specific embodiment is as follows: the DDoS detection method based on the improved Count-Min Sketch provided by the invention can detect common DDoS attacks on UDP Flood and SYN Flood in a high-speed network environment, and the flow frame is shown in fig. 1, and the detection steps of UDP Flood and SYN Flood are respectively described below.

The detection of UDP Flood comprises the following steps:

(1) acquiring an open flow data set containing UDP flow, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of flows is earlier than that of the second part of flows, and the two parts of flows contain common UDP flow and normal flow attacked by DDoS;

in one embodiment of the invention, a public data set CIC-DDoS2019 containing UDP flow collected by a university UIB on 12 th in 2018 and 11 th in 2018, 3 and 11 is obtained, wherein the flow collected on 12 th in 2018 is used as a first part of flow, and the flow collected on 11 th in 2018 is used as a second part of flow.

(2) Setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

in one embodiment of the present invention, the sampling ratio is set to 256, and as shown in fig. 2, a packet is first randomly extracted as a starting point for the incoming traffic, and then extracted every 256 packets.

(3) Extracting valid features from the UDP packets using UDP Sketch as shown in FIG. 3; the UDP Sketch is composed of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket and is used for storing a plurality of characteristics of a data packet, each counting bucket comprises 2 counters and 2 hash tables, the counters respectively occupy 1 byte, the hash tables respectively occupy 2 bytes, and a UDP Sketch counting bucket structure for UDP Flood detection is listed in a table 4. The UDP Sketch includes three operations of updating, value taking and feature extraction as shown in table 5, and the updating operation is divided into three steps: extracting binary information (IP, protocol number) of the data packet as a key, calculating subscript addresses of r counting buckets corresponding to the key by using a hash function, and adding 1 to corresponding counters of the r counting buckets. The value taking operation is divided into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, and taking out the minimum counting barrel value. The operation of extracting the features is divided into two steps: and using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch.

Table 4 UDP Sketch count bucket structure

Counter with a memory	Number of bytes occupied	Means of
			Rp	1 byte	Number of received UDP packets
Sp
		1 byte	Number of UDP packets to be sent
Hr				2 bytes	Hash table for recording IP address and port number distribution of receiving end
	Hs	2 bytes	Hash table for recording port number distribution of sending terminal

TABLE 5 Sketch operations

Operation of	Means of
		Update(key)	Adding 1 to the corresponding counter in the r counting buckets corresponding to the keys
GetMinBucket(key)	Taking out the minimum counting barrel value corresponding to the key
		GetFeature(key)	Taking out the characteristic value corresponding to the key

The method specifically comprises the following steps:

(3.1) judging whether the sampling data packet uses a UDP protocol or not according to the IP protocol number, if so, continuing to process according to the step (3.2), and if not, not processing the data packet and waiting for the next sampling data packet to arrive;

(3.2) extracting binary information (source IP, protocol number) and (sink IP, protocol number) of the data packet as keys respectively, carrying out hash calculation on the keys by using a secure hash algorithm SHA-1 to obtain a 160-bit hash address, and dividing the hash address into r pieces

And (4) carrying out updating operation after the hash address of the bit is obtained and the subscript addresses of r counting buckets in the UDP Sketch by key are obtained.

For a data packet, firstly mapping to a UDP Sketch structure by taking (source IP, protocol number) as a key, and updating sending counters in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to UDP Sketch structure once, and receiving counters in r counting buckets are updated;

and (3.3) considering that when the Server is attacked by UDP Flood, the ports of the Server end are accessed more intensively, and the ports of the Client end are distributed widely. To record this feature, we use two hash tables as shown in fig. 4, each hash table takes two bytes, the initial value of each location in the hash table is 0, and each cell in the hash table is used to indicate whether a port is accessed, and if so, it is set to 1. When the UDP data packet arrives, firstly (destination IP address, destination port number) is taken as key to carry out Hash calculation by using SHA-1, the subscript address of the corresponding Hash table Hr is obtained, and the value of the position is set to 1. And then (source port number) is taken as a key to carry out hash calculation by using SHA-1 to obtain a subscript address of Hs of a corresponding hash table, and the value of the position is set to be 1. When the UDP Flood attack occurs, the UDP packet is almost sent from the Client to the Server, the (IP address, port number) of the Client changes constantly, and the port number of the Server is fixed, so the UDP Flood attack is represented in the hash table as: 1 in the hash table Hr mapped by the Client is uniformly distributed, and 1 in the hash table Hs mapped by the Server is centrally distributed;

(3.4) considering that the process of providing domain name resolution service by DNS server and the process of UDP Flood attack suffered from server show the same characteristics: a plurality of clients send data packets to a Server, and due to the existence of asymmetric routes, a router may capture only one-way packets, and if only a DNS request packet can be captured but a DNS response packet cannot be captured, such a feature will interfere with detection of UDP Flood attacks, and therefore, a feature value needs to be added: the packet receiving rate Rp _ Spd is because the Server packet receiving rate in the DNS procedure is significantly smaller than the packet receiving rate in the DDoS attack procedure.

In addition, when the Server is attacked by UDP Flood, the packet receiving rate is much higher than the packet sending rate, so the characteristic value needs to be added: the packet transmission rate Sp _ Spd. The two feature values of Rp _ Spd and Sp _ Spd can be calculated according to Rp and Sp counters in the counting bucket when UDP Sketch performs feature extraction operation, and specific calculation formulas refer to (3-1) and (3-2), t₂And t₁Respectively representing the current time when the feature extraction operation is successfully executed and the last time when the feature extraction operation is successfully executed, and samplingtratio represents the sampling ratio set in the step (2).

(3.5) performing feature extraction operation on the UDP Sketch, and when the sum of the two counters of Rp and Sp reaches a Threshold value Threshold, extracting 2 counter values and 2 hash table values in a UDP Sketch counting bucket and Rp _ Spd and Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

(4) And (4) constructing a training set of UDPFlood classification according to the batch of samples containing the characteristic values obtained in the step (3), and judging whether the UDP flow in the samples is UDP Flood attack flow according to the IP address to mark as the flow is from an open flow data set containing DDoS flow.

In one embodiment of the present invention, the Threshold for the extracted feature operation is set to 100, and the training set part data is shown in table 6, where since the samples are all from UDP traffic, the key is directly represented by IP without storing the protocol number field. Rp, Sp, Hr, Hs, Rp _ Spd and Sp _ Spd are UDP flow characteristics, a Label of 1 indicates UDP flow, and a Label of 0 indicates normal flow.

TABLE 6UDP flow classification training set partial data

IP	Rp	Sp	Hr	Hs	Rp_Spd	Sp_Spd	Label
								192.168.50.1	100	0	16	16	30603	0	1
192.168.50.1	100	0	15	16	34929	0	1
								192.168.50.1	100	0	16	16	36269	0	1
192.168.50.1	100	0	16	16	35072	0	1
								192.168.50.1	99	1	16	16	28553	0	1
203.77.7.149	13	87	16	9	811	5428	0
								173.18.59.249	13	87	1	1	141	947	0
163.61.234.133	83	17	1	1	1223	250	0
								173.18.54.119	6	97	6	16	22	344	0
150.242.97.63	85	15	3	3	497	87	0

(5) According to the training set constructed in the step (4), a UDP flow classifier is constructed in an off-line mode by using a decision tree method and used for detecting UDP flow attacks;

(6) testing the constructed UDP flow classifier by using the second part of flow;

the method specifically comprises the following steps:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled UDP traffic by using UDP Sketch according to the step (3);

and (6.3) when the counter in the UDP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a UDP Flood flow classifier for detection, if the output is 1, indicating that the flow is UDP Flood attack flow, and if the output is 0, indicating that the flow is normal flow.

(7) And (4) directly applying the UDP Flood flow classifier tested according to the step (6) to UDP Flood attack detection in a high-speed network environment.

The detection of SYN Flood comprises the following steps:

(1) acquiring an open flow data set containing SYN flow, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of the flows is earlier than that of the second part of the flows, and the two parts of the flows contain common SYN flow and normal flow attacked by DDoS;

in an embodiment of the invention, a public data set CIC-DDoS2019 containing SYN Flood flows, which is acquired by a university at UIB on 12 th in 2018 and 11 rd in 2018, 3 and 11 th in 2018, is acquired, wherein the flows acquired on 12 th in 2018 are used as a first part of flows, and the flows acquired on 11 th in 2018, 3 and 11 th in 2018 are used as a second part of flows.

(2) Setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

in one embodiment of the present invention, the sampling ratio is set to 256, and as shown in fig. 2, a packet is first randomly extracted as a starting point for the incoming traffic, and then extracted every 256 packets.

(3) Extracting valid features for TCP packets using TCP Sketch as shown in FIG. 5; TCP Sketch consists of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket for storing a plurality of characteristics of a data packet, each counting bucket comprises 6 counters, each counting 1 byte, and Table 7 lists the TCP Sketch counting bucket structure for SYN Flood detection. The TCP Sketch includes three operations of updating, value taking and feature extraction as shown in table 5, and the updating operation is divided into three steps: extracting binary information (IP, protocol number) of the data packet as a key, calculating subscript addresses of r counting buckets corresponding to the key by using a hash function, and adding 1 to corresponding counters of the r counting buckets. The value taking operation is divided into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, and taking out the minimum counting barrel value. The operation of extracting the features is divided into two steps: and using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch.

TABLE 7TCP Sketch count bucket architecture

Counter with a memory	Number of bytes occupied	Means of
			R0	1 byte	Number of TCP control messages received
Rd	1 byte	Number of TCP messages with received data length larger than 0
			Rs	1 byte	Receiving TCP control message number containing SYN mark
S0
		1 byte	Number of TCP control messages sent
Sd				1 byte	The number of TCP messages with the transmitted data length larger than 0
	Ss	1 byte	Number of TCP control messages including SYN flag sent

The method specifically comprises the following steps:

(3.1) judging whether the sampling data packet uses a TCP (transmission control protocol) protocol according to the IP protocol number, if so, continuing to process according to the step (3.2), otherwise, not processing the data packet, and waiting for the next sampling data packet to arrive;

(3.2) binary extraction of data packetsThe group information (source IP, protocol number) and (sink IP, protocol number) are respectively used as keys, the keys are subjected to Hash calculation by using a secure Hash algorithm SHA-1 to obtain a 160-bit Hash address, and the Hash address is divided into r

And the hash address of the bit is used for executing the updating operation after the subscript addresses of r counting buckets in the TCP Sketch are obtained.

For a data packet, firstly mapping to a TCP Sketch structure by taking (source IP, protocol number) as a key, and updating a sending counter in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to TCP Sketch structure for the first time, and receiving counters in r counting buckets are updated;

(3.3) consider that the SYN Flood attack has such characteristics: when attacked by SYN Flood, the Server end shows that the packet receiving rate is much greater than the packet sending rate, so that 2 more characteristic values are required to be added: the two characteristic values can be calculated according to R0, Rd, S0 and Sd counters in a counting bucket when TCP Sketch performs characteristic extraction operation, and the specific calculation formula refers to (4-1) and (4-2), t₂And t₁Respectively representing the current time when the feature extraction operation is successfully executed and the last time when the feature extraction operation is successfully executed, and samplingtratio represents the sampling ratio set in the step (2).

(3.4) performing feature extraction operation on the TCP Sketch, and when the sum of the four counters R0, Rd, S0 and Sd reaches a Threshold, extracting the 6 counter values in the TCP Sketch counting bucket and the Rp _ Spd and the Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

(4) And (4) constructing a SYN Flood classification training set according to a batch of samples containing the characteristic values obtained in the step (3), and judging whether the TCP flow in the samples is SYN Flood attack flow according to the IP address to mark as the flow comes from an open flow data set containing DDoS flow.

In one embodiment of the present invention, the Threshold for the extracted feature operation is set to 100, and the training set part data is shown in table 8, where since the samples are all from TCP traffic, the key is directly represented by IP without storing the protocol number field. R0, Rd, Rs, S0, Sd, Ss, Rp _ Spd and Sp _ Spd are SYN flow characteristics, Label of 1 indicates SYN flow, and Label of 0 indicates normal flow.

TABLE 8SYN Flood traffic classification training set partial data

(5) According to the training set constructed in the step (4), a SYN Flood flow classifier is constructed in an off-line mode by using a decision tree method and is used for detecting the SYN Flood attack;

(6) testing the constructed SYN Flood flow classifier by using the second part of flow;

the method specifically comprises the following steps:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled TCP flow by using TCP Sketch according to the step (3);

(6.3) when the counter in TCP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a SYN flow classifier for detection, if the output is 1, indicating that the flow is a SYN flow attack flow, and if the output is 0, indicating that the flow is a normal flow.

(7) The SYN Flood flow classifier tested according to the step (6) can be directly applied to a high-speed network environment for SYN Flood attack detection.

The technical means disclosed in the invention scheme are not limited to the technical means disclosed in the above embodiments, but also include the technical scheme formed by any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and such improvements and modifications are also considered to be within the scope of the present invention.

Claims (6)

1. A UDP Flood attack detection method oriented to a high-speed network packet sampling data acquisition scene is characterized by comprising the following steps:

the method comprises the steps that (1) an open flow data set containing UDP flow is obtained, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of the flows is earlier than that of the second part of the flows, and the two parts of the flows contain common UDP flow and normal flow attacked by DDoS;

setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

step (3) extracting effective characteristics of the UDP data packet by using UDP Sketch; the UDP Sketch is composed of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket and is used for storing a plurality of characteristics of a data packet, each counting bucket comprises 2 counters and 2 hash tables, the counters respectively occupy 1 byte, the hash tables respectively occupy 2 bytes, and a UDP Sketch counting bucket structure for UDP Flood detection is listed in table 1; the UDP Sketch comprises three operations of updating, dereferencing and feature extraction shown in the table 2, wherein the updating operation is divided into three steps: extracting binary information (IP, protocol number) of a data packet as a key, calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, adding 1 to corresponding counters of the r counting barrels, and dividing the value taking operation into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, and taking out the minimum counting barrel value; the operation of extracting the features is divided into two steps: using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch;

table 1 UDP Sketch count bucket structure

Counter with a memory Number of bytes occupied Means of Rp 1 byte Number of received UDP packets Sp 1 byte Number of UDP packets to be sent Hr 2 bytes Hash table for recording IP address and port number distribution of receiving end Hs 2 bytes Hash table for recording port number distribution of sending terminal

TABLE 2 Sketch operations

Step (4) constructing a training set for UDP flow classification according to a batch of samples containing the characteristic values obtained in the step (3), and judging whether the UDP flow in the samples is UDP flow attack flow according to the IP address to mark as the flow is from an open flow data set containing DDoS flow;

step 5, according to the training set constructed in the step 4, a UDP flow classifier is constructed in an off-line mode by using a decision tree method and used for detecting UDP flow attacks;

step (6) testing the constructed UDP Flood flow classifier by using the second part of flow;

and (7) directly applying the UDP Flood flow classifier tested according to the step (6) to a high-speed network environment to carry out UDP Flood attack detection.

2. The UDP Flood attack detection method oriented to the high-speed network packet sampling data acquisition scene according to claim 1, wherein in the step (3), the method for extracting effective features by the UDP Sketch comprises the following steps:

(3.1) judging whether the sampling data packet uses a UDP protocol or not according to the IP protocol number, if so, continuing to process according to the step (3.2), and if not, not processing the data packet and waiting for the next sampling data packet to arrive;

(3.2) extracting binary information (source IP, protocol number) and (sink IP, protocol number) of the data packet as keys respectively, carrying out hash calculation on the keys by using a secure hash algorithm SHA-1 to obtain a 160-bit hash address, and dividing the hash address into r pieces

The hash address of the bit is used for executing updating operation after the subscript addresses of r counting buckets in the UDP Sketch of the key are obtained;

for a data packet, firstly mapping to a UDP Sketch structure by taking (source IP, protocol number) as a key, and updating sending counters in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to UDP Sketch structure once, and receiving counters in r counting buckets are updated;

(3.3) considering that when the Server is attacked by UDP Flood, the ports of the Server are accessed more intensively, the ports of the Client are distributed widely, two hash tables are used, each hash table occupies two bytes, the initial value of each position in each hash table is 0, each unit in each hash table is used for indicating whether a certain port is accessed or not, if the certain port is accessed, the unit is set to be 1, when a UDP data packet arrives, a destination IP address (destination IP address, destination port number) is taken as a key to perform hash calculation by using SHA-1 to obtain the corresponding Hr subscript address of the hash table, and the value of the position is set to be 1; then (source port number) is taken as a key to carry out Hash calculation by using SHA-1 to obtain a subscript address of Hs of a corresponding Hash table, the value of the position is set to be 1, when UDP Flood attacks occur, UDP data packets are almost sent to a Server end by a Client end, the (IP address and port number) of the Client end are constantly changed, and the port number of the Server end is fixed, so that the method is embodied in the Hash table as follows: 1 in the hash table Hr mapped by the Client is uniformly distributed, and 1 in the hash table Hs mapped by the Server is centrally distributed;

(3.4) considering that the process of providing domain name resolution service by DNS server and the process of UDP Flood attack suffered from server show the same characteristics: a plurality of clients send data packets to a Server, and due to the existence of asymmetric routes, a router may capture only one-way packets, and if only a DNS request packet can be captured but a DNS response packet cannot be captured, such a feature will interfere with detection of UDP Flood attacks, and therefore, a feature value needs to be added: the packet receiving rate Rp _ Spd is because the Server packet receiving rate in the DNS process is significantly smaller than the packet receiving rate in the DDoS attack process;

in addition, when the Server is attacked by UDP Flood, the packet receiving rate is much higher than the packet sending rate, so the characteristic value needs to be added: two feature values of the packet sending rates Sp _ Spd, Rp _ Spd and Sp _ Spd can be used for extracting features in UDP SketchDuring the characterization operation, the calculation is carried out according to the Rp and Sp counters in the counting barrel, and the specific calculation formulas refer to (1-1) and (1-2), t₂And t₁Respectively representing the current time of successfully executing the feature extraction operation and the last time of successfully executing the feature extraction operation, and the sampling ratio set in the step (2);

(3.5) performing feature extraction operation on the UDP Sketch, and when the sum of the two counters of Rp and Sp reaches a Threshold value Threshold, extracting 2 counter values and 2 hash table values in a UDP Sketch counting bucket and Rp _ Spd and Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

3. The UDP flow attack detection method oriented to the high-speed network packet sampling data acquisition scenario according to claim 1, wherein in the step (6), the method for testing the constructed UDP flow classifier by using the second part of traffic is as follows:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled UDP traffic by using UDP Sketch according to the step (3);

and (6.3) when the counter in the UDP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a UDP Flood flow classifier for detection, if the output is 1, indicating that the flow is UDP Flood attack flow, and if the output is 0, indicating that the flow is normal flow.

4. A SYN Flood attack detection method oriented to a high-speed network packet sampling data acquisition scene is characterized by comprising the following steps:

the method comprises the steps that (1) an open flow data set containing SYN flow is obtained, wherein the open flow data set contains flows collected in two different time periods, the collection time of the first part of the flows is earlier than that of the second part of the flows, and the two parts of the flows contain common SYN flow and normal flow attacked by DDoS;

setting a sampling ratio, and carrying out equidistant sampling on the first part of flow;

step (3) extracting effective characteristics of the TCP data packet by using TCP Sketch; the TCP Sketch is composed of r rows and c columns of two-dimensional arrays, Sketch [ i ] [ j ] represents a counting bucket and is used for storing a plurality of characteristics of a data packet, each counting bucket comprises 6 counters which respectively occupy 1 byte, a TCP Sketch counting bucket structure for SYN Flood detection is listed in Table 3, the TCP Sketch comprises three operations of updating, valuing and extracting characteristics as shown in Table 2, and the updating operation is divided into three steps: extracting binary information (IP, protocol number) of a data packet as a key, calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, adding 1 to corresponding counters of the r counting barrels, and dividing the value taking operation into two steps: according to a key formed by binary information (IP, protocol number), calculating subscript addresses of r counting barrels corresponding to the key by using a hash function, comparing r counting barrel values, taking out the minimum counting barrel value, and extracting characteristic operation, wherein the operation is divided into two steps: using a value taking operation to take out the minimum counting barrel value corresponding to the key, judging whether the corresponding counter reaches a threshold value, if so, storing all counter values into a file, and subtracting the minimum counting barrel value from all counting barrel values corresponding to the key in the Sketch;

TABLE 3 TCP Sketch count bucket architecture

Counter with a memory Number of bytes occupied Means of R0 1 byte Number of TCP control messages received Rd 1 byte Number of TCP messages with received data length larger than 0 Rs 1 byte Receiving TCP control message number containing SYN mark S0 1 byte Number of TCP control messages sent Sd 1 byte The number of TCP messages with the transmitted data length larger than 0 Ss 1 byte Number of TCP control messages including SYN flag sent

Step (4), constructing a SYN flow classified training set by a batch of samples containing the characteristic values obtained in the step (3), and judging whether the TCP flow in the samples is SYN flow attack flow according to the IP address to mark as the flow is from an open flow data set containing DDoS flow;

step 5, according to the training set constructed in the step 4, a SYN Flood flow classifier is constructed in an off-line manner by using a decision tree method and is used for detecting the SYN Flood attack;

step (6) testing the constructed SYN Flood flow classifier by using the second part of flow;

and (7) the SYN Flood flow classifier tested according to the step (6) can be directly applied to a high-speed network environment for SYN Flood attack detection.

5. The SYN Flood attack detection method oriented to the high-speed network packet sampling data acquisition scenario according to claim 4, wherein in step (3), the method for TCP Sketch to extract valid features from TCP data packets is as follows:

(3.1) judging whether the sampling data packet uses a TCP (transmission control protocol) protocol according to the IP protocol number, if so, continuing to process according to the step (3.2), otherwise, not processing the data packet, and waiting for the next sampling data packet to arrive;

(3.2) extracting binary information (source IP, protocol number) and (sink IP, protocol number) of the data packet as keys respectively, carrying out hash calculation on the keys by using a secure hash algorithm SHA-1 to obtain a 160-bit hash address, and dividing the hash address into r pieces

The hash address of the bit is used for executing updating operation after the subscript addresses of r counting buckets in TCP Sketch are obtained;

for a data packet, firstly mapping to a TCP Sketch structure by taking (source IP, protocol number) as a key, and updating a sending counter in r counting buckets; then (sink IP, protocol number) is used as key, mapping is carried out to TCP Sketch structure for the first time, and receiving counters in r counting buckets are updated;

(3.3) consider that the SYN Flood attack has such characteristics: when attacked by SYN Flood, the Server end shows that the packet receiving rate is much greater than the packet sending rate, so that 2 more characteristic values are required to be added: a packet receiving rate Rp _ Spd and a packet sending rate Sp _ Spd, which can be counted by R0, Rd, S0 and Sd counters in the counting bucket when TCP Sketch performs a feature extraction operationCalculated, the specific calculation formula refers to (2-1) and (2-2), t₂And t₁Respectively representing the current time when the feature extraction operation is successfully executed and the last time when the feature extraction operation is successfully executed, and samplingtratio represents the sampling ratio set in the step (2).

(3.4) performing feature extraction operation on the TCP Sketch, and when the sum of the four counters R0, Rd, S0 and Sd reaches a Threshold, extracting the 6 counter values in the TCP Sketch counting bucket and the Rp _ Spd and the Sp _ Spd calculated according to the counting bucket as a group of feature values to form a sample and storing the sample in a file.

6. The SYN Flood attack detection method oriented to the high-speed network packet sampling data acquisition scenario according to claim 4, wherein in the step (6), the method for testing the constructed TCP Flood flow classifier by using the second part of flow is as follows:

(6.1) sampling the second part of the flow according to the step (2);

(6.2) extracting features of the sampled TCP flow by using TCP Sketch according to the step (3);

(6.3) when the counter in TCP Sketch reaches a threshold value, generating a batch of samples containing IP addresses and characteristic values, sending the samples into a SYN flow classifier for detection, if the output is 1, indicating that the flow is a SYN flow attack flow, and if the output is 0, indicating that the flow is a normal flow.

Images