CN113242240B

CN113242240B - Method and device capable of detecting DDoS attacks of multiple types of application layers

Info

Publication number: CN113242240B
Application number: CN202110505288.8A
Authority: CN
Inventors: 周华春; 李颖之; 李坤; 杨天奇; 李丽娟; 沈琦
Original assignee: Beijing Jiaotong University
Current assignee: Beijing Jiaotong University
Priority date: 2021-05-10
Filing date: 2021-05-10
Publication date: 2022-07-01
Anticipated expiration: 2041-05-10
Also published as: CN113242240A

Abstract

The invention provides a method and a device capable of detecting DDoS attacks of various application layers. The device includes: the characteristic generation module generates effective characteristic information suitable for DDoS attacks of various application layers and transmits the effective characteristic information to the offline training module; the offline training module is used for obtaining various application layer DDoS attack detection models through training according to the effective characteristic information, and the detection sample set is used for training and verifying the various application layer DDoS attack detection models; the online detection module deploys a trained multi-type application layer DDoS attack detection model, the multi-type application layer DDoS attack detection model is used for detecting the network flow in real time, and the application layer DDoS attack detection result of the network flow is output. The invention can detect various application layer DDoS attacks including HTTP-Flood attack, HTTP-Get attack, HTTP-Post attack, CC attack and the like, can improve the application layer DDoS attack detection accuracy and reduce malicious flow.

Description

Method and device capable of detecting DDoS attacks of multiple types of application layers

Technical Field

The invention relates to the technical field of Internet, in particular to a method and a device capable of detecting DDoS attacks of multiple types of application layers.

Background

DDoS attacks refer to launching a distributed denial of service attack on one or more targets by using a client or server technology to combine multiple computers as an attack platform. Compared with the traditional DDoS attack based on a low-layer protocol, the application layer DDoS attack is realized by using the high-layer protocol and has the characteristic of difficult detection. The application layer DDoS attack is premised on normal TCP connection, real IP address and IP grouping, and the HTTP flow forming the attack does not have the symbolic characteristic of the traditional DDoS attack. The application layer DDoS attack has strong operability and low attack threshold, and brings a series of major risks such as customer loss, confidential file leakage, commercial loss and the like to network service providers. At present, a popular low-level detection system is difficult to judge whether a user request is from a normal user or an attacker, and the problem that application of DDoS attack is urgently to be solved is solved.

At present, a traditional detection method based on statistics is adopted in a DDoS attack detection method in the prior art, and a specific rule is set, so that an alarm is initiated when a condition value in the rule exceeds a threshold value set according to experience. The disadvantages of this method are: only abnormal flow and normal flow can be distinguished, and the specific types of application layer DDoS and application layer DDoS in the abnormal flow cannot be detected.

Disclosure of Invention

The embodiment of the invention provides a method and a device for detecting DDoS attacks of various application layers, so as to realize effective application layer DDoS attack detection on a network to be detected.

In order to achieve the purpose, the invention adopts the following technical scheme.

According to an aspect of the present invention, there is provided an apparatus for detecting multiple kinds of application layer DDoS attacks, including: the device comprises a feature generation module, an offline training module and an online detection module;

the characteristic generating module is used for generating effective characteristic information suitable for various application layer DDoS attacks and transmitting the effective characteristic information to the offline training module;

the offline training module is used for obtaining various application layer DDoS attack detection models through training according to the effective characteristic information, and training and verifying the various application layer DDoS attack detection models through a detection sample set;

the online detection module is used for deploying the trained multi-type application layer DDoS attack detection models, detecting the flow-through network traffic in real time by using the multi-type application layer DDoS attack detection models, and outputting the application layer DDoS attack detection results of the flow-through network traffic.

Preferably, the feature generation module is specifically configured to extract network traffic including multiple types of application layer DDoS attacks by using a feature extraction tool, generate multidimensional feature information corresponding to the network traffic of each type of application layer DDoS attack, analyze and screen the multidimensional feature information of each type of application layer DDoS attack by using a feature selection technology and an analysis method of a statistical chart, obtain effective feature information of each type of application layer DDoS attack, combine the effective feature information of all types of application layer DDoS attacks, and finally obtain the effective feature information of multiple types of application layer DDoS attacks.

Preferably, the valid feature information includes a time feature, a stream header feature and a payload feature; the time characteristics comprise stream duration, interval time for transmitting two data packets in the stream and derivative characteristics thereof, wherein the derivative characteristics comprise an average value, a maximum value, a minimum value, a standard deviation and a variance; the flow header characteristics include the number of flow bytes per second, the number of packets in the flow per second, and the forward/reverse packet header length; the payload characteristics include the number of forward/reverse packets per second, the packet length, and its SYN/FIN/RST flag bit count.

Preferably, the offline training module is specifically configured to mark effective information of multiple types of application layer DDoS attacks as different data streams according to flow quintuple information, where the quintuple information includes a source IP address, a destination IP address, a source port number, a destination port number, and a communication protocol, divide the marked data streams by using a set duration as a unit to obtain an offline detection sample set, and divide the offline detection sample set into an offline training sample set and an offline verification sample set in proportion;

integrating a plurality of strong learners by using a Stacking integration method, selecting the strong learners with front performance arrangement as base learning classifiers in the Stacking, and integrating each base learning classifier after classification training and testing to obtain a multi-type application layer DDoS attack detection model;

inputting the characteristic vectors in the offline training sample set in the appointed time period into the multi-type application layer DDoS attack detection model, training and classifying the input characteristic vector information by the model, judging the DDoS attack flow and the normal flow of different types of application layers, and verifying the output result of the multi-type application layer DDoS attack detection model by using the offline verification sample set.

Preferably, the indexes used for verification by the offline verification sample set include: the accuracy, the precision, the recall rate and the F1 value, and when the indexes are in the balance optimal value, the corresponding parameters are the optimal parameters of the DDoS attack detection model of the multiple application layers.

Preferably, the base learning classifier comprises a random forest model, an extreme random tree model and a LightGBM model.

Preferably, the online detection module is specifically configured to deploy a multiple application layer DDoS attack detection model at a network traffic entrance, capture, by using a traffic collection tool, a to-be-detected flow-through network traffic within a set time duration at the network traffic entrance in real time, and extract feature information corresponding to the flow-through network traffic; detecting the characteristic information of the flow-through network traffic according to a deployed multi-type application layer DDoS attack detection model, and identifying specific application layer DDoS attack types and normal traffic;

the output information of the DDoS attack detection model of the multiple types of application layers comprises: the method comprises the steps that source and destination IP addresses and flow labels which are detected as DDoS attack flows of different kinds of application layers are used for distinguishing different flow information subsequently, and the flow labels are used for indicating whether the flows are abnormal flows and indicating specific application layer DDoS attack kinds in detail;

the output information of the multi-type application layer DDoS attack detection model further comprises: and the accuracy, the precision, the recall rate and the F1 value verify indexes, and when the indexes are in balance and optimal, the online detection stage has optimal performance.

According to another aspect of the invention, a method for detecting multiple kinds of application layer DDoS attacks is provided and applied to the device, the method comprises the following steps:

extracting network flow containing multi-type application layer DDoS attacks by utilizing a characteristic extraction tool, and generating effective characteristic information suitable for the multi-type application layer DDoS attacks;

training according to the effective characteristic information to obtain various application layer DDoS attack detection models, and training and verifying the various application layer DDoS attack detection models by using a detection sample set;

the method comprises the steps of deploying trained multiple application layer DDoS attack detection models at a network entrance, acquiring detection flow at a flow-through network flow entrance to be detected in real time by using a real-time flow capture tool, identifying the detection flow by using the multiple application layer DDoS attack detection models, and outputting an application layer DDoS attack detection result of the flow-through network flow.

Preferably, the outputting the detection result of the DDoS attack on the application layer flowing through the network traffic includes:

the output information of the multi-type application layer DDoS attack detection model further comprises: the accuracy, the precision, the recall rate and the F1 value verify indexes, and when the indexes are in balance and optimal, the online detection stage has better performance.

Preferably, the accuracy rate is (TP + TN)/total sample, defined as the percentage of the result with correct prediction to the total sample, and represents the overall prediction accuracy degree;

the accuracy rate is TP/(TP + FP), which is defined as the probability of actually being a positive sample among all samples predicted to be positive, and represents the prediction accuracy in the positive sample result;

the recall ratio is TP/(TP + FN), defined as the probability of being predicted as a positive sample among the actually positive samples;

the F1 value is precision recall 2/(accuracy + recall), which represents the harmonic mean of precision and recall;

wherein TP represents a true sample, i.e., a positive sample predicted to be positive by the model; FP represents a false positive case, i.e., a negative sample predicted to be positive by the model; FN represents a false negative case, i.e., a positive sample predicted to be negative by the model; TN represents a true negative case, i.e., a negative sample predicted to be negative by the model.

According to the technical scheme provided by the embodiment of the invention, the embodiment of the invention provides a method for detecting DDoS attacks on various application layers, which can acquire various effective characteristic information and effectively improve the accuracy of subsequent model training; an optimal model suitable for subsequent flow detection can be obtained in the application layer DDoS attack detection model training stage; whether the flow-through traffic is application layer DDoS attack traffic or not and the specific type of the attack can be output online. The embodiment of the invention can detect various application layer DDoS attacks such as HTTP-Flood attack, HTTP-Get attack, HTTP-Post attack, CC attack and the like, can improve the application layer DDoS attack detection accuracy and reduce malicious flow.

Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a structural diagram of a device capable of detecting DDoS attacks of multiple application layers according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of acquiring traffic characteristic information according to an embodiment of the present invention;

fig. 3 is a schematic flowchart of offline training of a multi-type application layer DDoS attack detection model according to an embodiment of the present invention;

fig. 4 is an integrated model schematic diagram of an application layer DDoS offline model according to an embodiment of the present invention;

fig. 5 is a schematic flowchart of a device capable of detecting DDoS attacks of multiple types of application layers according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

For the convenience of understanding the embodiments of the present invention, the following description will be further explained by taking several specific embodiments as examples in conjunction with the drawings, and the embodiments are not to be construed as limiting the embodiments of the present invention.

The embodiment of the invention analyzes the characteristic behaviors of each application layer DDoS attack, and can detect the specific application layer DDoS attack types by adopting multi-classification detection based on machine learning.

The embodiment of the invention carries out detection experiments aiming at various application layer DDoS attacks, and specifically comprises HTTP-Post attacks, HTTP-Get attacks, HTTP-Flood attacks, CC attacks and other attacks. The HTTP-Post attack and HTTP-Get attack principle is that a large number of incomplete HTTP data packets, namely partial HTTP requests, are sent to an attacked target host or a web site, so that server resources are occupied and even crashed; the HTTP Flood attack is a Get request mode, and controls a large number of hosts to send a large number of requests such as images and files to a target server, so that the target server is inundated by the large number of requests and finally crashes; the principle of the CC attack is that a large number of high-frequency HTTP requests are sent to one or more webpages in a server by utilizing an HTTP protocol, so that the server is busy responding to attackers, consumes a large amount of computing resources, finally causes the server to crash, and cannot respond to user requests.

Example one

The structure diagram of a device capable of detecting DDoS attacks on multiple application layers provided by the embodiment of the invention is shown in fig. 1, and the structure diagram includes: the device comprises a feature generation module, an offline training module and an online detection module.

And the feature generation module 10 is configured to generate effective feature information suitable for various application layer DDoS attacks, and transmit the effective feature information to the offline training module.

And the offline training module 20 is configured to obtain multiple types of application layer DDoS attack detection models according to the effective feature information training, and train and verify the multiple types of application layer DDoS attack detection models by using a detection sample set.

The online detection module 30 is configured to deploy the trained multi-type application layer DDoS attack detection models, detect network traffic flowing through in real time by using the multi-type application layer DDoS attack detection models, and output an application layer DDoS attack detection result of the network traffic flowing through.

Specifically, the feature generation module provides a method for acquiring application layer DDoS traffic feature information. And extracting the network traffic containing the multi-type application layer DDoS attack by using a feature extraction tool, and generating multi-dimensional feature information corresponding to the network traffic of each application layer DDoS attack. Because various information extracted according to the flow data cannot be directly used for modeling or has the problem of characteristic information redundancy, the embodiment of the invention carries out characteristic screening and preprocessing operation on the original data and finally extracts effective characteristic information suitable for DDoS attack flow of all application layers.

The principles of each specific type of application layer DDoS attack are different, and the corresponding effective characteristic information is different. In order to further improve the detection rate of subsequent training models, the embodiment of the invention screens different effective characteristics for each kind of application layer DDoS attacks. Specifically, by using a feature selection technology and an analysis method of a statistical chart, analysis and screening are performed on the application layer DDoS attack feature sets of each type, so that effective feature information corresponding to each application layer DDoS attack is obtained.

The effective characteristic information comprises a time characteristic, a stream header characteristic and a payload characteristic; the time characteristics comprise stream duration, interval time for sending two data packets in the stream, derivative characteristics of the interval time and the like; the flow header characteristics include the number of flow bytes per second, the number of packets in the flow per second, the forward/reverse packet header length, etc.; the payload characteristics include the number of forward/reverse packets per second, the packet length and its flag bit count such as SYN/FIN/RST, etc. Wherein the derived features include mean, maximum, minimum, standard deviation, variance, and the like.

Further, the characteristic generating module may combine the effective characteristic information of each variety of application layer DDoS, and summarize the effective characteristic information into the effective characteristic information of the various types of application layer DDoS attacks.

Specifically, the offline training module provides an offline model training method for distributed denial of service (DDoS) attack detection of multiple application layers. Optionally, in the embodiment, the application layer DDoS attack detection model may select a detection model based on Stacking integration. And inputting the characteristic vectors of the detection sample set in the specified time period into the integrated detection model, training and classifying the input characteristic information by the model, and judging the DDoS attack flow and the normal flow of different types of application layers.

The detection sample set comprises a training sample set and a verification sample set, and the training sample set is used for training the DDoS attack detection model of various application layers to obtain an offline detection model; the verification sample set is used for optimizing the model parameters, and the effectiveness of the DDoS attack detection model of the application layer can be verified according to the verification sample set. Wherein the metrics for verification include: the accuracy, the precision, the recall rate and the F1 value, and when the indexes are in balance and optimal, the corresponding parameter values are the optimal parameters of the model.

Specifically, the online detection module provides a method for performing an online detection model of DDoS attack on an application layer by using an DDoS attack detection model on the application layer, and the method comprises the following steps: capturing the network traffic to be detected within a set duration, and extracting characteristic information corresponding to the network traffic to be detected; and detecting the network traffic to be detected according to the deployed application layer DDoS attack detection model, and identifying the specific application layer DDoS type and normal traffic.

And the network flow to be detected in the set time length is the network flow with the specified time length captured on line by using a flow collecting tool at a network flow inlet and is stored as a flow information file for subsequent conversion.

The characteristic information is obtained by extracting the characteristic information of the network traffic to be detected through a traffic characteristic extraction tool, the characteristic information is stored as a data set containing various application layer DDoS attack traffic and normal traffic, and a sample set for subsequent online detection is generated.

The output information of the DDoS attack detection model of the multiple types of application layers comprises: the source and destination IP addresses and the flow labels are detected as DDoS attack flows of different types of application layers, the source and destination IP of the DDoS attack flows of the different types of application layers are used for subsequently distinguishing different flow information, and the flow labels are used for indicating whether the flows are abnormal flows or not and indicating specific DDoS attack types of the application layers in detail.

The output information of the multi-type application layer DDoS attack detection model further comprises: and when the indexes are in balance and optimal, the online detection stage has better performance.

Fig. 2 is a schematic diagram for collecting DDoS attack feature information of multiple types of application layers according to an embodiment of the present invention. The process comprises the following steps: acquiring the DDoS experimental flow of multiple types of application layers in a specified time period, generating the characteristic information of the flow by adopting a flow characteristic extraction tool, and extracting the effective characteristic information of each application layer DDoS attack. And preprocessing the extracted effective characteristic information to remove redundant information in the characteristic information.

Optionally, the multiple kinds of application layer DDoS attacks include: Golden-Eye initiated HTTP-POST, Golden-Eye initiated HTTP-GET, Hulk initiated HTTP attack, CC attack, and the like.

Selecting basic flow characteristic information taking a time period including application layer DDoS attack as a specified time period, wherein the flow information comprises different kinds of application layer DDoS attack flow and normal flow.

Optionally, the network traffic in the specified time period is divided by taking the set duration as a unit, and the feature engineering construction method based on the sliding window mechanism derives a series of features by using the original features, such as the maximum value, the minimum value, the variance, the range difference, the standard deviation and the like in the window, so that the selection range of the features is expanded.

Optionally, a flow feature extraction tool, cif flow meter, or other tool is used to extract feature information of the flow, and the collected network flow information is converted into a sample set containing features and label information and stored in a detection sample set format for subsequent training.

The 82-dimensional feature information extracted by the CICFlowMeter can be divided into time features, stream header features and payload features according to the characteristics of the feature information, and partial feature information contents shown in the embodiment of the invention are shown by referring to the table 1, wherein the partial feature information contents comprise feature types, feature names and feature meanings.

As can be seen from table 1, the time characteristics may include a stream duration, an interval time between two packets sent in a stream and its derivatives, an interval time between forward/backward packets in a stream and its derivatives, an active time before a stream is idle and its derivatives, and so on. Flow header characteristics may include the number of bytes of flow per second, the number of packets in the flow per second, the forward/reverse packet header length, etc. The payload characteristics may include the number of forward/reverse packets per second, the packet length and its flag bit count such as SYN/FIN/RST, etc. Wherein the derived features may include mean, maximum, minimum, standard deviation, variance, and the like.

TABLE 1

And performing effective characteristic screening on the extracted flow characteristic information.

In order to ensure that the fitting degree is reduced under the condition of high accuracy, effective characteristic information of each application layer DDoS attack is screened, and characteristic information of application layer DDoS experimental flow in a specified time period is analyzed and compared, the embodiment of the invention combines a characteristic importance selection and threshold value statistics method to screen characteristics with obvious differences between different kinds of application layer DDoS attacks and normal, so that representative effective characteristics of the different kinds of application layer DDoS are obtained, and the characteristic dimension is reduced.

Optionally, the attack tool source code and the relevant parameters of the attack tool launch command are reviewed to further understand the principles of attack implementation. Summarizing and drawing an attack source code flow chart and an attack sequence chart, showing the sequence of message interaction between an attacker and a victim, and analyzing the relation between each application layer DDoS attack behavior and the distribution of corresponding characteristic values.

Optionally, the source code of the feature extraction tool cif cflowmeter is consulted, the overall working logic of the tool is understood in combination with the source code, a corresponding principle flow chart is drawn, the specific representation (threshold, quantity and the like) and the feature meaning of the feature are further analyzed in combination with the principle of each feature generation function and the detailed analysis of package capture tools such as Wireshark and the like, and the relationship between the screened feature and the attack behavior is summarized.

The step of feature screening may comprise: firstly, a feature selection method in feature engineering is utilized, three models of a random forest, an XGboost and an extreme random tree are utilized to carry out feature importance ordering, and features which are better represented in two or more models are screened. And secondly, observing by using a statistical chart method, respectively comparing distribution conditions of the DDoS on the probability distribution histogram of the normal application layer and the DDoS of a certain type of application layer, and determining a threshold limit for distinguishing the normal application layer from the attack application layer according to the distribution proportion range. The step of screening features can not only remove redundant features to reduce complexity, but also prevent an overfitting state of the model.

And respectively determining effective characteristics for various attacks of the application layer DDoS, including HTTP-POST initiated by Golden-Eye, HTTP-GET initiated by Golden-Eye, HTTP attack initiated by Hulk, CC attack and the like according to the screening steps. Summarizing the characteristic information which is better in all kinds of attacks to finally obtain the characteristic information which is effective to the DDoS attacks of various application layers, wherein the specific characteristic information is shown in a table 2, and the table 2 comprises characteristic types, characteristic names and characteristic meanings.

And performing characteristic preprocessing on the screened flow characteristic information.

Specifically, the obtained feature information is subjected to a feature preprocessing step, such as converting some quantitative features into simple numerical values by means of binarization, converting qualitative features which cannot be directly used into quantitative features, supplementing missing values, and the like. Through the preprocessing, the problems of characteristic information redundancy, characteristic information format inconsistency, missing values and the like contained in the characteristic information which is not preprocessed can be solved, and finally the characteristic values are converted into the characteristic information which can be used for post-training.

Fig. 3 is a schematic flow chart of offline training of a multi-type application layer DDoS attack detection model according to an embodiment of the present invention, and a specific processing procedure includes: and marking the characteristic information acquired at the last stage as different data flows according to the five-tuple information of the flow, wherein the five-tuple information is respectively a source IP address, a destination IP address, a source port number, a destination port number and a communication protocol. And dividing the marked data stream by taking the set time length as a unit to obtain an offline detection sample set.

TABLE 2

And dividing the offline detection sample set into an offline training sample set and an offline verification sample set in proportion, wherein samples used for subsequently training the application layer DDoS attack detection model come from the offline training sample subset. The offline training sample set includes label information. The label information is judged and marked according to the source IP address and the destination IP address and is used for indicating different kinds of application layer DDoS attack flow and normal flow. The output information of the application layer DDoS attack detection model comprises the application layer DDoS attack traffic detected as different types, the accuracy, the precision, the recall rate and the F1 value detected as normal traffic.

Specifically, an offline training sample set in the offline detection sample set is input into the integrated machine learning algorithm model, and the trained model can select a suitable model according to a specific experimental environment, a flow sample and the like. At present, a machine learning method is mostly adopted to train a model, the purpose of machine learning is to train a strong learner which is stable, has strong generalization capability and can perform better in all aspects, but in the actual training process, a better base learner which performs better in some aspects can be obtained. Through different ensemble learning, the classifiers in machine learning are combined, so that a strong learner with better performance and stronger generalization capability can be obtained, and the purpose of better detecting DDoS attack of an application layer is achieved.

Fig. 4 is an integrated model schematic diagram of an application layer DDoS offline model according to an embodiment of the present invention. Optionally, referring to fig. 4, a Stacking integration method is adopted to compare a plurality of strong learners, and three models, namely a random forest, an extreme random tree and a LightGBM, which are well represented are selected as a base learning classifier in the Stacking, and a detection model with higher detection performance is obtained after a new round of classification training and testing, so that the stability and the generalization of the detection model are improved. The principle of the Stacking integration method is that all trained models are used for testing and training data of a test set, and a predicted value of a training sample is used as a characteristic value in a new training set, so that a new data set is formed. After the data set is proportionally divided into a training set and a test set, a new model (such as logistic regression) is used for training the new training set again, and finally the test set is predicted. And carrying out classification training and testing on each base learning classifier to obtain a multi-type application layer DDoS attack detection model.

Optionally, the base learning classifier in the Stacking model may select a random forest model, and a Random Forest (RF) utilizes a Bagging integration method to reduce the variance of the model, aiming at constructing hundreds or even thousands of decision tree models (with a large depth). Each decision tree model randomly selects a part of data sets as a training set, and each node selects an optimal feature from the part of features to divide. The final overall classifier is obtained by performing weightless Voting classification on hundreds of decision trees.

Optionally, the base learning classifier in the Stacking model may select an extreme random tree model, and an extreme random tree (ET) algorithm is an integrated result of a plurality of decision trees by using Bagging, directly using an original training set, and randomly selecting a feature value to partition the decision trees. The final overall classifier is obtained by performing weightless Voting classification on hundreds of decision trees. The characteristics can ensure that the ET has good generalization capability, high detection precision and high detection speed.

Optionally, the base learning classifier in the Stacking model may select a LightGBM model, and the LightGBM utilizes a Boosting integration method, and through multiple iterations, each iteration generates a weak classifier (regression tree with a lower depth), and each classifier is trained on the basis of the residual error of the last classifier, so as to continuously reduce the deviation to improve the precision of the final classifier. And the final total classifier is obtained by weighting and summing the weak classifiers obtained by each round of training according to the precision of the weak classifiers, and the training speed is increased by using a histogram method.

A model with better performance can be obtained through parameter adjustment of the model, for example, parameters in three models of a random forest, an extreme random tree and a LightGBM are adjusted, and an application layer DDoS attack detection model is obtained through multiple times of training. When the optimal parameter is selected, the accuracy, the precision, the recall rate and the value F are considered, the values of the parameter are adaptively adjusted, and the value when the accuracy, the precision, the recall rate and the value F1 are balanced to be the optimal parameter, wherein:

the accuracy rate is (TP + TN)/total sample, defined as the percentage of the total sample that is the result of correct prediction, and represents the accuracy of the prediction as a whole. The higher the accuracy, the better the performance of the model.

The accuracy ratio TP/(TP + FP) is defined as the probability that the sample is actually positive among all samples predicted to be positive, and represents the accuracy of the prediction in the positive sample result. The higher the accuracy, the better the model performance.

The recall ratio TP/(TP + FN) is defined as the probability of being predicted as a positive sample among the actual positive samples. The higher the recall rate, the better the performance of the model.

F1 value precision recall 2/(accuracy + recall) represents the harmonic mean of precision and recall. The higher the F1 value, the better the model performance.

Wherein tp (true positive) represents a true positive example, i.e. a positive sample predicted to be positive by the model; fp (false positive) represents a false positive case, i.e., a negative sample predicted to be positive by the model; fn (false negative) represents a false negative case, i.e., a positive sample predicted to be negative by the model; tn (true negative) represents a true negative example, i.e., a negative sample predicted to be negative by the model.

Most of the current DDoS detection methods consider the DDoS detection methods as a two-classification problem, namely, whether the flow is normal or abnormal is judged. However, this viewpoint can only distinguish between abnormal traffic and normal traffic, and cannot detect a specific type of application layer DDoS or other mainstream DDoS in the abnormal traffic. The embodiment of the invention firstly carries out feature construction aiming at various mainstream attack types in abnormal flow, and identifies the DDoS attack and the mainstream attack types of an application layer by using a multi-classification method in an off-line model training stage.

Example two

Fig. 5 is a schematic workflow diagram of a device capable of detecting various application layer DDoS attacks on line according to an embodiment of the present invention, where a specific processing flow includes: in the stage of on-line capturing the flow at the network inlet, a real-time flow capturing tool is used for acquiring the detection flow at the network flow inlet; in the on-line characteristic generation stage, the generated flow file is read on line, and a corresponding detection sample set is generated by adopting a flow characteristic extraction tool; and in the online prediction stage, selecting effective characteristic information screened in the characteristic analysis stage as input, and performing online prediction on the experimental flow to be detected by using the trained application layer DDoS attack detection model.

Optionally, the automatic processes of online traffic capture, feature generation and online model detection are realized by using the SHELL script under the system. In the stage of on-line capturing flow, a command for capturing the flow at the network inlet on line is added into the script and is stored into a proper file format; in the online characteristic generation stage, the obtained flow information is input into an extraction tool, and a sample set containing multi-dimensional characteristic information is generated online; and in the online prediction stage, deploying an online detection model in the environment to realize the online prediction function.

Optionally, in the stage of capturing traffic at the network traffic entrance on line, a real-time traffic capture tool is used to obtain the detected traffic at the network traffic entrance.

The real-time traffic capturing tool can utilize a packet capturing soft Tcpdump carried by a Linux system, and the software can capture a specified protocol packet passing through a specified network card according to a specified interface, interval time and the like to obtain the network traffic to be detected within a specified time. In the embodiment of the invention, the specified interval time of Tcpdump is set to 60s, network flow can be acquired in real time once per minute and is stored into a flow format for subsequent processing.

Optionally, in the online feature generation stage, the generated flow file is read online, and a corresponding original data set is generated by using software such as a flow feature extraction tool, cif flowmeter, and the like. According to the flow file stored online every minute, a sample set containing 82-dimensional characteristic information can be generated every minute.

Optionally, the feature selection technology and the statistical graph method described above are used to screen the effective features of the original data set, delete the redundant features with low correlation, and generate the detection sample set required for subsequent training and the like on line.

The online detection sample set is divided into a training sample set and a verification sample set according to a certain proportion, and the detection sample comprises screened effective characteristic information and label information used for indicating whether the detection sample is detected as application layer DDoS attack flow.

And in the online prediction stage, selecting effective characteristic information screened in the characteristic analysis stage as input, and performing online detection on the experimental flow to be detected by using the trained application layer DDoS attack detection model.

The output information of the online application layer DDoS attack detection model comprises: and detecting the source and destination IP addresses and the prediction labels of the DDoS attack traffic of different application layers. The source and destination IP of the DDoS attack flow of the different kinds of application layers is used for subsequently distinguishing different flow information to generate a real label; the prediction label is used for indicating whether the flow is abnormal flow and indicating the specific application layer DDoS attack type in detail.

Optionally, the method and the device provided by the embodiment of the invention use accuracy, precision, recall rate and the F1 value as verification indexes and introduce malicious traffic reduction rate to verify the effectiveness of online detection. When the accuracy, the precision, the recall rate, the F1 value and the malicious traffic reduction rate are all better, it can be shown that the detection rate can be effectively improved and the malicious traffic can be reduced in the online detection stage, wherein:

the accuracy rate is (TP + TN)/total sample, defined as the percentage of the total sample that is the result of correct prediction, and represents the accuracy of the prediction as a whole. The higher the accuracy, the better the on-line detection effect.

The accuracy ratio TP/(TP + FP) is defined as the probability that the sample is actually positive among all samples predicted to be positive, and represents the accuracy of the prediction in the positive sample result. The higher the accuracy rate, the better the on-line detection effect.

The recall ratio TP/(TP + FN) is defined as the probability of being predicted as a positive sample among the actual positive samples. The higher the recall rate, the better the online detection effect.

F1 value precision recall 2/(accuracy + recall) represents the harmonic mean of precision and recall. The higher the F1 value, the better the on-line detection effect.

The malicious traffic reduction rate is (number of attack traffic before detection-number of attack traffic after detection)/number of total traffic data items, and is expressed according to the ratio of the difference of the number of attack traffic before and after detection to the total traffic data items, and the higher the malicious traffic reduction rate is, the better the online detection effect is.

Wherein tp (true positive) represents a true positive example, i.e. a positive sample predicted to be positive by the model; fp (false positive) represents a false positive case, i.e., a negative sample predicted to be positive by the model; FN (false negative) indicates a false negative case, namely a positive sample predicted to be negative by the model; tn (true negative) represents a true negative example, i.e., a negative sample predicted to be negative by the model.

In summary, the embodiment of the present invention can obtain the attack traffic and the normal traffic in the network on line, extract the multidimensional feature information of the traffic by using the traffic feature extraction tool, and in addition, screen the features by using the feature selection technology in the feature engineering and the threshold observation method of the statistical graph, delete the redundant features with low correlation, and reduce the feature dimension of the data set; the integrated machine learning model can be used for detecting the flow, the attack type of the application layer DDoS can be specifically identified on the basis of judging whether the flow is abnormal on line, the detection accuracy is improved, and the problem that the application layer DDoS attack is difficult to detect and identify in the prior art can be effectively solved. The flow through can be captured and detected on line, the purpose of monitoring the flow on line can be achieved, and the loss caused by DDoS is effectively reduced.

The embodiment of the invention can detect various application layer DDoS attacks such as HTTP-Flood attack, HTTP-Get attack, HTTP-Post attack, CC attack and the like, can improve the application layer DDoS attack detection accuracy and reduce malicious flow.

Those of ordinary skill in the art will understand that: the figures are merely schematic representations of one embodiment, and the blocks or flow diagrams in the figures are not necessarily required to practice the present invention.

From the above description of the embodiments, it is clear to those skilled in the art that the present invention can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, they are described in relative terms, as long as they are described in partial descriptions of method embodiments. The above-described embodiments of the apparatus and system are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An apparatus for detecting DDoS attacks on multiple application layers, comprising: the device comprises a feature generation module, an offline training module and an online detection module;

the online detection module is used for deploying a trained multi-type application layer DDoS attack detection model, detecting network flow passing through by using the multi-type application layer DDoS attack detection model in real time, and outputting an application layer DDoS attack detection result of the network flow passing through;

the feature generation module is specifically configured to extract network traffic including multiple types of application layer DDoS attacks by using a feature extraction tool, generate multi-dimensional feature information corresponding to the network traffic of each type of application layer DDoS attack, analyze and screen the multi-dimensional feature information of each type of application layer DDoS attack by using a feature selection technology and an analysis method of a statistical chart, obtain effective feature information of each type of application layer DDoS attack, combine the effective feature information of all types of application layer DDoS attacks, and finally obtain the effective feature information of multiple types of application layer DDoS attacks; wherein,

the valid feature information includes a temporal feature, a stream header feature, and a payload feature; the time characteristics comprise stream duration, interval time between sending two data packets in the stream and derivative characteristics thereof, wherein the derivative characteristics comprise an average value, a maximum value, a minimum value, a standard deviation and a variance; the flow header characteristics include the number of flow bytes per second, the number of packets in the flow per second, and the forward/reverse packet header length; the payload characteristics include the number of forward/reverse packets per second, the packet length, and its SYN/FIN/RST flag bit count.

2. The apparatus of claim 1, wherein:

the offline training module is specifically configured to mark effective information on multiple types of application layer DDoS attacks as different data streams according to quintuple information of traffic, where the quintuple information includes a source IP address, a destination IP address, a source port number, a destination port number, and a communication protocol, and divide the marked data streams by using a set duration as a unit to obtain an offline detection sample set, and divide the offline detection sample set into an offline training sample set and an offline verification sample set in proportion;

3. The apparatus of claim 2, wherein the off-line validation metrics for the set of samples comprises: the accuracy, the precision, the recall rate and the F1 value, wherein when the indexes are in balanced optimal values, the corresponding parameters are the optimal parameters of the DDoS attack detection model of the multiple application layers; wherein the F1 value represents the harmonic mean of the precision rate and the recall rate.

4. The apparatus of claim 2, wherein the base learning classifier comprises a random forest model, an extreme random tree model, and a LightGBM model.

5. The apparatus of claim 1, wherein:

the online detection module is specifically used for deploying a DDoS attack detection model of various application layers at a network flow inlet, capturing the to-be-detected flow-through network flow within a set time length at the network flow inlet by using a flow collection tool in real time, and extracting characteristic information corresponding to the flow-through network flow; detecting the characteristic information of the flow-through network traffic according to a deployed multi-type application layer DDoS attack detection model, and identifying specific application layer DDoS attack types and normal traffic;

the output information of the multi-type application layer DDoS attack detection model further comprises: the accuracy, precision, recall rate and F1 value verify indexes, and when the indexes are in balance and optimal, the indexes can indicate that the online detection stage has better performance; the value of F1 represents the harmonic mean of precision and recall.

6. A method for detecting DDoS attacks of various application layers, applied to the device of any one of claims 1 to 5, the method comprising:

extracting network flow containing multi-type application layer DDoS attacks by using a feature extraction tool, and generating effective feature information suitable for the multi-type application layer DDoS attacks;

7. The method of claim 6, wherein outputting the application layer DDoS attack detection result of the network traffic comprises:

the output information of the multi-type application layer DDoS attack detection model further comprises: the accuracy, the precision, the recall rate and the F1 value verify indexes, and when the indexes are in balance and optimal, the indexes show that the online detection stage has optimal performance; the value of F1 represents the harmonic mean of precision and recall.

8. The method of claim 7, wherein:

the accuracy rate is (TP + TN)/total sample, defined as the percentage of the result with correct prediction in the total sample, and represents the overall prediction accuracy;

the recall ratio TP/(TP + FN), defined as the probability of being predicted as a positive sample among the actual positive samples;

said F1 value ═ precision ═ recall × (precision + recall) 2, representing the harmonic mean of precision and recall;