CN112989540A - Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network - Google Patents
Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network Download PDFInfo
- Publication number
- CN112989540A CN112989540A CN202110388025.3A CN202110388025A CN112989540A CN 112989540 A CN112989540 A CN 112989540A CN 202110388025 A CN202110388025 A CN 202110388025A CN 112989540 A CN112989540 A CN 112989540A
- Authority
- CN
- China
- Prior art keywords
- data
- sru
- training
- network
- abnormal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/18—Network design, e.g. design based on topological or interconnect aspects of utility systems, piping, heating ventilation air conditioning [HVAC] or cabling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/02—CAD in a network environment, e.g. collaborative CAD or distributed simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Geometry (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Biology (AREA)
- Computer Hardware Design (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Analysis (AREA)
- Mathematical Optimization (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Mathematics (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention relates to a method and a system for detecting abnormal traffic based on an SRU (remote request Unit) network. Attribute mapping is carried out on the flow data, so that the data to be detected meets the requirements of a model data format; the abnormal traffic classification is realized by performing code conversion and dimension reduction processing on the traffic data, performing feature learning by using an SRU network and training a classifier. The method solves the problem that the existing abnormal flow detection algorithm based on deep learning is low in training speed.
Description
Technical Field
The invention relates to a traffic analysis and deep learning technology, in particular to a detection method and a detection system based on SRU network abnormal traffic.
Background
With the deep application and rapid development of mobile internet and internet of things in various fields, information bearing forms and tools are richer, and the internet application is deeper to the aspects of life of people. With the depth of informatization degree, the safety awareness of people is continuously improved, and information and network safety are more and more concerned. Abnormal flow generated by various bad motivations influences normal operation of various applications, and users face more new safety problems and further even influence daily work of people. The traditional preventive network security technology comprises a firewall, virus searching and killing [1] and the like, but some new invasion forms are difficult to judge. With the application scenes becoming richer and richer, the abnormal traffic becomes more complex than before, and the conventional prevention technology can only be effective to a certain extent. The traditional abnormal flow detection method is mostly in a static mode [1], and is difficult to correct and detect abnormal and new attack types in a complex and dynamic network, and the traditional method is low in feature extraction and detection efficiency. The anomaly detection model based on RNN variant LTSM is proposed in document [2], but the training speed is still limited, and the anomaly detection mechanism based on GRU is proposed in document [3], and proves to be more suitable as a memory unit of RNN than LSTM, and proves to be an effective simplification and improvement of LSTM, but the training time of the model is high. The SRU is optimized on the network structure and mainly comprises a forgetting gate, a resetting gate and a memory unit. The calculation method comprises the following steps:
where g (.) is the activation function.
RNN networks are well suited to the time series problem, but their structure limits the training speed of the model, and RNNs cannot do parallelization processing compared to CNNs. The SRU (simple recovery units) puts most of the operations into parallel processing, and only serializes the steps with a small amount of operations. LSTM and GRU, etc., use neural gates to control the flow of information, alleviating the problem of gradient disappearance (or explosion). In comparison, the increasing of highway connection between the circulation layers is added in the SRU; when normalizing the RNN, the SRU is outside the standard dropout, adding the argument dropout, which uses the same mask as dropout at time step t.
[1] The abnormal traffic detection algorithm of the Internet of things is researched by Bojie, Zhang Yong and the like [ J ] information technology and network security, 2019(2).
[2] Wangwei, network traffic classification based on deep learning and anomaly detection method study [ D ]. 2018.
[3] Tangrui, Li Qi network traffic anomaly detection based on deep learning [ EB/OL ]. Beijing: the Chinese scientific and technological paper is on-line [2020-03-30]. http:// www.paper.edu.cn/releaseper/content/202003-.
Disclosure of Invention
The invention aims to provide a method and a system for detecting abnormal traffic based on an SRU network, which solve the problem of low training speed of the existing abnormal traffic detection algorithm based on deep learning.
In order to achieve the purpose, the technical scheme of the invention is as follows: a detection method based on SRU network abnormal flow comprises the following steps:
data acquisition: collecting data network flow data according to an actual application scene;
data preprocessing: preprocessing the acquired data network flow data;
constructing and training an SRU network model: constructing an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and performing model training on a training data set;
abnormality detection: and (5) calling an SRU network model, and outputting the abnormal detection.
In an embodiment of the present invention, the data preprocessing includes: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
In an embodiment of the present invention, the SRU network model includes: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; training a softmax classifier for anomaly detection.
In an embodiment of the present invention, the SRU network model is called, and the anomaly detection output is specifically implemented as: and (4) outputting a classification decision result by calling a classifier constructed by training, and judging whether the flow is abnormal or not.
The invention also provides a detection system based on the SRU network abnormal flow, which comprises the following steps:
the data acquisition module is used for acquiring data network flow data according to an actual application scene;
the data preprocessing module is used for preprocessing the acquired data network flow data;
the SRU network model building and training module is used for building an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and carrying out model training on a training data set;
and the anomaly detection module calls the SRU network model and outputs a flow anomaly detection result.
In an embodiment of the present invention, the data preprocessing module includes: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
In an embodiment of the present invention, the SRU network model building and training module implements: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; a classifier for anomaly detection is trained.
In an embodiment of the present invention, the anomaly detection module is specifically implemented as: and (4) outputting a classification decision result by calling a softmax classifier constructed by training, and judging whether the flow is abnormal or not.
Compared with the prior art, the invention has the following beneficial effects: the invention solves the problem of low training speed of the existing abnormal flow detection algorithm based on deep learning, and achieves the following aims through the deep learning SRU network:
in the aspect of data preprocessing, data are normalized, and data with smaller proportion are subjected to unbalanced processing;
in the aspect of deep learning model construction, an SRU network is selected, and highway connections are added among loop layers in the SRU; when the RNN is normalized, the SRU is outside the standard dropout, the variation dropout is added, the same mask is used by the variation dropout at the time step t and the dropout, the model training performance is greatly improved, and compared with the traditional method, the feature recognition rate is high;
in the model implementation, a hierarchical cascaded SRU network structure is used for carrying out feature learning on flow data with time sequence features, and a softmax classifier is constructed to realize the classification detection of flow abnormity.
Drawings
Fig. 1 is a flow anomaly detection system framework based on SRU of the present invention.
Fig. 2 is a SRU two-layer network structure.
FIG. 3 is a classifier construction.
Detailed Description
The technical scheme of the invention is specifically explained below with reference to the accompanying drawings.
The invention provides a method for detecting abnormal traffic based on an SRU network, which comprises the following steps:
data acquisition: collecting data network flow data according to an actual application scene;
data preprocessing: preprocessing the acquired data network flow data;
constructing and training an SRU network model: constructing an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and performing model training on a training data set;
abnormality detection: and (5) calling an SRU network model, and outputting the abnormal detection.
The data preprocessing comprises the following steps: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
The SRU network model comprises: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; training a softmax classifier for anomaly detection.
The SRU network model is called, and the abnormal detection output is specifically realized as follows: and (4) outputting a classification decision result by calling a classifier constructed by training, and judging whether the flow is abnormal or not.
The invention also provides a detection system based on the SRU network abnormal flow, which comprises the following steps:
the data acquisition module is used for acquiring data network flow data according to an actual application scene;
the data preprocessing module is used for preprocessing the acquired data network flow data;
the SRU network model building and training module is used for building an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and carrying out model training on a training data set;
and the anomaly detection module calls the SRU network model and outputs a flow anomaly detection result.
The data preprocessing module comprises: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
The SRU network model construction and training module is realized as follows: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; training a softmax classifier for anomaly detection.
The anomaly detection module is specifically implemented as follows: and (4) outputting a classification decision result by calling a classifier constructed by training, and judging whether the flow is abnormal or not.
The following is a specific implementation of the present invention.
As can be seen from FIG. 1, the SRU network model of the present invention is composed of data acquisition, data preprocessing, data partitioning, model training, and anomaly detection. Data acquisition is the primary source of traffic data and is organized to form data sets. The data preprocessing mainly completes the attribute conversion of the data, and the normalization processing of the attributes mainly uses a minimum and maximum normalization method and unifies data dimensions to reduce the calculated amount and the influence caused by unit difference. The data is divided into training data set and test data set at the beginning of model construction. Constructing an SRU deep learning model, training and learning a training set user model, and performing feature learning, specifically including converting flow data by using OHE unique hot coding, performing dimensionality reduction processing on a feature vector, and performing model training and learning. After model training, anomaly detection is performed by using the test data set, and evaluation test is performed on the performance of the model.
The two-layer network structure based on the SRU is shown in fig. 2, and the processed time series traffic data features of the same type are input as inputs for pooling, dependency features between traffic packets are learned, and the second layer takes the first layer output of the last packet as input and the output of the SRU of the second layer as features.
The process of constructing the softmax classifier is shown in fig. 3. Aggregating the features obtained in the figure 2, classifying the features by a full connection layer and a softmax layer after the SRU unit, and taking the result of the classifier as the result of the flow detection.
And when the performance of the model meets the expected requirement, the model is used for an actual application scene.
The specific process is as follows:
(1) acquiring original data, and inputting data into a system to form a data set;
(2) converting non-numerical data into numerical data;
(3) normalizing each attribute value to a [0,1] interval according to a minimum maximization method;
(4) labeling the data set;
(5) dividing a data set into a training set and a test set;
(6) constructing a cascaded two-layer SRU deep learning network by using a tool;
(7) carrying out learning training on the SRU network, and carrying out feature learning;
(8) constructing a softmax classifier model on the basis of training;
(9) applying the test set to a classifier model to evaluate the performance of the model;
(10) and (5) actually applying the model.
The above are preferred embodiments of the present invention, and all changes made according to the technical scheme of the present invention that produce functional effects do not exceed the scope of the technical scheme of the present invention belong to the protection scope of the present invention.
Claims (8)
1. A detection method based on SRU network abnormal flow is characterized by comprising the following steps:
data acquisition: collecting data network flow data according to an actual application scene;
data preprocessing: preprocessing the acquired data network flow data;
constructing and training an SRU network model: constructing an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and performing model training on a training data set;
abnormality detection: and (5) calling an SRU network model, and outputting the abnormal detection.
2. The method according to claim 1, wherein the data preprocessing comprises: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
3. The method according to claim 1, wherein the SRU network model comprises: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; training a softmax classifier for anomaly detection.
4. The method for detecting the abnormal traffic based on the SRU network according to claim 3, wherein the SRU network model calls and the abnormal detection output is specifically realized as: and (4) outputting a classification decision result by calling a classifier constructed by training, and judging whether the flow is abnormal or not.
5. A detection system based on SRU network abnormal flow is characterized by comprising:
the data acquisition module is used for acquiring data network flow data according to an actual application scene;
the data preprocessing module is used for preprocessing the acquired data network flow data;
the SRU network model building and training module is used for building an SRU network model based on the preprocessed data, training a network, extracting and analyzing characteristics, and carrying out model training on a training data set;
and the anomaly detection module calls the SRU network model and outputs a flow anomaly detection result.
6. The method according to claim 5, wherein the data preprocessing module comprises: sorting the preprocessed data into a data set; performing attribute conversion, namely performing numerical value coding on various non-numerical data; performing attribute normalization processing, namely normalizing the data to a [0,1] interval by adopting a minimum and maximum normalization method; the imbalance data is optimized.
7. The method for detecting the abnormal traffic based on the SRU network according to claim 5, wherein the SRU network model building and training module realizes: transforming the traffic data using OHE one-hot encoding; performing dimensionality reduction on the feature vector; using the SRU network to perform feature learning; training a softmax classifier for anomaly detection.
8. The method for detecting the abnormal traffic based on the SRU network according to claim 7, wherein the abnormal detection module is specifically implemented as: and (4) outputting a classification decision result by calling a classifier constructed by training, and judging whether the flow is abnormal or not.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110388025.3A CN112989540A (en) | 2021-04-12 | 2021-04-12 | Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110388025.3A CN112989540A (en) | 2021-04-12 | 2021-04-12 | Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112989540A true CN112989540A (en) | 2021-06-18 |
Family
ID=76337866
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110388025.3A Pending CN112989540A (en) | 2021-04-12 | 2021-04-12 | Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112989540A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948166A (en) * | 2017-11-29 | 2018-04-20 | 广东亿迅科技有限公司 | Traffic anomaly detection method and device based on deep learning |
| CN109639739A (en) * | 2019-01-30 | 2019-04-16 | 大连理工大学 | A kind of anomalous traffic detection method based on autocoder network |
| CN110138787A (en) * | 2019-05-20 | 2019-08-16 | 福州大学 | A kind of anomalous traffic detection method and system based on hybrid neural networks |
| CN110351123A (en) * | 2019-06-19 | 2019-10-18 | 新华三大数据技术有限公司 | Link flow prediction technique, device and electronic equipment |
| CN111428789A (en) * | 2020-03-25 | 2020-07-17 | 广东技术师范大学 | Network traffic anomaly detection method based on deep learning |
| CN111967343A (en) * | 2020-07-27 | 2020-11-20 | 广东工业大学 | Detection method based on simple neural network and extreme gradient lifting model fusion |
-
2021
- 2021-04-12 CN CN202110388025.3A patent/CN112989540A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107948166A (en) * | 2017-11-29 | 2018-04-20 | 广东亿迅科技有限公司 | Traffic anomaly detection method and device based on deep learning |
| CN109639739A (en) * | 2019-01-30 | 2019-04-16 | 大连理工大学 | A kind of anomalous traffic detection method based on autocoder network |
| CN110138787A (en) * | 2019-05-20 | 2019-08-16 | 福州大学 | A kind of anomalous traffic detection method and system based on hybrid neural networks |
| CN110351123A (en) * | 2019-06-19 | 2019-10-18 | 新华三大数据技术有限公司 | Link flow prediction technique, device and electronic equipment |
| CN111428789A (en) * | 2020-03-25 | 2020-07-17 | 广东技术师范大学 | Network traffic anomaly detection method based on deep learning |
| CN111967343A (en) * | 2020-07-27 | 2020-11-20 | 广东工业大学 | Detection method based on simple neural network and extreme gradient lifting model fusion |
Non-Patent Citations (5)
| Title |
|---|
| 张文 等: "基于简单循环单元的深层神经网络机器翻译模型", 《中文信息学报》, vol. 32, no. 10, pages 0 - 3 * |
| 机器之心: "爆款论文提出简单循环单元SRU：像CNN一样快速训练RNN（附开源代码）", 《知乎》 * |
| 机器之心: "爆款论文提出简单循环单元SRU：像CNN一样快速训练RNN（附开源代码）", 《知乎》, 12 September 2017 (2017-09-12), pages 1 - 9 * |
| 黎佳玥 等: "基于深度学习的网络流量异常预测方法", 《计算机工程与应用》 * |
| 黎佳玥 等: "基于深度学习的网络流量异常预测方法", 《计算机工程与应用》, vol. 56, no. 06, 9 January 2020 (2020-01-09), pages 39 - 50 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | LSTM-based SQL injection detection method for intelligent transportation system | |
| Wu et al. | A network intrusion detection method based on semantic Re-encoding and deep learning | |
| KR102279983B1 (en) | Network Intrusion Detection Method using unsupervised deep learning algorithms and Computer Readable Recording Medium on which program therefor is recorded | |
| CN109831422B (en) | Encrypted flow classification method based on end-to-end sequence network | |
| CN114553545A (en) | Intrusion flow detection and identification method and system | |
| CN109639734B (en) | Abnormal flow detection method with computing resource adaptivity | |
| Zheng | Intrusion detection based on convolutional neural network | |
| CN110377605A (en) | A kind of Sensitive Attributes identification of structural data and classification stage division | |
| CN115811440B (en) | Real-time flow detection method based on network situation awareness | |
| CN112804253A (en) | Network flow classification detection method, system and storage medium | |
| Alshamy et al. | Intrusion detection model for imbalanced dataset using SMOTE and random forest algorithm | |
| Roshan et al. | Deep learning approaches for anomaly and intrusion detection in computer network: A review | |
| CN116633601A (en) | Detection method based on network traffic situation awareness | |
| Yujie et al. | End-to-end android malware classification based on pure traffic images | |
| Harbola et al. | Improved intrusion detection in DDoS applying feature selection using rank & score of attributes in KDD-99 data set | |
| CN117375896A (en) | Intrusion detection method and system based on multi-scale space-time feature residual fusion | |
| Qiao et al. | Malware classification method based on word vector of bytes and multilayer perception | |
| CN112818407A (en) | Video privacy protection method based on generation countermeasure network | |
| CN112989540A (en) | Method and system for detecting abnormal traffic based on SRU (sequence recovery Unit) network | |
| Xin et al. | Research on feature selection of intrusion detection based on deep learning | |
| CN114358177B (en) | Unknown network traffic classification method and system based on multidimensional feature compact decision boundary | |
| Dong et al. | Mutual Information-based Intrusion Detection Model for Industrial Internet. | |
| Tan et al. | Detection of malicious web requests using neural networks with multi granularity features | |
| Juvonen et al. | Anomaly detection framework using rule extraction for efficient intrusion detection | |
| CN112449025A (en) | IP scene prediction method based on self-attention mechanism |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |