WO2021258348A1 - Procédé et système de détection de flux anormal et support de stockage informatique - Google Patents

Procédé et système de détection de flux anormal et support de stockage informatique Download PDF

Info

Publication number
WO2021258348A1
WO2021258348A1 PCT/CN2020/098177 CN2020098177W WO2021258348A1 WO 2021258348 A1 WO2021258348 A1 WO 2021258348A1 CN 2020098177 W CN2020098177 W CN 2020098177W WO 2021258348 A1 WO2021258348 A1 WO 2021258348A1
Authority
WO
WIPO (PCT)
Prior art keywords
detection
data
model
result
flow
Prior art date
Application number
PCT/CN2020/098177
Other languages
English (en)
Chinese (zh)
Inventor
程肯
Original Assignee
深圳市欢太科技有限公司
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市欢太科技有限公司, Oppo广东移动通信有限公司 filed Critical 深圳市欢太科技有限公司
Priority to PCT/CN2020/098177 priority Critical patent/WO2021258348A1/fr
Priority to CN202080100505.9A priority patent/CN115606162A/zh
Publication of WO2021258348A1 publication Critical patent/WO2021258348A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/14—Network analysis or design
    • H04L41/142—Network analysis or design using statistical or mathematical methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/16—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks using machine learning or artificial intelligence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/18—Protocol analysers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/50—Testing arrangements

Definitions

  • the embodiments of the present application relate to the field of communication technology, and in particular, to a method and system for detecting abnormal traffic, and a computer storage medium.
  • Abnormal traffic is unexpected traffic carried by limited bandwidth resources. Abnormal traffic can reflect the existence of network abnormalities to a certain extent. Therefore, rapid and accurate detection of abnormal traffic in the network is extremely important for network protection. significance.
  • black and white list-based detection can be divided into black and white list-based detection, rule-based detection, machine learning model-based detection, and deep learning model-based detection.
  • black-and-white list-based detection and rule-based detection require continuous updating of existing lists and rules, which cannot be well adapted to real-time network traffic data.
  • Detection based on machine learning models and detection based on depth The detection of the learning model requires high requirements for the selection of model parameters and the quality of the training data. Therefore, when performing abnormal traffic detection on network traffic data with a large amount of data and strong real-time performance, the accuracy of the detection cannot be guaranteed, and the detection effect is not stability.
  • the embodiments of the present application provide an abnormal flow detection method and system, and a computer storage medium, which can improve the accuracy of abnormal flow detection and effectively improve the detection quality of abnormal flow.
  • an embodiment of the present application provides a method for detecting abnormal traffic, and the method includes:
  • a target detection result of the to-be-detected flow data is generated.
  • an embodiment of the present application provides a method for detecting abnormal traffic, the method including:
  • the first detection model is a model generated based on an unsupervised algorithm
  • a second detection model is obtained by training based on the feature data and the test result; wherein, the second detection model is a model generated based on a supervised algorithm.
  • an embodiment of the present application provides an abnormal traffic detection system.
  • the abnormal traffic detection system includes: a first acquisition part, an analysis part, a first extraction part, a determination part, and a generation part,
  • the first obtaining part is configured to obtain traffic data to be detected
  • the analysis part is configured to analyze the traffic data to be detected to obtain target structured data
  • the first extraction part is configured to perform feature extraction processing on the target structured data to obtain target feature data
  • the determining part is configured to determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, determine the second detection result corresponding to the target feature data based on the second detection model, and based on a preset
  • the rule base determines the third detection result corresponding to the target feature data; wherein, the self-encoding model is a model generated based on an unsupervised algorithm; the first detection model is a model generated based on an unsupervised algorithm; the second The detection model is based on a model generated by a supervised algorithm;
  • the generating part is configured to generate a target detection result of the flow data to be detected based on the first detection result, the second detection result, and the third detection result.
  • an embodiment of the present application provides an abnormal traffic detection system.
  • the abnormal traffic detection system includes: a second acquisition part, a second extraction part, an encoding part, and a training part,
  • the second obtaining part is configured to obtain first flow data
  • the second extraction part is configured to extract feature data corresponding to the first flow data
  • the encoding part is configured to perform encoding processing on the characteristic data through a self-encoding model to obtain encoded data; wherein the self-encoding model is a model generated based on an unsupervised algorithm;
  • the second acquisition part is further configured to obtain a first detection model through training of the encoded data; wherein, the first detection model is a model generated based on an unsupervised algorithm; and according to the first detection model and the pre- Set a marking strategy to obtain a test result corresponding to the first flow data;
  • the training part is configured to train to obtain a second detection model based on the feature data and the test result; wherein the second detection model is a model generated based on a supervised algorithm.
  • an embodiment of the present application provides an abnormal flow detection system, the abnormal flow detection system includes a processor, a memory storing executable instructions of the processor, and when the instructions are executed by the processor , To achieve the abnormal flow detection method as described in the first aspect.
  • an embodiment of the present application provides an abnormal flow detection system.
  • the abnormal flow detection system includes a processor, a memory storing executable instructions of the processor, and when the instructions are executed by the processor , To achieve the abnormal flow detection method as described in the second aspect.
  • an embodiment of the present application provides a computer-readable storage medium with a program stored thereon, which is applied to an abnormal traffic detection system, and when the program is executed by a processor, it realizes the following aspects: the first aspect and the second aspect The described abnormal flow detection method.
  • the embodiment of the application provides an abnormal flow detection method and system, and a computer storage medium.
  • the abnormal flow detection system obtains the flow data to be detected, analyzes the flow data to be detected, and obtains target structured data; performs feature extraction on the target structured data Process to obtain target feature data; determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, determine the second detection result corresponding to the target feature data based on the second detection model, and determine the target based on the preset rule library
  • the third detection result corresponding to the feature data; among them, the self-encoding model and the first detection model are models generated based on unsupervised algorithms; the second detection model is based on models generated by supervised algorithms; based on the first detection results and the second detection results And the third detection result is to generate the target detection result of the flow data to be detected.
  • the abnormal traffic detection system uses unsupervised algorithms and supervised algorithms to train and generate the auto-encoding model, the first detection model, and the second detection model, respectively, so that it can be based on the auto-encoding model and the first detection model.
  • the first detection model, the second detection model and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection result obtained is the risk judgment of the flow data to be detected by combining the unsupervised algorithm, the supervised algorithm and the preset rule library. , Can improve the accuracy of abnormal flow detection, and effectively improve the detection quality of abnormal flow.
  • Figure 1 is the first schematic diagram of the implementation process of the abnormal flow detection method
  • Figure 2 is a schematic diagram of the system structure of the abnormal flow detection system
  • Figure 3 is a second schematic diagram of the implementation process of the abnormal flow detection method
  • Figure 4 is the third schematic diagram of the implementation process of the abnormal flow detection method
  • Figure 5 is a schematic diagram of the structure of the self-encoder
  • Figure 6 is a schematic diagram of the composition structure of the marking system
  • Figure 7 is a schematic diagram of labeling processing
  • Figure 8 is a fourth schematic diagram of the implementation process of the abnormal flow detection method.
  • Figure 9 is a schematic diagram of the composition structure of an abnormal flow detection system
  • Figure 10 is the second schematic diagram of the composition structure of the abnormal flow detection system
  • Figure 11 is the third schematic diagram of the composition structure of the abnormal flow detection system.
  • abnormal traffic detection has been used as an effective network protection method, which can detect unknown network attack behaviors and provide important support for network situation awareness. In recent years, it has received more and more attention from researchers.
  • abnormal flow identification can be divided into full flow monitoring, encrypted flow monitoring, industrial control protocol flow detection, Transmission Control Protocol/Internet Protocol (TCP/IP) flow monitoring, Domain Name System (DNSDNS) ) Traffic monitoring, Hyper Text Transfer Protocol (HTTP) traffic monitoring, etc.; according to the detection method, it can be divided into black and white list-based detection, rule-based detection, machine learning model-based detection, and deep learning model-based detection Wait.
  • IP blacklist filtering can only identify abnormal traffic from the source IP in the existing IP blacklist, without any perception of IP changes;
  • Rule matching needs to analyze the samples one by one, because the attacker will often find ways to circumvent the existing rules, causing the original rules to become invalid. Therefore, the rule base needs to be dynamically updated, which consumes a lot of manpower and has a poor detection effect on unknown threats.
  • Deep learning technology has outstanding advantages in abnormal traffic detection.
  • the deep learning model can take the original data as input and can better describe the rich information of the data from the learned features, and improve the classification performance.
  • the model's parameter selection is not appropriate or the selected data quality is poor, which will have a greater detection effect of the model. Impact. For example, if the selected neural network model has a large number of layers, there may be slow convergence during the training process. If the selected neural network model has a small number of layers, the network may not be adjusted accurately during the training process. Parameters, it is not easy to obtain a detection model with higher accuracy.
  • the abnormal traffic detection system uses an unsupervised algorithm and a supervised algorithm to train and generate an auto-encoding model, a first detection model, and a second detection model, respectively.
  • the coding model, the first detection model, the second detection model, and the preset rule library perform abnormal traffic detection on the to-be-detected traffic data, and the detection results obtained are the unsupervised algorithm, the supervised algorithm and the preset rule library to achieve the to-be-detected traffic Data risk judgment can improve the accuracy of abnormal traffic detection and effectively improve the detection quality of abnormal traffic.
  • the abnormal flow detection system can collect, analyze, and extract the flow data, use the self-encoding model to encode the obtained high-dimensional discrete feature data, and then use the unsupervised isolated forest algorithm, that is, the first detection model for preliminary
  • the abnormal traffic detection is then combined with the marking system including the preset information library, the regular rule library and the review pattern library to generate more accurate labels for training the supervised decision tree algorithm to obtain the second detection model. Therefore, the self-encoding model, the first detection model, the second detection model, and the preset rule library can be used to make risk judgments on the traffic data by combining unsupervised algorithms, supervised algorithms, and voting in the preset rule library.
  • FIG. 1 is a schematic diagram of the implementation process of the abnormal flow detection method. As shown in FIG. 1, in the embodiment of the present application, the abnormal flow detection system performs abnormal flow detection. The method can include the following steps:
  • Step 101 Obtain the flow data to be detected, and parse the flow data to be detected to obtain target structured data.
  • the abnormal flow detection system when performing abnormal flow detection, may first obtain the flow data to be detected, and then may parse the flow data to be detected, so as to obtain target structured data corresponding to the flow data to be detected.
  • the abnormal traffic detection system can perform protocol analysis on the traffic data to be detected, so as to restore its original network behavior information.
  • the abnormal traffic detection system can parse out the information of the communication parties from the traffic data to be detected according to the protocol specification, which can specifically include specific information such as source IP, destination IP, source port, destination port, request content, and response information. Throw away some useless information, and finally a structured data format can be generated, that is, target structured data is generated to facilitate subsequent feature extraction processing.
  • the abnormal traffic detection system can capture network traffic data to obtain the traffic data to be detected.
  • equipment such as optical fiber splitters can be used to capture network traffic data transmitted in gateways and switches.
  • the abnormal traffic detection system described above can be any terminal with communication and storage functions, such as tablet computers, mobile phones, e-readers, remote controllers, and personal computers (PC). ), laptops, in-vehicle devices, Internet TVs, wearable devices, personal digital assistants (PDA), portable media players (PMP), navigation devices and other terminals.
  • PC personal computers
  • PDA personal digital assistants
  • PMP portable media players
  • the traffic data to be detected may be network traffic data captured by the abnormal traffic detection system in real time.
  • the abnormal traffic detection system may directly collect the traffic data to be detected from the network card. It can directly receive the flow data to be detected from other systems.
  • Step 102 Perform feature extraction processing on the target structured data to obtain target feature data.
  • the abnormal traffic detection system can perform feature extraction processing on the target structured data to obtain the target feature data of the traffic data to be detected.
  • the abnormal traffic detection system after the abnormal traffic detection system obtains the target structured data through analysis processing, it can perform feature extraction on each piece of data in the target structured data to obtain the characteristics of each piece of data. Information, the target feature data can be generated in the end.
  • the target characteristic data may include the basic characteristics of the traffic corresponding to the traffic data to be detected, the content characteristics of the traffic, the statistical characteristics based on the time window, and the statistical characteristics of the hosts it visits within the time window, etc. Class feature information.
  • each feature data in the target feature data may include the following four types of feature information:
  • the first type is the basic characteristics of the traffic, including the duration of the visit, the protocol type, And the number of bytes sent, etc.
  • the second category is the content characteristics of the traffic, which converts the requested content in the traffic into text vector features
  • the third category is the statistical characteristics based on the time window, including the frequency of access within the time window, and the time window The total number of bytes sent in the network and the number of connections with "Synchronize Sequence Numbers (SYN)" errors
  • the fourth category is the statistical characteristics of the host that it accesses in the time window, including the frequency of access, and the number of received The total number of bytes, the total number of connections with "SYN” errors, etc.
  • the abnormal traffic detection system performs feature extraction on each piece of data in the target structured data, obtains the above four types of feature information of the piece of data, and then generates a uniform length feature of the piece of data based on the above four types of feature information vector.
  • each feature data in the target feature data is a feature vector with the same length. That is, after the abnormal flow detection system performs feature extraction on the flow data to be detected, it can obtain the feature vector of uniform length corresponding to each piece of data in the flow data to be detected, thereby forming the target feature data.
  • Step 103 Determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, determine the second detection result corresponding to the target feature data based on the second detection model, and determine the target feature data corresponding to the target feature data based on the preset rule library.
  • the third detection result where the self-encoding model and the first detection model are models generated based on unsupervised algorithms; the second detection model is based on models generated by supervised algorithms.
  • the abnormal traffic detection system after the abnormal traffic detection system performs feature extraction processing on the target structured data to obtain the target feature data, it can determine the first detection result corresponding to the target feature data based on the auto-encoding model and the first detection model, At the same time, the second detection result corresponding to the target feature data is determined based on the second detection model, and the third detection result corresponding to the target feature data can also be determined based on the preset rule library.
  • the self-encoding model and the first detection model are models generated based on an unsupervised algorithm; the second detection model is a model generated based on a supervised algorithm.
  • the abnormal traffic detection system when it obtains the first detection result, it may first input the target feature data into the self-encoding model, thereby outputting the encoded data, and then inputting the encoded data into The first detection model, so that the first detection result can be determined.
  • the self-encoding model is a model generated based on an unsupervised algorithm.
  • the target feature data obtained is high-dimensional and sparse, in order to be more suitable for the first generation of unsupervised learning algorithms.
  • the abnormal traffic detection system needs to preprocess the high-dimensional sparse target feature data first.
  • the abnormal traffic detection system can encode the target feature data through the self-encoding model obtained by pre-training, so as to obtain low-dimensional continuous feature data, that is, encoded data.
  • the abnormal traffic detection system encodes the target feature data through the self-encoding model
  • the encoded data can be input into the first detection model to output the first detection result.
  • the strategy performs labeling processing on the unpredictable encoded data corresponding to the traffic data to be detected, so as to obtain the corresponding first detection result.
  • the abnormal traffic detection system can use the self-encoding model to encode the target feature data corresponding to the traffic data to be detected, and then input the encoded data into the first detection model for detection, that is, automatic labeling processing.
  • the abnormal traffic detection system can use a preset marking strategy to mark this part of the unpredictable data through the preset information database, the regular rule database and the review pattern database, and finally obtain the traffic data to be detected. The corresponding first test result.
  • the target feature data can be input to the second detection model, so that the second detection result can be output. That is, the second detection model is directly used to perform detection processing on the target feature data to obtain the second detection result.
  • the abnormal traffic detection system when it obtains the third detection result, it can use a preset rule library to perform matching processing on the target feature data, so as to obtain the third detection result.
  • the abnormal traffic detection system may perform matching processing on the target characteristic data with the preset rule database to obtain the matching result, and then may determine the matching result as the third detection result of the to-be-detected traffic data .
  • the abnormal flow detection system not only uses the first detection model generated based on the unsupervised algorithm and the second detection model generated based on the supervised algorithm, but also combines The preset rule library is used for matching processing, so as to obtain the first detection result, the second detection result and the third detection result corresponding to the traffic to be detected respectively.
  • the first detection result, the second detection result, and the third detection result are all normal flow or abnormal flow.
  • Step 104 According to the first detection result, the second detection result, and the third detection result, generate a target detection result of the flow data to be detected.
  • the abnormal traffic detection system obtains the first detection result, the second detection result and the second detection result corresponding to the traffic to be detected based on the self-encoding model, the first detection model, the second detection model, and the preset rule library. After the third detection result, the target detection result of the flow data to be detected can be further generated according to the first detection result, the second detection result, and the third detection result.
  • the abnormal traffic detection system may perform voting processing based on the first detection result, the second detection result, and the third detection result, that is, follow the principle of minority to majority, and then obtain the detection result.
  • the detection result is an abnormal flow
  • the first detection result, the second detection result, and the second detection result are abnormal flow
  • At least two of the detection result and the third detection result are normal flow, then it can be determined that the target detection result is normal flow.
  • the detection result can be determined to be an abnormal flow; if both the first detection result and the second detection result are If it is a normal flow, and the third detection result is an abnormal flow, then it can be determined that the detection result is a normal flow.
  • the abnormal flow detection system when the abnormal flow detection system determines the target detection result, it can also assign different weight values to the first detection result, the second detection result, and the third detection result, and then reuse These weight values perform a weighted operation on the first detection result, the second detection result, and the third detection result, and finally obtain the target detection result of the flow data to be detected.
  • the abnormal flow detection system can separately evaluate and set the credibility of different detection results generated by different models in advance, so that the first detection result and the second detection result obtained by different models When the result and the third test result are inconsistent, the final test result can be determined more accurately.
  • the abnormal flow detection system may first obtain a preset weight set when generating the target detection result of the flow data to be detected according to the first detection result, the second detection result, and the third detection result; where , The preset weight set includes different weight values corresponding to different detection results.
  • the preset weight set specifically includes: the first weight value of the first detection result is 0.3, the second weight value of the second detection result is 0.4, and the second weight value is 0.4.
  • the third weight of the three detection results is 0.3; then, the abnormal traffic detection system can use the preset weight set, the first detection result, the second detection result, and the third detection result to perform a weighting operation to obtain the target detection result, for example, when When the first detection result is abnormal traffic, the second detection result is normal traffic, and the third detection result is abnormal traffic, the first weight 0.3, the second weight 0.4, and the third weight 0.3 in the preset weight set are used to detect abnormal traffic After the system performs the weighting operation, it can be determined that the target detection result of the flow data to be detected is 40% of the normal flow and 60% of the abnormal flow, and then the target detection result can be determined to be the abnormal flow.
  • the abnormal flow detection system when the abnormal flow detection system generates the target detection result of the flow data to be detected based on the first detection result, the second detection result, and the third detection result, it may also obtain the preliminary detection result. After the weight set is set, no weighting operation is performed, but the detection result corresponding to the maximum weight is directly used as the target detection result.
  • the preset weight set specifically includes: the first weight value of the first detection result is 0.3, the second weight value of the second detection result is 0.4, and the third weight value of the third detection result is 0.3; when the first detection result is abnormal
  • the abnormal flow detection system can directly use the maximum weight, that is, the second detection result corresponding to the second weight as the target detection result of the flow data to be detected, namely The target detection result is normal flow.
  • the abnormal flow detection system in the process of detecting the flow data to be detected, can adopt the idea of integrated learning, combining the detection results of the output of the autoencoder, the first detection model, and the second detection model. , And the existing preset rule library matching algorithm to obtain the detection result to determine the target detection result of the traffic to be detected, so as to obtain better generalization performance than a single model.
  • the judgment method based on the detection algorithm has better flexibility, and the anomaly detection algorithm based on unsupervised learning is more suitable for the scene of traffic monitoring and has more Good adaptability; on the other hand, if only unsupervised learning detection algorithms are used, there may be problems with low detection accuracy and coverage. Therefore, the two main machine learning algorithms used by the abnormal traffic detection system are none Supervised learning (isolated forest algorithm) and supervised learning (decision tree algorithm). Because the training speed of isolated forest and decision tree algorithms is very fast, the entire abnormal traffic detection system can be trained and updated at the day level (hour level).
  • the latest trained detection model can be used to detect real-time traffic data, so that the user behavior of the day can be inferred and judged.
  • the regular rule database and the preset information database can be updated in time to ensure that the entire The real-time nature of the abnormal flow detection system.
  • the abnormal flow detection system obtains the flow data to be detected, analyzes the flow data to be detected, and obtains target structured data; performs feature extraction processing on the target structured data to obtain target feature data;
  • the self-encoding model and the first detection model determine the first detection result corresponding to the target feature data, determine the second detection result corresponding to the target feature data based on the second detection model, and determine the third detection result corresponding to the target feature data based on the preset rule library ;
  • the self-encoding model and the first detection model are models generated based on an unsupervised algorithm; the second detection model is based on a model generated by a supervised algorithm; according to the first detection result, the second detection result, and the third detection result, the waiting Detect the target detection result of the flow data.
  • the abnormal traffic detection system uses unsupervised algorithms and supervised algorithms to train and generate the auto-encoding model, the first detection model, and the second detection model, respectively, so that it can be based on the auto-encoding model and the first detection model.
  • the first detection model, the second detection model and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection result obtained is the risk judgment of the flow data to be detected by combining the unsupervised algorithm, the supervised algorithm and the preset rule library. , Can improve the accuracy of abnormal flow detection, and effectively improve the detection quality of abnormal flow.
  • FIG. 2 is a schematic diagram of the system structure of the abnormal flow detection system.
  • the abnormal flow detection system 10 may include a collection module 11, Module 12, extraction module 13, encoding module 14, and detection module 15.
  • the collection module 11 may be used to capture traffic data and store the acquired network traffic mirroring. Specifically, the collection module 11 can directly receive traffic data from the network card, and can also directly receive traffic data sent by other systems.
  • the analysis module 12 may perform protocol analysis on the captured network traffic data to obtain traffic data that can be processed by subsequent functional modules. It mainly analyzes the information of the communication parties from the network traffic data according to the protocol specification, including the source IP, destination IP, source port, destination port, request content and response information, and discards some useless information to generate a structured data format.
  • both the collection module 11 and the analysis module 12 are used for the preprocessing of the flow data.
  • the extraction module 13 may obtain characteristic data corresponding to each data sample based on the structured data obtained by analysis.
  • the generated characteristics mainly include four categories: the first category is the basic characteristics of the traffic, including the duration of the visit, the protocol type, and the number of bytes sent, etc.; the second category is the content characteristics of the traffic.
  • the content of the request is converted into text vector features; the third category is based on the statistical characteristics of the time window, including the frequency of access within the time window, the total number of bytes sent within the time window, and the number of connections with "SYN" errors; the fourth category It is the statistical characteristics of the host it visits in the time window, including the frequency of access, the total number of bytes received, the total number of connections with "SYN” errors, etc. These four types of characteristics are converted into uniform length feature vectors, Obtain characteristic data.
  • the encoding module 14 may be used to convert the high-dimensional sparse feature data extracted by the extraction module 13 into low-dimensional continuous feature data, that is, encoded data.
  • both the extraction module 13 and the encoding module 14 are used to obtain the characteristics of the traffic data.
  • the detection module 15 may include a first detection model 151 obtained based on unsupervised algorithm training, a second detection model 152 obtained based on supervised algorithm training, and a labeling system for implementing preset labeling strategies. 153, and a preset rule library 154. Specifically, when detecting the flow data, the detection module 15 may simultaneously use the first detection model 151, the second detection model 152, and the preset rule library 154 to generate different detection results respectively.
  • Figure 3 is a schematic diagram of the second implementation process of the abnormal flow detection method.
  • the abnormal flow detection method performed by the abnormal flow detection system may include the following steps:
  • Step 201 Collect data flow.
  • the collection module 11 may first obtain the flow data, where the collection module 11 may use equipment such as an optical fiber splitter to capture the flow data transmitted in the gateway and the switch.
  • Step 202 Analyze the data flow.
  • the parsing module 12 can analyze and process the flow data. Specifically, the flow data can be analyzed by protocol to restore its original network behavior information. At the same time, some useless information can be discarded. Generate structured data format.
  • Step 203 Feature extraction.
  • the extraction module 13 can perform feature extraction on the parsed structured data to obtain feature data of the data flow.
  • the characteristic data may include several types of characteristic information such as the basic characteristics of the traffic corresponding to the traffic data, the content characteristics of the traffic, the statistical characteristics based on the time window, and the statistical characteristics of the hosts visited within the time window.
  • Step 204 Perform abnormal traffic detection using the self-encoding model and the first detection model.
  • the first detection model since the first detection model is trained based on an unsupervised learning algorithm, the first detection model has a poor detection effect on the high-dimensional sparse feature information obtained after feature extraction, and the encoding module 14 requires The self-encoding model is first used to encode the feature information, and the obtained encoded data is low-dimensional continuous feature data, and then the detection module 15 then uses the first detection model to perform detection processing on the encoded data.
  • the first detection model may be obtained by training using the isolated forest algorithm.
  • the detection module 15 may use a marking system to implement marking processing according to a preset marking strategy.
  • the detection module 15 can obtain the corresponding detection result 1 after performing the flow abnormality detection on the flow data based on the self-encoding model, the first detection model, and the marking system.
  • Step 205 Use the second detection model to perform abnormal flow detection.
  • the detection module 15 can directly input the feature data obtained by the extraction module 13 into the second detection model, so that the detection result 2 corresponding to the flow data can be output.
  • the second detection model may be obtained by training using a decision tree algorithm.
  • Step 206 Perform abnormal traffic detection using a preset rule library.
  • the detection module 15 may use a preset rule library to perform matching processing on the characteristic data, so as to obtain the detection result 3 corresponding to the flow data.
  • Step 207 Generate a detection result.
  • the detection module 15 may obtain the detection result of the flow data according to the detection result 1, the detection result 2, and the detection result 3.
  • the detection result 1, the detection result 2, the detection result 3, and the detection result are all abnormal flow or normal flow.
  • the detection module 15 when the detection module 15 generates the detection result, if at least two of the detection result 1, the detection result 2, and the detection result 3 are abnormal traffic, then the detection result is the abnormal traffic; At least two of the result 1, the detection result 2 and the detection result 3 are normal flow, then the detection result is the normal flow.
  • the detection module 15 when the detection module 15 generates the detection result, it may also first obtain a preset weight set, where the preset weight set is used to determine the credibility of different detection results generated by different models. Perform evaluation and setting, and then use the respective weight values of detection result 1, detection result 2 and detection result 3 included in the preset weight set to further determine the final detection result.
  • the abnormal flow detection system obtains the flow data to be detected, analyzes the flow data to be detected, and obtains target structured data; performs feature extraction processing on the target structured data to obtain target feature data;
  • the self-encoding model and the first detection model determine the first detection result corresponding to the target feature data, determine the second detection result corresponding to the target feature data based on the second detection model, and determine the third detection result corresponding to the target feature data based on the preset rule library ;
  • the self-encoding model and the first detection model are models generated based on an unsupervised algorithm; the second detection model is based on a model generated by a supervised algorithm; according to the first detection result, the second detection result, and the third detection result, the waiting Detect the target detection result of the flow data.
  • the abnormal traffic detection system uses unsupervised algorithms and supervised algorithms to train and generate the auto-encoding model, the first detection model, and the second detection model, respectively, so that it can be based on the auto-encoding model and the first detection model.
  • the first detection model, the second detection model and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection result obtained is the risk judgment of the flow data to be detected by combining the unsupervised algorithm, the supervised algorithm and the preset rule library. , Can improve the accuracy of abnormal flow detection, and effectively improve the detection quality of abnormal flow.
  • FIG. 4 is the third schematic diagram of the implementation process of the abnormal flow detection method.
  • the abnormal flow detection method performed by the abnormal flow detection system may include the following steps:
  • Step 301 Obtain first flow data, and extract characteristic data corresponding to the first flow data.
  • the abnormal flow detection system may first obtain the first flow data, and then may perform feature extraction on the first flow data to obtain the feature data corresponding to the first flow data.
  • the abnormal traffic detection system may first perform model training before using the self-encoding model, the first detection model, and the second detection model to detect the traffic to be detected.
  • the first traffic data may be used as training data for model training.
  • the first flow data may be composed of multiple data samples.
  • the abnormal flow detection system may first extract the characteristic information corresponding to the first flow data. Specifically, the abnormal flow detection system may perform feature extraction on the first flow data to obtain feature data corresponding to the first flow data.
  • the abnormal flow detection system needs to analyze and process the first flow data first.
  • the useless information in the first flow data can be filtered out, and the problem of model inaccuracy caused by irregular data can be reduced as much as possible.
  • the abnormal flow detection system can parse the first flow data to obtain the first structured data with a unified structure.
  • the abnormal traffic detection system may perform protocol analysis on the first traffic data, thereby restoring the original network behavior information.
  • the abnormal traffic detection system can parse out the information of the communication parties from the first traffic data according to the protocol specification, which can specifically include specific information such as source IP, destination IP, source port, destination port, request content, and response information. Some useless information can be discarded, and finally a structured data format can be generated, that is, the first structured data corresponding to the first traffic data can be generated respectively.
  • the abnormal traffic detection system after the abnormal traffic detection system obtains the first structured data through analysis processing, it can perform feature extraction on each piece of data in the first structured data to obtain each piece of data.
  • the feature information can eventually generate feature data.
  • each feature data in the feature data may include the following four types of feature information:
  • the first type is the basic characteristics of the traffic, including the duration of the visit, the protocol type, and The number of bytes sent, etc.
  • the second category is the content characteristics of the traffic, which converts the requested content in the traffic into text vector features
  • the third category is the statistical characteristics based on the time window, including the frequency of access within the time window, and the time window The total number of bytes sent and the number of connections with "SYN" errors
  • the fourth category is the statistical characteristics of the host accessed within the time window, including the frequency of access, the total number of bytes received, and the appearance of "SYN" The total number of wrong connections, etc.
  • the abnormal traffic detection system performs feature extraction on each piece of data in the first structured data, obtains the above four types of feature information of the piece of data, and then generates a uniform length of the piece of data based on the above four types of feature information Feature vector.
  • each feature data in the feature data is a feature vector with the same length.
  • Step 302 Encoding the characteristic data through the self-encoding model to obtain encoded data; wherein the self-encoding model is a model generated based on an unsupervised algorithm.
  • the abnormal flow detection system after the abnormal flow detection system obtains the first flow data and extracts the characteristic data corresponding to the first flow data, it can first encode the characteristic data through the self-encoding model to obtain the encoded data .
  • the self-encoding model may be a model generated by the abnormal traffic detection system based on an unsupervised algorithm.
  • unsupervised algorithms are unsupervised learning. In real life, there is insufficient prior knowledge, so it is difficult to manually label categories or the cost of manual category labeling is too high. Naturally, I hope that computers can complete these tasks on behalf of humans, or at least provide some help. Solving various problems in pattern recognition based on training samples with unknown categories (not labeled) is called unsupervised learning.
  • PCA Principal component analysis
  • isometric mapping method local linear embedding method
  • Laplacian feature mapping method Hesse local linear embedding method
  • local tangent space arrangement Methods etc.
  • the abnormal traffic detection system after the abnormal flow detection system performs feature extraction on the first flow data, the obtained feature data is high-dimensional sparse, while the unsupervised learning algorithm discretizes the high-dimensional sparse type.
  • the data classification effect is not ideal. Therefore, when using the feature data of the first traffic data for model training, in order to be more suitable for the detection model generated by the unsupervised learning algorithm, the abnormal traffic detection system needs to first perform the high-dimensional sparse feature data. Pretreatment. Specifically, the abnormal traffic detection system can encode feature data through a self-encoding model obtained by pre-training, so as to obtain low-dimensional continuous feature data, that is, obtain encoded data.
  • the autoencoder model is an autoencoder (AE), where the autoencoder is a type of artificial neural network (Artificial Neural Networks, used in semi-supervised learning and unsupervised learning). ANNs), whose function is to perform representation learning on the input information by taking the input information as the learning target.
  • autoencoders can be divided into undercomplete autoencoders (undercomplete autoencoders), regularized autoencoders (regularized autoencoders), and variational autoencoders (Variational AutoEncoders, VAE).
  • the first two are discriminant models and rear The latter is a generative model.
  • the autoencoder can be a neural network with a feedforward structure or a recursive structure.
  • the autoencoder has the function of characterizing learning algorithms in a general sense, and is applied to dimensionality reduction and anomaly detection.
  • the autoencoder including convolutional layer construction can be applied to computer vision problems, including image denoising, neural style transfer, and so on.
  • Fig. 5 is a schematic diagram of the structure of the self-encoder.
  • the self-encoder may include two parts: an encoder and a decoder. Among them, the input sample X is mapped to the feature space Z through the encoder, that is, the encoding process, and then the abstract feature Z is mapped back to the original space through the decoder to obtain the reconstructed sample X 0 , that is, the decoding process.
  • the optimization goal is to train the encoder and the decoder at the same time by minimizing the reconstruction error, so as to learn the abstract feature representation Z for the sample input X 0.
  • Step 303 Obtain a first detection model through the encoded data training; wherein, the first detection model is a model generated based on an unsupervised algorithm.
  • Step 304 Obtain a test result corresponding to the first traffic data according to the first detection model and the preset marking strategy.
  • the abnormal traffic detection system after the abnormal traffic detection system encodes the characteristic data through the self-encoding model and obtains the encoded data, it may first use the encoded data for model training, thereby obtaining the first detection model. Then, through the first detection model and the preset marking strategy, the test result corresponding to the first flow data can be continuously obtained.
  • the first detection model may be a model generated by the abnormal traffic detection system based on an unsupervised algorithm.
  • the first detection model may be obtained by the abnormal traffic detection system based on the isolation forest algorithm training.
  • the abnormal traffic detection system may first detect the encoded data through the first detection model, so that part of the first traffic data can be marked based on the characteristic data.
  • the abnormal flow detection system can choose to perform detection processing on other first flow data according to the preset marking strategy, and finally complete the detection of all the first flow data and obtain the detection result. That is, the test result of the first flow data.
  • the abnormal flow detection system can use an unsupervised algorithm to classify the first flow data after obtaining the encoded data.
  • the abnormal flow detection system may obtain the first detection model based on the isolated forest algorithm, thereby realizing automatic marking processing for most of the first flow data.
  • Step 305 Train to obtain a second detection model based on the feature data and the test result; wherein, the second detection model is a model generated based on a supervised algorithm.
  • the abnormal flow detection system can use the characteristic data corresponding to the first flow data after obtaining the test result corresponding to the first flow data according to the first detection model and the preset marking strategy Perform model training with the test results to obtain a second detection model.
  • the second detection model may be generated by the abnormal traffic detection system based on supervised algorithm training.
  • a supervised algorithm that is, supervised learning
  • supervised learning is a machine learning task that infers a function from a labeled training data set, and the training data is composed of a set of training examples.
  • each example is a pair of an input object (usually a vector) and a desired output value (also called a supervised signal).
  • the supervised learning algorithm analyzes the training data and produces an inferred function, which can be used to map new examples. An optimal solution would allow the algorithm to correctly determine the class label when the label is not visible.
  • supervised learning is the most common method of machine learning. Among them, supervised learning can include Support Vector Machines, linear regression, logistic regression, naive Bayes, linear discriminant analysis, and decision trees. (decision trees) K-nearest neighbor algorithm, etc.
  • the abnormal traffic detection system when trained to obtain the second detection model based on the feature data and the test result, it can input the feature data and the test result into the decision tree algorithm to train to obtain the second detection model .
  • the abnormal traffic detection system when the abnormal traffic detection system obtains the second detection model based on the feature data and test results, it may preferably have a decision tree algorithm in supervised learning, specifically, the abnormal traffic detection system The feature data and test results can be input into the decision tree algorithm, and finally the second detection model can be trained.
  • the decision tree is a supervised machine learning algorithm, a tree-shaped decision diagram with additional probability results, and a graphical method of intuitive use of statistical probability analysis.
  • a decision tree is a predictive model, which represents a mapping between object attributes and object values. Each node in the tree represents the judgment condition of the object attribute, and its branches represent objects that match the node. The leaf nodes of the tree represent the prediction results to which the object belongs. Decision tree algorithms are often used to solve classification and regression problems.
  • the abnormal flow detection system obtains the test result corresponding to the first flow data through the first detection model and the preset marking strategy, and completes the marking processing of the first flow data.
  • the abnormal traffic detection system obtains more accurate labels of the first traffic data, so that these labels can be used to train the second detection model based on the supervised algorithm.
  • the abnormal flow detection system can optimize the decision tree algorithm to implement the training of the second detection model, and use the high-dimensional discrete features obtained after feature extraction on the first flow data, that is, the feature data as the initial feature. Take the marking result obtained through the first detection model and the preset marking strategy, that is, the test result as the label, and send the corresponding feature data and test result as the training data to the decision tree for supervised learning, and finally can be obtained by training The second detection model.
  • the advantage of the decision tree algorithm over other machine learning algorithms is that it is more interpretable, the algorithm is relatively simple, and it can realize fast discrimination.
  • the abnormal flow detection system in the process of the abnormal flow detection system using one-to-one corresponding feature data and test results as training data for supervised learning, if the second detection model is used to determine based on the feature data If the detection result is inconsistent with the corresponding test result, the abnormal flow detection system can feed back the detection result to the feature extraction process, step 301, and then perform feature extraction on the first flow data again to obtain new feature data, and then use the new According to step 302 to step 305 again, the first detection model and the second detection model are trained in sequence.
  • the abnormal flow detection system when it is performing model training, it can perform feedback processing according to the training output results, and adjust the weight values corresponding to different features in the feature extraction process, so as to be based on the first flow data. On top, continuously obtain new feature data, and continuously train the first detection model and the second detection model. Thus, a fully trained model can be obtained after multiple iterations of training and testing.
  • the method for the abnormal traffic detection system to obtain the test result corresponding to the first traffic data according to the first detection model and the preset marking strategy may include the following steps:
  • Step 304a Use the first detection model to perform detection processing on the encoded data to obtain an initial result; where the initial result includes: normal traffic, abnormal traffic, and unpredictability.
  • Step 304b Perform marking processing on the encoded data whose initial result is unpredictable according to a preset marking strategy to obtain a marking result; wherein the marking result includes normal traffic and abnormal traffic.
  • Step 304c Based on the initial result and the marking result, a test result is generated.
  • the abnormal traffic detection system encodes the characteristic data through the self-encoding model and obtains the encoded data
  • it may first use the first detection model to detect the encoded data to obtain the initial result.
  • the abnormal traffic detection system when the abnormal traffic detection system uses the first detection model to detect the encoded data, it may first input the encoded data into the first detection model (isolated forest), thereby The predicted score X of each flow sample in the first flow data can be output. Then, the abnormal flow detection system can normalize the predicted score X of each flow sample according to the following formula, and obtain the normalization to 0 ⁇ The prediction score X norm of 1 :
  • X norm (XX min )/(X max -X min ) (1)
  • X is the score predicted by the isolated forest algorithm
  • X min is the lowest score predicted by the training sample
  • X max is the highest score predicted by the training sample
  • X norm is the normalized prediction score
  • the flow sample can be considered to be a normal flow sample, and if the normalized prediction score X norm is close to 0, then it can be considered that the flow sample is an abnormal flow sample.
  • the initial result obtained can be one of normal flow, abnormal flow and unpredictable.
  • the abnormal flow detection system regards them as flow data samples with a higher degree of discrimination, which can be automatically marked by the first detection model It is abnormal traffic or normal traffic.
  • the abnormal traffic detection system considers the traffic data samples to be difficult to distinguish. Therefore, it is necessary to follow the preset marking strategy for this part of the difficult to distinguish and unpredictable traffic. The sample continues to be labeled.
  • the encoded data whose initial result is unpredictable may be input to the marking system , Finally output the corresponding marking result.
  • the marking result may be abnormal flow or normal flow.
  • FIG. 6 is a schematic diagram of the composition structure of the marking system.
  • the marking system may be composed of a preset information library, a regular rule library, and a review pattern library.
  • the abnormal traffic detection system can first match the coded data whose initial result is unpredictable with the preset information base and the regular rule base respectively. If the initial result is unpredictable coded data, it can match the preset information at the same time. Database and regular rule database, then the abnormal traffic detection system can determine that the marking result is abnormal traffic; if the initial result is unpredictable encoded data that cannot match the preset intelligence database, or cannot match the regular rule database, then the abnormal traffic detection system can Continue to determine the marking result according to the preset review strategy. Specifically, the abnormal traffic detection system can use the review pattern library to further perform marking processing, and finally obtain the marking result.
  • the abnormal traffic detection system can update the preset information database and the regular rule database in real time. Specifically, the abnormal traffic detection system can use data that cannot match the preset information database or the regular rule database to update the preset information database and the regular rule database in real time.
  • FIG. 7 is a schematic diagram of labeling processing.
  • the abnormal traffic detection system can first compare the data with preset information.
  • the database performs matching processing, where the preset intelligence database can be a threat intelligence database obtained by the abnormal traffic detection system from a third party. If the preset intelligence database matches the data, for example, the IP of the data matches the IP black in the preset intelligence database. If the data in the list matches, then you can continue to match the data with the regular rule database; if the preset information database does not match the data, then you can mark the data according to the preset review strategy, specifically , You can use the audit mode in the audit mode library to mark the data.
  • the preset review strategy can represent the use of historical review methods for labeling processing, or it can represent the review of data to achieve labeling processing.
  • the data when the abnormal traffic detection system matches the data with the regular rule database, if the preset information database matches the data, then the data can be directly determined as a black sample, that is, the data is marked It is abnormal traffic; if the preset information library does not match the data, the data can be marked according to the preset review strategy. Specifically, the data can be marked using the review mode in the review mode library.
  • the matching requirements are relatively strict. Therefore, if the data matches the regular rule base, then the data can be directly determined as a black sample, that is, abnormal. Traffic; and the data that cannot match the preset intelligence database and regular rule database, the abnormal traffic detection system can consider it to be a possible abnormal sample, so it needs to continue to mark the data according to the preset review strategy. Specifically, if the If the data review fails, it will be determined as a black sample, that is, an abnormal flow. If the data review is passed, it will be determined as a white sample, that is, a normal flow. Exemplarily, for data that cannot be matched with the preset information database and the regular rule database, a network security expert can review the data, so as to implement labeling processing on the data.
  • the abnormal flow detection system when the abnormal flow detection system generates the test result based on the initial result and the marking result, if the initial result is normal flow, then it can be determined that the test result is normal flow; if the initial result is abnormal If the initial result is unpredictable, after marking according to the preset marking strategy, if the marking result is normal traffic, then the test result can be determined to be normal traffic; if the marking result is normal If it is abnormal flow, then it can be determined that the test result is abnormal flow.
  • experiments prove that the first detection model can automatically mark a large amount of traffic data, and the abnormal traffic detection system only needs to mark 3% of the traffic data according to the preset marking strategy.
  • the abnormal flow detection system before the abnormal flow detection system obtains the first flow data and extracts the characteristic data corresponding to the first flow data, that is, before step 301, the abnormal flow detection system performs abnormal
  • the flow detection method may also include the following steps:
  • Step 306 Grab the network traffic data, and store the network traffic data in a preset storage space in a mirror image.
  • the abnormal traffic detection system can capture the network traffic data in real time, and then mirror the network traffic data obtained by the capture to a fixed space, that is, store it in a preset storage space.
  • the abnormal traffic detection system when the abnormal traffic detection system captures network traffic data, it can use equipment such as optical fiber splitters to capture network traffic data transmitted in gateways and switches, and obtain The network traffic mirroring is saved in the preset storage space.
  • the abnormal traffic detection system continuously captures and mirrors and stores network traffic data, and the network traffic data stored in the mirror can be used to train and test the model.
  • the first flow data when the abnormal flow detection system obtains the first flow data, the first flow data may be read from the network flow data stored in the preset storage space.
  • the first traffic data may be historical network traffic data pre-stored by the abnormal traffic detection system.
  • FIG. 8 is the fourth schematic diagram of the implementation process of the abnormal flow detection method.
  • the abnormal flow detection system encodes the characteristic data through the self-encoding model to obtain the encoded data Before, that is, before step 302, the method for abnormal flow detection by the abnormal flow detection system may further include the following steps:
  • Step 307 Obtain the second traffic data from the network traffic data stored in the preset storage space.
  • Step 308 Based on the unsupervised algorithm, use the second traffic data to train to obtain the self-encoding model.
  • the abnormal traffic detection system needs to train to obtain the self-encoding model before acquiring the encoded data corresponding to the characteristic data.
  • the abnormal traffic detection system may first obtain the second traffic data from the network traffic data stored in the preset storage space; then, it may use the second traffic data training based on an unsupervised algorithm to obtain Self-encoding model.
  • the autoencoder model (autoencoder) is an unsupervised machine learning algorithm widely used in anomaly detection. Normal flow samples are reconstructed and restored, but data points that are different from the normal distribution cannot be restored well. According to this feature, in the process of generating the auto-encoding model, the abnormal traffic detection system can use a small amount of normal traffic data as training samples, so that the resulting auto-encoding model can get a more uniform distribution for normal traffic samples, and for abnormal traffic samples It can't reach a more uniform distribution.
  • the second traffic data for training the self-encoding model is normal traffic data.
  • the abnormal flow detection system can read the flow data that has been marked as normal flow from the network flow data stored in the preset storage space, and then determine the flow data as the second flow data . That is, the second flow data includes multiple pieces of normal flow, but does not include abnormal flow.
  • the abnormal traffic detection system uses the second traffic data to train to obtain the autoencoder, it may only retain the encoding part of the autoencoder, that is, the autoencoder model may be the autoencoder obtained by training.
  • the encoding part of the encoder can convert the input high-dimensional discrete feature data into low-dimensional continuous feature data through the self-encoding model, that is, the feature data is encoded through the self-encoding model to obtain the encoded data.
  • the abnormal traffic detection system can use the self-encoding model, the first detection model, and the second detection model after completing the training of the self-encoding model, the first detection model, and the second detection model.
  • the detection model detects abnormal traffic.
  • the abnormal flow detection system can use the self-encoding model, the first detection model, the second detection model, and a preset rule library to perform detection processing on the flow data to be detected, so as to obtain the flow data to be detected. Detect the target detection result of the flow data.
  • the traffic data to be detected may be network traffic data captured by the abnormal traffic detection system in real time.
  • the abnormal traffic detection system may directly collect the traffic data to be detected from the network card. It can directly receive the traffic data to be detected sent by other systems.
  • the abnormal traffic detection system can obtain a fully trained autoencoder, a first detection model, and a second detection model.
  • the abnormal traffic detection system can adopt the idea of integrated learning, combining the detection results of the output of the autoencoder, the first detection model and the second detection model, and the existing preset rule library
  • the matching algorithm obtains the detection result to determine the target detection result of the traffic to be detected, so as to obtain better generalization performance than a single model.
  • the core of the abnormal traffic detection method proposed in this application is to detect the abnormal traffic in combination with the machine learning algorithm and the existing rule engine, thereby realizing a semi-supervised learning solution.
  • the abnormal traffic detection system uses the preset marking strategy to mark the data that cannot be recognized by the first detection model, it can be manually marked by network security experts to distinguish a small part of the less clear traffic data, and then Larger flow data, even if you only need to manually mark 0.1% of the flow data, it will be a lot of work.
  • the abnormal traffic detection system can be further improved by unsupervised algorithms such as isolated forests, that is, the first detection model can be optimized to achieve higher accuracy and coverage; on the other hand,
  • the abnormal traffic detection system can generate a large number of abnormal traffic with different characteristics by analyzing the existing abnormal traffic samples, using the method of generating a confrontation network, and training a supervised algorithm with higher accuracy based on the generated data and tags. Model.
  • the abnormal traffic detection method on the one hand, the algorithm-based judgment method has better flexibility, and the anomaly detection algorithm based on unsupervised learning is more suitable for traffic monitoring scenarios and has better
  • the abnormal traffic detection system only uses unsupervised learning detection algorithms Generate results with higher confidence as labels.
  • the abnormal traffic detection system introduces a preset marking strategy to accurately mark them, that is, through the detection algorithm and the preset marking strategy. In combination, the preset marking strategy helps the detection algorithm to improve its accuracy and coverage.
  • the abnormal traffic detection system uses the self-encoder obtained by training to encode the characteristic data, and the continuous characteristic data is then sent to the first detection model, which greatly improves the performance of the detection algorithm; on the other hand, the abnormal traffic detection system mainly uses two Two machine learning algorithms, namely unsupervised learning (isolated forest algorithm) and supervised learning (decision tree algorithm), due to the fast training speed of isolated forest and decision tree algorithms, the entire abnormal traffic detection system can achieve day-level (hours) Level) training and update. Therefore, in actual use, the latest trained detection model can be used to detect real-time traffic data, so that the user behavior of the day can be inferred and judged, and at the same time, the regular rule base and prediction can be updated in time. The establishment of an information database can ensure the real-time performance of the entire abnormal flow detection system.
  • the abnormal flow detection system obtains first flow data and extracts characteristic data corresponding to the first flow data; encodes the characteristic data through an auto-encoding model to obtain encoded data; wherein,
  • the self-encoding model is a model generated based on an unsupervised algorithm; the first detection model is obtained through data training after encoding; where the first detection model is a model generated based on an unsupervised algorithm; according to the first detection model and the preset labeling strategy, The test result corresponding to the first traffic data; the second detection model is obtained by training based on the characteristic data and the test result; wherein, the second detection model is a model generated based on a supervised algorithm.
  • the abnormal traffic detection system uses unsupervised algorithms and supervised algorithms to train and generate the auto-encoding model, the first detection model, and the second detection model, respectively, so that it can be based on the auto-encoding model and the first detection model.
  • the first detection model, the second detection model and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection result obtained is the risk judgment of the flow data to be detected by combining the unsupervised algorithm, the supervised algorithm and the preset rule library. , Can improve the accuracy of abnormal flow detection, and effectively improve the detection quality of abnormal flow.
  • FIG. 9 is a schematic diagram of the composition structure of an abnormal flow detection system.
  • the abnormal flow detection system 10 proposed in the embodiment of the present application may include a first acquisition Part 16, parsing part 17, first extracting part 18, determining part 19, generating part 110.
  • the first acquiring part 16 is configured to acquire traffic data to be detected
  • the parsing part 17 is configured to analyze the traffic data to be detected to obtain target structured data
  • the first extraction part 18 is configured to perform feature extraction processing on the target structured data to obtain target feature data
  • the determining part 19 is configured to determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, determine the second detection result corresponding to the target feature data based on the second detection model, and based on the prediction
  • the rule base determines the third detection result corresponding to the target feature data
  • the self-encoding model is a model generated based on an unsupervised algorithm
  • the first detection model is a model generated based on an unsupervised algorithm
  • the detection model is based on a model generated by a supervised algorithm
  • the generating part 110 is configured to generate a target detection result of the flow data to be detected based on the first detection result, the second detection result, and the third detection result.
  • the determining part 19 is specifically configured to input the target feature data into the self-encoding model and output the encoded data; input the encoded data into the first detection The model outputs the first detection result.
  • the determining part 19 is further specifically configured to input the target feature data into the second detection model, and output the second detection result.
  • the determining part 19 is further specifically configured to use the preset rule library to perform matching processing on the target feature data to obtain the third detection result.
  • the generating part 110 is specifically configured to: if at least two of the first detection result, the second detection result, and the third detection result are normal traffic, It is determined that the target detection result is normal flow; if at least two of the first detection result, the second detection result, and the third detection result are abnormal flow, it is determined that the target detection result is abnormal flow.
  • the generating part 110 is further specifically configured to obtain a preset weight set; use the preset weight set, the first detection result, the second detection result, and the The third detection result is weighted to obtain the target detection result.
  • the abnormal flow detection system 10 is a schematic diagram of the second structure of the abnormal flow detection system.
  • the abnormal flow detection system 10 proposed in the embodiment of the present application may also include a second acquisition part 111, a second extraction part 112, an encoding part 113, and training Part 114.
  • the second obtaining part 111 is configured to obtain first flow data
  • the second extraction part 112 is configured to extract feature data corresponding to the first flow data
  • the encoding part 113 is configured to perform encoding processing on the characteristic data through a self-encoding model to obtain encoded data; wherein, the self-encoding model is a model generated based on an unsupervised algorithm;
  • the second acquisition part 111 is further configured to obtain a first detection model through the encoded data training; wherein, the first detection model is a model generated based on an unsupervised algorithm; and according to the first detection model and Preset marking strategy to obtain the test result corresponding to the first flow data;
  • the training part 114 is configured to train to obtain a second detection model based on the feature data and the test result; wherein, the second detection model is a model generated based on a supervised algorithm.
  • the second extraction part 112 is specifically configured to parse the first traffic data to obtain structured data; perform feature extraction processing on the structured data to obtain the feature data.
  • the second acquiring part 111 is specifically configured to use the first detection model to perform detection processing on the encoded data to obtain the initial result; wherein, the initial The results include: normal traffic, abnormal traffic, and unpredictable; according to the preset marking strategy, the encoded data whose initial result is unpredictable is marked to obtain a marking result; wherein, the marking result includes normal Flow and abnormal flow; based on the initial result and the marking result, the test result is generated.
  • the second acquiring part 111 is further specifically configured to determine if the initial result is the unpredictable encoded data and matches the preset information database and the regular rule database
  • the marking result is abnormal traffic; if the initial result is the unpredictable encoded data and does not match the preset information library or the regular rule library, the marking result is determined according to a preset review strategy.
  • the second acquiring part 111 is further specifically configured to determine that the test result is a normal flow if the initial result is a normal flow; if the initial result is an abnormal flow , It is determined that the test result is an abnormal flow; if the marking result is a normal flow, the test result is determined to be a normal flow; if the marking result is an abnormal flow, the test result is determined to be an abnormal flow.
  • the training part 114 is specifically configured to obtain the first detection model by training based on the isolated forest algorithm and using the encoded data.
  • the training part 114 is further specifically configured to input the feature data and the test result into the decision tree algorithm, and train to obtain the second detection model.
  • the second obtaining part 111 is further configured to obtain the first flow data, and before extracting the characteristic data corresponding to the first flow data, capture the network flow data, and
  • the network traffic data is mirrored and stored in a preset storage space;
  • the second acquiring part 111 is also specifically configured to acquire the first traffic data from the network traffic data stored in the preset storage space.
  • the second acquiring part 111 is further configured to encode the characteristic data through a self-encoding model, and before acquiring the encoded data, store the data from the preset storage space. Obtain the second flow data from the network flow data; based on an unsupervised algorithm, use the second flow data to train to obtain the self-encoding model.
  • FIG. 11 is a schematic diagram of the third composition structure of an abnormal flow detection system.
  • the abnormal flow detection system 10 proposed in the embodiment of the present application may further include a processor 115 and a memory 116 storing executable instructions of the processor 115, and further Ground, the abnormal traffic detection system 10 may further include a communication interface 117 and a bus 118 for connecting the processor 115, the memory 116, and the communication interface 117.
  • the above-mentioned processor 115 may be an application specific integrated circuit (ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital signal processing device (Digital Signal Processing Device, DSPD). ), Programmable Logic Device (ProgRAMmable Logic Device, PLD), Field Programmable Gate Array (Field ProgRAMmable Gate Array, FPGA), Central Processing Unit (CPU), Controller, Microcontroller, Microprocessor At least one of. It is understandable that, for different devices, the electronic devices used to implement the above-mentioned processor functions may also be other, which is not specifically limited in the embodiment of the present application.
  • ASIC application specific integrated circuit
  • DSP Digital Signal Processor
  • DSPD Digital Signal Processing Device
  • PLD Programmable Logic Device
  • Field Programmable Gate Array Field ProgRAMmable Gate Array
  • CPU Central Processing Unit
  • Controller Microcontroller
  • Microprocessor At least one of. It is understandable that, for different devices, the electronic devices used to implement the above-mentioned processor functions
  • the abnormal traffic detection system 10 may also include a memory 116, which may be connected to the processor 115, where the memory 116 is used to store executable program code, the program code includes computer operation instructions, the memory 116 may include high-speed RAM memory, and It may also include non-volatile memory, for example, at least two disk memories.
  • the bus 118 is used to connect the communication interface 117, the processor 115, and the memory 116, and to communicate with each other among these devices.
  • the memory 116 is used to store instructions and data.
  • the above-mentioned processor 115 is configured to obtain the flow data to be detected, and parse the flow data to be detected to obtain target structured data; perform feature extraction processing on the target structured data , Obtain target feature data; determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, and determine the second detection result corresponding to the target feature data based on the second detection model, based on preset rules
  • the library determines the third detection result corresponding to the target feature data; wherein, the self-encoding model is a model generated based on an unsupervised algorithm; the first detection model is a model generated based on an unsupervised algorithm; the second detection The model is based on a model generated by a supervised algorithm; according to the first detection result, the second detection result, and the third detection result, a target detection result of the to-be-detected flow data is generated.
  • the above-mentioned processor 115 is further configured to obtain the first flow data, and extract the characteristic data corresponding to the first flow data; to encode the characteristic data through the self-encoding model , Obtain encoded data; wherein, the self-encoding model is a model generated based on an unsupervised algorithm; obtain a first detection model through training on the encoded data; wherein, the first detection model is generated based on an unsupervised algorithm Model; obtain the test result corresponding to the first flow data according to the first detection model and the preset marking strategy; obtain a second detection model based on the feature data and the test result training; wherein, the second The detection model is a model generated based on a supervised algorithm.
  • the aforementioned memory 116 may be a volatile memory (volatile memory), such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory.
  • volatile memory such as a random-access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory.
  • a storage device Read-Only Memory, ROM), flash memory (flash memory), hard disk (Hard Disk Drive, HDD) or solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memory, and Instructions and data are provided to the processor 115.
  • the functional modules in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be realized in the form of hardware or software function module.
  • the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this embodiment is essentially or correct
  • the part that the prior art contributes or all or part of the technical solution can be embodied in the form of a software product.
  • the computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer).
  • the aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes.
  • An abnormal traffic detection system uses an unsupervised algorithm and a supervised algorithm to train and generate an auto-encoding model, a first detection model, and a second detection model, respectively, so as to be based on the auto-encoding model ,
  • the first detection model, the second detection model, and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection results obtained are combined with the unsupervised algorithm, supervised algorithm and the preset rule library to achieve the flow data to be detected Risk judgment can improve the accuracy of abnormal traffic detection and effectively improve the detection quality of abnormal traffic.
  • the embodiment of the present application provides a computer-readable storage medium on which a program is stored, and when the program is executed by a processor, the method for detecting abnormal traffic as described above is implemented.
  • the program instructions corresponding to an abnormal flow detection method in this embodiment can be stored on storage media such as optical disks, hard disks, and USB flash drives.
  • storage media such as optical disks, hard disks, and USB flash drives.
  • a target detection result of the to-be-detected flow data is generated.
  • the first detection model is a model generated based on an unsupervised algorithm
  • a second detection model is obtained by training based on the feature data and the test result; wherein, the second detection model is a model generated based on a supervised algorithm.
  • this application can be provided as a method, a terminal, or a computer program product. Therefore, this application may adopt the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.
  • These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are used to generate A device for implementing functions specified in one process or multiple processes in the schematic flow chart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device.
  • the device realizes the functions specified in one process or multiple processes in the realization process schematic diagram and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment.
  • the instructions provide steps for implementing functions specified in one or more processes in the schematic diagram and/or one block or more in the block diagram.
  • the embodiment of the application proposes an abnormal flow detection method and system, and a computer storage medium.
  • the abnormal flow detection system obtains the flow data to be detected, analyzes the flow data to be detected, and obtains target structured data; characterizes the target structured data Extraction process to obtain target feature data; determine the first detection result corresponding to the target feature data based on the self-encoding model and the first detection model, determine the second detection result corresponding to the target feature data based on the second detection model, and determine based on the preset rule library
  • the third detection result corresponding to the target feature data; among them, the self-encoding model and the first detection model are models generated based on unsupervised algorithms; the second detection model is based on models generated by supervised algorithms; according to the first detection results, the second detection As a result and the third detection result, the target detection result of the flow data to be detected is generated.
  • the abnormal traffic detection system uses unsupervised algorithms and supervised algorithms to train and generate the auto-encoding model, the first detection model, and the second detection model, respectively, so that it can be based on the auto-encoding model and the first detection model.
  • the first detection model, the second detection model and the preset rule library perform abnormal flow detection on the flow data to be detected, and the detection result obtained is the risk judgment of the flow data to be detected by combining the unsupervised algorithm, the supervised algorithm and the preset rule library. , Can improve the accuracy of abnormal flow detection, and effectively improve the detection quality of abnormal flow.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Algebra (AREA)
  • Mathematical Optimization (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Pure & Applied Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

Selon les modes de réalisation, la présente invention concerne un procédé et un système de détection de flux anormal, et un support de stockage informatique. Le procédé de détection de flux anormal comprend les étapes suivantes : acquérir des données de flux devant subir une détection et analyser les données de flux devant subir une détection afin d'obtenir des données structurées cibles ; effectuer un traitement d'extraction de caractéristiques sur les données structurées cibles pour obtenir des données de caractéristiques cibles ; en fonction d'un modèle autocodant et d'un premier modèle de détection, déterminer un premier résultat de détection correspondant aux données de caractéristiques cibles, en fonction d'un deuxième modèle de détection, déterminer un deuxième résultat de détection correspondant aux données de caractéristiques cibles, et en fonction d'une base de règles prédéfinie, déterminer un troisième résultat de détection correspondant aux données de caractéristiques cibles, où le modèle autocodant et le premier modèle de détection sont des modèles produits en se basant sur un algorithme non supervisé, tandis que le deuxième modèle de détection est un modèle produit en se basant sur un algorithme supervisé ; et selon le premier résultat de détection, le deuxième résultat de détection et le troisième résultat de détection, produire un résultat de détection cible des données de flux devant subir une détection.
PCT/CN2020/098177 2020-06-24 2020-06-24 Procédé et système de détection de flux anormal et support de stockage informatique WO2021258348A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2020/098177 WO2021258348A1 (fr) 2020-06-24 2020-06-24 Procédé et système de détection de flux anormal et support de stockage informatique
CN202080100505.9A CN115606162A (zh) 2020-06-24 2020-06-24 异常流量检测方法和系统、及计算机存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/098177 WO2021258348A1 (fr) 2020-06-24 2020-06-24 Procédé et système de détection de flux anormal et support de stockage informatique

Publications (1)

Publication Number Publication Date
WO2021258348A1 true WO2021258348A1 (fr) 2021-12-30

Family

ID=79282432

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/098177 WO2021258348A1 (fr) 2020-06-24 2020-06-24 Procédé et système de détection de flux anormal et support de stockage informatique

Country Status (2)

Country Link
CN (1) CN115606162A (fr)
WO (1) WO2021258348A1 (fr)

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220038482A1 (en) * 2020-01-31 2022-02-03 Panasonic Intellectual Property Corporation Of America Anomaly detection method and anomaly detection device
CN114500334A (zh) * 2021-12-31 2022-05-13 钉钉(中国)信息技术有限公司 服务端应用架构的诊断方法及装置
CN114547970A (zh) * 2022-01-25 2022-05-27 中国长江三峡集团有限公司 一种水电厂顶盖排水系统异常智能诊断方法
CN114584391A (zh) * 2022-03-22 2022-06-03 恒安嘉新（北京）科技股份公司 异常流量处理策略的生成方法、装置、设备及存储介质
CN114615088A (zh) * 2022-04-25 2022-06-10 国网冀北电力有限公司信息通信分公司 一种终端业务流量异常检测模型建立方法及异常检测方法
CN114629699A (zh) * 2022-03-07 2022-06-14 北京邮电大学 基于深度强化学习的可迁移性网络流行为异常检测方法及装置
CN114679308A (zh) * 2022-03-21 2022-06-28 山东大学 一种基于双路自编码的未知流量识别方法及系统
CN114726581A (zh) * 2022-03-09 2022-07-08 同济大学 一种异常检测方法、装置、电子设备及存储介质
CN114745170A (zh) * 2022-04-07 2022-07-12 鹏城实验室 物联网异常实时检测方法、装置、终端及可读存储介质
CN114866338A (zh) * 2022-06-10 2022-08-05 阿里云计算有限公司 网络安全检测方法、装置及电子设备
CN115016433A (zh) * 2022-06-01 2022-09-06 哈尔滨工业大学（威海） 一种车载can总线流量异常检测方法及系统
CN115080965A (zh) * 2022-08-16 2022-09-20 杭州比智科技有限公司 基于历史表现的无监督异常检测方法及系统
CN115118514A (zh) * 2022-07-11 2022-09-27 深信服科技股份有限公司 一种数据检测方法、装置、设备及介质
CN115174190A (zh) * 2022-06-29 2022-10-11 武汉极意网络科技有限公司 一种基于网络流量的信息安全管控系统及方法
CN115174178A (zh) * 2022-06-28 2022-10-11 南京邮电大学 基于生成对抗网络的半监督网络流量异常检测方法
CN115250199A (zh) * 2022-07-15 2022-10-28 北京六方云信息技术有限公司 数据流检测方法、装置、终端设备以及存储介质
CN115278680A (zh) * 2022-07-29 2022-11-01 国网区块链科技(北京)有限公司 一种移动应用攻击检测方法、装置、设备和存储介质
CN115277098A (zh) * 2022-06-27 2022-11-01 深圳铸泰科技有限公司 一种基于智能学习的网络流量异常检测装置及方法
CN115296919A (zh) * 2022-08-15 2022-11-04 江西师范大学 一种边缘网关对特殊流量包计算方法及系统
CN115529162A (zh) * 2022-08-26 2022-12-27 中国科学院信息工程研究所 工控流量异常行为防护方法及系统
CN115694947A (zh) * 2022-10-26 2023-02-03 四川大学 基于对抗生成dqn的网络加密流量威胁样本生成机制方法
CN116132337A (zh) * 2023-04-04 2023-05-16 深圳行云创新科技有限公司 一种基于服务网格技术的接口流量异常检测方法
CN116319386A (zh) * 2023-05-17 2023-06-23 北京国信蓝盾科技有限公司 可用性及故障预测方法、装置、电子设备和介质
CN116582301A (zh) * 2023-04-17 2023-08-11 华中科技大学 基于拉普拉斯金字塔的工控网络异常流量检测方法和系统
CN116993663A (zh) * 2023-06-12 2023-11-03 阿里巴巴（中国）有限公司 图像处理方法、图像处理模型的训练方法
CN117061254A (zh) * 2023-10-12 2023-11-14 之江实验室 异常流量检测方法、装置和计算机设备
CN117151768A (zh) * 2023-10-30 2023-12-01 国网浙江省电力有限公司营销服务中心 一种生成式营销事件风控规则库的构建方法及系统
CN117349774A (zh) * 2023-10-24 2024-01-05 重庆邮电大学 一种基于大数据的区块链异常交易检测方法
CN117834299A (zh) * 2024-03-04 2024-04-05 福建银数信息技术有限公司 一种网络安全智能监督管理方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108563548A (zh) * 2018-03-19 2018-09-21 阿里巴巴集团控股有限公司 异常检测方法及装置
CN108985330A (zh) * 2018-06-13 2018-12-11 华中科技大学 一种自编码网络及其训练方法、异常用电检测方法和系统
CN111178523A (zh) * 2019-08-02 2020-05-19 腾讯科技（深圳）有限公司 一种行为检测方法、装置、电子设备及存储介质

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108563548A (zh) * 2018-03-19 2018-09-21 阿里巴巴集团控股有限公司 异常检测方法及装置
CN108985330A (zh) * 2018-06-13 2018-12-11 华中科技大学 一种自编码网络及其训练方法、异常用电检测方法和系统
CN111178523A (zh) * 2019-08-02 2020-05-19 腾讯科技（深圳）有限公司 一种行为检测方法、装置、电子设备及存储介质

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11876818B2 (en) * 2020-01-31 2024-01-16 Panasonic Intellectual Property Corporation Of America Anomaly detection method and anomaly detection device
US20220038482A1 (en) * 2020-01-31 2022-02-03 Panasonic Intellectual Property Corporation Of America Anomaly detection method and anomaly detection device
CN114500334A (zh) * 2021-12-31 2022-05-13 钉钉(中国)信息技术有限公司 服务端应用架构的诊断方法及装置
CN114500334B (zh) * 2021-12-31 2024-04-09 钉钉(中国)信息技术有限公司 服务端应用架构的诊断方法及装置
CN114547970A (zh) * 2022-01-25 2022-05-27 中国长江三峡集团有限公司 一种水电厂顶盖排水系统异常智能诊断方法
CN114547970B (zh) * 2022-01-25 2024-02-20 中国长江三峡集团有限公司 一种水电厂顶盖排水系统异常智能诊断方法
CN114629699B (zh) * 2022-03-07 2022-12-09 北京邮电大学 基于深度强化学习的可迁移性网络流行为异常检测方法及装置
CN114629699A (zh) * 2022-03-07 2022-06-14 北京邮电大学 基于深度强化学习的可迁移性网络流行为异常检测方法及装置
CN114726581A (zh) * 2022-03-09 2022-07-08 同济大学 一种异常检测方法、装置、电子设备及存储介质
CN114726581B (zh) * 2022-03-09 2023-06-20 同济大学 一种异常检测方法、装置、电子设备及存储介质
CN114679308A (zh) * 2022-03-21 2022-06-28 山东大学 一种基于双路自编码的未知流量识别方法及系统
CN114584391A (zh) * 2022-03-22 2022-06-03 恒安嘉新（北京）科技股份公司 异常流量处理策略的生成方法、装置、设备及存储介质
CN114584391B (zh) * 2022-03-22 2024-02-09 恒安嘉新（北京）科技股份公司 异常流量处理策略的生成方法、装置、设备及存储介质
CN114745170A (zh) * 2022-04-07 2022-07-12 鹏城实验室 物联网异常实时检测方法、装置、终端及可读存储介质
CN114745170B (zh) * 2022-04-07 2023-08-18 鹏城实验室 物联网异常实时检测方法、装置、终端及可读存储介质
CN114615088A (zh) * 2022-04-25 2022-06-10 国网冀北电力有限公司信息通信分公司 一种终端业务流量异常检测模型建立方法及异常检测方法
CN115016433A (zh) * 2022-06-01 2022-09-06 哈尔滨工业大学（威海） 一种车载can总线流量异常检测方法及系统
CN114866338A (zh) * 2022-06-10 2022-08-05 阿里云计算有限公司 网络安全检测方法、装置及电子设备
CN115277098A (zh) * 2022-06-27 2022-11-01 深圳铸泰科技有限公司 一种基于智能学习的网络流量异常检测装置及方法
CN115174178A (zh) * 2022-06-28 2022-10-11 南京邮电大学 基于生成对抗网络的半监督网络流量异常检测方法
CN115174178B (zh) * 2022-06-28 2023-07-04 南京邮电大学 基于生成对抗网络的半监督网络流量异常检测方法
CN115174190A (zh) * 2022-06-29 2022-10-11 武汉极意网络科技有限公司 一种基于网络流量的信息安全管控系统及方法
CN115174190B (zh) * 2022-06-29 2024-01-26 武汉极意网络科技有限公司 一种基于网络流量的信息安全管控系统及方法
CN115118514A (zh) * 2022-07-11 2022-09-27 深信服科技股份有限公司 一种数据检测方法、装置、设备及介质
CN115250199A (zh) * 2022-07-15 2022-10-28 北京六方云信息技术有限公司 数据流检测方法、装置、终端设备以及存储介质
CN115278680A (zh) * 2022-07-29 2022-11-01 国网区块链科技(北京)有限公司 一种移动应用攻击检测方法、装置、设备和存储介质
CN115296919A (zh) * 2022-08-15 2022-11-04 江西师范大学 一种边缘网关对特殊流量包计算方法及系统
CN115080965A (zh) * 2022-08-16 2022-09-20 杭州比智科技有限公司 基于历史表现的无监督异常检测方法及系统
CN115529162A (zh) * 2022-08-26 2022-12-27 中国科学院信息工程研究所 工控流量异常行为防护方法及系统
CN115694947B (zh) * 2022-10-26 2024-04-16 四川大学 基于对抗生成dqn的网络加密流量威胁样本生成机制方法
CN115694947A (zh) * 2022-10-26 2023-02-03 四川大学 基于对抗生成dqn的网络加密流量威胁样本生成机制方法
CN116132337A (zh) * 2023-04-04 2023-05-16 深圳行云创新科技有限公司 一种基于服务网格技术的接口流量异常检测方法
CN116132337B (zh) * 2023-04-04 2023-06-13 深圳行云创新科技有限公司 一种基于服务网格技术的接口流量异常检测方法
CN116582301B (zh) * 2023-04-17 2024-02-02 华中科技大学 基于拉普拉斯金字塔的工控网络异常流量检测方法、系统及计算机可读存储介质
CN116582301A (zh) * 2023-04-17 2023-08-11 华中科技大学 基于拉普拉斯金字塔的工控网络异常流量检测方法和系统
CN116319386A (zh) * 2023-05-17 2023-06-23 北京国信蓝盾科技有限公司 可用性及故障预测方法、装置、电子设备和介质
CN116993663A (zh) * 2023-06-12 2023-11-03 阿里巴巴（中国）有限公司 图像处理方法、图像处理模型的训练方法
CN116993663B (zh) * 2023-06-12 2024-04-30 阿里巴巴（中国）有限公司 图像处理方法、图像处理模型的训练方法
CN117061254B (zh) * 2023-10-12 2024-01-23 之江实验室 异常流量检测方法、装置和计算机设备
CN117061254A (zh) * 2023-10-12 2023-11-14 之江实验室 异常流量检测方法、装置和计算机设备
CN117349774A (zh) * 2023-10-24 2024-01-05 重庆邮电大学 一种基于大数据的区块链异常交易检测方法
CN117151768A (zh) * 2023-10-30 2023-12-01 国网浙江省电力有限公司营销服务中心 一种生成式营销事件风控规则库的构建方法及系统
CN117834299A (zh) * 2024-03-04 2024-04-05 福建银数信息技术有限公司 一种网络安全智能监督管理方法及系统

Also Published As

Publication number Publication date
CN115606162A (zh) 2023-01-13

Similar Documents

Publication Publication Date Title
WO2021258348A1 (fr) Procédé et système de détection de flux anormal et support de stockage informatique
US10880321B2 (en) Method and system for learning representations of network flow traffic
Wang et al. App-net: A hybrid neural network for encrypted mobile traffic classification
CN113535825A (zh) 一种基于云计算智能化的数据信息风控处理方法及系统
Weng et al. Multi-agent-based unsupervised detection of energy consumption anomalies on smart campus
WO2022227388A1 (fr) Procédé, appareil et dispositif de formation de modèle de détection d'anomalie de journal
CN112910859B (zh) 基于c5.0决策树和时序分析的物联网设备监测预警方法
CN113242207A (zh) 一种迭代聚类的网络流量异常检测方法
Li et al. Traffic identification of mobile apps based on variational autoencoder network
CN110855648A (zh) 一种网络攻击的预警控制方法及装置
Wang et al. Res-TranBiLSTM: An intelligent approach for intrusion detection in the Internet of Things
CN114697096A (zh) 基于空时特征和注意力机制的入侵检测方法
CN112839014A (zh) 建立识别异常访问者模型的方法、系统、设备及介质
CN116318928A (zh) 一种基于数据增强和特征融合的恶意流量识别方法及系统
CN112039997A (zh) 一种基于三重特征的物联网终端识别方法
US20220121942A1 (en) Method and system for cognitive information processing using representation learning and decision learning on data
CN116805039B (zh) 特征筛选方法、装置、计算机设备和数据扰动方法
Gao et al. The prediction role of hidden markov model in intrusion detection
CN110290101B (zh) 智能电网环境中基于深度信任网络的关联攻击行为识别方法
CN113918936A (zh) Sql注入攻击检测的方法以及装置
Xin et al. Research on feature selection of intrusion detection based on deep learning
Khan et al. Anomalous node detection in attributed social networks using dual variational autoencoder with generative adversarial networks
Hao et al. Adaptive Intrusion Detection Model Based on CNN and C5. 0 Classifier
Liu et al. A feature compression technique for anomaly detection using convolutional neural networks
Zhang et al. An Intelligent Edge Dual-Structure Ensemble Method for Data Stream Detection and Releasing

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20942318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 12/05/2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20942318

Country of ref document: EP

Kind code of ref document: A1