CN108965248B

CN108965248B - P2P botnet detection system and method based on traffic analysis

Info

Publication number: CN108965248B
Application number: CN201810565197.1A
Authority: CN
Inventors: 邹福泰; 张奕; 吴越
Original assignee: Shanghai Jiaotong University
Current assignee: Shanghai Shiyue Computer Technology Co ltd
Priority date: 2018-06-04
Filing date: 2018-06-04
Publication date: 2021-08-20
Anticipated expiration: 2038-06-04
Also published as: CN108965248A

Abstract

The invention discloses a P2P botnet detection system based on flow analysis, which belongs to the field of computer network safety and comprises a network flow receiving module, a communication structure chart construction module, a detection algorithm module, a P2P structure, a decision tree and Bayesian network machine learning method, a P2P botnet detection module, a database module and relevant data storage modules, wherein the network flow receiving module obtains network flow from different monitoring points, the detection algorithm module discovers the P2P structure by using a community discovery algorithm from the communication structure chart, the P2P botnet detection module is combined with tracking communication with other nodes, and the database module stores relevant data. The invention also discloses a P2P botnet detection method based on flow analysis. The invention does not take the port as a characteristic to prevent detection failure caused by port randomization, performs P2P botnet detection based on flow analysis, improves detection efficiency by filtering benign network flow, reduces data processing capacity by using packet length as a characteristic, can efficiently identify P2P botnet communication, and provides support for an intrusion detection system.

Description

P2P botnet detection system and method based on traffic analysis

Technical Field

The invention relates to the field of computer network security, in particular to a P2P botnet detection system and method based on flow analysis.

Background

Botnets (botnets) are a large number of hosts on the internet that are under the control of attackers, using C & C (command and control) channel control and issuing commands to hosts that exist within botnets. Botnets are commonly used to launch Distributed Denial-of-Service (DDoS) attacks, send spam, bitcoin mining, and other attacks.

P2P (Peer-to-Peer) Peer-to-Peer networks, also known as Peer-to-Peer technologies, have equal capabilities on each node, and typically do not rely on a central node. The dispersive nature of P2P networks makes them difficult to discover, and the distributed nature also handles single points of failure well. Since users began to have more and more powerful servers, storage space and bandwidth in the beginning of the 21 st century, the share of the internet with the P2P architecture increased sharply, and since the P2P model has been plagued by privacy, piracy, etc., the share of the internet that later occupied has decreased.

The P2P botnets are a wide variety of types, but the bottom most layer is the P2P fabric network consisting of peer bot host nodes. An attacker can issue commands to botnet hosts controlled by the attacker through setting the super agent node, and can also encrypt the commands in an asymmetric encryption mode so as to guide the botnet to continue attacking.

The initial botnet detection work was to detect traffic packet information for a particular port using a port and signature approach that would not work if an encrypted botnet was encountered or a randomized botnet was employed for the communication port.

Some subsequent detection methods use a graph clustering technology to detect P2P traffic, but do not analyze benign P2P traffic, and have the problems that the detection consumes large computing resources and the detection efficiency is low, and in addition, the method also has the problem that graphs cannot be expanded.

Accordingly, those skilled in the art have endeavored to develop a P2P botnet detection system based on traffic analysis that does not rely on communication port detection, to distinguish between benign and malignant P2P traffic, and to improve detection efficiency. In addition, the botnet structure obtained by the system is further ensured to be expanded by further tracking on the basis of the botnet mined by the system.

Disclosure of Invention

In view of the above-mentioned drawbacks of the prior art, the technical problem to be solved by the present invention is to distinguish benign and malignant P2P traffic, thereby improving the detection efficiency. In addition, the botnet structure obtained by the system is further ensured to be expanded by further tracking on the basis of the botnet mined by the system.

In order to achieve the above object, the present invention provides a P2P botnet detection system based on traffic analysis, which includes a network traffic receiving module, a communication structure diagram constructing module, a detection algorithm module, and a database module, wherein:

a network flow receiving module: acquiring network traffic from programs distributed at different monitoring points, filtering out benign network traffic according to a filtering rule, and recording uncertain traffic in a database module;

the communication structure diagram construction module: reading uncertain flow records and constructing a communication structure chart;

a detection algorithm module: analyzing and researching the generated communication structure diagram, and judging whether the IP address of the node in the communication structure diagram is a botnet host or not;

a database module: and storing the calculation results of the network flow receiving module, the communication structure chart constructing module and the detection algorithm module.

Further, the filtering rules contain well-established network addresses.

Further, the filtering rules include benign P2P network addresses detected by the present system.

Further, the communication structure diagram building module is configured to: and constructing a weighted directed graph by taking the monitored IP addresses as nodes and the communication among the nodes as edges, and performing subgraph segmentation by utilizing a community discovery algorithm to find a P2P structural subgraph.

Further, the weight of the weighted directed graph can be obtained by calculating the communication similarity between the node and other connected nodes.

Further, the communication similarity is calculated by using the time stamp TS and the packet payload length PS.

Further, the detection algorithm module is configured to: integrating the communication between nodes in a communication structure chart into a session between the nodes, calculating the difference value of a session start timestamp and a session end timestamp to obtain a session duration TD, the total length APS of a data packet effective load sent by a source IP A to a target IP B, the total length BPS of a data packet effective load sent by the source IP B to the target IP A, and the number of times NOC of communication between the two parties, taking the APS, the BPS, the NOC and the TD as characteristics, adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network to detect malicious P2P traffic, adding a benign P2P network into a filtering rule of a traffic receiving module, and storing the detection result of a P2P botnet into a database module.

Further, the detection algorithm module is configured to: when detecting that the nodes in the botnet communicate with other nodes which do not belong to the botnet, tracking the conversation between the two nodes, calculating the values of APS, BPS, NOC and TD and the conversation characteristics of the botnet to carry out similarity judgment, and if the values are similar, adding the values into the previous P2P botnet structure and recording the similarity.

The invention also provides a P2P botnet detection method based on flow analysis, which comprises the following steps:

step 1: for a known communication structure diagram, acquiring a key field group of related flow in the communication structure diagram, wherein the key field group comprises a source IP address SIP, a destination IP address DIP, a time stamp TS and a data packet effective load length PS in primary communication;

step 2: using the SIP, DIP, TS and PS obtained in the step 1 as characteristics, carrying out sub-graph segmentation on the communication structure chart by using a community discovery algorithm, discovering a P2P structure therein, and obtaining P2P structure flow;

and step 3: traversing key field groups belonging to P2P structure flow, grouping the communication with the same IP addresses of both communication parties into a group, namely, the SIP and DIP of the A data packet are respectively the same as the DIP and SIP of the B data packet, and grouping A and B into a group;

and 4, step 4: traversing each group in the step 3, sorting the data in the group according to a time rule, and dividing the data in the group into sessions based on time;

and 5: calculating a difference value of a session starting timestamp and a session ending timestamp to obtain a session duration time TD, wherein SIP is a total length APS of a data packet effective load sent from A to DIP as B, SIP is a total length BPS of the data packet effective load sent from B to DIP as A, and the number of times NOC of communication between the two parties;

step 6: taking APS, BPS, NOC and TD as a quadruple, and adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network to detect malicious P2P flow, thereby detecting a P2P botnet;

and 7: and (6) storing the detection result of the step (6) into a database.

And 8: reading a detected host IP in a P2P botnet, and acquiring a key field group comprising a source IP address SIP, a destination IP address DIP, a timestamp TS and a packet payload length PS in one communication when detecting that the IP is communicated with other IPs which do not belong to the botnet;

and step 9: tracking the next communication, and calculating to obtain quadruples including APS, BPS, NOC and TD;

step 10: and (4) carrying out similarity comparison on the quadruple acquired in the step (9) and the quadruple of the previous botnet, if the quadruple is malicious traffic, adding a new node into the P2P botnet, and recording the new node in a database.

In the preferred embodiment of the present invention, the recognized benign traffic of 10000 before the Alexa rank and the normal network traffic of the benign P2P network detected by the present system are selected from the filtering rules, which greatly reduces the amount of data to be processed in the subsequent steps.

The invention discloses a P2P botnet detection system and method based on network flow, which centralizes the network flow monitored by different nodes distributed on a network, takes the network flow as input, and takes the detection result of the botnet as output. The invention does not take the ports of both communication parties as characteristics for detection, avoids the failure of a detection method caused by port randomization, performs P2P botnet detection based on flow analysis, improves the detection efficiency by filtering benign network flow, does not take the specific effective load of a flow packet as the characteristics, but selects the length of the flow packet as the characteristics, reduces the data processing capacity, and can efficiently identify P2P botnet communication. Meanwhile, the invention can further track based on the existing botnet result of the system, thereby realizing the characteristic that the detection result can be expanded. The final detection result can enable the P2P botnet communication behavior to be recognized, and can provide support for some intrusion detection systems.

The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.

Drawings

FIG. 1 is a system diagram of a preferred embodiment of the present invention;

FIG. 2 is a flow chart of a network traffic receiving module in accordance with a preferred embodiment of the present invention;

FIG. 3 is a process of detecting a P2P botnet in the detection algorithm module of a preferred embodiment of the present invention;

FIG. 4 is a diagram of the P2P botnet tracking process in the detection algorithm module in accordance with a preferred embodiment of the present invention;

FIG. 5 is a flow chart of a preferred embodiment of the present invention.

Detailed Description

The technical contents of the preferred embodiments of the present invention will be more clearly and easily understood by referring to the drawings attached to the specification. The present invention may be embodied in many different forms of embodiments and the scope of the invention is not limited to the embodiments set forth herein.

In the drawings, structurally identical elements are represented by like reference numerals, and structurally or functionally similar elements are represented by like reference numerals throughout the several views. The size and thickness of each component shown in the drawings are arbitrarily illustrated, and the present invention is not limited to the size and thickness of each component. The thickness of the components may be exaggerated where appropriate in the figures to improve clarity.

As shown in fig. 1, the system is composed of four modules, which are a network traffic receiving module, a communication structure diagram constructing module, a detection algorithm module, and a database module.

The system sets a monitoring program at each peer node to collect network traffic. The collected flow enters a network flow receiving module, benign flow is filtered out according to a filtering rule, and the remaining uncertain network flow is stored in a database module;

the communication structure chart building module reads the uncertain flow records, and builds a weighted directed communication structure chart by taking the monitored IP addresses as nodes and the communication between the nodes as edges;

the detection algorithm module analyzes and explores the generated communication structure diagram, judges whether the IP address of the node in the communication structure diagram is a botnet host or not and tracks the botnet again;

the database module stores the calculation results of the network flow receiving module, the communication structure diagram constructing module and the detection algorithm module, and comprises the steps of storing the filtered network flow, the monitored benign P2P network structure, the P2P botnet structure and the characteristics of the botnet structure;

as shown in fig. 2, the flow of the network traffic receiving module is as follows:

the network traffic receiving module acquires network traffic collected by using NETFLOW from monitoring points of peer nodes in a traversing manner;

judging whether the traffic belongs to recognized benign traffic ranked 10000 before Alexa ranking, and if so, discarding the network traffic;

if not, the next step is carried out, whether the network traffic belongs to benign P2P network traffic which is detected by the system is judged, and if the network traffic belongs to benign P2P network traffic which is detected by the system, the network traffic is discarded;

if not, storing the flow into a database;

and continuously judging the next network flow until the traversal is completed.

Through the screening of the network flow receiving module, the network flow needing further judgment is greatly reduced.

As shown in fig. 3, the P2P botnet detection process in the detection algorithm module is as follows:

the detection algorithm module calculates the communication similarity between the nodes in the graph and other connected nodes according to the communication structure graph constructed by the communication structure graph construction module, wherein the communication similarity is calculated by adopting a time stamp TS and a data packet effective load length PS. The calculated node communication similarity is used as a weight of an edge, a community discovery algorithm is used for carrying out subgraph segmentation, and a P2P structural subgraph is found;

integrating communications between the nodes into a session between the nodes;

calculating values of APS, BPS, NOC and TD by using TS and PS values in communication, the number of communication times in conversation and the number of exchange packets, taking the values as characteristics, and detecting malicious P2P traffic by adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network;

the benign P2P network is placed into the filtering rules of the traffic receiving module and the P2P botnet detection results are stored into the database.

As shown in fig. 4, the P2P botnet tracking process in the detection algorithm module is as follows:

the P2P mode botnet tracking process is established on the basis of the P2P botnets detected by the system, and as time goes on, the botnets may infect more and more hosts, the scales of the botnets become larger and larger, and the problem of expansibility can be well solved by the tracking process of the system.

Firstly, acquiring communication traffic of nodes in a botnet and other nodes which do not belong to the botnet;

tracking the communication between the two nodes and establishing a session;

calculating the values of APS, BPS, NOC and TD, and judging the similarity with the conversation characteristics of the botnet;

if so, it is added to the previous P2P botnet structure and recorded.

As shown in fig. 5, the steps of the P2P botnet detection method based on network traffic are as follows:

step S1: for a known communication structure diagram, acquiring a key field group of related flow in the communication structure diagram, wherein the key field group comprises a source IP address SIP, a destination IP address DIP, a time stamp TS and a data packet effective load length PS in primary communication;

step S2: using the SIP, DIP, TS and PS obtained in the step S1 as features, carrying out sub-graph segmentation on the communication structure chart by using a community discovery algorithm, discovering a P2P structure therein, and obtaining P2P structure flow;

step S3: traversing key field groups belonging to P2P structure flow, grouping the communication with the same IP addresses of both communication parties into a group, namely, the SIP and DIP of the A data packet are respectively the same as the DIP and SIP of the B data packet, and grouping A and B into a group;

step S4: traversing each group in the step S3, sorting the data in the group according to a time rule, and dividing the data in the group into sessions based on time;

step S5: calculating a difference value of a session starting timestamp and a session ending timestamp to obtain a session duration time TD, wherein SIP is a total length APS of a data packet effective load sent from A to DIP as B, SIP is a total length BPS of the data packet effective load sent from B to DIP as A, and the number of times NOC of communication between the two parties;

step S6: taking (APS, BPS, NOC, TD) as a quadruple, and adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network to detect malicious P2P flow, thereby detecting a P2P botnet;

step S7: storing the detection result of the step S6 in a database;

step S8: reading a detected host IP in a P2P botnet, and acquiring a key field group comprising a source IP address SIP, a destination IP address DIP, a timestamp TS and a packet payload length PS in one communication when detecting that the IP is communicated with other IPs which do not belong to the botnet;

step S9: tracking the next communication and calculating to obtain a quadruple containing (APS, BPS, NOC, TD);

step S10: and (4) carrying out similarity comparison on the quadruple acquired in the step (S9) and the quadruple of the previous botnet, if the quadruple is malicious traffic, adding a new node into the P2P botnet, and recording the new node in a database.

The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.

Claims

1. The utility model provides a P2P botnet detecting system based on flow analysis which characterized in that, includes network flow receiving module, communication structure picture construction module, detection algorithm module, database module, wherein:

the network flow receiving module acquires network flow from programs distributed at different monitoring points, filters out benign network flow according to a filtering rule and records uncertain flow in the database module;

the communication structure chart constructing module reads the uncertain flow records and constructs a communication structure chart;

the detection algorithm module analyzes and explores the communication structure diagram and judges whether the IP address of the node in the communication structure diagram is a botnet host or not;

the database module stores the calculation results of the network flow receiving module, the communication structure diagram constructing module and the detection algorithm module;

the filtering rules comprise recognized benign network addresses;

the filtering rules comprise benign P2P network addresses detected by the system;

the communication structure diagram building module is configured to: constructing a weighted directed graph by taking the monitored IP addresses as nodes and the communication among the nodes as edges, and performing subgraph segmentation by utilizing a community discovery algorithm to find a P2P structural subgraph;

the detection algorithm module is configured to: integrating the communication among the nodes of the communication structure chart into a session among the nodes, calculating the difference value of a session starting timestamp and a session ending timestamp to obtain a session duration time TD, a total effective load length APS of a data packet sent from a source IP to a target IP as B, a total effective load length BPS of the data packet sent from the source IP as B to the target IP as A, and communication times NOC between the two parties, taking the APS, the BPS, the NOC and the TD as characteristics, detecting malicious P2P traffic by adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network, adding a benign P2P network into the filtering rules of the traffic receiving module, and storing the detection result of the P2P botnet into the database module.

2. The traffic analysis-based P2P botnet detection system of claim 1, wherein the weighted directed graph is capable of being derived by computing communication similarities of the node with other connected nodes.

3. The traffic analysis-based P2P botnet detection system of claim 2, wherein the communication similarity is calculated using a timestamp TS, a packet payload length PS.

4. The traffic analysis-based P2P botnet detection system of claim 1, wherein the detection algorithm module is configured to: when detecting that the nodes in the botnet communicate with other nodes which do not belong to the botnet, tracking the conversation between the two nodes, calculating the values of the APS, the BPS, the NOC and the TD and the conversation characteristics of the botnet to carry out similarity judgment, and if the values are similar, adding the values into the previous P2P botnet structure and recording the similarity.

5. A P2P botnet detection method based on traffic analysis is characterized by comprising the following steps:

step 2: using the SIP, DIP, TS and PS obtained in the step 1 as features, carrying out sub-graph segmentation on the communication structure diagram by using a community discovery algorithm, discovering a P2P structure therein, and obtaining P2P structure flow;

and step 3: traversing key field groups belonging to the P2P structure flow, and grouping the communication with the same IP addresses of both communication parties into a group;

and 4, step 4: traversing each group in the step 3, sorting the group internal data according to a time rule, and dividing the group internal data into sessions based on time;

and 5: calculating the difference value between the session starting timestamp and the session ending timestamp in the step 4 to obtain the duration time TD of the session, wherein SIP is the total length APS of the effective load of the data packet sent from A to DIP as B, SIP is the total length BPS of the effective load of the data packet sent from B to DIP as A, and the number of times NOC of communication between the two parties;

step 6: taking the APS, the BPS and the NOCTD as a quadruple, and adopting an integrated classifier combining two machine learning methods of a decision tree and a Bayesian network to detect malicious P2P flow so as to detect a P2P botnet;

and 7: storing the detection result of the step 6 into a database;

and 8: reading the detected host IP in the P2P botnet, and acquiring a key field group comprising a source IP address SIP, a destination IP address DIP, a timestamp TS and a packet payload length PS in one communication when the IP is detected to be communicated with other IPs which do not belong to the botnet;

and step 9: tracking the next communication, and calculating to obtain a quadruple containing the APS, the BPS, the NOC and the TD;

step 10: and (3) carrying out similarity comparison on the quadruple acquired in the step (9) and the quadruple of the prior botnet, and if the quadruple is malicious traffic, adding a new node into the P2P botnet and recording the new node in a database.