RU2480937C2 - System and method of reducing false responses when detecting network attack - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: system comprises the following modules: a control module for storing statistics on previous network attacks for correcting filtering rules for filtering centres; collectors for compiling filtering rules based on traffic information from the filter centres and sensors; filter centres for filtering traffic based on filtering rules; sensors for aggregating traffic information for further transmission to collectors.

EFFECT: reduced false responses when detecting network attack.

14 cl, 6 dwg

Description

Technical field

The invention relates to systems for determining distributed network attacks, and more specifically, to reduce false positives when determining a network attack by analyzing the behavioral interaction of users with a protected resource.

State of the art

Currently, almost all companies and organizations, one way or another, are represented on the Internet, and many use the Internet as a business tool. At the same time, the Internet does not provide adequate protection for default services. In addition, today it is impossible to protect yourself from a number of Internet threats with personal protective equipment, such as firewalls, anti-virus systems, antiviruses, etc. A vivid example of such threats are DDoS attacks.

A DoS attack presents an attack on a computer system in order to bring it to failure, that is, the creation of conditions under which legitimate (legitimate) users of the system cannot gain access to the resources provided by the system (servers), or this access is difficult. The motives for such attacks can vary widely - they can serve as elements of competition, a means of extortion, revenge, expressing discontent, demonstrating opportunities and attracting attention, which is most often interpreted as cyber terrorism. If the attack is carried out simultaneously from a large number of computers, they speak of a DDoS attack (from the English Distributed Denial of Service, a distributed denial of service attack). There are two types of DDoS attacks: bandwidth attacks and attacks on applications.

Bandwidth attacks - an attacker acts by filling up communication channels, allocated bandwidths and equipment with a large number of packets. Routers, servers, and firewalls selected as a victim, each of which has only limited processing resources, may become inaccessible to process correct transactions under the action of an attack or fail under heavy load. The most common form of a bandwidth-filled attack is a packet-based avalanche attack in which a large number of apparently reliable packets of TCP, User Datagram Protocol (UDP), or Internet Message Control Protocol (ICMP) are routed to a specific point.

Attacks on applications — an attacker exploiting the features of the behavior of computer interaction protocols (TCP, HTTP, etc.), as well as the behavior of services and applications, captures the computing resources of the computer on which the attacked object operates, which prevents the latter from processing legitimate transactions and requests . Examples of attacks on applications are attacks with half-open HTTP connections and erroneous HTTP connections. More details about such attacks can be found in the article on the Cisco website http://www.cisco.com/web/RU/products/ps5887/products_white_paper0900aecd80 lle927.html.

As a rule, a DDoS attack is carried out using a botnet, which is also called a network of “zombie” computers. A botnet (also a botnet) is a network of infected computers infected with a malicious program that allows you to remotely manage infected computers without the knowledge of their users. Programs that allow you to perform such actions are called bots.

Figure 1 shows the algorithm of such attacks. From the botnet master computer 100, signals are sent to botnet control computers 110, each of which tracks a significantly larger number of computers 120 on which the bots are installed. The use of botnet management computers 110 greatly complicates the definition of a botnet master computer 100, while also increasing the potential size of the botnet to millions of machines or more. Further, bots on computers 120 initiate a DDoS attack on service 130. By service is meant any web service that provides users with certain services or resources. Examples of services include an online store or a file server. From the outside it may seem that, for example, at a certain point in time, the online store experiences an influx of buyers, with whom it is unable to cope. In a DDoS attack on service 130, both its owners and its potential customers suffer in the first place.

It should be noted that to solve the problem of DDoS attacks, various technologies are used, for example, using the collection of statistics of collected data to build a user profile. Collecting such statistics can be used to separate potential bots. In the application US 20090031244 describes a method of collecting statistics when transmitting data by the user. In the application US 20080010247 describes a system for collecting information about user actions and drawing up the final profile using a set of rules. EP 2109282 describes the possibility of constructing a histogram of user requests (as a profile) for further actions to cut off traffic.

After determining the operation of potential bots, a botnet can be determined on the basis of identifying the specifics of their requests or identifying anomalies in traffic. For example, in applications and patents WO 06039529, US 20090037592, WO 06039529, US 7478429 the fact of the operation of bots is determined by identifying duplicate GET requests, incorrect carriage returns, or DNS server overloads. US 7,426,634 describes a method for prohibiting new connections when defining an attack. Possible zombie computers (bots) are identified by their MAC addresses. EP 2109279 defines an attack by comparing the average number of requests from a country with the current number of requests. In patents US 7626940, US 7602731 discusses the option of analyzing DNS queries and identifying anomalies for further counteraction to possible attacks. GB 2393607 describes a method for detecting anomalies and subsequent filtering. In the application US 20080022405, the definition of anomalies is based on finding executable parts in the queries.

One of the methods for solving the problem of DDoS attacks is to clear the traffic directed to the service. For example, in applications and patents WO 06039529, US 20090037592, WO 06039529, US 7478429 describe a system for detecting the congestion status of a server (service) using SYN-Flood attacks. US Pat. No. 7,058,015 describes a system of sensors responsible for monitoring traffic, routers, and a control device that redirects traffic in accordance with defined policies. The method described in the application US 20040064738 involves the use of a proxy server to analyze traffic to the server. In the application US 20030172289 describes a method of controlling the flow of traffic using its redirection and further processing.

Another option to deal with unwanted traffic is routing to black holes. The black hole routing process is used by the service provider to block all traffic addressed to the target at the earliest possible point. Traffic removed from the route is routed to a black hole to protect the provider's network and its other clients. Routing to the “black holes” cannot be called a good solution, because along with malicious traffic, attacks are rejected and trustworthy packets.

It also describes methods and systems that allow you to determine the attack on the application. In the patent US 7272854 there is a definition of the type of application (IM type), which may be a DoS attack. In the application US 20070258438 describes the possibility of detecting a possible attack on a web service. Patents KR 7061017, KR 7077517 describe a method for determining an attack on a web application.

You can also use firewalls to combat DDoS attacks. This has its advantages, since firewalls are in close proximity to the protected resource, which, however, does not save from an attack on the exhaustion of the channel bandwidth. The filtering methods in this case are quite primitive, and if a DDoS attack is carried out using allowed ports and protocols, the firewall in front of it will be completely helpless.

Finally, one of the longest used technologies is the use of black and white lists (hereinafter we will use these words without quotes), which are used to share access as a ban (black list, http://en.wikipedia.org: / wiki / Blacklist (computing)), and for permission (white list, http://en.wikipedia.org/wiki/Whitelist).

However, one of the main problems of existing systems and methods remains the problem of false positives. False positives can disrupt the operation of a web service - for example, by not allowing ordinary visitors to an online store to use its services. Thus, a system is required that can deal with possible DDoS attacks, while at the same time allowing ordinary users to use the capabilities of web services without any problems.

An analysis of the prior art and the possibilities that arise when combining them in one system allows you to get a new result, namely a system to reduce false positives when determining a network attack.

SUMMARY OF THE INVENTION

Thus, the technical result of the present invention is to reduce false positives when determining a network attack by analyzing the behavioral interaction of users with the protected resource.

According to one implementation option, a network traffic filtering system is provided, comprising: a control module connected to collectors, purification centers and sensors and designed to store statistics of previous network attacks to adjust filtering rules for purification centers; the said collectors are associated with treatment centers and sensors and are designed to draw up filtering rules based on traffic information from treatment centers and sensors; said cleaning centers are designed to filter traffic based on filtering rules; said sensors are intended for aggregating traffic information for further transmission to collectors.

In one embodiment, the service is a web service that provides users with certain services or resources.

In another embodiment, the statistics of previous network attacks include: statistics of the average and peak load of the channel during a network attack, information about malicious activity on the Internet, the number of bot networks participating in the network attack, time since the start of the network attack, duration of the network attack, geography network attack.

In yet another embodiment, the control module uses white and black lists of IP addresses to adjust filtering rules.

In one embodiment, white and black lists of IP addresses are set based on behavioral criteria, which include analysis: the number of requests and sessions established from one IP address, the number of requests without confirmation from one IP address, the number of requests of the same data from one IP-addresses, the number of connections without continuing information exchange.

In yet another embodiment, the treatment centers are connected to the main communication channels through channels with high throughput.

In another embodiment, each cleaning center consists of at least a proxy server for redirecting traffic and a filtering router for filtering traffic.

In one embodiment, the sensors are in close proximity to the service.

According to another embodiment, a method for filtering network traffic is provided, comprising the steps of: redirecting traffic to a service to sensors and cleaning centers; process all service requests on sensors with further aggregation of the received information; update the filtering rules on the collectors, using the information received from the sensors; Correct updated filtering rules using the control module based on statistics from previous network attacks; filter traffic at cleaning centers using the specified filtering rules.

In one embodiment, the service is a web service that provides users with certain services or resources.

In another embodiment, the statistics of previous network attacks include: statistics of the average and peak load of the channel during a network attack, information about malicious activity on the Internet, the number of bot networks participating in the network attack, the time from the start of the network attack, the duration of the network attack, geography of network attack.

In another embodiment, the control module uses white and black lists of IP addresses to adjust filtering rules.

In one embodiment, white and black lists of IP addresses are set based on behavioral criteria, which include analysis: the number of requests and sessions established from one IP address, the number of requests without confirmation from one IP address, the number of requests of the same data from one IP-addresses, the number of connections without continuing information exchange.

In yet another embodiment, the treatment centers are connected to the main communication channels through channels with high throughput.

In another embodiment, each cleaning center consists of at least a proxy server for redirecting traffic and a filtering router for filtering traffic.

In one embodiment, the sensors are in close proximity to the service.

Brief Description of the Drawings

Additional objectives, features and advantages of the present invention will be apparent from reading the following description of an embodiment of the invention with reference to the accompanying drawings, in which:

Figure 1 shows the algorithm of the DDoS attack.

Figure 2 shows a simplified diagram of the construction of the present system.

Figure 3 shows a detailed diagram of the present system.

Figure 4 illustrates the use of black and white lists when filtering traffic.

Figure 5 illustrates the way the system works when a DDoS attack is detected.

6 illustrates the use of various levels of aggregation of the obtained data.

Description of Embodiments

The objects and features of the present invention, methods for achieving these objects and features will become apparent by reference to exemplary embodiments. However, the present invention is not limited to the exemplary embodiments disclosed below, it can be embodied in various forms. The essence described in the description is nothing more than the specific details provided to assist the specialist in the field of technology in a comprehensive understanding of the invention, and the present invention is defined only in the scope of the attached claims.

Figure 2 shows a simplified diagram of the construction of the present system. To protect the service 130, cleaning centers 210 are used, which filter traffic - both from computers 120 on which the bots are installed, and from ordinary users 220 who are trying to access the service 130.

To protect against attacks on the bandwidth, the traffic to the service 130 will be redirected through the cleaning centers 210, which should be connected as close as possible to the main communication channels through the channels with high bandwidth. Due to this, it is supposed to significantly distribute traffic without overloading the communication channels leading to the service 130.

To protect against attacks on applications 230, the system will form a model of average traffic to service 130, and then, during the attack, it will cut off spurious traffic based on such a model. Next, spurious traffic will mean traffic generated by bots on computers 120. The legitimate traffic is the data stream from ordinary users 220.

Figure 3 shows a detailed diagram of the present system. It consists of a control module 320, collectors 310, cleaning centers 210 and sensors 330. Let us dwell in more detail on their interaction.

Traffic A to service 130 can go both from computers 120 on which bots are installed, and from ordinary users 220. The traffic itself is duplicated to redirect it to cleaning centers 210 and to sensors 330. On sensors 330, all requests for the service are processed 130 with further aggregation of the information received. As a result, from the sensors 330 to the collector 310, a brief selection of information on all requests is already received. Before considering the main functions of collector 310, it is worth mentioning the second traffic stream that goes to the cleaning centers 210. The cleaning centers 210 themselves are made in the form of two devices - a proxy server 210a and a filtering router 210b. The task of the proxy server 210a is to transfer the filtered traffic B to the service 130 itself. The decision to pass traffic from one computer or another (and this can be either computer 120 with a bot or a regular user 220) is made using a filtering router 210b. The filtering rules are passed to the filtering router 210b from the collector 310. Now we consider each element of the system in more detail.

The control module 320 monitors the health of all other modules (primarily collectors 310), tracking their likely congestion. The control module 320 is able to monitor the channel load statistics (current day, day of the week, month), as well as malicious activity on the Internet, allowing you to build the geography of current attacks and store and collect statistics of previous attacks (number, duration, peak and average loads). Based on this information, for each attack, its descriptive characteristics can be obtained, such as: the number of bot networks involved, time since the start of the attack, geography of the attack. Based on this information, the control module can adjust the filtering rules (the "filtering profile") that are used by the cleaning centers 210. For this purpose, the allowable volumes of transmitted data, the allowable number of packets depending on the protocol, etc. are calculated. The types of parameters considered are given in the table. Also, the control module 320 stores lists of black / white addresses. About them it is worth mentioning separately.

Figure 4 illustrates that when using black and white lists of addresses (or simply black and white lists), they are prevailing over filtering rules. This means that if the computer address is in the white list (check at step 440), then traffic from it will never be blocked (that is, it will not be filtered at step 450), and if it is in the black list (step 420), vice versa - all traffic is blocked (step 430).

White and black lists of addresses can be created manually as the system administrator or automatically based on statistical and behavioral criteria. Examples of the formation and correction of these lists are considered, for example, in US Pat. No. 7,640,589. Under the behavioral criteria, one can consider the analysis of the number of requests and sessions established from one IP address, the number of requests without confirmation from one IP address, the number of requests of the same data from one IP -addresses, the number of connections without continuing information exchange.

The collector 310 performs statistical processing and aggregation of traffic information from the cleaning centers 210, as well as collecting aggregated information from the sensors 330. It is the collector 310 that generalizes the statistics of legitimate traffic (both from the cleaning centers 210 and from the sensors 330) into the so-called a "filtering profile" (hereinafter without quotes), on the basis of which, in the event of an attack by the cleaning center 210, a decision is made to filter out spurious traffic. At the same time, the control module 320 is able to adjust this filtering profile, which avoids false positives.

Cleaning Center 210, as a rule, represents a separate server, connected as close as possible to the main communication channels through channels with high bandwidth. In one embodiment, the cleaning center 210 may consist of a proxy server 210a and a filtering router 210b in order to separate functions to achieve higher efficiency. The proxy server 210a redirects traffic to the service 130. The filtering router 210b makes a decision to pass one traffic or another based on the data transmitted from the collector 310 (i.e., clearing the spurious traffic that the bots generate). Thus, the cleaning center 210 filters traffic A, leaving in the redirected traffic B only legitimate requests from ordinary users 220.

Sensors 330 are located in close proximity to the service 130, from which the traffic is mirrored (shown in Fig. 3 as another traffic arrow A), and they perform statistical processing of traffic information in order to aggregate traffic information to provide it to collector 310.

It is worth noting that the above system can work both during a DDoS attack and outside it (Fig. 5). Outside the attack, the operation of the system is aimed at collecting statistical information and tracking anomalies (step 510). The collection of statistical information is required to create a filtering profile (block 520). If significant deviations of the traffic are detected at step 530 from the profile, the system switches to the mode of counteracting the DDoS attack and starts filtering traffic at steps 540-550. At step 560, it is checked whether the current filtering profile, which is created by the collector 310 and can be supplemented at step 570, is relevant using the control module 320, which has the necessary statistical information on known past attacks. When determining the end of the attack at step 580, the algorithm returns to step 510.

In one embodiment, the filtering profile is constructed in relation to traffic coming from a specific individual user of the resource, and evaluates the parameters of such traffic for compliance with the calculated normal parameters. To detect anomalies, an anomaly detection profile is used, which is built in relation to traffic directed towards the resource and estimates the total parameters of such traffic for compliance with the established threshold values. To build profiles, the same data set is used, which is interpreted differently for both profiles.

The data used to build the profile have different levels of data aggregation, which allow you to analyze input data at different levels.

Aggregation level number Locked Key Fixed values one 1) IP address of the client of the protected resource 1) The number of bytes received 2) IP address of the protected resource 2) The number of bytes sent 3) Protocol / port (service) 3) Number of received packets 4) Timestamp 4) Number of packets sent 5) The number of received packets with the only SYN flag set (for TCP) 2 1) Country code for the IP address of the client of the protected resource 1) The number of bytes received 2) IP address of the protected resource 2) The number of bytes sent 3) Protocol / port (service) 3) Number of received packets 4) Timestamp 4) Number of packets sent 5) The number of received packets with the only SYN flag set (for TCP) 6) The number of unique IP addresses of clients of the protected resource 3 1) IP address of the protected resource 1) The number of bytes received 2) Protocol / port (service) 2) The number of bytes sent 3) Timestamp 3) Number of received packets 4) Number of packets sent 5) The number of received packets with the only SYN flag set (for TCP) 6) The number of unique IP addresses of clients of the protected resource four 1) Resource Group Identifier 1) The number of bytes received 2) Protocol / port (service) 2) The number of bytes sent 3) Timestamp 3) Number of received packets 4) Number of packets sent 5) The number of received packets with the only SYN flag set (for TCP) 6) The number of unique IP addresses of clients of the protected resource 5 1) Customer ID 1) The number of bytes received 2) Protocol / port (service) 2) The number of bytes sent 3) Timestamp 3) Number of received packets 4) Number of packets sent 5) The number of received packets with the only SYN flag set (for TCP) 6) The number of unique IP addresses of clients of the protected resource

6 illustrates the use of different levels of aggregation (in this example, from 3 to 5), which allows you to track data at different levels - starting from the selected client, and going down to certain services. For example, you can track statistics not only for Client 1, but also for one of its HTTP services, such as Site 1.

The anomaly detection profile is a set of threshold values of a certain value S describing normal traffic for one of the aggregation levels (for example, a client or service). A threshold value can be set for each hour of the day and a specific day of the week to prevent possible false alarms. The value of S can be any of the fixed values, such as, for example, the total number of incoming packets or the number of unique IP addresses of users.

The present description sets forth the main inventive concept of the authors, which cannot be limited to those hardware devices that were mentioned earlier. It should be noted that hardware devices are primarily designed to solve a narrow problem. Over time and with the development of technological progress, such a task becomes more complicated or evolves. New tools are emerging that are able to fulfill new requirements. In this sense, these hardware devices should be considered from the point of view of the class of technical problems they solve, and not purely technical implementation on a certain elemental base.

Claims (14)

1. A system for filtering network traffic to protect the service from network attacks, containing:
a) the control module is connected to collectors, cleaning centers and sensors and is designed to store statistics of previous network attacks to adjust filtering rules for cleaning centers;
b) the said collectors are connected to treatment centers and sensors and are designed to draw up filtering rules based on traffic information from treatment centers and sensors;
c) the mentioned cleaning centers are designed to filter traffic based on the filtering rules, while the cleaning centers are connected to the main communication channels through channels with high throughput;
d) the said sensors are designed to aggregate traffic information for further transmission to the collectors. 2. The system according to claim 1, in which the service is a web service that provides users with certain services or resources. 3. The system according to claim 1, in which statistics of previous network attacks include:
a) statistics of average and peak channel load during a network attack,
b) information about malicious activity on the Internet,
c) the number of bot networks involved in a network attack,
d) the time since the start of the network attack,
d) the duration of the network attack,
e) the geography of the network attack. 4. The system according to claim 1, in which the control module uses white and black lists of IP addresses to adjust filtering rules. 5. The system according to claim 4, in which white and black lists of IP addresses are set based on behavioral criteria, which include analysis:
a) the number of requests and sessions established from one IP address,
b) the number of requests without confirmation from one IP address,
c) the number of requests of the same type of data from one IP address,
d) the number of connections without continuing information exchange. 6. The system according to claim 1, in which each cleaning center consists of at least a proxy server for redirecting traffic and a filtering router for filtering traffic. 7. The system according to claim 1, in which the sensors are in the immediate vicinity of the service. 8. A method for filtering network traffic to protect a service from network attacks, comprising the steps of:
(i). redirect traffic to the service to sensors and cleaning centers;
(ii). process all service requests on sensors with further aggregation of the received information;
(iii). update the filtering rules on the collectors using the information received from the sensors;
(iv). Correct updated filtering rules using the control module based on statistics from previous network attacks;
(v). filter traffic at the cleaning centers using the specified filtering rules, while the cleaning centers are connected to the main communication channels through channels with high throughput. 9. The method of claim 8, wherein the service is a web service that provides users with specific services or resources. 10. The method of claim 8, in which the statistics of previous network attacks includes:
g) statistics of the average and peak channel load during a network attack,
h) information about malicious activity on the Internet,
i) the number of bot networks involved in a network attack,
j) time since the start of the network attack,
l) the duration of the network attack,
m) the geography of the network attack. 11. The method of claim 8, in which the control module uses white and black lists of IP addresses to adjust filtering rules. 12. The method according to claim 10, in which the white and black lists of IP addresses are set based on behavioral criteria, which include analysis:
e) the number of requests and sessions established from one IP address,
f) the number of requests without confirmation from one IP address,
g) the number of requests of the same type of data from one IP address,
h) the number of connections without continuing information exchange. 13. The method of claim 8, in which each cleaning center consists of at least a proxy server for redirecting traffic and a filtering router for filtering traffic. 14. The method according to claim 9, in which the sensors are in the immediate vicinity of the service.

Images