PL241005B1 - Method and system of adaptive creation of network traffic filtering rules on a network device, detecting anomalies and automatically fighting DDoS attacks - Google Patents

Description

PL 241 005 B1

Description of the invention

The field of technology

The present invention relates to a method and a system for adaptively creating network traffic filtering rules on a network device. The method automatically detects and suppresses anomalies, especially volumetric attacks (DDoS). The method relates to the use of the existing network infrastructure, after the necessary modifications related to the use of the system according to the invention, to protect this infrastructure against the risks associated with network congestion due to exceeding the physical capacity of data transfer within the infrastructure in which the method and system is used.

This solution is an adaptive (unattended learning) control system that independently and automatically detects, tracks, isolates and suppresses multi-dimensional and time-varying network traffic disturbances caused by volumetric DDoS attacks. The solution immunizes the protected network against disturbances in the packet transmission process by analyzing the signal representations of packet flows in the time and frequency domains, and on this basis carrying out adaptive synthesis of signals controlling their flow rate on an ongoing basis. The control process is carried out by the flow observer system.

Introduction

ICT security is directly related to the technical solutions behind the way of organizing communication in the existing ICT systems, especially in computer networks. The most common threats are related to the deliberate and hostile use of existing protocols and basic vulnerabilities of commonly available network services. These vulnerabilities include, firstly, the naturally occurring asymmetry between the size of the query sent to the server and the size of the response provided by the server, and secondly, the naturally limited performance of stateful protocol handling mechanisms, especially the TCP protocol. The use of commonly known and generally available network mechanisms enables the effective construction of many varied variants of volumetric attacks of the DDoS type (DDoS - Distributed Denial of Service). For this reason, an increase in the number of attacks directed against many targets and using many of their vulnerabilities at the same time is projected, i.e. multidimensional attacks with complex dynamics.

Reports showing the ICT security landscape in the world prove that DDoS attacks are the dominant threat on the market of network services such as telnet, http, https, dns and game server services. The number of attacks grows from year to year, and with it the scale of attacks measured in bits per second (bps) and packets per second (pps) grows sharply. Current data also show the increasing complexity of the dynamics of attacks. The vast majority of DDoS attacks last no longer than 30 minutes, with attacks lasting less than 5 minutes predominating among them. These results clearly define the response time requirements that must be met by attack detection systems and ICT security incident response teams.

The basic tool for carrying out attacks are the networks of so-called bots (botnets), i.e. computers connected to the network and infected with software prepared to carry out a coordinated attack according to a specific scenario. An increase in the advancement of algorithms used by bots to mask presence, coordination and execution of multidimensional attacks should be expected.

The currently used methods of detecting DDoS attacks use signature mechanisms that compare the contents of network packets or statistical characteristics of flows with a library of known attacks patterns. The signatures most often describe:

• statistical distribution of selected packet attributes (from the sample defined by the initial training period), • acceptable source or destination addresses (packet attribute values), • conditions for the correctness of the frame structure or the packet data content (often only after decryption).

Detection of anomalies in the network traffic consists in carrying out a test of matching a packet or set of packets to the pattern or in finding that the network flow exceeds the set threshold values (which define a priori known norm). The network flow is most often identified on the basis of only five parameters of the network packets it consists of: address

The source device address, the target device address, the source device port, the target device port, and the transmission protocol designation. The effectiveness of such solutions depends on the fact that the attack pattern database is up-to-date and the attack-free healthy movement patterns.

Gathering knowledge about these kinds of patterns is of course important and necessary, but it requires a long learning process. Matching the packet content to patterns also requires a lot of computing power and thus reduces system throughput. The signature approach is based on the assumption that the dynamics of the protected and attacking systems is slower than the pattern learning process.

The systems used so far after detecting an attack (signature method) usually redirect the disrupted traffic to a dedicated network or cloud in order to clean it, or completely block the contaminated flow of packets. The disadvantage of this solution is the long response time to the attack, e.g. resulting from the propagation of network routing rules, and the cutting off of normal network connections in dirty traffic. As a result, paradoxically, it may contribute to increasing the effectiveness of a detected attack.

State of the art

There are many known solutions for the detection and suppression of volumetric attacks, but the principle of their operation is significantly different from the principle of the method according to the invention. The available solutions are characterized by the following functional features:

• detection of attacks only on the basis of their known signatures (known patterns), • protection of only selected and defined network addresses, • complete rejection of packets matching the attack pattern, • traffic cleaning carried out outside the customer's network in the service provider's cloud, • traffic requiring filtration is routed to the external traffic cleaning center, the so-called scrubbing center (reaction time depends on the propagation of routing tables), • deep packet inspection (looking at the data stored in the fields of the OSI application layers), • addresses in the network specified and assigned by the operator are protected, • no rules for adaptive control of policer / shaper settings (this degrades the accuracy of link resource allocation or completely prevents flow rate regulation), • no mechanisms for forecasting the speed of isolated flows, • no mechanisms for automatic calculation of safe allocations of link resources to flows, • no signal analysis mechanisms (digital processing of signals representing packet flows) , • carrying out tasks in a strictly defined network environment (no possibility to implement a solution based on selected communication protocols with devices or distribution of filtration rules), • established protocol for writing packet filtration rules.

The solution presented in the application EP3253025 is known from the state of the art. According to this method, the controller collects network traffic statistics from the first network packet relay device. If the global statistics of network traffic flowing to the destination IP network address exceeds the set level, the controller delivers the DDoS preventive policy rule to the second packet forwarding device. Accordingly, the second packet forwarding device performs operations on the traffic flowing to the destination network address according to the predetermined rules according to the predetermined preventive policy sent by the controller. As a result, the impact of the attack is limited and network security is improved. Characteristic for this solution is that the distribution of filtration or routing rules takes place after exceeding the alarm thresholds determined on the basis of traffic statistics, and that the identification of flows directed to the indicated IP address and exceeding the threshold flows is made on the basis of statistical analysis of network traffic data to the indicated IP address.

A more advanced solution is known from the international application number WO17065627. The method according to this application consists in conducting the initial learning phase with supervision, during which classifiers are created on the basis of the training set with full-length network flows, which are then verified on the basis of the training set with limited network flows, which enables the determination of a correction model. Then, in the prediction phase, packets of unclassified network flow are received, which are limited, and further, as a result of classification according to the model built in the training phase on limited flows, and after applying the correction model, classification results are obtained. The results of such a classification

PL 241 005 B1 can be used to isolate flows that are the carrier of network traffic anomalies characteristic, for example, for volumetric attacks.

The disadvantages of this solution are the necessity to perform a relatively long learning phase, and the classification is based on, inter alia, statistical data for the adopted training set. Only in the prediction phase is the classification of network flows observed in real-time network traffic. The analyzed network packet parameters are: source address, destination address, source and destination port as well as the type of transmission protocol and statistics obtained on their basis. The effectiveness of the classification depends on the frequency of updating the prediction model.

Another prior art solution is described in US10122740. The described method is based on the cyclical generation of models and configurations and their updating. The model defines a threshold for at least one of the signals. Detection of an anomaly in the traffic is based on the application of the model to the signaling data or the matching of at least a portion of the signaling data to the traffic anomaly pattern obtained from the centralized analytics server. When an anomaly is detected, the process of limiting and mitigating the anomaly follows. This enables dynamic and adaptive network traffic analysis and anomaly detection, including detection enhancements independent of human intervention.

Anomaly detection is based on the analysis of deviations from the statistical model stored on a central analytical server and based on operator-defined thresholds.

Patent application US8307430 discloses a solution that is based on an analysis of deviations from a given statistical model. If the spectral density of the selected parameter deviates from the uniform distribution, a decision is made to detect an anomaly or a potential attack.

The solution is based on the monitoring of signals with given statistical characteristics, i.e. it is a signature method.

Another known from the prior art is the solution disclosed in the document No. US2018152474. One or more performance indicators associated with one or more thresholds to be configured are user-defined. Based on historical information on the flow of network traffic related to previously detected malicious activity, characteristic values for one or more performance indicators are identified. The threshold values are automatically configured based on the identified characteristic values.

It is a solution based on blocking an attack detected by the signature method. Attack suppression parameters as well as alarm thresholds are entered by the operator.

In the prior art, document US2018241762A can also be found, which describes a solution for detecting an anomaly in network traffic based on a measure of the difference between a new anomaly and known anomalies.

Anomaly detection based on known patterns classifies this solution as based on a signature model. The classification and isolation of flows is approved by the operator, who is also responsible for initiating the processes of protection of network resources against detected anomalies.

Moreover, the solution presented in the application US8234401 is known from the state of the art. It consists in adapting the so-called policers to the level of use of the monitored network resource. The flow of network packets observed by the corresponding policer induces the level of utilization of the monitored network resource.

There is a proportional control rule using the feedback signal in the form of the load on the selected resource. The solution uses one model of policer operation regulation and does not allow shaping the spectrum of network traffic or forecasting the value of the policer output signal. The original value of the monitor resource usage level must be set by the operator.

US7120931 discloses a solution for a method of generating filters based on data incoming to a network device. In this solution, network data incoming to the device is divided into multiple network flows. Then, aggregated summaries of the network flows are created for each of the distributed network flows. Aggregated summaries of network flows are sent to the flow analyzer, where they are analyzed for the characteristics of potentially harmful network flows. The result of the analysis is the generation or tuning of a network filter for the analyzed aggregated network flow summary, which prevents potentially harmful flows from passing through the network device.

PL 241 005 Β1

The flow analysis is carried out on the basis of statistics of aggregated data on network packet attributes included in the flow samples. The filtration resolution is increased on the basis of statistical analysis after detection of anomalies in selected observed subflows. According to the cited document, the network filter is understood to mean both the separation of the network flow (under the flow) and the control of its dynamics, i.e. the suppression of packets belonging to the selected packet flow.

The solutions known from the state of the art usually divide the analyzed network traffic into flows on the basis of network packet parameters such as: source address, destination address, source port and destination port, type of transmission protocol, or statistics obtained on their basis. However, this limitation is not reflected in theoretical knowledge of network engineering.

The solutions known from the state of the art do not guarantee effective detection and suppression of network traffic anomalies, especially DDoS attacks. The desired solution must be able to rapidly track the ever-changing sources of anomalies or the changing attack vectors. The solution should be able to take actions anticipating attack scenarios in a safe and reliable manner. In addition, the system should strengthen the network operator's ability to respond to the observed events within several seconds, e.g. by preparing effective means of influencing the attack, and providing a properly filtered data set (hiding the technical complexity of the system).

The technical problem posed was solved thanks to the use of formal concepts of the theory of predicate calculus and inductive reasoning (machine learning), set theory, control theory and signal processing, and terminology derived from selected technical standards.

The formal definitions and glossary of terms used are presented below.

A network packet is called an OSI telecommunications protocol data unit (PDU). The field values included in the protocol headers and the parameters characterizing the packet transmission will together form the inference domain on which the knowledge building processes are carried out. The list of network packet parameters observed in the system depends on the adopted packet sampling standard (P. Phaal, S. Panchen, N. McKee, InMon Corporation's sFlow: A Method for Monitoring Traffic in Switched and Routed Networks, 2001, IETF, RFC 3176; B Claise, Ed., Cisco Systems NetFlow Services Export Version 9, 2004, IETF, RFC 3954).

Let Fi, i = 1, ..., 11, denote the set of numbers representing the contents of the available header fields of the network protocols and the parameters characterizing the network packet. The Cartesian product F = F _t xf ₂ x ... xp _n will be called the packet parameter domain (inference domain).

A subset of the packet parameter domain, ^F o ^c F, reconstructed from the set of observed network packets constitutes network traffic.

A function of the form ^{F i =} i, -, n assigning a number from the set Fi to the packet P ^EF will be called the packet attribute.

The subset of Si c Ft, i = 1, ..., n corresponding to the attribute ai will be called a selector.

Cartesian product of selectors T = χ ... χ s _n defines the term. A term consisting of a single selector ^s t = defined by a single-element set and a Cartesian product 7 (0 = Fi x ... x W, x ... x F _n defines the base term.

ΤΛ7 __ I I TM ψϊ

For any term defined by the sum of Cartesian products selekto ₌ CL P e W.

ditch, the pointer function ^w (0, pg W, specifies the packet filter.

The set of observed packets satisfying the conditions defined by term W, F [vr] = (pe F _o | 5 _W (p) = 1} cf defines the packet flow.

Function of the form: xe-> n specified for any subset of network traffic packets

W ^c ^ (Foloraz of the parameter Θ e Θ. Determines the numerical characteristic of the flow.

The basic numerical characteristics of the sets of packets observed by a given filter determine the packet flow rate expressed in bits or packets per second.

Let h ε IR, h> 0 denote the time interval in which packet samples are collected. Consider the selector = (th.t] ^c F _k which selects the packets observed in the sampling period preceding the time t ε K and term m

W (t) = X ... X sjt) X ... χ s ^)

1 = 1

PL 241 005 Β1

If U is a set of configuration parameter values for packet rate limiters (policers, shapers) of a network device (firewallI), then the control rule corresponding to the term W assigns the vector of numerical flow characteristics configuration parameters of the network device. The control algorithm calculates the value of the control rule.

Let 0 (^^) = ^e denote the vector of numerical flow characteristics P [W1. The filter rule assigns the control rule to packets belonging to the specified flow: Wemh M ^ W])) ^{r J} if h ICC

The flow dynamics model is a sequence of functions describing the time evolution of the numerical flow characteristics according to the following system of difference equations:

x (t + 1) = fX (t), u (c), v (i) | O) Xt _o ) = y (t) = βζΟΙθ) f = t ₀ , t ₀ + l, t ₀ + 2, ...

where x (t) denotes the (abstractly understood) flow state, u (t) value of the control signal, v (t) random disturbances in the flow state, y (t) observed numerical flow characteristics, e (t) random measurement disturbances, and θ vector identified in the process of adaptation of model parameters.

According to the adopted concept, the system can be treated as a learning (adaptive) multi-layer network packet flow control system resistant to disturbances (failures / attacks / anomalies / irregularities) caused by a selected class of phenomena (DDoS attacks), equipped with diagnostic mechanisms identifying and isolating sources of anomalies , acquiring (generating) knowledge (declarative and procedural) about the observed phenomena.

The essence of the invention

According to the invention, a method of adaptively creating network traffic filtering rules on a network device that detects anomalies by itself, and automatically suppresses volumetric attacks (DDoS), which on at least one network device, based on actual network flows and after separating them into isolated packet flows, recognizes potentially malicious network flows, and then configures or fine-tunes network filters and packet discipline measures whose filtering rules can be propagated to other network devices, and selects isolated packet flows associated with at least one configured or tuned network filter for further analysis.

The method is carried out cyclically in given sampling periods, and consists of the following steps:

a) From the input interface and the output interface, packets are sampled from actual network flows on at least one network device with at least one network traffic sampler and the packet samples are stored in a sample buffer.

b) The Do detector, based on the packet samples from the sample buffer, determines the packet flow isolation network filter definitions in memory.

c) Each isolated packet flow in this way has thresholds determined in the flow observer system and these are stored in memory.

d) Then, for each isolated packet flow, the predicted characteristics of the flows in the predetermined forecast horizon are determined in the automatic flow control system or in the flow observer system and stored in the memory, each new isolated packet flow being predicted with predetermined values.

e) In addition, for each isolated packet flow, it is determined in the flow observer system whether the previous threshold values determined in the previous sampling period have been exceeded by comparing the parameters of the isolated packet flow with the previous threshold values stored in the memory, and if the previous threshold values are exceeded, signals are generated. alarms saved in memory.

PL 241 005 B1

f) The actual network flows that correspond to isolated packet flows that do not meet the threshold values are subjected to a process of automatic regulation and dynamics shaping using the automatic flow control system and packet discipline measures and network filters.

Preferably, in step b) the packet flows are isolated based on any of the available attributes of the packet sample stored in the sample buffer and the network filter definitions are stored in memory.

The process of regulating and shaping the dynamics consists, in particular, in the determination of the new signals controlling the packet disciplining means on the basis of the current control signals and on the basis of the adaptive control method.

Preferably, in step f), on the basis of the packet samples from the input interface and the output interface taken from the sample buffer, the threshold values, the alarm signals, the predicted flow characteristics, the definition of the network filters and the current control signals, new signals controlling the disciplining means are determined by adaptive control. packets that are written to memory.

Preferably, step f) is followed by step g), in which, for isolated packet flows processed in the automatic flow control system based on the control signals, filtering rules are created which are performed on at least one network device by means of network filters or packet discipline means.

Preferably, in step g), the automatic flow control system propagates the created filtration rules to other network devices.

Preferably, by means of the detector D0 in step b) the network filter definitions are determined by:

determination of at least one ordered list corresponding to the selected attribute of packets in the set of packet samples available in the sample buffer sorted in descending order due to the selected flow parameters, which constitute the given evaluation criteria, for each packet attribute a collective list obtained by aggregating ordered lists is determined and saved is in the detector's memory, the base term is created for each attribute of the packet from the collective lists and stored in the memory, the activity counter is increased for each base term stored in the memory, the activity counter value is checked in the memory, if at least one counter exceeds the set value activity, a network filter definition is created that corresponds to the base term, the counter of which has exceeded the activity value.

Preferably, detector D0 is run cyclically at predetermined sampling periods (o1).

Preferably, the given evaluation criteria are the number of bytes, the number of packets, the number of bits per second (bps), the number of packets per second (pps), the number of logical flows (flows) or the average number of bytes in the packet (bpp) recorded in the given period. sampling.

Preferably, in steps c), d), e), f) the observer circuit or the automatic flow control circuit creates time series corresponding to the isolated packet flows by stored network filter definitions describing the changes in packet attributes over time, each the time series contains information about changes to any available attribute of the packet samples stored in the packet buffer during a predetermined sampling period and is stored in memory and is used to determine the flow characteristics and in steps c), d), e) and f).

Preferably, step b) is followed by step b1), in which the packet flow isolated by detector D0 is divided, by the first order detector D1, into packet subflows such that: for each isolated packet flow, the corresponding time series are taken from the memory, time series separated due to a given feature are selected and a combination of separate time series corresponding to packet flows isolated in the Do detector (7) is determined, on the basis of the combination of separated time series, terms composed of base terms are determined and the corresponding subflows of packets in samples are searched for packets written to the sample buffer;

if the composite term packet subflow exists in the packet sample samples stored in the buffer, a new network filter definition is created, which isolates the identified packet flow, and stored in memory.

PL 241 005 B1

Preferably, the selection of at least two separated time series takes place due to the maximum linear (orthogonal) independence or due to the minimum correlation (stochastic independence) in the signal space of the analyzed at least two time series in the time or frequency domain.

Preferably, step b1) is followed by step b2) in which the isolated packet flows are split, by a higher order detector Di, based on a combination of downstream filter definitions to form new higher order network filter definitions that isolate further sub-packet flows:

for each isolated packet flow, corresponding time series are downloaded from the memory, correlated time series or similar are selected according to another given criterion, a new term is built consisting of the product of terms building the definitions of network filters isolating packet flows to which the detected correlated time series or similar correspond time series, and then the search for the corresponding new complex term is the packet flows in the packet samples stored in the sample buffer, if a complex term subflow exists in the collected packet samples, a network filter definition is created that isolates the identified packet flow and is stored in memory.

Preferably, at least two time series are qualified as similar because of their correlation with each other or because of their similarity in the shape of the waveform.

Preferably, in step g), suitably suppressing the network data packet flow isolated by the network data packet by the packet discipline means comprises discarding the packet.

Preferably, in step g), suitably suppressing the network data packet flow isolated by the network data packet by means of packet discipline comprises buffering the packet.

Preferably, the individual steps of the method are performed on different network devices or with at least one computer connected to the network via a suitable network interface.

In the method according to the invention, the state of the flow is strictly determined by the values of the network filter definitions stored in the memory and on the basis of the current control signals (16), the values of alarm signals or their absence, threshold values and forecast flow characteristics and is controlled by the observer system, starting from flow isolation state by means of an appropriate detector, through the state of flow observation by means of the observer system, to the state of flow control operated by the automatic flow control system, or to the state of flow termination, in the flow observer system, each of the mentioned flow states corresponds to a specific value of the rules filtering implemented on the network device by means of network filters and appropriate packet discipline measures.

Adaptive network traffic filtering rule-making for at least one network device to self-detect anomalies and automatically suppress volumetric attacks (DDoS), including memory, processor or controller, and at least one interface, characterized in that it includes a network traffic sampling system for the input network interface and the output network interface network interface performing the function of reading and collecting packet samples of the actual network flow and writing them to a dedicated sample buffer connected to a detector circuit (Do) that isolates packet flows by reading packet samples stored in the buffer and generating network filter definitions and saving them in memory. The memory is connected to the observer circuit, which reads from the memory information about the network filter definitions and the corresponding packet samples stored in the sample buffer, and then determines the threshold values and stores them in the memory.

The observer system determines whether the predetermined threshold values have been exceeded by comparing the isolated packet flow parameters with the previous threshold values stored in the memory, and if the previous threshold values have been exceeded, alarm signals are generated and stored in the memory. Moreover, the observer system predicts the characteristics of the flows at the given point

The forecast horizon and stores them in the memory. At the same time, the memory is connected to the automatic flow control system regulating and shaping the dynamics of real network flows that correspond to isolated packet flows that do not meet the set threshold values, controlling the means of packet discipline and network filters. At the same time, the automatic flow control system is connected to at least one other network device to which it transmits the filtering rules created on the basis of the network filter definitions read from the memory and signals controlling packet disciplining means. Network filter definitions and packet discipline control signals are translated into packet filtering rules and sent to packet discipline means and network filters, thus isolating network traffic packet flows and shaping their dynamics in the protected network area.

Preferably, the automatic flow control system reads from the memory the alarm signals (forecast and observed) flow characteristics, network filter definitions, current signals controlling the packet discipline means and retrieves the threshold values. The automatic flow control is also connected to the sample buffer, from which it reads packet samples from the input interface and the output interface, and adjusts adaptively by determining new signals controlling the packet disciplining means for the respective isolated packet flows and storing them in memory.

Preferably, between the sample buffer and the flow observer system, there is a first order detector system D1 connected in parallel with detector D0, which is connected to the memory.

Preferably, between the sample buffer and the flow observer system, there is a higher order detector system Di connected in parallel with detector D0, which is connected to the memory.

Preferably, between the sample buffer and the flow observer system, there is a first order detector system D1 connected in series downstream of detector D0, which is connected to the memory.

Preferably, between the sample buffer and the flow observer system, there is a higher order detector system D1 connected in series with the first order detector D1, which is connected to the memory.

Preferably, the observer circuit, based on the definition of the line filters, the alarm signals and the determined threshold values, triggers a first-order detector D1 or higher D1, which isolates new subflows and stores new network filter definitions.

Preferably, the observer system based on the definition of the network filters, the alarm signals and the determined thresholds is configured to activate the automatic flow control system which performs the dynamics control and shaping function indicated by the isolated packet flow observer system.

Preferably, the automatic flow control system is configured to communicate with other network devices.

Advantageously, the individual components of the circuit can be implemented as special purpose integrated circuits (ASICs) or as single integrated circuit modules or as programmable logic gates (FPGAs) or as a computer consisting of at least a processor, memory, mass memory and a suitable network interface. connected to at least one network device.

The invention also relates to a computer program product with program coding means which are stored on a computer readable storage medium for performing the inventive method when the computer program product is executed on a computer.

Advantageous Effects of the Invention

The developed solution is an adaptive (learning in unattended mode) control system that independently and automatically detects, tracks and suppresses non-stationary and multidimensional disturbances in network traffic caused by volumetric DDoS attacks. The solution autonomously detects packet flows posing a potential threat to the protected network, isolates them with a set of automatically generated and distributed network filters with increasing resolution over time, identifies models of dynamics of signal representations of observed packet flows, and on the basis of the identified dynamics models it generates and adapts control algorithms shaping the dynamics of isolated flows, it also calculates the safe share of isolated flows in the capacity of the protected network. The control process is carried out by a specially developed mechanism for managing the state of network flows.

PL 241 005 B1

The innovativeness of the developed solution to the problem of fast detection and effective suppression of volumetric DDoS attacks with variable characteristics consists in reducing this issue to the problem of signal analysis and predictive adaptive control.

In this approach, packet flows are converted into appropriately defined signals (time series), which are treated as observed outputs of the process subject to control (the process of supervised transmission of network packets). Packet flows to be monitored are selected on the basis of autonomously performed detection of signals that stand out from the network traffic. This task requires neither a database of known attacks, nor knowledge of the typical characteristics of the monitored traffic. The invented system, thanks to the use of signal analysis techniques, is able to independently notice signals that require supervision and detect by which packet flows these signals are generated. As a result of this analysis, network filters are generated that allow to isolate a set of packets that require supervision from the network traffic. The accuracy of the filters is constantly improved (using signal analysis techniques), which also increases the accuracy of isolating potential attacks. The invented method of increasing the filter resolution consists in solving a properly defined signal separation and clustering task, examining selected signal features seen in the time and frequency domains.

The purpose of the system is to maintain isolated signal flows (i.e. packet flows) at a safe level and to shape their dynamics in a way that guarantees the security of the protected network (preventing overloading of links and network services). The packet flows (signals) observed in the network, however, have very complex dynamics of a random and time-varying nature. DDoS attacks, both known and unknown, are characterized by such dynamics. Additionally, the mechanisms of filtering and queuing packets on network devices are characterized by complex dynamics. In each of the above cases, the dynamics models are not known a priori. Therefore, in order to achieve the assumed control objective, the developed invention learns the model of the observed motion on an ongoing basis and very quickly (with a sampling frequency of single seconds), and also selects the principles of its operation (control) to the observed situation. The learning process consists in the identification of the dynamics models of the isolated signals (i.e. with the predicted flow characteristics) and in the synthesis of the control signals adjusted to the detected dynamics. Each network filter generated automatically by system detectors, isolating packet flows, is related to a traffic discipline mechanism in the form of packet discipline measures, e.g. policer, and an adaptive control algorithm (automatic flow control system) with its settings. It should be emphasized that the use of the traffic discipline mechanism alone, without the control system for its settings, is not enough to achieve the assumed goal of network protection. The algorithm of adaptive control of the settings of the discipline mechanism allows to precisely regulate the flow rate of packets coming from the associated network filter. Such control allows for effective regulation and shaping of the dynamics of supervised flows. The essence of the regulation is the precise limitation of the rates of indicated packet flows. The essence of shaping the dynamics is to let flows with appropriate spectral characteristics to the protected network, i.e. free from, for example, too sudden jumps in the flow rate (causing saturation of queues or an increase in the variance of response times of network services). Only this type of control allows for accurate management of the use of free network link resources, and as a result, effective protection of the network against overload. It should also be noted that the tide dynamics models used in this context allow:

• making accurate short-term forecasts, • detection of anomalies difficult to notice in long-term statistics (eg related to the processes of coordinated communication or periodic characteristics of algorithmically generated traffic), • synthesis of control algorithms adjusting to the a priori unknown characteristics of network traffic and network devices.

Thanks to the applied approach, the invented system does not require a set of training data or a long pre-learning process. It is able to work from the moment of commissioning. None of the solutions available on the market exhibit the features of a system designed on the basis of the recommendations of the control theory, signal theory and the features of the unsupervised learning system, distinguishing the invented system. In particular, contrary to known solutions, the developed system enables:

• ensuring the resistance of the protected network to sudden changes in the dynamic characteristics of attacks, • protection against volumetric attacks with unknown / non-existent signatures, • maintaining high availability of services in the presence of large-scale volumetric attacks (including wide area networks), • quick identification, isolation and suppression of flows saturating protected lines, • unattended operation, the system independently selects flows requiring monitoring, • forecasting volumetric attacks, • extracting knowledge about new attacks based on frequency analysis and analysis of dynamics models, • identification of the dynamics model of traffic disciplining systems (packet disciplining measures) by network devices, • optimal automation of network traffic data processing, • minimization of response time to an attack, data processing time, calculations and configuration changes of network devices, • accurate and stable speed regulation of packet inflow under conditions of uncertainty, • obtaining system performance results with guaranteed quality indicators related to the effectiveness of attack detection and the effectiveness of shaping the dynamics of network flows, • increasing the resistance of protected services to attack by selective rejection of packets and shaping the dynamics of their transmission, • cleaning of traffic inside the client's network or directly on the client device, • privacy protection thanks to the inspection of only L2-L4 layers, • reduction of costs by local cleaning of traffic and extending the functionality of client devices installed in the network, • transparent operation and installation of the system that does not require modifications to the network architecture or routing policies, • the ability to implement the solution in the most popular network environments (incl. flowspec + xFlow, netconf + xFlow, ssh + xFlow) thanks to the flexibility and transparency of the invented architecture.

Figures

The subject matter of the invention in an exemplary embodiment is shown in the drawing, in which Fig. 1 shows a diagram of an exemplary implementation of an adaptive flow controller circuit. the diagram of the observed flows, Fig. 4 shows the system with the use of an appropriately configured database and virtualized services, i.e. independent of the protected network area and / or network device, Fig. 5 shows the dirty-clean interface system with a pair of internal interfaces performing dynamics shaping tasks of flows by means of packet discipline, Fig. 6 shows the concept of an anomaly detection accuracy enhancement mechanism, Fig. 7 shows an implementation example of an iterative mechanism for increasing the anomaly detection accuracy, Fig. 8 shows an example of the implementation of the flow isolation and discipline mechanism. Implementation example

The method autonomously detects actual network flows 3 posing a potential threat to the protected network on at least one network device 4, isolates packet flows 9 by means of a set of automatically generated and distributed network filter definitions 8. The network filter definitions 8 serve to configure the network filters 19 on at least one network device and may be distributed to other network devices 27 using dedicated control protocols, for example, BCP (Bridging Control Protocol). The system generates definitions of 8 network filters and configures 19 network filters with increasing resolution over time, identifies models of dynamics of signal representations of observed packet flows,

It generates and adapts control signals 16 shaping the dynamics of the isolated flows by means of packet discipline 17 on the basis of the predicted characteristics of the flows 14, i.e. the identified dynamics models, also calculates the safe shares of the isolated flows in the capacity of the protected network by determining the threshold values 12. Signals together, the control rules 16 that shape the dynamics of the isolated flows and the definitions 8 of the network filters 9 that isolate the network flows 9 constitute the filtration rules 18. The process flow diagram is shown in Fig. 2.

According to an embodiment, the method starts with sampling the packets. Based on the data from the input interface 1 and the output interface 2, packets are sampled from the actual network flows 3. The network traffic data comes from sampler 5 of network device 4, but can also come from multiple devices and be sent to a selected network device with a pattern according to of the invention or a computer with a network interface adapted to perform the steps of the method of the invention. At least one network traffic sampler 5 on at least one network device 4 records packet samples a1 in sample buffer 6.

The scope of data on sampled packets stored in sample buffer 6 depends on the packet sampling standards used (e.g. sFlow or netFlow). Any of the available x1 attributes can be used to isolate packets, e.g. t-start, t-end, duration, src address, dst address, src port, dst port, protocol, flag, src as, dst as, tos, mac address, packet samples a1 stored in the sample buffer 6.

On the basis of the packet samples a1 from the sample buffer 6, the detector D0 7 determines the definitions of 8 network filters that isolate the packet flows 9, which definitions 8 are stored in the memory 10. Detector D0 isolates from the observed set of network traffic packet samples a1 flows that stand out from the due to the given traffic statistics, e.g. bps, pps, flows, bpp. The isolated packet flows 9 are described by a network filter definition 8, which in the case of detector D0 7 distinguishes one single packet attribute x1, e.g. destination IP address or protocol type.

For each isolated packet flow 9 through any detector D0, D1, Di, a process is created in the automatic flow control system 13. The automatic flow control system 13 is a controller responsible for the implementation of the control process, whose task is to shape the dynamics of the real network flows 3 between the input and output interfaces (in the dirty-clean system) by updating the control signals 16 and, consequently, updating the filtration rules 18. Diagram of the flow isolation system, in which, on the one hand, the result of the detectors' operation provides the definition of 8 network filters, and on the other hand, the automatic control system The flow is used to control the control signals 16, is shown in Fig. 8. The flowchart of the automatic flow control system is presented in Fig. 1.

According to an exemplary embodiment, in each iteration of the adaptive control process, identification of the flow dynamics model is performed at every sampling period o1. The sampling period may be from 0.1 s to 15 s, preferably 5 seconds in this example.

In the flow observer system 11, for each isolated packet flow 9, threshold values 12 are determined and stored in the memory 10. In the flow observer system 11 or in the automatic flow control system 13, the forecasted characteristics of the flows 14 in a given forecast horizon are determined. The determined flow characteristics are stored in a memory 10, with each new isolated packet flow 9 being predicted with preset values.

If the previous threshold values 121 determined in the previous sampling period o1 have been exceeded, after comparing the parameters of the isolated packet flow 9 with the previous threshold values 121 stored in the memory 10, alarm signals 15 stored in the memory 10 are generated.

Then, for the identified dynamics model, i.e. the predicted flow characteristics 14 of the isolated packet flows 9, a synthesis of the algorithms for determining the control signals 16 and the algorithms for predicting the characteristics of the flows 14, for example the intensity of network traffic, is performed. Appropriate configuration of the network device 4 allowing for control (mitigation) is designed automatically, taking into account the specifications set by the system administrator. The determination of the control signals 16 is a pattern for shaping the dynamics of packet flow and defines the basic parameters for the suppression of anomalies or attacks using packet discipline 17. Fig. 1 shows a block diagram of an embodiment of the present invention.

PL 241 005 B1

The actual network flows 3 that correspond to the isolated packet flows 9 that do not meet the threshold values 12, i.e. generate alarm signals, are subjected to a process of automatic regulation and dynamics shaping by means of automatic flow control 13 and packet discipline 17 and network filters 19.

The automatic flow control system 13 comprises control mechanisms which periodically determine successive values of the control signals 16 of the packet discipline means 17, i.e. the work parameters of the policer or shaper (e.g. token-bucket algorithms), on the basis of the forecasted and predicted network traffic sampling period o1. the observed response of packet sources to the imposed restrictions on their forwarding rate.

The purpose of the disclosed control process is to shape the flow dynamics in accordance with the specification set by the system operator, e.g. suppression of components with a given signal band associated with the selected packet flow. The solution allows for forecasting and early suppression of DDoS attacks, correcting the work of packet disciplining measures 17 or, in other words, mechanisms for transmitting packets of network devices (policers and shapers), as well as conducting advanced diagnostics of network traffic dynamics based on the analysis of the predicted flow characteristics 14 and the models obtained on their basis dynamics. Controlling the work of packet discipline 17 measures (policers / shapers) is a key part of the attack suppression process. It allows to obtain the required speed and accuracy of damping, as well as to shape the flow characteristics (time and frequency) in accordance with the assumptions defined by the operator.

An adaptive regulator in the form of an automatic flow control system by selecting control signals from 16 packet disciplining measures 17, a policer or a shaper, for example the operating parameters of tokenbucket algorithms, is designed to protect the link covered by the system operation, e.g. keeping the flow rates at safe levels. The task of determining safe levels (upper limits) of the flow rates is carried out automatically by the mechanism of determining the threshold values then set to perform the regulation. The threshold values 12 are determined in the system of the flow observer 11 and in the system of automatic flow control 13 based on the solution of the nonlinear optimization problem, which determines the fair distribution of resources in the protected network. The upper bounds on packet flow rates thus determined ensure that the protected link is not saturated and that its usability from the perspective of each flow is maximized. They are also used in the automatic flow damping process. If the flow exceeds the upper limit of the flow rate, alarm signals 15 are triggered, which influences the system's decision to include flow in the automatic shaping of the automatic flow control system 13.

The system of automatic flow control based on packet samples from the input interface 1 and the output interface 2 taken from the sample buffer 6, threshold values 12, alarm signals 15, predicted flow characteristics 14, definition of 8 network filters and on the basis of current control signals 16 is determined by the adaptive control method new packet discipline means control 161 which are stored in memory 10.

Selected packet samples from input interface 1 and output interface 2 taken from sample buffer 6 corresponding to isolated packet flows 9, thresholds 12, alarm signals 15, predicted flow characteristics 14, network filter definitions 8 and current control signals 16 are stored in memory 10. The memory 10 may, in particular, be organized as a database of flows. The automatic flow control system uses a database of flows containing data on the medium-term distribution of traffic on the link. Thanks to the applied solution, the network administrator does not have to calculate the levels limiting the speed of each monitored flow on his own, the invented system performs this task automatically during operation, and access to the data is provided by the database.

The automatic flow control system 13 is responsible, in particular, for creating the hardware abstraction layer and provides translation functions of logical formulas in the form of control signals 16 and network filter definitions 8. The abstraction layer mediates communication between the higher-level control layer and the network infrastructure layer of the system. It is responsible for mapping the state of network switches and other network devices in the control layer and for transferring between layers of control signals. The automatic flow control system 13 is also responsible for transforming the filtering rules expressed in the language of reading traffic samples (e.g. pcap-filter) into the filtering rules of the operating system of the network device. The applied solution allows for automatic configuration of filtration tools and flow shaping on network devices

PL 241 005 B1 (e.g. network client), as well as the implementation of advanced network control concepts using machine learning and artificial intelligence (SI) tools. The automatic flow control 13 can also propagate the created filter rules 18 to other network devices 27.

For isolated packet flows 9 processed in automatic flow control 13 from control signals 16, filtering rules 18 are created which are performed on at least one network device 4 by means of network filters 19 or packet discipline 17. it should be understood that the control signals 16 and the network filter definitions 8 serve to appropriately configure the available packet discipline means 17 and the network filters 19 so that the actual network flows 3 are subjected to filtering and dynamics modeling. Automatic flow control 13 can propagate the created filter rules 18 to other network devices 27, e.g. using the BCP protocol.

The invention according to an embodiment records and stores the characteristics of the monitored network flows, such as network filter definitions 8, threshold values 12, predicted flow characteristics 14, alarm signals 15, control signals 16, in a time series database 26 supported by the management mechanisms invented for this purpose. the state of network flows of the flow observer system 11.

The individual circuits responsible for carrying out the method according to the invention may be implemented as special purpose integrated circuits (ASICs) or as single integrated circuit modules or as programmable logic gates (FPGAs) or as a computer consisting of at least a processor, memory, mass memory and a suitable network interface connected to at least one network device. In particular, the individual circuits may be software modules that perform corresponding functions on a suitably configured network device or computer, consisting of at least a processor, memory, mass memory and a suitable network interface connected to at least one network device as a computer program product with program coding means, which are stored on a computer-readable storage medium for performing the method when the computer program product is executed on a computer. Moreover, the software modules may be distinguished processes running in the operating system of a suitably configured network device or computer with a network interface.

According to an exemplary embodiment, the method of managing the state of flows facilitates effective monitoring of the security of the protected network and improves the response to events threatening its security. The exemplary system analyzes information contained in traffic samples received from at least one network device 4 or from selected network devices. The analysis is carried out by the detector systems D0, D1, Di and the flow observer system 11. When any detector detects a pattern of network traffic that requires monitoring, the system creates an object representing the set of packets matching the detected pattern. This object is also called the isolated packet flow 9. The flow is strictly defined by the network filter definition 8 (pattern) made of packet headers (terms) attributes and generated by any of the system detectors. An object created in this way may be in one of four states during its lifetime in the system. Four flow states can be distinguished: flow isolation state, flow monitoring state, flow suppression state, and flow suppression state. The migration between states can be described by a finite automaton, the state graph of which is presented in Fig. 3. The flow observer system 11 is responsible for the change of the state of the flow.

The flow entering the database by any Di detector is in an isolated state. This is the initial state of the flow from which it is automatically transferred by the flow observer to the monitoring state. The flow in this state is sampled (with a predetermined sampling rate) by the flow observer system 11, which, analyzing the packet headers, calculates their signal representation as well as time and frequency characteristics. The collected measurement results are saved every sampling period in the flow database. At the same time, the observer process checks the conditions for changing the state of each of the registered flows. The transition condition is a logical product of alarm signals that are generated for each flow in the system. The alarm signals indicate violation of the set limitations by the supervised network flow signal, e.g. related to the automatically identified upper limit of its value or a change in power in a selected part of its spectrum. If

Once the conditions for flow transition to the damping state are met, the flow observer circuit 11 changes the state of the flow. The flow in the suppressed state is sampled (at a predetermined sampling rate) by the sampling system 5. In addition, however, for the flow in this state, the controller creates a network filter 19 for packets on the network device and configures packet discipline 17 to shape dynamics flow. Calculated control parameters and measurements of observed characteristics are saved in the database every sampling period. The flow in the damped state, the characteristics of which do not violate the safety standards, can be transferred by the flow observer system 11 back to the monitoring state. From this state, the flow may also enter the damped state again. The change of state will occur and will be saved in the database if the safety standards are breached again. The flow may repeatedly switch between monitoring and suppressing states.

If the flow in the monitored state meets the safety conditions, it will be transferred by the flow observer system 11 to the quiescent state. In this state, monitoring of its parameters is completed and the data related to this flow is removed from the system.

An exemplary implementation of the database system is shown in Fig. 4.

The control process according to an embodiment of the invention is performed by managing the states of the network flows. Each of the mentioned flow states s1 corresponds to a specific value of the filtration rules 18 implemented on the network device 4 by means of network filters 19 and packet discipline measures 17. The flow observer system is responsible for the transitions between states and the current state control 11. The flow state s1 is strictly defined by the saved values in the definition memory of 8 network filters and the current control signals 16, the values of alarm signals 15 or their absence, threshold values 12 and the model of flow dynamics 14. The example of the invention describing the control process by means of the s1 flow states does not limit the invention only to the distinguished states. The control process can be described in a completely different way while maintaining key functionalities. The control process consists of the following activities:

• automatic and autonomous identification of isolated packet flows 9 requiring monitoring and identification of disturbances in monitored packet flows, • automatic isolation of sources of disturbances (attacks) by creating and sending to network devices a configuration describing the filtering rules of 18 filtered real network flows 3, • identification of the flow dynamics model and model of dynamics of packet discipline mechanisms in order to calculate forecasts of traffic volume for each flow based on the identified dynamics model, creating forecasted flow characteristics 14, • adaptive suppression of isolated disturbances (attack mitigation) dynamically tuning parameters of the filtering rules 18, shaping the dynamics of selected network flows 3 in a way that prevents saturation of protected links.

The above process is carried out periodically with a predetermined sampling frequency in sampling periods o1, which, for example, may have values from 1 s to 15 s. The sampling periods may be different for individual systems, and the sampling system 5 should then have a predetermined sampling period greater than or equal to the longest sampling period or being the least common multiple of different sampling periods. The different sampling periods may be, for example, a period of 1 second for the flow observer system 11, and 5 seconds for the automatic flow control system 13. The adaptive control system according to the invention forms a closed loop with at least one network device 4 (e.g. switches or routers). control using the packet samples a1 from the output interface 2 as feedback signals to regulate the network load of traffic requiring supervision by synthesized filtering rules 18. Based on the feedback signals, the dynamics of the actual network flows 3 potentially threatening the protected links are shaped. Importantly, packets included in the shaped network flows 3 are not completely rejected, but sent in an amount that guarantees the security of the protected network. The safe share of the supervised network flows in the total network traffic is calculated automatically based on the solution of the equitable link sharing task. An example of its implementation is shown in Fig. 2.

PL 241 005 B1

The system communicates with network devices connected to it (switches, routers), but it must consist of at least one network device 4, every specified sampling period in order to collect a1 packet samples of network traffic from the indicated interfaces and configure 18 packet filtration rules. These tasks are performed using standard protocols and functions of remote configuration of the network device provided by the hardware manufacturer (e.g. netconf, flowspec).

According to a further example, implementing the inventive method on a network device 4 replaces this network device 4 (e.g. a switch or a router) equipped with network input 1 and output 2 interfaces, standard sampling circuits 5, network filters 19 and packet discipline 17 to - conditional sending (policing / shaping) packets into an advanced device shaping the dynamics of packet flows (observed in the L2-L4 OSI layers). The extension of the functions of the network device 4 can be realized by modifying the device architecture with the necessary circuits according to the invention or by implementing appropriate procedures according to the SDN architecture. The network engineering method used extracts two types of network interfaces on the network device 4:

• Inbound interface 1, dirty: interfaces that receive packets that will be analyzed by the system, • Outbound interface 2, clean type: interfaces that send packets analyzed by the system, providing a feedback signal to the system showing the effectiveness of the system's actions.

The system takes samples (e.g. sFlow or netFlow) of a1 packets going to both types of interfaces (1, 2). The samples from the dirty input interface 1 are used to isolate packet flows 9, sets of packets matching an automatically built pattern by the system, requiring observation (potential attacks), and modeling the network traffic related to the observed actual network flow of 3 packets in the system. Based on the identified model (dynamics) of packet flow and predicted flow characteristics 14, the system makes further decisions about shaping the forwarding speed of packets that make up the network flow 3.

The packet samples from the clean output interface 2 are used by the system to determine the effectiveness of the flow rate shaping process. The shaping of the flow dynamics is carried out by means of filtering rules 18 (policing / shaping) assumed on the dirty-type interfaces or on the interfaces between dirty-clean interfaces. An example of the dirty-clean control system is shown in Fig. 5,

Transparent network architecture based only on data from the L2-L4 layer of the OSI model using commonly available network technologies allows for easy connection of client devices to input interfaces and output interfaces for package distribution. The use of advanced network engineering methods allows the system services to be delivered even on a single network device 4, transforming the firewall device into an intelligent network security device.

Contrary to the signature techniques of the prior art comparing observed network traffic to known patterns, the invented mechanism uses signal analysis techniques to perform tasks of automatically building filtering rules 18 that isolate actual network flows 3 that need to be monitored.

According to the invention, the detection of flows being the source of anomalies in the network traffic or the DDoS attack carrier consists in the analysis of a set of packet samples a1, including data from the L2-L4 layers of the OSI model, flowing through the observed input interface 1. The system consists of detectors connected to each other, with at least of one detector D0, the system preferably having at least one first order detector D1 and / or at least one higher order detector D1. The detector D0 extracts from the observed set of network traffic packet samples a1 flows that differ due to the set traffic statistics, for example, the number of bytes, the number of packets, the number of bits per second (bps), the number of packets per second (pps), the number of logical flows. .flows) or average number of bytes in the packet (bpp) recorded in the given sampling period o1.

The isolated packet flows 9 extracted in this way are described by a network filter definition 8 composed of a single base term 23 (containing a single packet attribute x1, e.g. destination IP address or protocol type). The detector performs a multi-criteria analysis of the observed data in order to isolate flows bearing the features of an anomaly in network traffic or an attack.

PL 241 005 B1

For this purpose, a multiple voting mechanism is used, in which the candidates (in parallel elections) are flows defined by a single base term 23 (the first list of candidates: dst ip A, dst ip B, the second list: src port A, src port B, etc. .), and traffic statistics voters (bps, pps, flows, bpp, etc.).

Lists 20 ordered by voters from the list of candidates are aggregated by detector D0 into aggregate lists 21, combining flows that are simultaneously distinct in many respects. This produces a set of base terms 23 that correspond to flows potentially requiring further observation. For each base term 23 stored in the memory 10, the activity counter 24 is incremented. Then the value of the activity counter 24 in the memory 10 is checked. If at least one counter exceeds the preset activity value 25, a definition 8 of the network filter is created, which corresponds to the base term 23, the counter of which is exceeded the required activity value 25. Detector D0 is activated cyclically at set sampling periods o1.

In order to improve the detection of the flows requiring observation, it is advantageous to use the representation of packet samples a1 in the form of time series 26 that describe the time series changes of attributes x1, each time series 26 containing information about the changes of any available attribute x1 of packet samples a1 recorded in the packet buffer 6 in a given sampling period o1. The time series 26 are used to derive the predicted characteristics of the flows 14, to build dynamics models, and to detect the flows in the detectors. The time series are formed by the flow observer 11 or the automatic flow control 13 and correspond to the isolated packet flows 9 by the definitions of the 8 network filters stored in the memory 10.

On the basis of the set of flows extracted by the detector D0, the higher-order detector, D1, builds a new, more precisely defined set of flows. This set consists of flows being a combination of flows separated by detector D0 7. Such a combination of definitions of 8 network filters is a solution to a properly defined signal separation task. For each isolated packet flow 9 through detector D0 7, corresponding time series 26 are taken. Then, time series 26 separated by a given feature are selected and a combination of the separated time series is determined. The choice of at least two time series 26 separated from each other takes place either because of the maximum linear (orthogonal) independence or because of the minimum correlation (stochastic independence) in the signal space of the analyzed at least two time series 26 in the time or frequency domain. The determination of the combination of the separated time series 26 consists in the calculation of the matrix mixing the time series 26 corresponding to the base terms 23 (formed by the detector D0 7). Conventional time and frequency domain signal analysis techniques known to those skilled in the art are used to calculate the scramble matrix. The hash matrix is then used to synthesize the complex terms 32 from the base terms 23 and the corresponding packet subflows 29 are found in the packet samples a1 stored in the sample buffer 6. In the collected motion samples, the detector D1 searches for flows that are logical product of terms corresponding to the distinguishing elements of the hash matrix. . If a packet subflow 29 described by a complex term 32 exists in the packet samples a1 stored in the buffer 6 samples, a new definition 8 of the network filter is created, which isolates the identified packet flow 9 and is stored in the memory 10. The filters thus formed increase the resolution of the flow observation, i.e. they divide the traffic distinguished by the detector D0 into its components requiring monitoring. Detector D1 is activated cyclically at predetermined sampling periods o1 depending on whether detector D0 has isolated new packet flows or depending on the status of flows analyzed by the flow observer system 11.

Then the higher order detector Di, builds a new subset of the flows by performing an analysis of the observed flows targeting time series clustering 26 but this time based on the subset determined by any other detector, preferably at least one detector D1. The higher order detector selects correlated time series 26 or the like according to another predetermined criteria, e.g. The new definitions of 8 network filters are a logical product of the terms building the definitions of the detector D1 network filters that make up the identified cluster. The new complex term 32 is the product of the terms building the definitions 8 of the packet flow isolating network filters 9 to which the detected correlated time series or similar time series correspond. Then it looks for a match for the new term

If a subflow 29 described by complex term 32 exists in the collected packet samples a1, a network filter definition 8 is created, which isolates the identified packet flow 9. As a result, the network filter definitions 8 divide the traffic distinguished by any other detector, preferably by at least one detector D1, into its constituent components, increasing the resolution of the filtration. The detector Di is activated cyclically at given sampling periods o1 depending on whether another lower order detector has isolated new packet flows or depending on the state of the flows analyzed by the flow observer system 11.

The procedure of increasing the filtration resolution, using the analysis in the time and frequency domain of the extended signal base in the form of time series 26 (representing packet flows), can be repeated in the iterative process, building the hierarchy of detectors D0, D1, Di, which in subsequent steps isolates from the observed traffic network actual network flows 3, more and more precisely defined filtration rules 18. This mechanism is shown in Fig. 6.

The applied approach allows for autonomous detection of complex and time-varying attack vectors, and for adaptive building of mechanisms to suppress complex attacks with variable dynamics. A schematic diagram of the iterative mechanism for increasing the detection accuracy is shown in Fig. 7.

According to an embodiment, suitably suppressing the network data packet flow 3 isolated by the network filter 19 by the packet discipline 17 comprises discarding the packet. Adequate suppression of the network data packet flow 3 isolated by the network filter 19 by packet discipline 17 includes packet buffering.

The individual steps of the method are performed on different network devices or by means of at least one computer connected to the network via a suitable network interface. The sampling periods used in various network devices may differ from each other.

The inventive system for adaptive generation of network traffic filtering rules for at least one network device self-detecting volumetric anomalies and attacks (DdoS), comprising memory 10, processor or controller and at least one network interface, further comprises a sampling circuit 5 of input 1 and output 2 interface. , sample buffer 6, detector circuit D0 7, observer circuit 11, automatic flow control circuit 13, and packet discipline means 17 and network filters 19. Packet discipline means 17 and network filters 19 may be located in the same network device or in different devices. networks equipped with them.

The sampling circuit 5 performs the function of reading and collecting packet samples a1 of the actual network flow 3 and writes them to a dedicated sample buffer 6. This buffer is connected to the detector circuit D0 7 that isolates the packet flows 9 by reading the 6 sample packets a1 stored in the sample buffer and generating the definitions of 8 network filters and storing them in the memory 10. Moreover, the memory 10 is connected to the observer circuit 11, which reads from the memory 10 information about the definitions of the 8 network filters and about the corresponding packet samples a1 stored in the sample buffer 6 and then determines the threshold values 12 and stores them in memory 10, determines whether predetermined thresholds 12 have been exceeded by comparing the parameters of isolated packet flow 9 with previous threshold values 121 stored in memory 10, and if previous thresholds 121 are exceeded, alarm signals 15 stored in memory 10 are generated, over then the observer system 11 forecasts the characteristics of the flows 14 in a given forecast horizon and stores them in the memory 10.

At the same time, the memory 10 is connected to the automatic flow control system 13 regulating and shaping the dynamics of the real network flows 3, which correspond to isolated packet flows 9 which do not meet the preset threshold values 12, controlling the means of disciplining the packets 17 and network filters 19. Automatic flow control system 13 it is connected to at least one other network device 27 which is configured such that it is possible to transmit the filter rules 18, created on the basis of the definitions read from the memory 10 and the signals controlling 16 packet discipline measures, the filter rules 18. packet discipline control 16 are translated into packet filtering rules 18 and are sent to packet discipline means 17 and network filters 19, thus isolating network traffic packet flows 3 and shaping their dynamics in the protected network area.

Claims (28)

PL 241 005 B1 According to an exemplary embodiment, the automatic flow control system 13 reads from the memory 10 alarm signals 15, (predicted and observed) flow characteristics 14, network filter definitions 8, current signals controlling the packet discipline means, and retrieves the threshold values 12. The automatic flow control 13 is also coupled to sample buffer 6, from which it reads packet samples a1 from input interface 1 and output interface 2, and adjusts adaptively to determine new packet discipline means control signals 16 for corresponding isolated packet flows 9 and stores them in memory 10. Between the sample buffer 6 and the flow observer system 11, there is a first order detector system D1 28 connected in parallel with the detector D0 7, which is connected to the memory 10. It is also possible that between the sample buffer 6 and the flow observer system 11 there is a connected in parallel with the detector D0 7 a higher order detector circuit Di 30, which is connected to the memory 10. Moreover, the invention does not exclude the possibility that between the sample buffer 6 and the flow observer system 11, there is a first order detector system D1 28 connected in series downstream of the detector D0 7 which is connected to the memory 10. Also, it is possible that between the sample buffer 6 and the flow observer system 11, there is a higher order detector system D1 28 connected in series with the first order detector D1 28, which is connected to the memory 10. The observer circuit 11 can be configured such that based on the definition of 8 mains filters, alarm signals 15 and the determined threshold values 12, it triggers a first order detector D1 28 or higher D1 30 which isolates new subflows 29 and stores new 8 filter definitions in memory 10. network. Furthermore, the observer system 11, based on the definition of 8 network filters, alarm signals 15 and the determined threshold values 12, can be configured to activate the automatic flow control system 13, which performs the dynamics control and shaping function indicated by the isolated packet flow observer circuit 11. the flow control 13 is configured to communicate with other network devices 27. Application The invention can be used as an element of a security system for ICT networks or network traffic exchange points. It can be used, among others as: • a tool for monitoring the network security condition, • a tool for actively reacting to network security incidents, • a tool for generating knowledge about network flows, • a decision support system for ICT security operators, • a tool for controlling selected parameters of network service quality. Patent claims 1. A method of adaptive creation of network traffic filtering rules on a network device that automatically detects anomalies and automatically suppresses volumetric attacks (DDoS), which on at least one network device (4) based on real network flows (3) and after their separation into isolated packet flows (9), recognizes potentially harmful network flows, and then configures or fine-tunes network filters (19) and packet discipline measures (17), whose filtering rules (18) can be propagated to other network devices (27), and selects for further analysis isolated packet flows (9) associated with at least one configured or tuned network filter (19), characterized in that at given sampling periods (o1) cyclically a) based on data from the input interface (1) and the output interface (2) packets from real network flows (3) on at least one network device (4) at p by at least one network traffic sampling system (5) and packet samples (a1) are saved in the sample buffer (6), b) on the basis of packet samples (a1) from the sample buffer (6), the definitions (8) of network filters (9) isolating the packet flows (9) are determined using the detector Do (7), which definitions (8) are stored in the memory (10) , c) for each isolated packet flow (9), threshold values (12) are determined in the flow observer system (11) and stored in the memory (10), PL 241 005 B1 d) for each isolated packet flow (9), the predicted characteristics of the flows (14) in a given forecast horizon are determined in the automatic flow control system (13) or in the flow observer system (11) and stored in the memory (10), each new isolated packet flow (9) is forecast with the values set by default, e) for each isolated packet flow (9), it is determined in the flow observer system (11) whether the previous threshold values (121) determined in the previous sampling period (o1) have been exceeded by comparing the parameters of the isolated packet flow (9) with the earlier threshold values (121) stored in the memory (10) and if the previous thresholds (121) have been exceeded, alarm signals (15) stored in the memory (10) are generated, f) the actual network flows (3) that correspond to isolated packet flows (9) that do not meet the threshold values (12) are subject to the process of automatic regulation and dynamics shaping using the automatic flow control system (13) and packet discipline measures (17) and network filters (19). 2. The method according to p. The method of claim 1, characterized in that in step b) the packet flows (9) are isolated based on any of the available attributes (x1) of the packet sample (a1) stored in the sample buffer (6). 3. The method according to p. A method according to claim 1 or 2, characterized in that in step f) based on packet samples from the input interface (1) and the output interface (2) taken from the buffer, samples (6), threshold values (12), alarm signals (15), predicted features flows (14), network filter definitions (8) and based on the current control signals (16), new packet discipline means (161) control signals (161) are determined by the adaptive control method, which are stored in the memory (10). 4. The method according to any of claims 1 to 4 1-3, characterized in that step f) is followed by step g), in which filtering rules (18) are created for the isolated packet flows (9) processed in the automatic flow control (13) on the basis of control signals (16) which are performed on at least one network device (4) using network filters (19) or packet discipline (17). 5. The method according to p. 4. The method of claim 4, characterized in that in step g) the automatic flow control (13) propagates the created filtration rules (18) to other network devices (27). 6. The method according to any one of claims 1 to 6 The method according to 1-5, characterized in that by means of the detector Do (7) in step b) the network filter definitions are determined by: determination of at least one ordered list (20) corresponding to the selected attribute (x1) of packets in the set of packet samples available in the sample buffer (6) ranked in descending order due to the selected flow parameters, which constitute the given evaluation criteria, for each attribute (x1) The collective list (21) is determined by aggregating the ordered lists and is stored in the detector memory (22), for each attribute (x1) of the packet placed on the collective lists (21), the base term (23) is created and stored in the memory (10), for each base term (23) stored in the memory (10), the activity counter (24) is incremented, the value of the activity counter (24) in the memory (10) is checked, if at least one counter exceeds the preset activity value (25) a definition (8) of the network filter is created that corresponds to the base term (23) whose counter has exceeded the activity value (25). 7. The method according to any one of claims 1 to 7 The method of 1-6, characterized in that the detector Do is cyclically triggered at predetermined sampling periods (o1). 8. A method according to any one of claims 1 to 8 1-7, characterized in that the given evaluation criteria are the number of bytes, the number of packets, the number of bits per second (bps), the number of packets per second (pps), the number of logical flows (flows) or the average number of bytes in the packet ( bpp) recorded in a given sampling period (o1). 9. A method according to any one of claims 1 to 9 1-3, characterized in that In steps c), d), e), f) the observer circuit (11) or the automatic flow control circuit (13) creates time series (26) corresponding to the isolated packet flows (9) through the definitions (8). ) network filters stored in memory (10) describing the changes of packet attributes (x1) occurring over time, each time series (26) containing information about changes to any available attribute (x1) of packet samples (a1) stored in the packet buffer (6) in a predetermined sampling period (o1) i are stored in the memory (10) and are used to determine the characteristics of the flows (14) and in steps c), d), e) and f). A method according to any one of claims 1 to 10 A process according to any of the claims 1-9, characterized in that step b) is followed by step b1), in which the packet flow (9) isolated by the detector Do (7) is divided by a first-order detector Di (28) into sub-packet flows (29) so that: for each isolated packet flow (9), the corresponding time series (26) are downloaded from the memory (10), the time series separated due to the given feature are selected and the combination of separate time series corresponding to the packet flows (9) is determined isolated in the Do detector (7), on the basis of the combination of separated time series, the terms composed (32) of the base terms (23) are determined and the corresponding sub-flows of packets (29) are searched for in the packet samples (a1) stored in the sample buffer (6) ; if a packet subflow (29) described by a complex term (32) exists in packet samples (a1) stored in the sample buffer (6), a new network filter definition (8) is created, which isolates the identified packet flow (9), and is stored in memory (10). 11. The method according to any one of claims 1 to 11 1-10, characterized in that the selection of at least two separated time series (26) takes place due to the maximum linear (orthogonal) independence or due to the minimum correlation (stochastic independence) in the signal space of the analyzed at least two time series (26) in the time or frequency domain. 12. The method according to any one of claims 1 to 12 1-11, characterized in that step b1) is followed by step b2), in which the isolated packet flows (9) are divided by a higher order detector Di (30) on the basis of a combination of lower order network filter definitions (8) forming new definitions (8) of higher order network filters that isolate further subflows of packets (29): for each isolated packet flow (9), the corresponding time series (26) are retrieved from the memory (10), time series (26) correlated or similar are selected according to another given criterion, a new complex term (32) is built from the product of building terms definitions (8) of network filters that isolate packet flows (9) to which the detected correlated time series or similar time series correspond, and then search for the corresponding new complex term (32) packet flows in packet samples (a1) stored in the sample buffer (6) if a complex term subflow (29) exists in the collected packet samples (a1), a network filter definition (8) is created, which isolates the identified packet flow (9), and stored in memory (10). 13. A method according to any one of claims 1 to 13 The method according to any of the claims 1-12, characterized in that at least two time series (26) are qualified as similar because of their correlation with each other or because of their similarity in the shape of the waveform. 14. A method according to any one of claims 1 to 14 4-13, characterized in that in step g), suitably suppressing the network data packet flow (3) isolated by the network filter (19) by packet discipline (17) comprises discarding the packet. 15. A method according to any one of claims 1 to 15 4-14, characterized in that in step g), corresponding suppression of the network data packet flow (3) isolated by the network filter (19) by packet discipline (17) includes packet buffering. 16. A method according to any one of claims 1 to 16 Method according to any of the claims 1-11, characterized in that the individual method steps are performed on different network devices or with at least one computer connected to the network via a suitable network interface. PL 241 005 B1 17. The method, characterized in that the state of the flow (s1) is strictly determined by the values of network filter definitions (8) stored in the memory (10) and on the basis of the current control signals (16), the values of alarm signals (15) or their absence, values threshold (12) and predicted flow characteristics (14) and is controlled by the observer system (11), starting from the isolation state (sw) of the flow by means of an appropriate detector, through the observation state (s) of the flow by means of the observer system (11) , to the flow control (sk) state, operated by the automatic flow control system (13), or to the flow expiration (se) state, in the flow observer system (11), each of the mentioned flow states (s1) corresponds to a specific value of the filtration rules (18) implemented on the network device by means of network filters (19) and appropriate packet discipline (17). 18. Adaptive network traffic filtering rule making system for at least one network device self-detecting anomalies and automatically suppressing volumetric attacks (DDoS), comprising memory, processor or controller and at least one network interface, characterized in that it comprises a network traffic sampling device of the input interface (1) network and output interface (2) of the network performing the function of reading and collecting packet samples (a1) of the actual network flow (3) and writing them to a dedicated sample buffer (6) connected to the isolating detector (Do) (7) packet flows (9) by reading packet samples (a1) stored in the buffer (6) and generating network filter definitions (8) and storing them in the memory (10), the memory (10) being connected to the observer circuit (11) which reads from memory (10) information about network filter definitions (8) and the corresponding samples packets (a1) stored in the sample buffer (6) and then determines the threshold values (12) and stores them in the memory (10), determines whether the determined threshold values (12) have been exceeded by comparing the parameters of the isolated packet flow (9) with the previous threshold values (121) stored in the memory (10) and if the previous threshold values (121) have been exceeded, alarm signals (15) stored in the memory (10) are generated, moreover, the observer circuit (11) forecasts the characteristics of the flows (14) in the predetermined forecast horizon and stores them in the memory (10), at the same time the memory (10) is connected with the automatic flow control system (13) regulating and shaping the dynamics of real network flows (3), which correspond to isolated packet flows (9) that do not meet the set threshold values ( 12), controlling the means of disciplining packets (17) and network filters (19), at the same time the automatic flow control system (13) is connected to the to at least one other network device (27), to which it transmits the filtering rules (18), created on the basis of the definitions (8) of the network filters and the control signals (16) of the packet disciplining means (16) read from the memory (10), network filters and packet discipline control signals (16) are translated into packet filtration rules (18) and sent to packet discipline means (17) and network filters (19), thus isolating network traffic packet flows (3) and shaping their dynamics in a protected area of the network. 19. The system according to p. 18, characterized in that the automatic flow control system (13) reads from the memory (10) alarm signals (15), (predicted and observed) flow characteristics (14), network filter definitions (8), current control signals (16), disciplining packets and retrieving threshold values (12), the automatic flow control (13) is also connected to the sample buffer (6), from which it reads packet samples (a1) from the input interface (1) and the output interface (2) and adaptively adjusts by determining new packet discipline means (16) control signals for the respective isolated packet flows (9) and stores them in memory (10). 20. A system according to any one of the preceding claims. A first order detector system Di (28) connected in parallel with the detector Do (7), which is connected to the memory (10), is provided between the sample buffer (6) and the flow observer system (11). PL 241 005 B1 21. A system according to any one of the preceding claims A system according to 18-20, characterized in that between the sample buffer (6) and the flow observer system (11) there is a higher order detector system Di (30) connected in parallel with the detector Do (7), which is connected to the memory (10). 22. System according to any one of claims A first order detector system Di (28) connected in series downstream of detector Do (7), which is connected to the memory (10), is provided between the sample buffer (6) and the flow observer system (11). 23. A system according to any one of claims 1 to 23. 18-22, characterized in that between the sample buffer (6) and the flow observer system (11) there is a higher order detector system Di (28) connected in series with the first order detector Di (28), which is connected to the memory (10 ). 24. A system as claimed in any one of claims 1 to 24. 18-23, characterized in that the observer system (11), based on the definition (8) of the mains filters, alarm signals (15) and the determined threshold values (12), activates a first-order detector D1 (28) or a higher order Di (30), which isolate new subflows (29) and store in memory (10) new line filter definitions (8). 25. The system according to any of claims 1 to 25 18-24, characterized in that the observer circuit (11) based on the definition (8) of the line filters, the alarm signals (15) and the determined threshold values (12) is configured to activate the automatic flow control circuit (13) that performs the control function and shaping the dynamics of the isolated packet flow (9) indicated by the observer system (11). The system of claim 26 The method of claim 12, characterized in that the automatic flow control circuit (13) is configured to communicate with other network devices (27). The system of claim 27 12. A method according to claim 12, characterized in that the individual elements of the circuit can be implemented as special purpose integrated circuits (ASICs) or as modules of one integrated circuit or as programmable logic gates (FPGAs) or as a computer consisting of at least a processor, memory, mass memory. and a suitable network interface connected to at least one network device. 28. A computer program product with program coding means stored on a computer-readable storage medium for performing the method of any one of claims 1 to 12 when the computer program product is executed on a computer or network device.