IL309415A - Detecting and Mitigating a Distributed Denial of Service Attack - Google Patents

Description

DDOS DETECTION AND MITIGATION FIELD OF THE PRESENTLY DISCLOSED SUBJECT MATTER The presently disclosed subject matter relates to detection and mitigation of cybersecurity threats.

BACKGROUND A Distributed Denial of Service (DDoS) attack is a malicious attempt to disrupt the operation of resources such as websites, online services, or networks, by bombarding them with a massive volume of traffic over a short period of time. Facilitators of DDoS attacks (also referred to herein as "attackers") often control a network of compromised computers and other IoT devices (e.g., webcams, Smart TVs, Smart home hubs, etc.), known as botnets, referred to herein collectively as "bots" or "bad actors"). These bots are unknowingly infected with malware, allowing the attacker to control the botnet and use it for carrying out the attack. The bots in the botnet overwhelm the target resource with a high volume of traffic, including, for example, various types of network requests, such as HTTP requests, pings, etc.

The goal of a DDoS attack is to make the targeted resource (e.g., bandwidth, firewall capacity, servers, etc.) unavailable for serving legitimate users. DDoS attacks are initiated for various reasons, such as financial gain, political motives, revenge, or simply wreaking havoc. The consequences of DDoS attacks include service disruption, financial loss, harm to reputation, and user frustration.

Known countermeasures and defensive strategies for mitigating DDoS attacks involve various technologies and strategies, including, for example, rate limiting, traffic filtering, IP blacklisting, traffic scrubbing, etc.

GENERAL DESCRIPTION The presently disclosed subject matter includes a computer‐implemented method and computer system for detecting and mitigating DDoS attacks on different network layers of the Open Systems Interconnection (OSI) model, including for example, L3, L4, L7. – 2 – According to the disclosed technique, logs that are automatically generated by various network components, which are part of a network topology of one or more monitored web‐domains, are continuously collected and recorded. The data extracted from the logs (herein below "log‐data") are used for detecting DDoS attacks.

The log‐data is used for generating various models that enable to identify data that indicates an occurring or imminent DDoS attack (also referred to herein as "DDoS indicators"). This includes models that represent normal healthy behavior of client‐devices operated by legitimate users (actors) interacting with web‐domain components (e.g., services). After their generation the models are used for processing log‐data collected from the relevant network components, and identifying deviations from the normal behavior which indicate an occurring or imminent DDoS attack. In response to the identification of an occurring or imminent DDoS attack, one or more actions (also referred to herein as "attack mitigating actions") may be automatically executed by the system.

Other commonly known DDoS detection and/or mitigation services utilize network traffic, i.e., the data, which is transmitted through network devices and applications related to network services such as Internet Service Providers (ISP), Content Delivery Networks (CDN), Firewalls (FW), Load‐Balancers (LB), Web‐Application Firewalls (WAF), Rate Limiting (RL), Bot Mitigation (Bot), Scrubbing Centers and more. This includes for example, HTML files, JavaScript files, videos, emails, VoIP, etc. This requires the implementation of various changes to the components, including for example, addition of hardware devices, installation of new software, and/or the reconfiguration of existing software.

Unlike these common approaches, DDoS detection and mitigation services as disclosed herein does not require to make extensive modification to the network. As DDoS detection and mitigation disclosed herein does not rely on network traffic, but rather on logs, which are automatically generated by various network components and can be made available, a customer (e.g., web‐domain owners) who wishes to use these DDoS detection and mitigation services is not required to make extensive – 3 – changes to network components in the relevant network topology. The computer system disclosed herein can be implemented as an external system that receives the logs from the relevant network components and is capable of providing the DDoS detection and mitigation services without or with little changes to the network components.

Furthermore, as DDoS detection and mitigation is implemented using models (including but not limited to ML models) generated based on log‐data that is generally common to all network components and vendors providing network services (e.g., ISP vendors, CDN vendors, Firewall vendors, Web‐Application Firewall vendors, etc.), the models which are generated are vendor independent. This provides a significant technical advantage in the ability of customers to switch between vendors and avoid vendor lock‐in. Since the same models can be implemented in network components of different vendors, and following their implementation in network components of one vendor, the respective security posture can be similarly applied in network components of another vendor, the method and system disclosed herein facilitates easy and efficient migration between network component vendors, which enables to immediately apply the security posture of the customer in the components of the new vendor following migration.

According to certain examples, the computer system and methods disclosed herein also includes, following detection of a suspected DDoS attack, automatic reconfiguration of security settings in one or more network components. Reconfiguration of security settings includes, for example, the modification of security thresholds dedicated for increasing limitations on the access to monitored web‐domains. In some examples, once it is determined that the attack has subsided, security settings are automatically reverted to the baseline values. As further explained below, reconfiguration of security settings is implemented in a manner that has a limited or no impact on the quality of service provided to legitimate users.

According to a first aspect of the presently disclosed subject matter there is provided a computer‐implemented method of distributed denial of service (DDoS) – 4 – attack detection and mitigation in a monitored web‐domain of a customer, the method comprising: continuously obtaining customer‐specific log‐data from logs generated by network components of the monitored web‐domain; wherein the logs are collected from the network components without requiring direct access and processing of network traffic transferred through the network components of the monitored web‐domain; processing the customer‐specific log‐data and identifying, based on the customer‐specific log‐data, data indicative of an occurring or pending DDoS attack; automatically executing one or more mitigating actions dedicated for mitigating the DDoS attack. In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xxv) below, in any desired and technically possible combination or permutation: i. wherein the network components include devices and applications implementing one or more of the following network components: Content Delivery Network (CDN); Web Application Firewall; Load Balancer; Virtual Server; Bot Mitigation service (Bot), and Web Application server. ii. wherein the network components include devices and applications implementing one or more of the following types of network components: Content Delivery Network (CDN); Web Application Firewall; Load Balancer; Virtual Server; and Web Application server; the method comprising: prioritizing customer‐specific log‐data received from different types of network components according to a distance of the component from a network edge, where the closer a network component is to the network edge, the higher is the priority of a respective log‐date. iii. wherein processing the customer‐specific log‐data includes generating at least one customer‐specific model based on customer‐specific log‐data obtained from the network components of the monitored web‐domain configured for determining whether customer‐specific log‐data received from network components 30 – 5 – of the monitored web‐domain represents normal behavior or behavior that indicates an occurring or pending DDoS attack. iv. wherein the at least one customer‐specific model includes a request rate model representing a normal range of request rates; the method comprising: determining, based on the customer‐specific log‐date, real‐time request rate; in case it is determined that the real‐time request rate deviates from a request rate threshold defined by the request rate model, identifying a DDoS indicator. v. wherein the request rate model is configured to apply different respective thresholds to different annual dates, weekdays, and/or times. vi. wherein the at least one customer‐specific model includes a bandwidth model representing normal bandwidth usage patterns; the method comprising: determining, based on the customer‐specific log‐date, real‐time bandwidth usage; in case it is determined that the real‐time bandwidth usage deviates from a bandwidth threshold defined by the request rate model, identifying a DDoS indicator. vii. wherein applying at least one customer‐specific model comprises: classifying log‐data to respective Uniform Resource Locator (URI) paths to thereby obtain URI‐specific log‐data; and applying one or more URI‐specific models, each URI‐specific model generated based on the customer‐specific log‐data obtained from a respective URI‐path and configured for determining whether respective log‐data associated with the respective URI‐path represent normal network behavior or network behavior that indicates an occurring or pending DDoS attack. viii. wherein the one or more URI‐specific models include one or more of: ‐ URI‐specific request rate model, configured for determining whether respective customer‐specific log‐data associated with a specific URI‐path indicate normal request rate; ‐ URI grouping model, configured for determining whether groups of URI‐paths in the monitored web‐domain, which are expected to be accessed 30 – 6 – together, are indeed accessed simultaneously; ‐ URI‐specific bandwidth model, configured for determining whether bandwidth usage detected in real‐time at a specific URI‐path deviates from common bandwidth usage in the specific URI‐path; and ‐ URI‐specific engagement period model configured for determining whether respective customer‐specific log‐data, associated with a specific URI‐path, indicate normal engagement period with the specific URI‐path. ix. wherein processing the log‐data includes applying at least one universal model generated based on universal log‐data obtained from network components of multiple independent monitored web‐domains; the universal model is configured for determining whether the universal log‐data indicates an occurring or pending DDoS attack. x. wherein the at least one universal model includes an IP anomaly detection model dedicated for identifying suspicious IP addresses, by cross‐referencing IP addresses extracted from the customer‐specific log with IP addresses that have been previously identified as participants in DDoS attacks. xi. wherein the at least one universal model includes a botnet detection model dedicated for identifying IP addresses being part of a botnet by cross‐referencing IP addresses extracted from the customer‐specific log with IP addresses that have been previously identified as participants of a botnet. xii. wherein, following identification of one or more IP addresses which are part of a botnet, automatically executing at least one of the following mitigating actions: blocking all IP addresses recorded as part of the botnet; hardening an access rate threshold for accessing the monitored web‐domain; and activating bot mitigation. xiii. determining attack capacity of the botnet; determining whether the botnet is disruptive to operability of the monitored web‐domain based on a comparison between the attack capacity of the botnet and bandwidth capacity of the monitored web‐domain; and avoiding execution of certain 30 – 7 – mitigation actions if the botnet is not disruptive to the operability of the monitored web‐domain. xiv. wherein the botnet is a ransom botnet, the method comprising: analyzing a ransom email received from an attacker and identifying information in the email identifying the attack; identifying a respective ransom botnet which has been previously associated with the attacker; and automatically executing at least one mitigating action targeting the botnet. xv. wherein applying, on the customer‐specific log‐data, multiple customer‐specific models and at least one universal model, each model providing a respective data output indicative as to whether or not a DDoS attack is occurring or pending; and confirming an occurring or pending DDoS attack based on compilation of the respective data outputs. xvi. wherein determining whether the DDoS attack is disruptive to operability of the monitored web‐domain; and avoiding execution of part or all mitigation actions if the DDoS attack is not disruptive to the operability of the monitored web‐domain. xvii. wherein automatically determining a security posture of the monitored web‐domain, including setting respective request rate thresholds to different network components of the monitored web‐domain. xviii. wherein automatically determining a security posture of the monitored web‐domain includes automatically determining request rate thresholds to specific FQDNs and/or URI‐paths in the monitored web‐domains. xix. wherein the one or more mitigation actions include automatically reconfiguring one or more security posture parameters including one or more of: ‐ automatically hardening a request rate limitation in the monitored web‐domain; ‐ automatically blocking one or more IP addresses; and ‐ automatically activating bot mitigation. xx. wherein, following execution of the mitigation action, continuing the 30 – 8 – processing of the log‐data; and automatically stopping the execution of the one or more mitigation actions, if data indicating that the DDoS attack has ceased or subsided, is identified. xxi. wherein automatically reverting security posture parameters to baseline values following identification of the data indicating that the DDoS attack has ceased. xxii. wherein automatically executing one or more mitigating actions includes sending warnings to one or more Internet Service Providers (ISPs) identified as a source of IP addresses that were identified to take part in the DDoS attack. xxiii. wherein automatically executing more or more mitigating actions includes executing one or more life‐cycle actions, comprising sending traffic targeted towards IP addresses identified in the logs; continuously analyzing the logs to determine whether the IP addresses continue to appear in the logs; once the part or all of the IP addresses cease to appear in the logs, determining an offensive capability of the IP addresses based on an amount of traffic sent towards them. xxiv. wherein, following migration of the customer from a first network components vendor to a second network component vendor, applying the determined security posture while operating with the first network component vendor at the network components of the second network components vendor. xxv. wherein the network components are of a first network component vendor, the method comprising following migration of the customer from the first network components vendor to a second network component vendor, applying the at least one customer‐specific model determined while operating with the first network component vendor at the network components of the second network components vendor. The presently disclosed subject matter further contemplates a computer program product comprising a computer readable storage medium retaining a program of instructions, which, when read by a computer processor, causes the computer processor to perform a method according to the first aspect disclosed above. 30 – 9 – The presently disclosed subject matter further contemplates a non‐transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method according to the first aspect disclosed above. The presently disclosed subject matter further contemplates a computer system comprising one or more computers, each computer comprising at least one processing circuitry, the system being operatively connectable to a plurality of network components in a network topology of one or more monitored web‐domains of a monitored customer and configured to execute a computer‐implemented method of distributed denial of service (DDoS) attack detection and mitigation in accordance with the first aspect disclosed above. The computer program product, the non‐transitory program storage device, and the system can include, where applicable, one or more of features (i) to (xxv) listed above, mutatis mutandis, in any technically possible combination or permutation. BRIEF DESCRIPTION OF THE DRAWINGS To understand the presently disclosed subject matter and to see how it may be carried out in practice, the subject matter will now be described, by way of non‐limiting examples only, with reference to the accompanying drawings, in which: Fig. 1 is a high‐level block diagram schematically illustrating computer network 10 configured with DDoS detection and mitigation system 20 , in accordance with an example of the presently disclosed subject matter; Fig. 2 is a high‐level flowchart showing operations carried out as part of the DDoS detection and mitigation process, in accordance with an example of the presently disclosed subject matter; Fig. 3 is a block diagram schematically illustrating system 20 in more detail, in accordance with an example of the presently disclosed subject matter; Fig. 4 is a flowchart showing operations carried out as part of the DDoS detection and mitigation process in more detail, in accordance with an example of the – 10 – presently disclosed subject matter; Fig. 5 is a flowchart showing operations carried out as part of logs ingestion, in accordance with an example of the presently disclosed subject matter; Fig. 6 is a block diagram illustrating components of mitigation orchestrator 353 in accordance with an example of the presently disclosed subject matter; and Fig. 7 is a flowchart of attack life‐cycle operations carried out, according to some examples of the presently disclosed subject matter.

DETAILED DESCRIPTION In the drawings and descriptions set forth, identical reference numerals indicate those components that are common to different embodiments or configurations. Elements in the drawings are not necessarily drawn to scale.

Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that, throughout the specification, discussions utilizing terms such as "obtaining", "collecting", "processing", "executing", "determining" or the like, include an action and/or processes of a computer that manipulate and/or transform data into other data, said data represented as physical quantities, e.g. such as electronic quantities, and/or said data representing the physical objects.

The terms "computer", "computer system", "computer device", "computerized device" or the like, should be expansively construed to include any kind of hardware‐based electronic device with one or more data processing circuitries. A processing circuitry can comprise, for example, one or more processors operatively connected to computer memory of any suitable sort, loaded with executable instructions for executing operations, as further described below.

The one or more processors referred to herein can represent, for example, one or more general‐purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, a given processor may be one of: a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, – 11 – processor implementing other instruction sets, or a processor implementing a combination of instruction sets. The one or more processors may also be one or more special‐purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a graphics processing unit (GPU), a network processor, or the like.

The memories referred to herein can comprise for example, one or more of the following: internal memory, such as, e.g., processor registers and cache, etc., main memory such as, e.g., read‐only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc. The operations in accordance with the teachings herein may be performed by a computer specially constructed for the desired purposes, or by a general‐purpose computer specially configured for the desired purpose by a computer program stored in a computer readable storage medium.

As used herein, the phrase "for example", "such as", "for instance" and variants thereof, describe non‐limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases", or variants thereof, means that a particular feature, structure, or characteristic described in connection with the embodiment(s), is included in at least one embodiment of the presently disclosed subject matter. Thus, the appearance of the phrase "one case", "some cases", "other cases", or variants thereof, does not necessarily refer to the same embodiment(s).

It is appreciated that certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately, or in any suitable sub‐combination.

In various examples of the presently disclosed subject matter, fewer, more, – 12 – and/or different stages than those shown in Figs. 2 , 4 , 5 , and 7 may be executed. In embodiments of the presently disclosed subject matter, one or more stages illustrated in the figures may be executed in a different order, and/or one or more groups of stages may be executed simultaneously.

Figs. 1 , 3 , and 6 illustrate general schematics of the system architecture in accordance with certain non‐limiting examples of the presently disclosed subject matter. Elements in Figs. 1 , 3 , and 6 can be made up of any combination of software and hardware and/or firmware that performs the functions as defined and explained herein. Elements in Figs. 1 , 3 , and 6 may be centralized in one location or dispersed over more than one location. For example, each one of computers 21 , 23 , and 25 can be located at a different geographical location, remote from the other module. Furthermore, in some examples of the presently disclosed subject matter, the system may comprise fewer, more, and/or different elements than those shown in Figs. 1 , 3 , and 6 . For example, Figs. 1 and 3 show several separate computers, each dedicated for executing certain functions of the system, however it would be clear to any person skilled in the art that the functionalities of the system can be otherwise divided. For instance, in an alternative system‐design, different functions assigned to computer 21 or computer 23 can be otherwise distributed to several computers. For instance, customer‐specific‐models generator 331 , and universal model generator 333 , which are both illustrated as part of computer 23 , can be otherwise implemented in separate computers, and data enrichment module 315 , which is illustrated as part of computer 21 can be otherwise implemented as part of other computers in system 20 , e.g., computer 23 or 25 .

Likewise, various elements described as distributed over different computers can be otherwise consolidated into a single computer device. For example, the functionalities of computer 25 and computer 23 can be consolidated and implemented in a single computer device. In some examples, system 20 is implemented on a single computer.

It is noted that use of the term "model" in terms such as "customer‐specific – 13 – models" and "universal models" should be broadly construed to include a computer program specifically designed for detecting a DDoS attack. Such models can be implemented using a diverse range of approaches, including logic‐based approaches and artificial intelligence (AI), including machine learning (ML) techniques.

The term 'customer' as used herein refers to an individual, company, enterprise, or the like, which owns or operates one or more web‐domains.

The term "monitored customer" as used herein refers to customers whose web‐domains are receiving DDoS detection and mitigation services from the system disclosed herein, and the term "monitored web‐domain" refers to a web‐domain receiving DDoS detection and mitigation services from the system disclosed herein.

The term "web‐domain" includes, but is not limited to, various forms of the customer's online presence such as official websites, web applications, and API (Application Programming Interface) services. For instance, a customer (e.g., a company) may have multiple monitored web‐domains, including a marketing website ('www.domain.com'), a web application for user interaction ('app.domain.com'), and a platform for programmable interactions ('api.domain.com')." A network topology of a specific web‐domain refers to the arrangement and interconnection of network components, including devices, services, and applications, that are relevant to the operation of that web‐domain. The specific network layout can vary, depending on the organization's infrastructure and particular demands. As mentioned above, according to the presently disclosed subject matter, logs generated by a monitored web‐domain are collected from the network components which constitute its network topology. Such network components are referred to herein as "relevant network components".

Bearing the above in mind, attention is drawn to Fig. 1 , which is a high‐level diagram schematically illustrating a computer network topology 10 configured with DDoS detection and mitigation services, in accordance with certain examples of the presently disclosed subject matter. Fig. 1 is a general non‐limiting example which demonstrates various principles of the presently disclosed subject matter. As would – 14 – be clearly understood to the skilled person, various modifications, alterations, and adaptations to the disclosed network may be made (e.g., by adding certain network components or removing certain network components) without departing from the scope of the subject matter disclosed herein.

Fig. 1 shows client computers which are connected to a network topology 10 of a certain web‐domain. A client can be any type of computer device with internet connection such as a personal computer, laptop computer, Smartphone, tablet, Smart watch, other wearable devices, electric car, aircraft, etc. Fig. 1 illustrates a first type of clients, which are legitimate clients or "good actors" ( 11a ), and a second type of clients, which are malicious clients or "bad actors" ( 11b ). As mentioned above, bad actors are often implemented by bots. Notably, the figure illustrates many more bad actors than good actors (e.g., greater by 3 orders of magnitude or more). This demonstrates a common scenario observed in DDoS attacks, where a large number of malicious clients overwhelm the target resources with a high volume of traffic, preventing legitimate clients from being properly served.

In general, clients (both legitimate and malicious) issue requests directed to a certain resource (e.g., HTTP requests, HTTPS requests) in a certain web‐domain. Such requests may include, for example, requesting to access a certain website (e.g., https://app.domain.com), watch a certain video, retrieve certain information (download a file), etc. The requests are transmitted over the network topology through a sequence of network components in the topology leading from the source client device to the desired destination, some of which are described in Fig. 1 .

Normally clients connect to the network through Internet Service Providers (ISPs; 12 ) which provide and manage Internet connectivity. Different clients connect to the network through different ISPs. The ISPs often offer services such as routing traffic to the correct destination, Domain Name System (DNS) services, hosting services (e.g., mail hosting), security services, etc.

Fig. 1 shows various network components (e.g., services) which are commonly part of a network topology. Each such component is implemented over various – 15 – computer devices and applications. As shown in Fig. 1 , requests are forwarded from the ISP to the target destination in the network. In some examples, requests are forwarded to a Content Delivery Network (CDN; 13 ). A CDN consists of servers strategically positioned across various locations, aimed to expedite delivery of web content according to a user's geographical location. By storing content on these servers, CDNs minimize delays, boost website speed, and provide added protection against large traffic influxes and potential cyber threats, ensuring a quicker and safer online experience for users. An advanced version of CDNs is an Application Delivery Network (ADN). Examples of CDN/ADN vendors include CloudFlare, Akamai, Edgio, Fastly, Imperva, AWS, GCP, and Azure.

CDNs often apply various security modules dedicated for inspecting traffic to ensure secure and reliable delivery of web content. CDN security modules include for example: IP controls 131 configured to control which users are granted access to the content. This includes content authentication, geographic restriction, implementing Access Control Lists (ACLs) such as black listening, whitelisting, etc.; Web‐Application Firewall (WAF; 133 ) configured to monitor and filter HTTP traffic to and from a web application. It can detect and block malicious requests, such as SQL injection, cross‐site scripting (XSS), and other web‐based threats; Rate‐Limit (RL; 135 ) configured to limit the number of requests a user (or IP address) can make within a specified time frame. This helps to mitigate DDoS attacks on a target resource (e.g., website); and Bot Mitigation (Bot; 137 ) configured to identify and block malicious bots, such as scrapers and crawlers while allowing legitimate bots (like search engine crawlers) to access the target resource. Bot mitigation techniques include, for example, JavaScript injection and Captcha.

Notably, while in some network topologies a CDN is not implemented, DDoS detection and mitigation as disclosed does not necessitate a CDN for its proper operation. – 16 – From the CDN the request is forwarded to the Customer Origin 15 , which is the original server or location where the requested content or application data is located. The request can be transmitted through a Firewall 151 and/or to a Load Balancer 153 and finally reaches an appropriate web‐site nodes 155 1‐n.

Firewall (FW) 151 is a transport layer (L3 or L4) FW, which can be an on‐prem FW or cloud‐based, and is configured to examine header information of data packet being transmitted in the network to confirm validity of the communication. FW tasks include, for example, stateful inspection, port filtering (allowing or denying traffic based on source and destination IP addresses and/or protocols and/or port numbers), TCP/UDP session tracking, protection against TCP attacks, and traffic rate limiting.

Load Balancer 153 can be implemented by hardware or software (on L4 or L7) and is configured to distribute network traffic across multiple servers to avoid the overwhelming of a any single server with too much traffic. Traffic distribution to multiple servers ensures that applications run efficiently and can maintain high availability and reliability. In some cases, a load balancer can also take part in protecting against malicious attacks. For example, Load Balancer 153 can provide WAF integration, which can protect against web‐based attacks. Load Balancer 153 can also mitigate certain types of DDoS attacks by distributing the attack traffic across a large pool of servers.

The requests pass through Firewall 151 and are distributed by the Load Balancer 153 to a respective server, where the request is processed and the relevant data is provided, e.g., requested data is transmitted back to the requesting client 11a .

Fig. 1 also shows system 20 (also referred to as "DDoS attacks detection and mitigation system") which is operatively connected to network 10 and is configured to detect and mitigate DDoS attacks in the network. As known in the art, different network components produce logs which record different types of information related to various activities and events which occur within the network components. As schematically illustrated in Fig. 1 , CDNs 13 , Firewalls 151 , Load Balancers 153 , and various applications running on the web servers 155 , generate respective logs ( 160 ). – 17 – System 20 is configured to provide DDoS detection and mitigation services to monitored web‐domains operating in network 10 (e.g., "domain.com"). As disclosed herein, system 20 is configured to obtain logs from various network components in the network topology extending from the Internet edge, where clients 11a , 11b are located, and the target web‐domain servers 155 1‐n and provide DDoS detection and mitigation services to web‐domain servers 155 1‐n. System 20 is configured to continuously receive and ingest logs from various network components located along relevant network topology, and implements DDoS detection and mitigation, which relies on log‐data extracted from the logs. System 20 is further configured to respond to detected DDoS attacks by executing and managing various DDoS attack mitigation actions.

As mentioned above, one advantage of this approach is that it does not require from system 20 to be integrated within the network to monitor the network traffic that is being transferred through the network components along the relevant network topology. This considerably simplifies integration of the system when initially connecting to a network for providing services for a new customer.

By way of example, system 20 is shown to include several computers, each computer being configured to perform certain tasks, including ingestion computer 21 , security management module 23 , and automatic security facilitator computer 25 . A detailed description of the operations of each computer is provided below with respect to the following figures.

Fig. 2 is a flowchart showing high‐level operations carried out by system 20 as part of the DDoS detection and mitigation process, according to some examples of the presently disclosed subject matter. When a new customer joins the DDoS detection and mitigation services provided by system 20 (block 201 ), it provides system 20 with permission to collect logs generated by the relevant network components within the customer's network topology.

System 20 is configured to continuously collect and process the logs (block 203 ). The log‐data extracted from the logs is used for monitoring behavior and – 18 – identifying data indicating a current or pending security threat. In some examples, log‐data is used for generating one or more models, each model being dedicated for determining a certain type of information that enables the detection and classification of incoming DDoS attacks based on the continuous stream of log‐data input (block 205 ). The models are designed to represent normal, healthy operation of network components. By creating models that represent standard log‐data patterns and behaviors exhibited by the different network components under regular conditions, the models establish a comprehensive baseline of 'normal log‐files' behavior. This representation serves as a benchmark against which deviations or anomalies in log‐ data can be effectively detected.

Once the models are ready and available, the trained models are applied on new log‐data which is continuously obtained from the network components and used for detecting and classifying irregular behavior that indicates DDoS attacks (block 207 ). In some examples, operations described with reference to blocks 205 and 207 are executed by security management computer 23 .

Following detection of a current or pending DDoS attack, system 20 is configured to automatically execute one or more DDoS mitigation actions (block 209 ). In some examples, security control computer 23 is configured to generate and send instructions to one or more of the relevant network components dedicated for updating their configuration in order mitigate an ongoing DDoS attack. Various specific examples of DDoS mitigations operations are provided below with reference to Fig. 4 .

In some examples, following detection of a DDoS attack, system 20 is further configured to issue one or more attack life‐cycle operations (block 211 ). Automatic security facilitator computer 25 can be configured to execute operations, such as informing attackers' ISPs of the ongoing attacks, and monitoring attack scale, as further described below.

The process described with reference to Fig. 2 can be implemented as a continuous process where logs are continuously being obtained from the relevant network components, and the extracted log‐data is used for generating/updating the – 19 – different models, monitoring the behavior and identifying and mitigating DDoS attacks.

Fig. 3 is a block diagram showing a more detailed view of components in system 20 , according to some examples of the presently disclosed subject matter. Fig. 3 shows various sub‐components comprised in computers 21, 23, and 25. These sub‐ components are presented purely for illustrative purposes to facilitate a clearer understanding of the disclosed subject matter, while the description of sub‐components within system 20 , as shown, is non‐limiting, and various alternative configurations and designs are contemplated to be within the scope of the present disclosure.

Each computer comprises one or more processing circuitries (not explicitly shown) configured to execute several functional modules or units in accordance with computer‐readable instructions implemented on a non‐transitory computer‐readable storage medium of some sort. Such functional modules or units are referred to hereinafter as comprised in the computer.

Fig. 4 is a flowchart showing, in more detail, operations carried out by system 20 as part of the DDoS detection and mitigation process, according to some examples of the presently disclosed subject matter. For ease of understanding and by way of example only, operations shown in Fig. 4 are described below in conjunction with the components of Figs. 1 and 3 .

As explained above, to provide DDoS detection and mitigation services, system 20 is configured to continuously obtain logs (including layer‐7 logs) from network components of a monitored web‐domain (block 401 ). According to some examples of the presently disclosed subject matter, the relevant network components are configured to provide (e.g., transmit) logs to system 20 . For example, in case of CDNs, this may involve configuring the CDN to send the generated logs to a prescribed network address, e.g., of ingestion computer 21 or a database server which is accessible to ingestion computer 21 and is used for storing the raw logs. In other cases (e.g., in network components other than CDNs) it may require some reconfiguration – 20 – in the components such as installing a dedicated software agent (implementing for example dedicated APIs) that enables the relevant components to transmit the respective logs to a desired target computer. Notably, according to some examples, the only data which is obtained by system 20 is the logs, and no changes are required to be made to the network components or architecture, aside from providing the logs.

Logs which are obtained from the relevant network components are ingested (block 403 ) e.g., by ingestion computer 21 . Fig. 5 is a flowchart of operations carried out as part of the ingestion, according to some examples of the presently disclosed subject matter. Ingestion includes processing the logs and extracting from the logs the relevant log‐data (block 501; e.g., by logs processing unit 311 ).

During parsing of the logs, at least a minimal set of data, which is assumed to exist in all types of logs, is extracted. Any further information, such as request headers, response headers, HTTP method, or any other data, may be added and extracted to the customer’s unique data sources for the customer’s custom model.

Ingestion may further include standardizing the data according to a unified format (block 503 ; e.g., by logs standardization module 313 ) Notably, layer‐3, layer‐and layer‐7 logs of different network components and/or different vendors may have different formats and may comprise overlapping as well as nonoverlapping data. A non‐exhaustive list of examples of formats that may be used in layer‐7 logs include: JSON, XML, CSV, TSV, Web Server Access Logs, CDN Logs, Cloud Traffic Logs, Firewall Logs, Load‐Balancer Logs, Application Logs, etc. It is therefore required to standardize the format of the log‐data extracted from the logs. In some examples, different standardization schemes can be implemented for different vendors, depending on the vendor specific format, to thereby obtain a uniform format for all vendors.

While the specific data in layer‐7 logs may differ from one vendor to another, the minimal overlapping log‐data with respect to the requesting client generally includes: ‐ Timestamp ‐ The date and time the log event occurred.

‐ Client IP address ‐ The source IP address of the client making the request. – 21 – ‐ Request URL/Host ‐ The URL or host name being requested.

‐ Response code ‐ HTTP response codes like 200, 302, 404 etc.

‐ Bytes sent/received ‐ The size of the request and response.

‐ User agent ‐ The user agent string identifying the client software.

‐ Referrer ‐ The referring URL that linked to the request.

‐ Session ID ‐ A unique identifier for the session.

‐ Action ‐ The action taken, like ALLOW or DENY for firewalls.

‐ Protocol ‐ The protocol used like HTTP, HTTPS, FTP etc.

‐ HTTP Method ‐ URI or data that defines the URI Standardization of the log‐data may also be necessary for the generation of the universal ML model, which is created based on log‐data extracted from network communication of different customers. As different customers, owning different web‐domains, may use network components provided by different vendors and therefore have respective logs in different formats, standardization is needed to enable to process all logs uniformly.

As discussed, logs can be obtained from different network components, including CDN services 13 , firewall services 151 , load balancer services 153 , and servers 155 . However, since the logs are used for monitoring behavior of clients (both good and bad), it is preferable to obtain logs generated as close as possible to the Internet edge, where the client is located, as these logs comprise data which is the least modified, and accordingly can be more easily used for identifying the actual clients which generated the data and monitor the behavior at the client's domain. For example, an IP address obtained from a log‐file of an outer layer closer to the Internet edge (e.g., CDN) is most likely to be the true IP address of the client.

Thus, logs (or respective log‐data) are prioritized according to the type of the network device which generated the files (block 505; e.g., by data enrichment module – 22 – 315 in computer 21 ), where the closer the network component is to the Internet edge, the better it is. In some examples, if sufficient log‐data is obtained from a network component with higher priority, log‐data from other network components with lower priority is not used.

According to one example, log priority from top to bottom is: CDN logs, load balancer logs and application logs (i.e., running on servers). Firewall logs are an exception as they stand last in the priority chain because the firewall is layer‐4 or layer‐3, which only show a connection being made, and not the actual layer‐7 application requests (HTTP/HTTPS/WebSocket, etc.), and therefore include less data.

In some examples, ingestion also includes the classification of logs (or respective log‐data) according to their respective URI‐paths (e.g., Fully Qualified Domain Names (FQDN) and/or endpoints). The URI path and the IP address can be obtained from the logs, as explained above. As elaborated below with respect to block 403 in Fig. 4 , classification of logs to specific URI‐paths enables the generation of URI‐specific models.

Standardized log‐data are stored in one or more dedicated databases (e.g., implemented on an appropriate data‐storage server 317 ; block 509 ). In some examples, log‐data extracted from the logs is stored in a database dedicated for storing the log‐data of each specific customer individually (also referred to a "customer‐specific database"), for example, by assigning one or more identifiable records in a database to each customer. Log‐data can also be stored in a database designed to allow aggregation of log‐data obtained from network components of different customers (also referred to herein as a "universal database"). Log‐data stored in customer‐specific databases can be used for generating and updating customer‐specific models, and log‐data stored in the universal database can be used for generating and updating one or more universal models, as further disclosed below.

The operations described above with respect to blocks 401 and 403 are continuously repeated for the purpose of monitoring the behavior of all the monitored web‐domains, including generating and updating respective models and using the – 23 – models for detecting and mitigating DDoS attacks, as further described below.

Reverting to Fig. 4 , at block 407 the log‐data is used for generating one or more customer‐specific models (e.g., by customer‐specific‐models generator 331 in security management computer 23 ). These models are dedicated for characterizing, based on the log‐data, legitimate and illegitimate behavior observed with respect to a particular monitored web‐domain. The models enable to determine whether recorded log‐data represent normal, healthy behavior associated with the web‐domain, or abnormal behavior that indicates an occurring or pending DDoS attack. As different web‐domains are characterized by different network activity, generating customer‐specific models for each monitored web‐domain, based on customer‐specific log‐data generated in relevant network components, provides the ability to accurately identify abnormal activity in each monitored web‐domain. In some examples, if sufficient log‐data is obtained from more than one type of component, corresponding models are generated for each one of the components individually.

In some examples, different models are generated based on the continuous flow of log‐data, each model dedicated for determining particular information that serves as an indicator (referred to herein also as "DDoS indicator") of an ongoing or pending DDoS attack.

Some non‐limiting examples of customer‐specific models include: Request rate model ‐ determining and monitoring request rate observed in real‐time. According to this example, the normal request rate (e.g., number of requests per 5 minutes or per 5 seconds) observed in different network components in the network topology of a monitored web‐domain during normal operation, is recorded. Based on the recorded request rate, a respective normal range of request rates can be determined, and a request rate threshold value can be defined accordingly. During runtime, the request rate is monitored in real‐time, based on the log‐data which includes information indicative of request, and a DDoS indicator is identified, in case the observed request rate deviates from the threshold. Notably, a threshold can be defined as a range of values. – 24 – Determining the request rate based on the log‐data can be accomplished for example, by assigning logs into time windows (e.g., sliding time windows) of a predefined length (e.g., 60 seconds) and counting the number of requests made by each unique IP address in each time window. By summing the number of requests made by different unique IP addresses in each time window, an average request rate can be calculated, and an upper limit and lower limit can be determined. Time/date dependent request rate model ‐ Comparing time/date dependent request rate observed in real‐time to normal request rate. According to this example, the request rate (request rate range) which is commonly observed at specific times during the day and/or specific days during the week, and/or specific dates during the year, is monitored and recorded. Based on the recorded request rate, different respective threshold values can be determined. For example, one threshold value can be defined for working days, during working hours, a second threshold value can be defined for working days, for non‐working hours, and a third threshold value can be defined for weekends and holidays, etc. Notably, these thresholds are customer‐ specific, as different web‐domains observe different behavior at different times and dates, and enable to customize the thresholds according to the specific behavior that characterizes each web‐domain. Bandwidth model ‐ Monitoring and determining common bandwidth usage served and/or received. For example, during normal data communication in the monitored web‐domain, the size of each request (e.g., in Megabytes) can be obtained, for example, from the CDN logs, Load‐balancer, or web‐server logs. Based on the size of the requests, respective average values of communication volumes at different times and dates can be calculated. Assuming bandwidth logs or web‐server logs usage during a certain period (e.g., a 5 minute or 10 minute period) at a certain web domain has a certain common average value, bandwidth usage detected by the model in real‐time that deviates from this value can indicate a suspected DDoS attack or other malicious activity. This approach assists in exposing DDoS attack facilitators that implement a DDoS attack while maintaining the request rate of each bad actor (having – 25 – a respective IP address) below the request rate limit. Similar to request rate limits, bandwidth usage can also be time/date dependent. As further discussed below, bandwidth modelling can also be used during a DDoS attack to determine, based on the collective bandwidth, which is being occupied by the bad actors, whether the attack interrupts the operation of the web‐domain, to thereby help decide how to respond to the attack accordingly. In addition to the above, according to some examples, different customer‐specific models are generated for different FQDN and paths (e.g., application endpoints) in the same domain. For example, the Fully Qualified Domain Names (FQDN) of a certain web‐domain can be identified as well as the different URI‐paths in each FQDN. Consider the following illustrative example of web‐domain "www.domain.com, which includes the following Uniform Resource Identifiers (URIs): https://app.domain.com/auth/login https://app.domain.com/auth/register https://app.domain.com/auth/forgotpassword https://app.domain.com https://app.domain.com/contact/email https://app.domain.com/support https://support.domain.com https://www.domain.com In the above example, there are three FQDN, including: www.domain.com, app.domain.com and support.domain.com, where, in the app.domain.com FQDN the following endpoints exists, where each endpoint defined by a respective URI‐path: /auth/login /auth/register /auth/forgotpassword Thus, according to this example, as part of the generation of customer‐specific models, logs obtained from network devices associated with the monitored web‐ 30 – 26 – domain (e.g., "domain.com") are classified according to the respective FQDN and paths as mentioned above with respect to block 507 . Each subset of log‐data classified to specific target URI‐path (e.g., FQDN or endpoint) is used for generating a respective URI‐specific model dedicated for determining whether the log‐data represent normal and legitimate behavior observed in a particular URI‐path in the monitored web‐ domain. The URI‐specific models can include for example: URI‐specific request rate model ‐ determining and monitoring request rate of a specific URI‐path. The normal request rate observed in different URI‐paths is monitored and recorded. Based on the recorded request rate, a respective request rate threshold value can be determined. During inference, the request rate in different URI‐paths is monitored in real‐time, and deviations of the observed request rate from the threshold, which exceeds a certain value, is identified as a DDoS indicator. URI grouping model ‐ identifying and recording based on the log‐data groups of URIs which are frequently accessed together. This model operates by analyzing patterns of access to various URI‐paths, identifying instances where specific paths are consistently accessed together e.g., followed in succession. For example, an access to a certain application may always start at a certain FQDN (e.g., /auth/login) followed by a set of three other FQDN, which is application or web‐domain dependent.

During inference, the model determines, in real‐time, based on the log‐data, whether URI‐paths that are expected to be accessed in a certain way are indeed accessed as expected. If one or more access requests made by a certain actor (identified by a respective IP address) deviates from the common pattern, i.e., does not involve URIs which are commonly accessed together, a DDoS indicator is identified, and the requesting actor is classified as suspicious or malicious.

URI‐specific bandwidth model ‐ Monitoring and modeling common bandwidth usage served and/or received observed at particular URI‐paths. As explained above, this information can be used for identifying anomalies in behavior, even if the observed request rate is within limits. – 27 – URI‐specific engagement model ‐ monitoring and modeling period of engagement in activity, which is commonly observed at particular URI‐paths. For example, assume the average time range of engagement of an IP address with a certain URI‐path (e.g., a certain endpoint) is 1 to 3 minutes. An observed engagement period that deviates from this range can indicate a suspected DDoS attack or other malicious activity. As before, this approach assists in exposing attackers that take part in an attack, while maintaining the request rate of each bad actor (having a respective IP address) below the request rate limit. An engagement period can be determined, for example, by recording the period starting from a first appearance of a legitimate IP address in a log, until the log is no longer found in the logs. Notably, the models described herein can be used for determining that an IP address is legitimate when an indication of malicious activity is not detected. At block 409 one or more universal models are generated (e.g., by universal‐ models generator 333 in security management computer 23 ). Unlike the customer‐ specific models, universal models are generated using log‐data collected from network components which are associated with multiple, independent web‐domains of different monitored customers (also referred to herein as "universal log‐data"), providing a broader and more diverse data set for the universal model creation. The universal models provide DDoS indicators which are relevant to all customers which operate in the same network. Examples of universal models include: IP anomaly detection model ‐ designed to identify suspicious IP addresses, when monitoring a certain web‐domain, by cross‐referencing IP addresses extracted from customer‐specific logs with IP addresses that have been previously identified as participants in DDoS attacks. As mentioned above, log‐data includes the IP addresses assigned to the clients (good and bad), which access the web‐domain. According to some examples a blacklisting IP database ( 360 ) is created and used for storing suspicious IP addresses. IP addresses which are identified while taking part in a DDoS attack occurring at any one of the monitored web‐domains of different monitored customers, are stored in the database. As part of the detection of DDoS attack 30 – 28 – executed by system 20 , IP addresses extracted from the logs obtained from the network components of the monitored web‐domains are compared against IP addresses stored in the IP database, to identify suspicious IP addresses.

The IP anomaly detection model can be further configured to identify suspicious IP addresses based on additional data extracted from the logs. For example, the browser type and version can be obtained from the user agent in the log‐data. The usage of an outdated (e.g., obsolete) browser version typically raises suspicions of malicious intent, since it is unlikely that it is being used by a legitimate user.

Botnet detection model ‐ a model dedicated to identifying IP addresses being part of a botnet by cross‐referencing IP addresses extracted from the logs with IP addresses that have been previously identified as participants of a botnet. According to the presently disclosed subject matter a botnet database ( 361 ) is created, dedicated for creating groups of IP addresses, which are identified to take part in botnet attacks occurring at any one of the monitored web‐domains of different monitored customers.

During an attack on a web‐domain, IP addresses that take part in the attack are stored, where the different IP addresses are associated together to create a botnet (also referred to herein below as "deduced botnet"). Deduced botnets are created based on the observed collaborative engagement of IP addresses in an attack, indicating the possibility that they are part of a botnet. If different IP addresses are repeatedly identified together during different DDoS attacks, this further supports the possibility that they are part of the same botnet. In some examples, a validity score can be assigned to each deduced botnet based on the number of times (e.g., average number) the IP addresses in the botnet have been identified to take part together in an attack.

As further disclosed below, the botnet database can be used for identifying the first IP addresses (acting for example as "sniffers") that access a web‐domain, before the full DDoS attack takes place. This enables the system to prepare and mitigate the full attack before it occurs by performing various preventive actions. For – 29 – example, if a sniffer IP address is identified indicating an imminent botnet attack of a familiar botnet, all IP address recorded as part of the botnet can be blocked before the full‐scale DDoS attack occurs, thus causing the attack to fail.

In addition, during a botnet attack, the attack capacity of the botnet (i.e., the maximal bandwidth occupied by the botnet) can be determined and stored with the respective botnet. During an attack this information can be used for determining whether an ongoing attack jeopardizes the operation of the web‐domain which is being attacked, and execute a response accordingly. Maximal bandwidth occupied by a botnet can be determined for example, by extracting from logs, generated by network components in relation to bots in a botnet, information about bytes sent/received and combining this information to determine the overall attacking bandwidth of the botnet. At block 411 data enrichment models are generated. Data enrichment, which can be otherwise considered as another type of universal model, includes the use of additional data, obtained from data sources other than the logs, to further enhance the detection and classification abilities of system 20 . Data enrichment can be executed for example by data enrichment module 315 .

One example of data enrichment includes using data obtained from third party vendors. For example, the IP database ( 360 ) and the botnet database ( 361 ) can be enriched using information about IP addresses obtained from third party vendors with respect to the reputation of various IP addresses. Data enrichment can also include the identification of the source ISP of a suspicious IP address, as further discussed below.

A second example of data enrichment that can be applied in conjunction with a universal model is related to ransom attacks, where the targeted domain receives a ransom email or other messages which generally include threats of a pending cybersecurity attack such as a DDoS attack, requesting various demands, often involving payment, in return for holding back the attack.

Ransom DDoS attacks often involve a preliminary stage characterized by an – 30 – attack on a smaller scale ("Demo Attack") aiming to demonstrate, to the targeted domain, the ability and intent of the attacker. Many times, ransom messages (e.g., ransom emails) are sent to the targeted web‐domain as part of a preliminary ransom DDoS attack. According to the presently disclosed subject matter, IP addresses which are identified during ransom attacks are recorded and associated with information identifying the attacker, which is extracted from the ransom email. This information can be stored for example in the IP database ( 360 ) or the botnet database ( 361 ), thus creating groups of IP addresses which are associated with a certain attack or attacker (referred to herein as "ransom botnets").

In some examples, in response to receiving a ransom email at a monitored web‐domain, the email is transmitted to system 20 where it is processed (using for example natural language processing e.g., by ransom mail analyzer analysis module 381 ), and if the ransom email is identified as being related to a ransom botnet stored in the database, the IP addresses associated with the ransom botnet in the database can be identified and blocked. This can be done for example during a demo attack, causing the demo attack to fail, and thereby dissuading the attacker from continuing with the full‐scale attack that normally follows.

Based on the log‐data analysis and the different models, customer‐specific security posture is determined (block 413 ). This includes for example automatically determining request rate thresholds for different network components and/or FQDNs and/or specific URI‐paths in monitored web‐domains, based on the observed normal behavior deduced from the logs. The customer‐specific security posture is automatically applied to the various network components of the monitored web‐domain (block 415 ). In some examples automatic posture configuration module 335 can be operatively connected to various network components in the network routes of the monitored web‐domain (e.g., CDNs, Firewalls, Load Balancers, etc.) and configured to provide instructions controlling various parameters that govern the security posture in these devices. The network components can be configured to be accessible and controllable by automatic posture configuration module 335 for this purpose. 30 – 31 – In some examples, at least part of the models (including customer‐specific and universal models) can be implemented as machine learning models (including one or more of supervised, unsupervised, and reinforcement learning) which can be repeatedly trained as new log‐data, is obtained from the network (where training can be carried out as part of the modeling operations of block 205 ). One or more machine learning classifiers can be implemented for determining, based on features extracted from the logs, whether a DDoS attack is occurring or is imminent. In some examples, log‐data, such as IP address, user‐agents, URIs, ASNs, HTTP Methods, Query Strings, etc. are used as input for training the ML models.

The training dataset can be used for training a suitable classifier machine learning model, such as support vector machine (SVM), random forest, or logistic regression, as well as neural networks. If the training dataset is sufficiently large, a deep neural network can be used for classification, such as a convolutional network with dense layers. In some examples, part or all of the DDoS indicators specified above can be used as input features to an ML classifier. An ML classifier according to the presently disclosed subject matter can be implemented as a binary classifier providing output indicating the probability that a DDoS attack is occurring or is imminent. Alternatively, or additionally, an ML classifier can be implemented as a multi‐class classifier, which provides output indicating, in addition to whether a DDoS attack is occurring or is imminent, also a classification which suggests a recommended response to the attack.

Turning to the detection and classification (block 207 ;) of malicious attacks, during inference log‐data is continuously obtained from the relevant network components of the monitored web‐domain and ingested. The models, including both customer‐specific models and universal models, are applied on the ingested log‐data to detect DDoS attacks (block 417; e.g., by DDoS detection and classification module 351 ). In some examples, one or more enrichment models are also used for this purpose.

In some examples, one or more URI‐specific models are applied as well. To this – 32 – end, logs are classified according to their respective URI‐paths, and the relevant subset of log‐data is applied for DDoS detection and mitigation in the respective URI‐path. When URI‐specific models are applied, request rate model can be applied on log‐data obtained from logs addressed to specific URI‐paths, each corresponding to a respective endpoint. As explained above, different endpoints can be assigned with a respective request rate threshold. It can thus be determined whether the request rate observed at particular URI‐paths exceeds the respective request rate threshold.

In some examples, the output of different models and other data can be consolidated, where the final decision as how to respond is made, based on the compiled output of the models. For instance, the models and other data can be used for checking if an IP address is listed in an IP blacklisting database, determining if its access rate exceeds a predefined threshold, evaluating whether the engagement duration surpasses a specified limit, and considering enrichment data, such as the identification of outdated web browsers. The final decision regarding the occurrence of a DDoS attack is reached by collectively weighing these outputs. System 20 may apply multiple models on the log‐data and assign to each output of each model ("DDoS indicator") a respective score, where, in case the score of a single output or the accumulative score of multiple outputs is above a certain threshold, a positive identification of a DDoS attack is determined. Different models may be assigned with different weights, determined based on their respective importance in determining an attack.

If a DDoS attack threat is detected (block 419 ), instructions to execute actions dedicated for mitigating the threat (herein below "mitigating actions") can be automatically generated by the system (block 423 ; e.g., by mitigation orchestrator 353 ). In some examples, mitigation orchestrator 353 is configured to analyze the characteristics of the threat and provide instructions to the system to execute certain mitigation actions. The mitigation actions are selected according to the specific type of threat which has been identified. Fig. 6 is a block diagram illustrating components of mitigation orchestrator 353 . According to the illustrated example, module 353 comprises a mitigation decision logic 30 – 33 – 601 , disruption analysis module 603 , and health monitor module 605 .

Mitigation decision logic module 601 is configured responsive to receiving an indication (e.g., from detection and classification module 351 ) that a DDoS is occurring or is pending, to decide which mitigation action to execute in response to the relevant threats.

Assuming for example a few suspicious IP addresses are identified, the threat is analyzed, and one or more suitable mitigation actions are selected. If, for example, the IP addresses are identified by the request rate model indicating that their request rate is greater than the acceptable request rate threshold, the mitigation actions can include automatically hardening the request rate limitation and/or blocking the identified IP address, and/ or automatically activating bot mitigation.

As explained above, the request rate thresholds are defined based on the normal request rate observed in different network components during normal operation. Therefore, hardening the request rate threshold (i.e., reducing the number of requests allowed per a given period), although it may effectively restrict access for bad actors who exceed this threshold, would have minimal to no impact on good actors who operate within these established limits. This advantage is even more substantial when applying URI‐specific request rate thresholds, since the threshold assigned to each URI accurately represents the commonly observed request rate in the respective URI‐path.

If, however, the IP addresses are identified as suspicious by the IP anomaly detection model, indicating that the IP addresses have previously participated in a DDoS attack, while their request rate is below a threshold (e.g., engaging in a large scale and highly distributed DDoS attack), increasing the request rate limitation would be ineffective. In such cases the mitigation actions may include blocking the suspicious IP addresses.

This example also demonstrates the advantage in the combined operation of the customer‐specific model which enables the accurate request rate update and the universal model which enables the accurate identification and blocking of suspicious – 34 – IP addresses. Notably, hardening the request rate has an advantage over blocking of specific IP addresses, since the former prevents the attacker from using other bots assigned with different IP addresses, instead of those which were blocked.

If the option of bot mitigation is made available by the web component, it can be also automatically activated responsive to detection of suspicious IP addresses. This includes, for example, automatically activating Captcha or JavaScript injections.

If blocking is not possible, for example in case IP blocking is unavailable or disabled, or if blocking of an IP address does not effectively mitigate the attack, mitigation actions may include applying an attack life‐cycle operation on the source ISPs, as further explained below.

In case URI‐specific models are applied and a DDoS attack is detected at a particular URI‐path, appropriate mitigation actions can be selected. It can be decided whether to apply one or more mitigation actions in the particular URI‐path alone, in additional URI‐paths, in one or more particular FQDNs, or in all web‐domain assets, for example, whether to harden the request rate threshold and/or block access of IP addresses only to the particular URI‐path or other URI‐paths as well. This provides greater granularity when applying DDoS mitigation and helps in limiting degradation of service to particular URI‐paths.

According to some examples, before any mitigating action is carried out, it is determined whether a disruption in the operation of the monitored web‐domain is detected (block 421 ; e.g., by disruption analysis module 603 ). If a disruption is not detected, certain actions may be avoided and the system may provide a warning instead. For example, in case one or more suspicious IP addresses are identified as being part of botnet which has been previously observed to take part in a DDoS attack (e.g., based on the information in database 361 ) and information indicating the expected attack bandwidth of the botnet is available in the database, the bandwidth is compared with the bandwidth of the targeted web‐domain and its Firewall capacity. If the attack bandwidth of the botnet does not present a risk to the operability of web‐domain, request rate limiting update may not be implemented. – 35 – In general, if a disruption to the operation of the web‐domain is not detected, this may indicate that the current security posture of the web‐domain can effectively thwart the attack, and further intervention is not required. Avoiding the execution of a mitigation action, which often involves resource‐intensive processes, can be advantageous, as it effectively minimizes the expenditure of valuable computer resources and prevents potential disruptions to the normal operation of a network. Once a threat is detected and classified as one which does not disturb the operability of the network, it is continuously monitored, and, if a disruption is identified, certain mitigating actions that were avoided may be executed. In some examples, determining whether a detected DDoS attack is disruptive also includes determining whether the state of the network component is operational and accessible. For example, health module 605 in mitigation orchestrator 353 can be configured to execute an "IS UP" query to a frequently accessed endpoint in the web‐domain. In some cases, mitigation actions are executed only if the "IS UP" query returns negative feedback. Health monitor module 605 can be configured to perform additional operations related to the health of the monitored web‐domains. This includes, for example, robotic process automation (RPA) or scraping technologies used for identifying how a web application behaves even before the first log is received. This allows to cross‐correlate with the logs received and test the application unobtrusively, to determine if the site is down due to DDoS indicators. Health monitor module 605 can be configured to perform health monitoring operations repeatedly, e.g., every minute. At block 425 one or more mitigating actions are automatically executed (e.g., by attack mitigation module 371 ). The one or more mitigation actions include reconfiguring one or more security posture parameters, such as: ‐ Automatically and dynamically reconfiguring security modules of CDN/ADN network components, including: o reconfiguring IP control modules 131 , including for example, blocking suspicious IP addresses and updating Access Control Lists 30 – 36 – (ACLs); these may include IP addresses listed in the blacklisting IP database 360 or botnet database 361 and IP addresses that were classified as suspicious (e.g., based on anomalous behavior detected in the monitored web‐domain). o reconfiguring Web Application Firewall module 133 . o reconfiguring rate limit module 135 , for example, by hardening request rate limitation, the allowed frequency of requests in a certain time frame is reduced, thereby preventing malicious agents from executing the attack in its full capacity. As explained above, mitigation actions including request rate limitation, and IP blocking can be applied to specific URIs. o reconfiguring bot mitigation module 137 . ‐ Automatically and dynamically reconfiguring Firewall 151 and Load Balancer 153 . In case an appropriate API exists and system 20 is given the needed permission, the above operations can be done automatically, e.g., by instructions generated by attack mitigation module 371 executed at the relevant network component. In some examples, if an API and/or permission is not available, a recommendation suggestion to perform this action can be displayed at a computer device, alerting the relevant stakeholders (e.g., on a dashboard by reporting and presentation module 357 ). At block 427 it is determined whether the mitigating actions have successfully mitigated the threat. For example, following the execution of the mitigation actions, logs are continuously ingested, and the log‐data is analyzed to determine whether requests made by the bad actors are either diminished or stopped altogether. These operations can be carried out for example by threat monitoring module 379 configured to provide feedback, e.g., to mitigation orchestrator 353 , indicating whether the mitigation has been successful or not. According to examples of the presently disclosed subject matter, if the threat has been mitigated, the system automatically reverts to the baseline security posture configurations (block 431 ). In one example, if the request rate limitation has been 30 – 37 – hardened, responsive to determining that the attack has subsided, the request rate limitation is changed back to the baseline values before the attack (e.g., as defined by the security posture of the web‐domain). In another example, if a specific IP address has been blocked, the system may continue and block the IP address as long as it appears in logs. Once the IP address no longer appears in the logs, the IP address is allowed to access the system once more. This enables the system to automatically revert to its full capacity following mitigation of the attack. To enable automatic roll‐back to previous (e.g., baseline) security posture, changes made to the security posture are documented and version controlled, allowing complete audit trail of the actions made to achieve the security posture. Automatic reconfiguration of the security posture, including increasing and decreasing security measures and/or releasing blocked IP addresses, can be carried out for example, by automatic (security) posture configuration module 335 . Automatic posture configuration module 335 can be responsive to mitigation orchestrator 335 to apply roll back. Alternatively, if it is determined at block 429 that the threat has not been mitigated, additional mitigation actions may be carried out. According to one example, certain attack life‐cycle actions are applied. These measures include warnings, which may be sent to the source ISPs of the bad actors (e.g., by ISP notification module 373 ). For example, threat monitoring module 379 may provide feedback to mitigation orchestrator 353 , indicating that mitigation has been unsuccessful. As mentioned above, in some examples determination of a source ISP of a certain IP address can be done as part of enrichment, e.g., by data enrichment module 315 . Alternatively, or additionally, determination of a source ISP can be carried out by ISP notification module 373 . This can be accomplished by a "who is" query applied on an IP address which returns an Autonomous System Network (ASN) identifier. The ASN can be used to identify the ISP and vendor. Once the ISP and vendor are known, a warning can be sent to the vendor. The warning can notify the vendor that its ISP is being used for executing a DDoS attack, requesting the vendor to execute measures for stopping the attack. Identification of the ASN and generation and transmission of 30 – 38 – warnings can be performed, for example, with the help of generative or contextual AI. Warnings can be sent through email, voice message, telephone message, or any other means.

At block 435 , according to some examples, the system is configured to wait for feedback from the notified ISPs for a predefined period (e.g., a few minutes). If no response is received (or a received response is not satisfactory) and the warning has not caused the attack to subside, then attack life‐cycle actions are executed (e.g., by attack life‐cycle module 375 ). Execution of the attack life‐cycle actions can include the activation of multiple computer devices for executing attack life‐cycle actions dedicated for monitoring the bad actors and determining their attacking abilities, such as their attack bandwidth. Notably, in other examples attack‐life‐cycle actions are executed independently of the warning, e.g., without the issuance of a warning or without waiting for feedback.

Devices used for executing the attack life‐cycle actions (also referred to herein as "triage devices") can be implemented for example by dedicated triage servers (e.g., cloud machines) configured to execute attack life‐cycle actions on the offending malicious IP addresses. Attack life‐cycle module 375 can be configured to dynamically activating multiple triage servers (physical machines and/or virtual machines) configured to receive triage data, including identifying the targets (e.g., network paths leading to the offensive devices) and validating the accuracy of the DDoS attack's capability such as bandwidth. Triage servers can be dynamically added or reduced, according to the required triage activities. During execution of the attack life‐cycle operations, logs are continuously obtained and processed until a sufficient accurate result of the bad actors (IP addresses originating from the offensive devices) is identified in the logs, and the monitored web‐domain is up and operative. Following execution of attack life‐cycle operations, if the DDoS attack has been mitigated, the system automatically reverts to the baseline security posture (block 431 ). Otherwise, the attack life‐cycle operations may continue (block 435).

Fig. 7 is a flowchart of attack life‐cycle operations carried out, according to – 39 – some examples of the presently disclosed subject matter. Operations described with reference to Fig. 7 can be executed for example by attack life‐cycle module 375 .

As explained above, attack life‐cycle action can be triggered responsive to failure to receive a satisfactory reaction from a notified ISP. As explained above, analysis of logs can be used for identifying IP addresses which participate in a DDoS attack, e.g., as part of a botnet. In some examples a subset of IP addresses is selected from the attacking IP addresses extracted from the logs (block 701 ). In other examples, the operations are performed on all IP addresses.

Life‐cycle actions are initiated targeting the IP addresses in the subset (block 703 ). This includes, for example, sending traffic (e.g., for example UDP/TCP/ICMP traffic) towards the source of the IP addresses.

The logs are continuously monitored and analyzed to determine whether the IP addresses in the subset continue to appear in the logs (block 705 ). If some or all of the IP addresses continue to appear in the logs, the amount/volume of traffic that targets the IP addresses is increased, e.g., by a predefined incremental value (block 707 ).

If the IP addresses (or some predefined percentage of the addresses) no longer appear in the logs, this indicates that the traffic sent towards them has succeeded in overwhelming them and has stopped their own traffic. Based on the amount of traffic that was used for incapacitating the offensive ability of the IP addresses in the subset, the offensive capability of the subset and/or botnet is determined. This information can be stored, e.g., as part of a respective universal model specifying the IP addresses and their respective estimated offensive capability. This information can be used when mitigating future DDoS attacks implemented using these IP addresses, as explained above.

Information on the monitored web‐domains can be presented to various stakeholders, e.g., at any one of: Security Operations Center, Network Operations Center, Cyber Security Operations Center, and so forth. For example, reporting and presentation module 357 can be configured generate, present, and update different – 40 – information including ongoing status of monitored web‐domains, including real‐time security posture, granular representation of particular URI‐paths and respective security posture, detected threats and their level of disruption, suggested/executed mitigation actions, and the mitigation actions outcomes. In some examples reporting and presentation module 357 is configured to generate and present a dashboard which provides a consolidated graphical representation of the data. The dashboard can present alerts and root cause analysis of detected attacks as they occur, as well as the required mitigation actions. A well‐defined dashboard presentation enables stakeholders to proactively monitor cybersecurity threats. The interface offers actionable insights for informed decision‐making and resource allocation, and, in some examples, allows stakeholders to manually execute operations and/or override the automatic decisions made by system 20 if needed, enabling adaptive threat response. As mentioned above, the presently disclosed subject matter allows for efficient migration between different network components vendors (e.g., switching between CloudFlare and Akamai for receiving CDN/ADN services). Since the log‐files of different vendors generally provide the same data, log‐files generated by network components of different vendors can be similarly used for generating models, monitoring, and mitigating DDoS attacks. Accordingly, in case a security posture of a given web‐domain has been determined while using the services of one vendor, the same security posture and the same models can be implemented when using the services of a different vendor. This enables to implement the security postures and models determined when operating with a first vendor immediately following migration to another vender, with no or very little delay.

It further allows for an automatic transition between vendors which requires very little or no human intervention as compared to existing techniques (in some cases a different standardization scheme is applied adapted for the new vendor, as mentioned above.). Responsive to a request to change from a first vendor to a second vendor, the migration between the vendors can be executed automatically by connecting to the new vendor, and automatically setting the security posture and 30 – 41 – models by copying the security posture and models from the first vendor and implementing them at the second vendor.

It will also be understood that the system according to the presently disclosed subject matter may be a suitably programmed computer. Likewise, the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the method of the presently disclosed subject matter. The presently disclosed subject matter further contemplates a machine‐readable non‐transitory memory tangibly embodying a program of instructions executable by the machine for executing the method of the presently disclosed subject matter.

It is to be understood that the presently disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the present presently disclosed subject matter.

Claims (29)

– 42 – CLAIMS:

1. A computer‐implemented method of distributed denial of service (DDoS) attack detection and mitigation in a monitored web‐domain of a customer, the method comprising: continuously obtaining customer‐specific log‐data from logs generated by network components of the monitored web‐domain; wherein the logs are collected from the network components without requiring direct access and processing of network traffic transferred through the network components of the monitored web‐domain; processing the customer‐specific log‐data and identifying, based on the customer‐specific log‐data, data indicative of an occurring or pending DDoS attack; automatically executing one or more mitigating actions dedicated for mitigating the DDoS attack.

2. The method of claim 1, wherein the network components include devices and applications implementing one or more of the following network components: Content Delivery Network (CDN); Web Application Firewall; Load Balancer; Virtual Server; Bot Mitigation service (Bot), and Web Application server.

3. The method of claim 1, wherein the network components include devices and applications implementing one or more of the following types of network components: Content Delivery Network (CDN); Web Application Firewall; Load Balancer; Virtual Server; and Web Application server; the method comprising: prioritizing customer‐specific log‐data received from different types of network components according to a distance of the component from a network edge, where the closer a network component is to the network edge, the higher the priority of a respective log‐date.

4. The method of any one of the preceding claims, wherein processing the customer‐specific log‐data includes generating at least one customer‐specific model based on customer‐specific log‐data obtained from the network components of the monitored web‐domain configured for determining whether customer‐specific log‐data received from network components of the monitored web‐domain represents 30 – 43 – normal behavior or behavior that indicates an occurring or pending DDoS attack.

5. The method of claim 4, wherein the at least one customer‐specific model includes a request rate model representing a normal range of request rates; the method comprising: determining, based on the customer‐specific log‐date, real‐time request rate; in case it is determined that the real‐time request rate deviates from a request rate threshold defined by the request rate model, identifying a DDoS indicator.

6. The method of claim 5, wherein the request rate model is configured to apply different respective thresholds to different annual dates, weekdays, and/or times.

7. The method of any one of claims 4 to 6, wherein the at least one customer‐specific model includes a bandwidth model representing a normal bandwidth usage patterns; the method comprising: determining, based on the customer‐specific log‐date, real‐time bandwidth usage; in case it is determined that the real‐time bandwidth usage deviates from a bandwidth threshold defined by the request rate model, identifying a DDoS indicator.

8. The method of any one of claims 4 to 7, wherein applying at least one customer‐specific model comprises: classifying log‐data to respective Uniform Resource Locator (URI) paths to thereby obtain URI‐specific log‐data; and applying one or more URI‐specific models, each URI‐specific model generated based on the customer‐specific log‐data obtained from a respective URI‐path and configured for determining whether respective log‐data associated with the respective URI‐path represent normal network behavior or network behavior that indicates an occurring or pending DDoS attack.

9. The method of claim 8, wherein the one or more URI‐specific models include one or more of: ‐ URI‐specific request rate model, configured for determining whether 30 – 44 – respective customer‐specific log‐data associated with a specific URI‐path indicate normal request rate; ‐ URI grouping model, configured for determining whether groups of URI‐paths in the monitored web‐domain, which are expected to be accessed together, are indeed accessed simultaneously; ‐ URI‐specific bandwidth model, configured for determining whether bandwidth usage detected in real‐time at a specific URI‐path, deviates from common bandwidth usage in the specific URI‐path; and ‐ URI‐specific engagement period model configured for determining whether respective customer‐specific log‐data associated with a specific URI‐path, indicate normal engagement period with the specific URI‐path.

10. The method of any one of claims 4 to 9, wherein processing the log‐data includes applying at least one universal model generated based on universal log‐ data obtained from network components of multiple independent monitored web‐domains; the universal model is configured for determining whether the universal log‐ data indicates an occurring or pending DDoS attack.

11. The method of claim 10, wherein the at least one universal model includes an IP anomaly detection model dedicated for identifying suspicious IP addresses, by cross‐referencing IP addresses extracted from the customer‐specific log with IP addresses that have been previously identified as participants in DDoS attacks.

12. The method of claims 10 and 11, wherein the at least one universal model includes a botnet detection model dedicated for identifying IP addresses being part of a botnet by cross‐referencing IP addresses extracted from the customer‐specific log with IP addresses that have been previously identified as participants of a botnet.

13. The method of claim 12 further comprising: following identification of one or more IP addresses which are part of a botnet, automatically executing at least one of the following mitigating actions: blocking all IP addresses recorded as part of the botnet; hardening an access rate threshold for accessing the monitored web‐domain; 30 – 45 – and activating bot mitigation.

14. The method of claims 12 and 13 further comprising: determining attack capacity of the botnet; determining whether the botnet is disruptive to operability of the monitored web‐domain based on a comparison between the attack capacity of the botnet and bandwidth capacity of the monitored web‐domain; and avoiding execution of certain mitigation actions if the botnet is not disruptive to the operability of the monitored web‐domain.

15. The method of any one of claims 12 to 14, wherein the botnet is a ransom botnet, the method comprising: analyzing a ransom email received from an attacker and identifying information in the email identifying the attack; identifying a respective ransom botnet which has been previously associated with the attacker; and automatically executing at least one mitigating action targeting the botnet.

16. The method of any one of claims 1 to 14 comprising: applying on the customer‐specific log‐data, multiple customer‐specific models and at least one universal model, each model providing a respective data output indicative as to whether or not a DDoS attack is occurring or pending; and confirming an occurring or pending DDoS attack based on compilation of the respective data outputs.

17. The method of any one of claims 1 to 13 comprising: determining whether the DDoS attack is disruptive to operability of the monitored web‐domain; and avoiding execution of some or all mitigation actions if the DDoS attack is not disruptive to the operability of the monitored web‐domain.

18. The method of any one of claims 1 to 17, comprising: automatically determining a security posture of the monitored web‐domain, including setting respective request rate thresholds to different network components of the monitored web‐domain. 30 – 46 –

19. The method of claim 18, wherein automatically determining a security posture of the monitored web‐domain includes automatically determining request rate thresholds to specific FQDNs and/or URI‐paths in the monitored web‐domains.

20. The method of any one of the preceding claims wherein the one or more mitigation actions include automatically reconfiguring one or more security posture parameters including one or more of: ‐ automatically hardening a request rate limitation in the monitored web‐domain; ‐ automatically blocking one or more IP addresses; and ‐ automatically activating bot mitigation.

21. The method of any one of the preceding claims comprising: following execution of the mitigation action, continuing the processing of the log‐data; and automatically stopping the execution of the one or more mitigation actions, if data indicating that the DDoS attack has ceased or subsided is identified.

22. The method of claim 21 comprising: automatically reverting security posture parameters to baseline values following identification of the data indicating that the DDoS attack has ceased.

23. The method of any one of the preceding claims, wherein automatically executing one or more mitigating actions includes sending warnings to one or more Internet Service Providers (ISPs) identified as a source of IP addresses that were identified to take part in the DDoS attack.

24. The method of any one of the preceding claims, wherein automatically executing more or more mitigating actions includes executing one or more life‐cycle actions, comprising sending traffic targeted towards IP addresses identified in the logs; continuously analyzing the logs to determine whether the IP addresses continue to appear in the logs; once some or all of the IP addresses cease to appear in the logs, determining an offensive capability of the IP addresses based on an amount of traffic sent towards them.

25. The method of any one of claims 18 to 24 comprising following migration of the customer from a first network components vendor to a second 30 – 47 – network components vendor, applying the determined security posture while operating with the first network components vendor at the network components of the second network components vendor.

26. The method of any one of claims 4 to 25, wherein the network components are of a first network components vendor, the method comprising following migration of the customer from the first network components vendor to a second network components vendor, applying the at least one customer‐specific model determined while operating with the first network components vendor at the network components of the second network components vendor.

27. A computer program product comprising a computer readable storage medium retaining a program of instructions, which, when read by a computer processor, causes the computer processor to perform a method according to any one of claims 1 to 26.

28. A non‐transitory program storage device readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method according to any one of claims 1 to 26.

29. A computer system comprising at least one processing circuitry configured to execute a method according to any one of claims 1 to 26.