WO2022012054A1 - 一种动态预防流量攻击的方法、系统、设备及存储介质 - Google Patents

一种动态预防流量攻击的方法、系统、设备及存储介质 Download PDF

Info

Publication number
WO2022012054A1
WO2022012054A1 PCT/CN2021/077807 CN2021077807W WO2022012054A1 WO 2022012054 A1 WO2022012054 A1 WO 2022012054A1 CN 2021077807 W CN2021077807 W CN 2021077807W WO 2022012054 A1 WO2022012054 A1 WO 2022012054A1
Authority
WO
WIPO (PCT)
Prior art keywords
traffic
processing
token
service
token generation
Prior art date
Application number
PCT/CN2021/077807
Other languages
English (en)
French (fr)
Inventor
田琳
Original Assignee
苏州浪潮智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州浪潮智能科技有限公司 filed Critical 苏州浪潮智能科技有限公司
Priority to US18/013,782 priority Critical patent/US20230319102A1/en
Publication of WO2022012054A1 publication Critical patent/WO2022012054A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/215—Flow control; Congestion control using token-bucket
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/22—Traffic shaping
    • H04L47/225—Determination of shaping rate, e.g. using a moving window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/50—Queue scheduling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1408—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic by monitoring network traffic
    • H04L63/1425—Traffic logging, e.g. anomaly detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441—Countermeasures against malicious traffic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/14—Network architectures or network communication protocols for network security for detecting or protecting against malicious traffic
    • H04L63/1441—Countermeasures against malicious traffic
    • H04L63/1458—Denial of Service
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/20—Network architectures or network communication protocols for network security for managing network security; network security policies in general

Definitions

  • the invention relates to the field of traffic attack prevention, in particular to a method, system, device and storage medium for dynamically preventing traffic attacks.
  • the application system needs to deal with the traffic attack on the application system in scenarios such as a large amount of data processing required by the background logic or unexpected request volume (such as a series of centralized operations of a large number of users), and these traffic attacks will affect the stability of the application system.
  • the system's CPU (Central Processing Unit, central processing unit) utilization rate, memory utilization rate, message transmission channel, protocol chain, etc. bring great load challenges.
  • current limiting is to limit the concurrency of traffic through certain means for scarce resources (such as CPU, memory, etc.) or according to business needs, so as to effectively protect the stability of the application system.
  • current limiting algorithms are mainly divided into three types: counter/time window method, leaky bucket method and token bucket method.
  • the counter/time window method achieves current limiting by setting the maximum number of accesses per unit time. For example, if QPS (Queries-per-second, query rate per second) is set to 60, the interface can only be accessed per second. 60 times. However, if the traffic peaks in the first 1% of the time, the latter 99% of the time will be silent and blocked from the door. This defect is also known as the "spike effect".
  • QPS Quality-per-second, query rate per second
  • the leaky bucket method effectively avoids the "stab effect".
  • the leaky bucket method is realized by a fixed-size queue and the method of periodically fetching elements, just like a leaky bucket containing water, which only limits the rate of water output from the container. When the water intake rate is too high, it will overflow, that is, the flow of rejected. However, leaky buckets cannot really handle sudden traffic peaks, and the efficiency is not high.
  • the token bucket method is based on the bucket prototype, but unlike the leaky bucket, there is no water outlet.
  • the flow is controlled by generating the token rate plus the volume of the token bucket, which effectively solves the problem of low efficiency of leaky buckets. If the bucket volume is 100 and the token generation rate is 50 per second, it means that when the bucket is full of tokens, it can carry a maximum traffic of 100. Traffic is also processed at a constant rate of 50 per second if flood peaks are consistently high, so peak traffic is handled efficiently without being overwhelmed.
  • the assessment of the token generation rate in the token bucket method is difficult to evaluate. Generally, it is manually set based on human experience, which is difficult to be dynamic, cannot be automatically adjusted based on the resources of hardware devices, and lacks flexibility. Moreover, the content of the bucket is generally fixed and small. When the bucket is full, the traffic data is directly discarded. In some scenarios where a fixed frequency and a fixed amount of data burst suddenly in a jagged shape, the discarded data will inevitably cause business errors.
  • the present invention proposes a method, system, device and storage medium for dynamically preventing traffic attacks.
  • the present invention can limit the current, which can effectively deal with the burst period of traffic, so that the application system can provide services to the maximum extent. on and stabilize the operation of the application system.
  • the present invention adopts the following technical solutions:
  • a method for dynamically preventing traffic attacks including:
  • a queue mechanism When receiving traffic peaks, a queue mechanism is used to buffer unprocessed traffic
  • the token generation speed is dynamically controlled according to the real processing speed per unit time.
  • the type of the queue is determined according to the service, and when the service can determine the maximum amount of traffic peaks, the queue is an unbounded queue;
  • the queue is a bounded queue, and the queue policy uses the policy of rejecting or waiting when the queue is full.
  • the dynamic control token generation speed according to the real processing speed per unit time is specifically:
  • control token generation speed according to the amount of denial of service is specifically:
  • the idle ratio and the number of processed traffic per unit time of the business thread pool are obtained;
  • the amount of processed traffic in the previous unit of time is used as the token generation rate
  • the token generation rate formula is:
  • Token generation rate (current number of tokens*2>current number of tokens/(100%-idle ratio))? Current number of tokens/(100%-idle ratio): Current number of tokens*2.
  • the preset reserved value is set to 10% of the maximum processing capacity of the system.
  • the size of the service thread pool is determined according to the user service.
  • the present invention also proposes a system for dynamically preventing traffic attacks, including:
  • the traffic buffer device is used to buffer the unprocessed traffic by using a queue mechanism when receiving traffic peaks;
  • the token bucket device is used to generate a token, and according to the token, take the unprocessed traffic packets in the traffic buffer device, and put them into the processing device for execution;
  • a processing device used for invoking the business processing flow and processing the traffic data
  • the control device is used for dynamically controlling the token generation speed in the token bucket device according to the real processing speed per unit time.
  • the processing device obtains the idle ratio and the processing traffic quantity within the unit time of the service thread pool according to the start time and end time of the working thread;
  • the control device calculates the difference between the incoming flow and the taken flow in the last unit time, obtains the denial of service amount, and controls the token generation speed according to the denial of service amount:
  • the amount of processing traffic of the processing device in the previous unit time is taken as the token generation rate
  • the denial of service amount is a positive value, it is judged whether the idle ratio of the processing device reaches the system processing bottleneck, and when the processing bottleneck is reached, the total processing amount of the processing device minus the preset reserved value is used as the token generation rate;
  • the token generation rate formula is:
  • Token generation rate (current number of tokens*2>current number of tokens/(100%-idle ratio))? Current number of tokens/(100%-idle ratio): Current number of tokens*2.
  • the present invention also provides a device for dynamically preventing traffic attacks, including:
  • the processor is configured to implement the steps of the above-mentioned method for dynamically preventing traffic attacks when executing the computer program.
  • the present invention also provides a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method for dynamically preventing traffic attacks are implemented.
  • the invention solves the problem that the token bucket capacity of the classical token bucket algorithm is fixed and small, and the zigzag burst suddenly occurs in some fixed frequency and fixed data volume.
  • it provides a flexible adjustment method for the number of tokens that are difficult to calculate and quantify. It can dynamically respond to traffic peaks and dynamically control token buckets without manual experience intervention.
  • the token generation rate can ensure the stable operation of the system.
  • the invention effectively reduces the utilization of resources such as CPU, protects the stable operation of the application program, and ensures that the business can occupy system resources in a shrinkable manner, thereby preventing the network traffic shaping of traffic attacks.
  • FIG. 1 is a schematic flowchart of a method for dynamically preventing traffic attacks according to an embodiment of the present invention
  • FIG. 2 is a schematic structural diagram of a system for dynamically preventing traffic attacks according to an embodiment of the present invention
  • FIG. 3 is a schematic diagram of a usage flow of the system for dynamically preventing traffic attacks according to an embodiment of the present invention.
  • an embodiment of the present invention discloses a method for dynamically preventing traffic attacks, including:
  • a queue mechanism When receiving traffic peaks, a queue mechanism is used to buffer unprocessed traffic
  • the token generation speed is dynamically controlled according to the real processing speed per unit time, so as to ensure that the business can shrink the occupation of system resources.
  • the type of the queue is determined according to the service, and when the service can determine the maximum amount of traffic peaks, the queue is an unbounded queue;
  • the queue policy uses the policy of rejecting or waiting when the queue is full. After reaching the critical capacity, packets that cannot be stored are discarded to prevent excessive memory usage.
  • the described dynamic control token generation speed according to the real processing speed per unit time is specifically:
  • the idle ratio and the number of processed traffic per unit time of the business thread pool are obtained;
  • the amount of processed traffic in the previous unit of time is used as the token generation rate
  • the token generation rate formula is:
  • Token generation rate (current number of tokens*2>current number of tokens/(100%-idle ratio))? Current number of tokens/(100%-idle ratio): Current number of tokens*2. That is: determine whether twice the current number of tokens is greater than the maximum capacity of the processing device (ie: the current number of tokens/(100%-idle ratio)), if so, use the maximum capacity of the processing device as the token generation rate, if not , using 2 times the current number of tokens as the token generation rate.
  • the size of the service thread pool is determined according to the user service.
  • the preset reserved value is designed to protect the system from being continuously fully loaded, causing other applications to be unavailable or its own application to be restricted from being used by the system.
  • the preset reserve value can be set to 10% of the maximum processing capacity of the system. For example, if the processing device can process a maximum of 10,000 events per second, it is recommended to define the maximum capacity value as 9000, that is, the preset reserve value It is 1000 to ensure that the system will not continuously reach the peak value, thus ensuring the stability of operation.
  • an embodiment of the present invention further discloses a system for dynamically preventing traffic attacks, including:
  • the traffic buffer device is used to buffer the unprocessed traffic by using a queue mechanism when receiving traffic peaks;
  • the token bucket device is used to generate a token, and according to the token, take the unprocessed traffic packets in the traffic buffer device, and put them into the processing device for execution;
  • a processing device used for invoking the business processing flow and processing the traffic data
  • the control device is used for dynamically controlling the token generation speed in the token bucket device according to the real processing speed per unit time.
  • the token bucket device takes the traffic packets and puts them into the processing device for execution, and at the same time, generates tokens at a rate given by the control device.
  • the capacity of the token bucket is consistent with the generation rate per unit time to ensure that there is no backlog of data in the token bucket.
  • the token bucket thread continuously fetches packets from the traffic buffer device.
  • a token When fetching a message, a token needs to be obtained first.
  • the message When a token exists, the message is put into the processing device for processing; when no token is available, the thread changes to a waiting state.
  • the token bucket device making rate interface can receive the rate given by the control device to dynamically generate the token.
  • the control device is used to dynamically control the generation rate of the token, so as to achieve the goal of non-human intervention. For example, in a single-core single-block CPU scenario and a multi-core multi-CPU scenario, the processing rate of traffic must be different. At this time, if it is difficult to make the program run to an optimal state based on human experience, it is only possible to simply reduce the rate to A certain safe low speed to ensure the stability of the system. And through the algorithm of the control device, the effect of no manual experience intervention can be achieved, and the token generation rate of the token bucket can be dynamically controlled.
  • the control device obtains the denial of service amount according to the difference between the inflow and outflow quantities per unit time on the traffic buffer device. If the denial of service volume is negative, it means that the current traffic has not reached the traffic peak that can be accommodated by the application system. At this time, the processing traffic volume per unit time of the processing device in the previous unit time is used as the generation rate of the token in the token bucket, and the automatic Reduce the occupancy of resources such as CPU.
  • the denial of service amount is a positive value, it means that the current traffic has exceeded the token generation rate defined by the current control device based on the amount of traffic processed per unit time last time. At this time, it is necessary to determine whether the processing bottleneck of the system has been reached again according to the idle ratio of the processing device. , and then expand the token rate.
  • the total processing amount of the processing device minus the reserved value is used as the token generation rate in the token bucket.
  • the running performance is deliberately reduced by a part of the value.
  • the CPU utilization rate is: (unit time – CPU idle time per unit time) * 100%
  • the reserved value is Time to deliberately let the CPU idle.
  • the token generation rate formula is formulated as follows:
  • Token generation rate (current number of tokens*2>current number of tokens/(100%-idle ratio))? Current number of tokens/(100%-idle ratio): Current number of tokens*2.
  • the processing device invokes the processing logic of the service to process the traffic data. Specifically, the processing device creates an aspect and invokes the processing logic of the service. In the aspect, the start time is recorded before the processing logic is called, and the completion time is recorded after the processing is completed. Configure the thread pool. The size of the thread pool is selected based on user business needs. Provides the idle ratio of the thread pool per unit time and the number of processed traffic based on the start time and end time of the worker thread.
  • the embodiment of the present invention also discloses a device for dynamically preventing traffic attacks, including:
  • the processor is configured to implement the steps of the above-mentioned method for dynamically preventing traffic attacks when executing the computer program.
  • An embodiment of the present invention further discloses a storage medium, where a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method for dynamically preventing traffic attacks are implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Hardware Design (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

一种动态预防流量攻击的方法、系统、设备及存储介质，所述方法包括：当接收流量洪峰时，采用队列机制缓冲未处理的流量；根据令牌拿取未处理的流量报文，调用业务处理流程，处理流量数据；依据单位时间内真实处理速度动态控制令牌生成速度。所述系统包括：流量缓冲装置，用于当接收流量洪峰时，采用队列机制缓冲未处理的流量；令牌桶装置，用于生成令牌，根据令牌拿取流量缓冲装置中未处理的流量报文，放入处理装置中执行；处理装置，用于调用业务处理流程，处理流量数据；控制装置，用于依据单位时间内真实处理速度，动态控制令牌桶装置中令牌生成速度。本发明有效应对流量爆发期，使应用系统在最大限度提供服务的基础上稳定运行。

Description

一种动态预防流量攻击的方法、系统、设备及存储介质
本申请要求于2020年07月17日提交至中国专利局、申请号为202010695440.9、发明名称为“一种动态预防流量攻击的方法、系统、设备及存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及流量攻击预防领域，具体涉及一种动态预防流量攻击的方法、系统、设备及存储介质。
背景技术
应用系统需要应对后台逻辑所需的大量数据处理或者非预期的请求量(如大量用户的一系列操作集中式爆发)等场景对应用系统进行的流量攻击，这些流量攻击将对应用系统的稳定性、系统的CPU(Central Processing Unit，中央处理器)使用率、内存使用率、报文传输通道、协议链等带来极大的负载挑战。
在高并发大流量的应用系统中，一般会通过三个利器来控制流量，他们分别是：缓存、限流和降级。其中，限流就是针对稀缺资源(如CPU、内存等)或者根据业务需要等场景，通过一定的手段来限制流量的并发，从而有效的保护应用系统的稳定性。限流的实现方式有多种，目前，限流的算法主要分为计数器/时间窗口法、漏桶法和令牌桶法三种类型。
计数器/时间窗口法通过在单位时间内设置最大访问数达到限流的目的，例如，设定QPS(Queries-per-second，每秒查询率)为60，则限定该接口每秒只能被访问60次。但是，假如流量在前1％的时间内达到了峰值，则后面99％的时间都将会是静默状态，被阻断在门外，这种缺陷也被称作“突刺效应”。
相比计数器法，漏桶法有效地避免了“突刺效应”，漏桶法通过固定 大小的队列加上定时取元素的方式实现，就像一个盛水的漏桶，只限制容器出水的速率，当进水速率过大时就会溢出，也就是拒绝的流量。但是漏桶不能真正处理突发的流量洪峰，效率不高。
令牌桶法是基于桶的原型，但是和漏桶截然不同，没有出水口。通过生成令牌速率加令牌桶的容积来控制流量，有效解决了漏桶效率不高的问题。如桶的容积为100，令牌产生速率为50每秒，那么就代表桶中令牌满的时候，最大能承载100的流量。如果洪峰一直居高不下，也会以每秒50的恒定速率处理流量，所以有效地处理了洪峰流量而不至于被压垮。
但是，令牌桶法中令牌生成速率的评定很难评估，一般都是根据人的经验手动设定，很难动态化，不能基于硬件设备的资源进行自动调整，灵活性不足。而且桶内容量一般固定且较小，当桶内满时，流量数据直接抛弃，在某些固定频率、固定数据量锯齿状突然爆发的场景下，该丢弃数据势必会造成业务错误。
发明内容
为了解决上述技术问题，本发明提出了一种动态预防流量攻击的方法、系统、设备及存储介质，通过本发明进行限流，可以有效应对流量爆发期，使应用系统在最大限度提供服务的基础上并且使应用系统的运行趋于稳定。
为实现上述目的，本发明采用以下技术方案：
一种动态预防流量攻击的方法，包括：
当接收流量洪峰时，采用队列机制缓冲未处理的流量；
根据令牌拿取未处理的流量报文，调用业务处理流程，处理流量数据；
依据单位时间内真实处理速度动态控制令牌生成速度。
进一步地，所述队列的类型根据业务确定，当业务能够确定流量洪峰最大量时，队列为无界队列；
当业务不能确定流量洪峰最大量时，队列为有界队列，队列策略使用队列满时拒绝或等待策略。
进一步地，所述依据单位时间内真实处理速度动态控制令牌生成速度具体为：
计算上一单位时间流入流量和拿取流量的差值，得到拒绝服务量；
根据拒绝服务量控制令牌生成速度。
进一步地，所述根据拒绝服务量控制令牌生成速度具体为：
根据工作线程开始时间、结束时间得出业务线程池单位时间内的空闲比和处理流量数量；
当拒绝服务量为负值时，以上一单位时间的处理流量数量作为令牌生成速率；
当拒绝服务量为正值时，判断空闲比是否达到系统处理瓶颈，当达到处理瓶颈时，以处理总量减去预设保留值作为令牌生成速率；
当未达到处理瓶颈时，令牌生成速率公式为：
令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。
进一步地，所述预设保留值设置为系统最大处理能力的10％。
进一步地，所述业务线程池的大小根据用户业务确定。
本发明还提出了一种动态预防流量攻击的系统，包括：
流量缓冲装置，用于当接收流量洪峰时，采用队列机制缓冲未处理的流量；
令牌桶装置，用于生成令牌，根据令牌拿取流量缓冲装置中未处理的流量报文，放入处理装置中执行；
处理装置，用于调用业务处理流程，处理流量数据；
控制装置，用于依据单位时间内真实处理速度，动态控制令牌桶装置中令牌生成速度。
进一步地，所述处理装置根据工作线程开始时间、结束时间得出业务线程池单位时间内的空闲比和处理流量数量；
所述控制装置计算上一单位时间流入流量和拿取流量的差值，得到拒绝服务量，根据拒绝服务量控制令牌生成速度：
当拒绝服务量为负值时，以上一单位时间处理装置的处理流量数量作为令牌生成速率；
当拒绝服务量为正值时，判断处理装置的空闲比是否达到系统处理瓶颈，当达到处理瓶颈时，以处理装置的处理总量减去预设保留值作为令牌生成速率；
当未达到处理瓶颈时，令牌生成速率公式为：
令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。
本发明还提出了一种动态预防流量攻击的设备，包括：
存储器，用于存储计算机程序；
处理器，用于执行所述计算机程序时实现如上所述动态预防流量攻击的方法的步骤。
本发明还提出了一种存储介质，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述动态预防流量攻击的方法的步骤。
本发明的有益效果是：
本发明通过提出一种动态预防流量攻击的方法、系统、设备及存储介质，解决了经典令牌桶算法的令牌桶容量固定且较小，在某些固定频率、固定数据量锯齿状突然爆发的场景下，由于丢弃数据造成业务错误的问题，为难以计算量化的令牌数量提供了灵活的调整方法，可以在无人工经验干预的情况下，伸缩性动态应对流量洪峰，动态控制令牌桶的令牌生成速率，保证系统的稳定运行。本发明有效降低CPU等资源的利用，保护应用程序稳定运行，保障业务可收缩地占用系统资源，从而预防了流量攻击的网络 流量整形。
附图说明
图1是本发明实施例动态预防流量攻击的方法流程示意图；
图2是本发明实施例动态预防流量攻击的系统结构示意图；
图3是本发明实施例动态预防流量攻击的系统的使用流程示意图。
具体实施方式
为能清楚说明本方案的技术特点，下面通过具体实施方式，并结合其附图，对本发明进行详细阐述。下文的公开提供了许多不同的实施例或例子用来实现本发明的不同结构。为了简化本发明的公开，下文中对特定例子的部件和设置进行描述。此外，本发明可以在不同例子中重复参考数字和/或字母。这种重复是为了简化和清楚的目的，其本身不指示所讨论各种实施例和/或设置之间的关系。应当注意，在附图中所图示的部件不一定按比例绘制。本发明省略了对公知组件和处理技术及工艺的描述以避免不必要地限制本发明。
如图1所示，本发明实施例公开了一种动态预防流量攻击的方法，包括：
当接收流量洪峰时，采用队列机制缓冲未处理的流量；
根据令牌拿取未处理的流量报文，调用业务处理流程，处理流量数据；
依据单位时间内真实处理速度动态控制令牌生成速度，保障业务可收缩的占用系统资源。
具体地，所述队列的类型根据业务确定，当业务能够确定流量洪峰最大量时，队列为无界队列；
当业务不能确定流量洪峰最大量时，队列为有界队列，队列策略使用队列满时拒绝或等待策略，在到达临界容量后丢弃无法存放的报文，防止过度使用内存。
同时，记录单位时间流入和流出的数据数目。
所述依据单位时间内真实处理速度动态控制令牌生成速度具体为：
计算上一单位时间流入流量和拿取流量的差值，得到拒绝服务量；
根据拒绝服务量控制令牌生成速度。
所述根据拒绝服务量控制令牌生成速度具体为：
根据工作线程开始时间、结束时间得出业务线程池单位时间内的空闲比和处理流量数量；
当拒绝服务量为负值时，以上一单位时间的处理流量数量作为令牌生成速率；
当拒绝服务量为正值时，判断空闲比是否达到系统处理瓶颈，当达到处理瓶颈时，以处理总量减去预设保留值作为令牌生成速率；
当未达到处理瓶颈时，令牌生成速率公式为：
令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。即：判断当前令牌数的2倍是否大于处理装置的最大能力(即：当前令牌数/(100％-空闲比))，若是，使用处理装置的最大能力作为令牌生成速率，若不是，使用当前令牌数的2倍作为令牌生成速率。
所述业务线程池的大小根据用户业务确定。
所述预设保留值是为了保护系统不是持续性的满载，造成其他应用无法使用或者自身应用被系统限制无法使用的情况而设计。优选地，所述预设保留值可以设置为系统最大处理能力的10％，例如，如果处理装置每秒最大能处理一万条事件，那么建议将最大能力值定义为9000，即预设保留值为1000，保证系统不会持续的达到峰值，从而能够保证运行的稳定性。
如图2所示，本发明实施例还公开了一种动态预防流量攻击的系统，包括：
流量缓冲装置，用于当接收流量洪峰时，采用队列机制缓冲未处理的流量；
令牌桶装置，用于生成令牌，根据令牌拿取流量缓冲装置中未处理的流量报文，放入处理装置中执行；
处理装置，用于调用业务处理流程，处理流量数据；
控制装置，用于依据单位时间内真实处理速度，动态控制令牌桶装置中令牌生成速度。
如图3所示，令牌桶装置拿取流量报文放入处理装置中执行，同时，以控制装置给定的速率生成令牌。在拿取流量报文时，需要先获取桶中的令牌。同时，令牌桶的容量和单位时间的生成速率保持一致，保证令牌桶中没有数据的积压。
当流量缓冲装置中存在流量报文时，令牌桶线程不断从流量缓冲装置拿取报文。
拿取报文时需要先获取令牌，存在令牌时，将报文放入处理装置中进行处理；当无令牌可用时，线程变更为等待状态。
令牌桶装置制作速率接口可接收控制装置给定的速率动态生成令牌。
当生成的令牌数量已经达到了令牌桶容量时，停止令牌的生成。一旦令牌桶容量不满，继续生成令牌，放入令牌桶中。
所述控制装置用来动态控制令牌的生成速率，从而达到非人为干预的目标。比如，在单核单块CPU场景和多核多CPU场景下，流量的处理速率一定是不同的，此时如果依据人为的经验很难使程序运行达到最优状态，仅能简单地将速率降低到一定安全的低速，保证系统的稳定。而通过控制装置的算法，可以达到无人工经验干预的效果，动态控制令牌桶的令牌生成速率。
具体地，控制装置依据流量缓冲装置上一单位时间的流入和流出数量差值，得出拒绝服务量。若拒绝服务量为负值，说明当前流量未达到应用系统所能容纳的流量洪峰，此时使用上一单位时间处理装置单位时间内的处理流量数量作为令牌桶中令牌的生成速率，自动缩小CPU等资源的占 用。
若拒绝服务量为正值，说明当前流量已经超出了基于上次单位时间处理流量数量定义的当前控制装置生成令牌速率，此时需要再次根据处理装置的空闲比判定是否已经达到系统的处理瓶颈，然后进行令牌速率的扩容。
若已经达到了系统处理瓶颈，此时使用处理装置的处理总量减去保留值作为令牌桶中令牌生成速率。为降低CPU等资源的利用，保护应用程序稳定运行，特意将运行性能降低一部分的值，比如CPU利用率为：(单位时间–单位时间内CPU空闲时间)*100％，所述保留值即为想要刻意让CPU空闲的时间。
若未达到系统处理瓶颈，基于处理装置的空闲比及当前令牌数，制定令牌生成速率公式如下：
令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。
所述处理装置调用业务的处理逻辑处理流量数据。具体地，处理装置制作切面，调用业务的处理逻辑。切面中，在调用处理逻辑之前记录开始时间，处理完成后记录完成时间。配置线程池，线程池大小基于用户业务所需选定。提供接口根据工作线程开始时间、结束时间得出的线程池单位时间内的空闲比、处理流量数量。
本发明实施例还公开了一种动态预防流量攻击的设备，包括：
存储器，用于存储计算机程序；
处理器，用于执行所述计算机程序时实现如上所述动态预防流量攻击的方法的步骤。
本发明实施例还公开了一种存储介质，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述动态预防流量攻击的方法的步骤。
上述虽然结合附图对本发明的具体实施方式进行了描述，但并非对本发明保护范围的限制。对于所属领域的技术人员来说，在上述说明的基础上还可以做出其它不同形式的修改或变形。这里无需也无法对所有的实施方式予以穷举。在本发明的技术方案的基础上，本领域技术人员不需要付出创造性劳动即可做出的各种修改或变形仍在本发明的保护范围以内。

Claims (10)

  1. 一种动态预防流量攻击的方法，其特征在于，包括：
    当接收流量洪峰时，采用队列机制缓冲未处理的流量；
    根据令牌拿取未处理的流量报文，调用业务处理流程，处理流量数据；
    依据单位时间内真实处理速度动态控制令牌生成速度。
  2. 根据权利要求1所述的动态预防流量攻击的方法，其特征在于，所述队列的类型根据业务确定，当业务能够确定流量洪峰最大量时，队列为无界队列；
    当业务不能确定流量洪峰最大量时，队列为有界队列，队列策略使用队列满时拒绝或等待策略。
  3. 根据权利要求1所述的动态预防流量攻击的方法，其特征在于，所述依据单位时间内真实处理速度动态控制令牌生成速度具体为：
    计算上一单位时间流入流量和拿取流量的差值，得到拒绝服务量；
    根据拒绝服务量控制令牌生成速度。
  4. 根据权利要求3所述的动态预防流量攻击的方法，其特征在于，所述根据拒绝服务量控制令牌生成速度具体为：
    根据工作线程开始时间、结束时间得出业务线程池单位时间内的空闲比和处理流量数量；
    当拒绝服务量为负值时，以上一单位时间的处理流量数量作为令牌生成速率；
    当拒绝服务量为正值时，判断空闲比是否达到系统处理瓶颈，当达到处理瓶颈时，以处理总量减去预设保留值作为令牌生成速率；
    当未达到处理瓶颈时，令牌生成速率公式为：
    令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。
  5. 根据权利要求4所述的动态预防流量攻击的方法，其特征在于，所 述预设保留值设置为系统最大处理能力的10％。
  6. 根据权利要求3所述的动态预防流量攻击的方法，其特征在于，所述业务线程池的大小根据用户业务确定。
  7. 一种动态预防流量攻击的系统，其特征在于，包括：
    流量缓冲装置，用于当接收流量洪峰时，采用队列机制缓冲未处理的流量；
    令牌桶装置，用于生成令牌，根据令牌拿取流量缓冲装置中未处理的流量报文，放入处理装置中执行；
    处理装置，用于调用业务处理流程，处理流量数据；
    控制装置，用于依据单位时间内真实处理速度，动态控制令牌桶装置中令牌生成速度。
  8. 根据权利要求7所述的动态预防流量攻击的系统，其特征在于，所述处理装置根据工作线程开始时间、结束时间得出业务线程池单位时间内的空闲比和处理流量数量；
    所述控制装置计算上一单位时间流入流量和拿取流量的差值，得到拒绝服务量，根据拒绝服务量控制令牌生成速度：
    当拒绝服务量为负值时，以上一单位时间处理装置的处理流量数量作为令牌生成速率；
    当拒绝服务量为正值时，判断处理装置的空闲比是否达到系统处理瓶颈，当达到处理瓶颈时，以处理装置的处理总量减去预设保留值作为令牌生成速率；
    当未达到处理瓶颈时，令牌生成速率公式为：
    令牌生成速率＝(当前令牌数*2>当前令牌数/(100％-空闲比))？当前令牌数/(100％-空闲比):当前令牌数*2。
  9. 一种动态预防流量攻击的设备，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述计算机程序时实现如权利要求1至6任一项所述动态预防流量攻击的方法的步骤。
  10. 一种存储介质，其特征在于，所述存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至6任一项所述动态预防流量攻击的方法的步骤。
PCT/CN2021/077807 2020-07-17 2021-02-25 一种动态预防流量攻击的方法、系统、设备及存储介质 WO2022012054A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/013,782 US20230319102A1 (en) 2020-07-17 2021-02-25 Method, System and Device for Dynamically Preventing Traffic Attacks, and Storage Medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010695440.9 2020-07-17
CN202010695440.9A CN112003795B (zh) 2020-07-17 2020-07-17 一种动态预防流量攻击的方法、系统、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022012054A1 true WO2022012054A1 (zh) 2022-01-20

Family

ID=73467684

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/077807 WO2022012054A1 (zh) 2020-07-17 2021-02-25 一种动态预防流量攻击的方法、系统、设备及存储介质

Country Status (3)

Country Link
US (1) US20230319102A1 (zh)
CN (1) CN112003795B (zh)
WO (1) WO2022012054A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114726798A (zh) * 2022-02-28 2022-07-08 福建星云电子股份有限公司 一种锂电池测试通道限流方法及系统
CN114827033A (zh) * 2022-04-15 2022-07-29 咪咕文化科技有限公司 数据流控方法、装置、设备与计算机可读存储介质
CN116095013A (zh) * 2022-12-29 2023-05-09 北京开科唯识技术股份有限公司 一种服务请求限流方法、装置及存储介质
CN117061526A (zh) * 2023-10-12 2023-11-14 人力资源和社会保障部人事考试中心 一种基于全局与局部业务访问控制的访问高峰防拥塞方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112003795B (zh) * 2020-07-17 2022-12-20 苏州浪潮智能科技有限公司 一种动态预防流量攻击的方法、系统、设备及存储介质
CN113472682B (zh) * 2021-06-30 2022-08-23 广东电网有限责任公司 一种热点对象流媒体源自动伸缩方法及装置
CN114124829A (zh) * 2021-09-26 2022-03-01 新华三技术有限公司 一种业务转发控制方法、装置及电子设备
CN114745333B (zh) * 2022-03-03 2023-05-30 珠海高凌信息科技股份有限公司 一种ims系统自适应流量控制方法
CN115941286B (zh) * 2022-11-11 2023-07-04 南京鼎山信息科技有限公司 一种应用于物联网和直播平台的数据处理方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009012811A1 (en) * 2007-07-23 2009-01-29 Telefonaktiebolaget Lm Ericsson (Publ) Controlling traffic in a packet switched comunications network
CN103929366A (zh) * 2014-04-02 2014-07-16 华为技术有限公司 流量控制方法、装置及设备
CN108768873A (zh) * 2018-05-29 2018-11-06 腾讯科技（深圳）有限公司 一种流量控制方法及相关设备
CN109729013A (zh) * 2017-10-30 2019-05-07 深圳市中兴微电子技术有限公司 一种流量整形中添加令牌的方法、装置及计算机可读存储介质
CN112003795A (zh) * 2020-07-17 2020-11-27 苏州浪潮智能科技有限公司 一种动态预防流量攻击的方法、系统、设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109274745B (zh) * 2018-09-28 2021-07-30 石家庄良村热电有限公司 一种边缘节点优化计算的物联网系统和方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009012811A1 (en) * 2007-07-23 2009-01-29 Telefonaktiebolaget Lm Ericsson (Publ) Controlling traffic in a packet switched comunications network
CN103929366A (zh) * 2014-04-02 2014-07-16 华为技术有限公司 流量控制方法、装置及设备
CN109729013A (zh) * 2017-10-30 2019-05-07 深圳市中兴微电子技术有限公司 一种流量整形中添加令牌的方法、装置及计算机可读存储介质
CN108768873A (zh) * 2018-05-29 2018-11-06 腾讯科技（深圳）有限公司 一种流量控制方法及相关设备
CN112003795A (zh) * 2020-07-17 2020-11-27 苏州浪潮智能科技有限公司 一种动态预防流量攻击的方法、系统、设备及存储介质

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114726798A (zh) * 2022-02-28 2022-07-08 福建星云电子股份有限公司 一种锂电池测试通道限流方法及系统
CN114827033A (zh) * 2022-04-15 2022-07-29 咪咕文化科技有限公司 数据流控方法、装置、设备与计算机可读存储介质
CN114827033B (zh) * 2022-04-15 2024-04-19 咪咕文化科技有限公司 数据流控方法、装置、设备与计算机可读存储介质
CN116095013A (zh) * 2022-12-29 2023-05-09 北京开科唯识技术股份有限公司 一种服务请求限流方法、装置及存储介质
CN116095013B (zh) * 2022-12-29 2023-07-25 北京开科唯识技术股份有限公司 一种服务请求限流方法、装置及存储介质
CN117061526A (zh) * 2023-10-12 2023-11-14 人力资源和社会保障部人事考试中心 一种基于全局与局部业务访问控制的访问高峰防拥塞方法
CN117061526B (zh) * 2023-10-12 2023-12-12 人力资源和社会保障部人事考试中心 一种基于全局与局部业务访问控制的访问高峰防拥塞方法

Also Published As

Publication number Publication date
US20230319102A1 (en) 2023-10-05
CN112003795A (zh) 2020-11-27
CN112003795B (zh) 2022-12-20

Similar Documents

Publication Publication Date Title
WO2022012054A1 (zh) 一种动态预防流量攻击的方法、系统、设备及存储介质
EP2422495B1 (en) Dynamic adjustment of connection setup request parameters
WO2011137727A1 (zh) 一种报文的传输方法和系统
CN105978821B (zh) 网络拥塞避免的方法及装置
CN105721332A (zh) 一种基于改进的wred的拥塞控制方法和装置
CN113381944A (zh) 系统限流方法、装置、电子设备、介质和程序产品
CN115412497A (zh) 一种bbr拥塞控制算法的性能优化方法
CN110753004B (zh) 一种数据传输动态限流方法、系统、装置及可读存储介质
CN113315720A (zh) 一种数据流控制方法、系统及设备
Mehra et al. Resource management for real-time communication: Making theory meet practice
CN109274550A (zh) 一种iSCSI自适应IO队列深度匹配方法
WO2023155815A1 (zh) 一种报文处理方法、装置、终端设备及存储介质
CN116319569A (zh) 网络参数更新方法、网络参数更新装置、介质及电子设备
CN113098793B (zh) 分布式服务系统限流方法及分布式服务系统
WO2022001430A1 (zh) 一种高吞吐量流处理方法、装置及计算机可读存储介质
WO2021013260A1 (zh) 一种网络传输控制方法及装置
CN110535785B (zh) 一种发送频率的控制方法、装置和分布式系统
CN114024913A (zh) 一种网络性能优化方法、装置、设备以及存储介质
WO2024131421A1 (zh) 消息中间件的自适应动态限流方法、设备及介质
TWI735520B (zh) 調整元件邏輯執行緒數量的方法及裝置
CN109413489A (zh) 串行式的多线程弹幕分发方法、装置、设备和存储介质
CN113726685B (zh) 一种通用与专用混合流量控制方法、计算机设备和存储介质
TWI813417B (zh) 封包處理裝置以及封包處理方法
WO2024098952A1 (zh) 网络流量的限速方法、装置、中控设备及存储介质
CN115604044A (zh) 网卡功能开关状态调整方法、装置、电子设备及可读介质

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21842554

Country of ref document: EP

Kind code of ref document: A1