CN114598658A - Flow limiting method and device - Google Patents

Flow limiting method and device Download PDF

Info

Publication number
CN114598658A
CN114598658A CN202210222044.3A CN202210222044A CN114598658A CN 114598658 A CN114598658 A CN 114598658A CN 202210222044 A CN202210222044 A CN 202210222044A CN 114598658 A CN114598658 A CN 114598658A
Authority
CN
China
Prior art keywords
token
tokens
token bucket
time
request
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210222044.3A
Other languages
Chinese (zh)
Inventor
申大雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Inspur Cloud Information Technology Co Ltd
Original Assignee
Inspur Cloud Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Inspur Cloud Information Technology Co Ltd filed Critical Inspur Cloud Information Technology Co Ltd
Priority to CN202210222044.3A priority Critical patent/CN114598658A/en
Publication of CN114598658A publication Critical patent/CN114598658A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/215Flow control; Congestion control using token-bucket
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention relates to the technical field of web application, and particularly provides a flow limiting method.A user request firstly reaches an interceptor, and after being processed by the interceptor, a token application operation is executed, wherein a token generation controller generates a token according to a certain rule in the token application operation; when the user has frequent access and the tokens are insufficient, temporarily storing the token into a queue for waiting, and detecting the number of the tokens in a token bucket; when enough tokens exist in the token bucket, the request is forwarded again. Compared with the prior art, the method adopts a trigger type token generation mode, triggers the rule of generating the token when requesting access each time, and reduces the occupation of resources such as server threads, CPUs (central processing units) and the like.

Description

Flow limiting method and device
Technical Field
The invention relates to the technical field of web application, and particularly provides a flow limiting method and device.
Background
With the rapid development of computer technology, various new technologies and new architecture ideas are applied to network products, the system is developed from traditional single-machine deployment to current distributed deployment, more and more traditional architectures are changed towards micro-service architectures and distributed architectures, and the traditional industry is also changed towards an internet plus mode. This puts requirements on both quality of service access and speed of service access. In highly concurrent scenarios, the three most common technical approaches are throttling, buffering, and destaging.
The purpose of current limiting is to protect the system by limiting the rate of concurrent access requests or limiting the rate of requested data within a time window, and once the rate of limitation is reached, service, queuing or waiting can be denied, the availability and stability of the system can be maintained, and slow or down operation of the system due to sudden increase in traffic can be prevented.
The token bucket algorithm is a common current-limiting algorithm, and the principle is that a system puts tokens into a bucket at a constant speed, if a request needs to be processed, the tokens need to be obtained from the bucket firstly, when no token is available in the bucket, service is refused, and the token bucket algorithm realizes request frequency limitation, capacity limitation and the like according to the rate frequency of the tokens by issuing the tokens.
The traditional token bucket algorithm needs to configure a special thread to add tokens into the token bucket at regular time, and the server thread and CPU resource occupation is large. The traditional flow limitation is mostly carried out in a single machine range, cannot be applied to a distributed environment, and cannot share the flow among multiple nodes.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a flow limiting method with strong practicability.
A further technical task of the present invention is to provide a flow restriction device that is reasonably designed, safe and practical.
The technical scheme adopted by the invention for solving the technical problem is as follows:
a flow limiting method, all user requests arrive at an interceptor first, after being processed by the interceptor, a token applying operation is executed, the token applying operation generates a token by a token generating controller according to a certain rule, when the token applying triggers the token generating rule, the token generating controller generates a certain number of tokens, the tokens are put into a token bucket, a token bucket server issues the tokens required by the user requests, and the requests are forwarded to each application server to execute a data access operation;
when the user has frequent access and the tokens are insufficient, temporarily storing the token into a queue for waiting, and detecting the number of the tokens in a token bucket; when enough tokens exist in the token bucket, the request is forwarded.
Further, before the user request reaches the interceptor, setting the maximum token number of the token bucket, the interval time of supplementing tokens by the token bucket and the quantity of supplementing tokens in each time interval of the token bucket;
and for different types of API services, setting the number of tokens required by different service access according to different sizes of occupied network resources and database resources.
Further, when the http request reaches the server, a request interceptor intercepts the http request to obtain the current limiting policy flag information of the http request, queries whether the current limiting policy flag information exists in a token bucket server, initializes a token bucket if the current limiting policy flag information does not exist, puts a preset maximum token number of the token bucket into the token bucket, sets expiration time of the token, and sets writing time of the token as current time.
Further, if the current-limiting policy flag information exists, reading token writing time corresponding to the current-limiting policy flag information in the token bucket server, comparing the token writing time with a current timestamp, if a time interval between the current timestamp and the last token writing time is greater than or equal to an interval time for supplementing tokens by the token bucket, calculating the number of tokens which can be added according to the time interval between the current timestamp and the last token writing time according to the rate of adding tokens, supplementing the tokens into the token bucket, and updating the writing time of the tokens to be the current time.
Further, if the time interval between the current timestamp and the last token writing time is less than the interval time of token supplement of the token bucket, the number of tokens required by the current http request is judged, if the number of the required tokens is less than or equal to the remaining number of the current token bucket, the tokens required by the http request are removed from the token bucket, the http request is forwarded, and the application server executes specific operation.
Further, if the required token quantity is larger than the residual quantity of the current token bucket, setting a queuing queue, setting the length of the queuing queue and the expiration time of queue elements, and if the token quantity required by the http request is larger than the residual quantity of the current token bucket, putting the http request into the queuing queue.
Preferably, when the http request deposited in the queue reaches an expiration time, the request is directly rejected and removed from the queue.
Further, by setting a timer, detecting the number of tokens in the token bucket by using a probe, and when the number of tokens stored in the token bucket is greater than or equal to the number of tokens required by the request, the token bucket server sends the number of tokens required by the request to the application server for execution.
Further, if the number of tokens in the token bucket is smaller than the number of tokens required by the http request, calculating a time interval between a current timestamp and the time when the token bucket generates tokens last, if the time interval is larger than the interval time when the token bucket supplements the tokens, directly supplementing the tokens in the token bucket to the maximum capacity, updating the token generation time to be the current timestamp, sending the number of tokens required by the request, and forwarding the request to the application server to execute subsequent operations.
A flow restriction device, comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to perform a flow restriction method.
Compared with the prior art, the flow limiting method and the flow limiting device have the following outstanding beneficial effects:
(1) the invention improves the traditional token bucket algorithm, adopts a trigger type token generation mode, triggers the rule of generating the token when requesting access each time, and reduces the occupation of resources such as server threads, CPU and the like. The token bucket algorithm used by the invention can realize smooth flow limitation on user access and control the data transmission rate to achieve the purpose of flow limitation.
(2) The invention also arranges a request interceptor, a token generation controller, a token bucket server and the like, which can break through the limitation of the current limiting function of single machine deployment and realize the current limiting requirement of the multi-node distributed environment.
And introducing a queuing queue to cache the http request which does not acquire the token in a high concurrency scene, and after a sufficient number of tokens are generated, processing the http request to allow a flow access spike within a certain range.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a flow restriction method;
fig. 2 is an architecture diagram of a flow restriction method.
Detailed Description
The present invention will be described in further detail with reference to specific embodiments in order to better understand the technical solutions of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A preferred embodiment is given below:
as shown in fig. 1 and 2, in the traffic limiting method in this embodiment, all user requests first reach an interceptor, and after being processed by the interceptor, a token application operation is executed, where the request interceptor is located at the uppermost layer of the system and belongs to a gateway layer.
In the token application operation, a token generation controller generates a token according to a certain rule, and Redis cache or memory cache of the memory cache can be used for ensuring the consistency, the robustness, the low time delay and the like of the data request.
When the token applies for triggering the token generation rule, the token generation controller generates a certain number of tokens according to parameters such as the size of a token bucket, the token generation rate, the last generation time of the tokens and the like set by the system, and the tokens are put into the token bucket. And issuing tokens required by the user request by the token bucket server, forwarding the request to each application server, and executing data access operation.
And allowing a part of the traffic requests to be in the token bucket, temporarily storing the traffic requests into a queuing queue to wait in the case of insufficient tokens, and detecting the token quantity of the token bucket by using a probe. When enough tokens exist in the token bucket, the request is forwarded again.
The specific operation is as follows:
before a user request reaches an interceptor, setting the maximum token number of a token bucket, the interval time of token bucket for supplementing tokens and the quantity of the token bucket for supplementing tokens in each time interval;
and for different types of API services, setting the number of tokens required by different service access according to different sizes of occupied network resources and database resources.
When the http request reaches the server, a request interceptor intercepts the http request to obtain the http request mark, such as the user ip and the access api, as the mark information of the current limiting policy.
And inquiring whether the token bucket server has the mark information of the current-limiting strategy, if not, initializing the token bucket, putting the preset maximum token quantity of the token bucket into the token bucket, setting the expiration time of the token, and setting the writing time of the token as the current time.
The maximum token quantity of the token bucket is at least 2 times larger than the supplementary token quantity of the token bucket in each time interval, and the actual situation is generally determined according to the current limiting threshold value supported by the system. The expiration time of the token is greater than 2 times the interval between replenishing tokens by the token bucket.
If the current limit strategy mark information exists, reading token writing time corresponding to the current limit strategy mark information in a token bucket server, comparing the token writing time with a current time stamp, if the time interval between the current time stamp and the last token writing time is larger than or equal to the interval time for supplementing tokens by the token bucket, calculating the number of tokens which can be added by the time interval between the current time stamp and the last token writing time according to the rate of adding the tokens, supplementing the tokens into the token bucket, and updating the writing time of the tokens to be the current time.
The time interval between the current timestamp and the last token writing time is calculated according to the token adding rate, and the number of tokens which can be added in the time interval is calculated as follows:
Figure BDA0003533959550000061
wherein currentTimestamp refers to a timestamp of current access, lastTimestamp refers to a timestamp of last token update, tokenInterval refers to an interval time for supplementing tokens by a token bucket, and tokenCountPer refers to the number of the supplementing tokens by the token bucket at each interval. newTokenCoun requires rounding down.
The number of tokens eventually added to the token bucket is:
resultCount=min(newTokenCount,maxTokenCountstoredTokenCount)
wherein, maxTokenCount refers to the maximum number of tokens supported by the token bucket, and storedTokenCount refers to the number of tokens remaining in the current token bucket. And replenishing the tokens with the number of resultCount into the token bucket, and updating the writing time of the tokens to be the current time.
And if the time interval between the current timestamp and the last token writing time is less than the interval time of token supplement of the token bucket, judging the number of tokens required by the current http request, if the number of the required tokens is less than or equal to the residual number of the current token bucket, removing the tokens required by the http request from the token bucket, forwarding the http request, and executing specific operation by the application server.
If the number of required tokens is larger than the residual number of the current token bucket, setting a queuing queue, and setting the length of the queuing queue and the expiration time of a queue element. And if the number of tokens required by the http request is larger than the remaining number of the current token bucket, putting the request into a queue.
When the http request stored in the queue reaches the expiration time, the request is directly rejected and removed from the queue.
The number of tokens in the token bucket is detected by using a probe through setting a timer, and when the number of tokens stored in the token bucket is larger than or equal to the number of tokens required by the request, the token bucket server sends the number of tokens required by the request to the application server for execution.
If the number of tokens in the token bucket is smaller than the number of tokens required by the http request, calculating the time interval between the current timestamp and the time for generating the tokens by the token bucket at last, if the time interval is larger than the interval time for replenishing the tokens by the token bucket, replenishing the tokens in the token bucket to the maximum capacity directly, updating the token generation time to be the current timestamp, sending the number of tokens required by the request, and forwarding the request to the application server to execute subsequent operations.
A flow restriction device, comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor is configured to invoke the machine readable program to perform a flow restriction method.
The above embodiments are only specific ones of the present invention, and the scope of the present invention includes but is not limited to the above embodiments, and any suitable changes or substitutions that are consistent with the flow limiting method and apparatus claims of the present invention and are made by those of ordinary skill in the art shall fall within the scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (10)

1. A flow limiting method is characterized in that all user requests firstly reach an interceptor, after being processed by the interceptor, a token application operation is executed, the token application operation generates a token by a token generation controller according to a certain rule, when the token application triggers the token generation rule, the token generation controller generates a certain number of tokens, the tokens are put into a token bucket, a token bucket server issues the tokens required by the user requests, and the requests are forwarded to each application server to execute data access operation;
when the user has frequent access and the tokens are insufficient, temporarily storing the token into a queue for waiting, and detecting the number of the tokens in a token bucket; when enough tokens exist in the token bucket, the request is forwarded.
2. The traffic limitation method according to claim 1, wherein before the user request reaches the interceptor, the maximum number of tokens in the token bucket, the interval time for replenishing tokens in the token bucket, and the number of replenishing tokens in each interval in the token bucket are set;
and for different types of API services, setting the number of tokens required by different service access according to different sizes of occupied network resources and database resources.
3. The traffic limiting method according to claim 2, wherein when the http request reaches the server, a request interceptor first intercepts the http request to obtain the current limiting policy flag information of the http request, queries whether the current limiting policy flag information exists in the token bucket server, if not, initializes the token bucket, puts a preset maximum token number of the token bucket into the token bucket, sets an expiration time of the token, and sets a write time of the token as a current time.
4. The method according to claim 3, wherein if there is the current-limiting policy flag information, reading a token write time corresponding to the current-limiting policy flag information in the token bucket server, comparing the read time with a current time stamp, if a time interval between the current time stamp and a last token write time is greater than or equal to an interval time for supplementing tokens by the token bucket, calculating the number of tokens that can be added according to a rate of adding tokens, and supplementing the tokens into the token bucket, wherein the write time for updating the tokens is the current time.
5. The traffic limitation method according to claim 4, wherein if the time interval between the current timestamp and the last write time of the token is less than the interval time of the token bucket for supplementing the token, the number of tokens required for the current http request is determined, and if the number of required tokens is less than or equal to the remaining number of the current token bucket, the tokens required for the http request are removed from the token bucket, the http request is forwarded, and the application server performs a specific operation.
6. The traffic throttling method of claim 5, wherein if the number of required tokens is greater than the remaining number of the current token bucket, setting a queue, setting the length of the queue and the expiration time of the queue element, and if the number of required tokens for an http request is greater than the remaining number of the current token bucket, placing the request in the queue.
7. A method according to claim 6, characterized in that when the http request stored in the queue reaches an expiry time, the request is directly rejected and removed from the queue.
8. The traffic limitation method according to claim 6 or 7, wherein a probe is used to detect the number of tokens in the token bucket by setting a timer, and when the number of tokens stored in the token bucket is greater than or equal to the number of tokens required by the request, the token bucket server issues the number of tokens required by the request and forwards the request to the application server for execution.
9. The method of claim 8, wherein if the number of tokens in the token bucket is less than the number of tokens required for the http request, calculating a time interval between a current timestamp and a time when the token bucket last generates tokens, if the time interval is greater than the time interval when the token bucket supplements the tokens, directly supplementing the tokens in the token bucket to a maximum capacity, updating the token generation time to the current timestamp, and sending the number of tokens required for the request, and forwarding the request to the application server for subsequent operations.
10. A flow restriction device, comprising: at least one memory and at least one processor;
the at least one memory to store a machine readable program;
the at least one processor configured to invoke the machine readable program to perform the method of any of claims 1 to 9.
CN202210222044.3A 2022-03-07 2022-03-07 Flow limiting method and device Pending CN114598658A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210222044.3A CN114598658A (en) 2022-03-07 2022-03-07 Flow limiting method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210222044.3A CN114598658A (en) 2022-03-07 2022-03-07 Flow limiting method and device

Publications (1)

Publication Number Publication Date
CN114598658A true CN114598658A (en) 2022-06-07

Family

ID=81807046

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210222044.3A Pending CN114598658A (en) 2022-03-07 2022-03-07 Flow limiting method and device

Country Status (1)

Country Link
CN (1) CN114598658A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115396377A (en) * 2022-07-29 2022-11-25 天翼云科技有限公司 Method, device and equipment for optimizing service quality of object storage and storage medium
CN115473852A (en) * 2022-07-26 2022-12-13 青岛海尔科技有限公司 Current limiting method for user side input request, storage medium and electronic device
CN117009955A (en) * 2023-09-26 2023-11-07 北京安普诺信息技术有限公司 IAST and RASP probe control method and device based on code vaccine

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101005453A (en) * 2006-12-26 2007-07-25 华为技术有限公司 Method and device for refreshing token bucket in flow limiting technology
WO2015149488A1 (en) * 2014-04-03 2015-10-08 深圳市中兴微电子技术有限公司 Method and apparatus for limiting rate by means of token bucket, and computer storage medium
CN108768873A (en) * 2018-05-29 2018-11-06 腾讯科技(深圳)有限公司 A kind of flow control methods and relevant device
CN109729013A (en) * 2017-10-30 2019-05-07 深圳市中兴微电子技术有限公司 The method, apparatus and computer readable storage medium of token are added in a kind of traffic shaping
CN111447150A (en) * 2020-02-29 2020-07-24 中国平安财产保险股份有限公司 Access request current limiting method, server and storage medium
CN113765820A (en) * 2020-10-30 2021-12-07 北京沃东天骏信息技术有限公司 Token bucket-based current limiting method, token bucket-based current limiting device, token bucket-based computing equipment and token bucket-based current limiting medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101005453A (en) * 2006-12-26 2007-07-25 华为技术有限公司 Method and device for refreshing token bucket in flow limiting technology
WO2015149488A1 (en) * 2014-04-03 2015-10-08 深圳市中兴微电子技术有限公司 Method and apparatus for limiting rate by means of token bucket, and computer storage medium
CN109729013A (en) * 2017-10-30 2019-05-07 深圳市中兴微电子技术有限公司 The method, apparatus and computer readable storage medium of token are added in a kind of traffic shaping
CN108768873A (en) * 2018-05-29 2018-11-06 腾讯科技(深圳)有限公司 A kind of flow control methods and relevant device
CN111447150A (en) * 2020-02-29 2020-07-24 中国平安财产保险股份有限公司 Access request current limiting method, server and storage medium
CN113765820A (en) * 2020-10-30 2021-12-07 北京沃东天骏信息技术有限公司 Token bucket-based current limiting method, token bucket-based current limiting device, token bucket-based computing equipment and token bucket-based current limiting medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陈刚,孟学军,邵烽: "航天测控通信系统网络技术", 30 September 2020, 国防工业出版社, pages: 108 - 109 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115473852A (en) * 2022-07-26 2022-12-13 青岛海尔科技有限公司 Current limiting method for user side input request, storage medium and electronic device
CN115396377A (en) * 2022-07-29 2022-11-25 天翼云科技有限公司 Method, device and equipment for optimizing service quality of object storage and storage medium
CN115396377B (en) * 2022-07-29 2024-03-12 天翼云科技有限公司 Method, device, equipment and storage medium for optimizing service quality of object storage
CN117009955A (en) * 2023-09-26 2023-11-07 北京安普诺信息技术有限公司 IAST and RASP probe control method and device based on code vaccine
CN117009955B (en) * 2023-09-26 2024-01-09 北京安普诺信息技术有限公司 IAST and RASP probe control method and device based on code vaccine

Similar Documents

Publication Publication Date Title
CN114598658A (en) Flow limiting method and device
Guo et al. Clio: A hardware-software co-designed disaggregated memory system
US10999381B2 (en) System and method for throttling service requests having non-uniform workloads
Harlap et al. Addressing the straggler problem for iterative convergent parallel ML
CN115210693A (en) Memory transactions with predictable latency
US9419904B2 (en) System and method for throttling service requests using work-based tokens
US8954986B2 (en) Systems and methods for data-parallel processing
US8195798B2 (en) Application server scalability through runtime restrictions enforcement in a distributed application execution system
US11641395B2 (en) Fault tolerant systems and methods incorporating a minimum checkpoint interval
EP3758311B1 (en) Techniques to facilitate a hardware based table lookup
CN112866136B (en) Service data processing method and device
Xie et al. Pandas: robust locality-aware scheduling with stochastic delay optimality
CN114928579B (en) Data processing method, device, computer equipment and storage medium
CN106164888A (en) The sequencing schemes of network and storage I/O request for minimizing interference between live load free time and live load
US11936568B2 (en) Stream allocation using stream credits
CN111857992B (en) Method and device for allocating linear resources in Radosgw module
CN110838987B (en) Queue current limiting method and storage medium
Simon et al. High-Performance Match-Action Table Updates from within Programmable Software Data Planes
WO2020076394A1 (en) Resource allocation using restore credits
Oh et al. Lalca: Locality-aware lock contention avoidance for nvme-based scale-out storage system
US10540341B1 (en) System and method for dedupe aware storage quality of service
US11816205B2 (en) Detecting and handling attacks on processes executing within a trusted execution environment
Jeon et al. Large graph processing based on remote memory system
JP3932965B2 (en) Multi-threaded program
CN117311910B (en) High-performance virtual password machine operation method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination