CN116095013B

CN116095013B - Service request current limiting method, device and storage medium

Info

Publication number: CN116095013B
Application number: CN202211741649.XA
Authority: CN
Inventors: 李文成; 左劼; 李昌盛; 秦川
Original assignee: Beijing Kaike Weizhi Technology Co ltd
Current assignee: Beijing Kaike Weizhi Technology Co ltd
Priority date: 2022-12-29
Filing date: 2022-12-29
Publication date: 2023-07-25
Anticipated expiration: 2042-12-29
Also published as: CN116095013A

Abstract

The invention relates to the technical field of information and provides a service request current limiting method, a device and a storage medium. The aim of self-adaptively adjusting the distribution quantity of tokens according to the processing capacity of a service system is achieved, and the purpose of limiting the flow of service requests is achieved. The main scheme includes that after a token management module receives a request of a service system for applying a token, a token maximum use threshold value of a token bucket corresponding to the type of a current service request is obtained, if the use amount exceeds the threshold value, a new token is refused to be issued, otherwise, a transaction token is generated, and the transaction token, a transaction token generation time stamp and a token bucket ID are sent to the service system; after the business system obtains the transaction token, executing the subsequent business logic, after the processing of the business logic is completed, returning the transaction token generation time command and the token bucket ID information by the business system, and subtracting 1 from the number of the corresponding token buckets according to the token bucket ID; and counting the usage information of the transaction tokens, and dynamically adjusting the maximum usage threshold according to the counting result.

Description

Service request current limiting method, device and storage medium

Technical Field

The invention relates to the technical field of information and provides a service request current limiting method, a device and a storage medium.

Background

In many service processing systems, the entrance for receiving the service request has a flow limiting function, and by using this function, the processing pressure given to the back-end service system can be limited, so as to reduce the risk of system breakdown. For example, in a financial payment service system, a service gateway receives a new service request and then processes the new service request through system modules such as a payment system, a core system, a wind control system, a back-end payment channel and the like. When the system resources of a certain system module are insufficient, the conditions of slowing down the request processing, overtime request processing, incapability of timely processing new requests and the like can occur, and a series of chain reactions can be generated, so that the user experience is finally affected. The traffic limiting function is added at the entrance for receiving the service request, and the service traffic is reasonably controlled according to the processing capacity of the rear system module, which is a common practice accepted by the industry. There are also many solutions to this problem currently in the industry.

The token bucket based throttling scheme is one scheme that uses more. The scheme maintains a "bucket" (hereinafter "token bucket") that can hold a number of tokens at the entrance of request reception and places tokens into the bucket at a certain rate. Newly placed tokens will be discarded if the token bucket reaches maximum capacity. When the system receives a new service request, it tries to acquire a token from the token bucket, if the token is available, the request can be executed, otherwise the request is refused or the request is put on hold.

There is also a flow restriction scheme based on the leaky bucket method, which maintains a "bucket" like a funnel (hereinafter "leaky bucket") with a certain capacity. The system puts a drop of water into the leaky bucket each time a new request is received, and the request enters a queuing state to wait for the system to process. While the leaky bucket "drips" outwards at a constant rate, indicating that a request in line needs to be processed. If the leaky bucket is already full of water, the newly received request may be denied or suspended.

The existing schemes can effectively achieve the purpose of flow control, and can guarantee the safe and stable operation of the rear system to a certain extent. However, these schemes have the problems that the processing condition of the rear system cannot be perceived and the flow rate cannot be dynamically adjusted according to the operation condition of the system.

1. The rear system processing situation cannot be perceived. The main emphasis of the existing flow control scheme is to limit the received request, so that the safety and stability of a rear system are guaranteed. This approach has a drawback: when the service processing speed of the rear system is reduced due to some problems, or the program is failed or even down to cause the service processing to be interrupted, the system can not bear the current flow pressure, at the moment, the front flow control program can not sense the condition and normally release a new request, and the flow control is meaningless.

2. The flow cannot be dynamically adjusted according to the system operation condition. By knowing the existing schemes, we know that there is a concept of "tokens" in the existing schemes, and the number of the tokens is fixed, and is determined at the beginning of the system operation, and the system operation cannot be conveniently adjusted. Such problems may result in a failure to reasonably utilize system resources, waste, or the like as described in point 1. For example, when the service processing efficiency of the rear system is high, the flow can be actually increased in the flow control program, and the system capacity is fully utilized; when the service processing efficiency of the rear system is lower, the flow can be reduced in the flow control program, so that the purpose of protecting the safety and stability of the system is achieved.

In systems using similar schemes, the system can only be protected by means of health detection of applications, monitoring of system resources, etc. In the micro-service scene, a fusing mechanism is also introduced to protect the system. All the modes can achieve the purpose of protecting the system, but have obvious delay and can not discover the problems in time. The stability of the system will also be somewhat affected by the introduction of new programs.

Disclosure of Invention

The invention aims to realize self-adaptive adjustment of the distribution quantity of tokens according to the processing capacity of a service system, thereby achieving the purpose of limiting the flow of service requests.

In order to achieve the above purpose, the present invention adopts the following technical means:

a service request throttling method, comprising the steps of:

step 1, when a service system receives a new service request, applying for a transaction token from a token management module, wherein the token management module firstly judges whether a corresponding token bucket exists in the type of the current service request, if not, performs resource initialization, generates the token bucket, and sets a maximum use threshold of the transaction token of the initial token bucket;

step 2, after receiving a request of a service system for applying a token, a token management module firstly acquires a maximum token use threshold value of a token bucket corresponding to the type of the current service request from a threshold management module, then compares the maximum token use threshold value with the number of currently issued tokens, refuses to issue new tokens if the use amount exceeds the threshold value, otherwise generates a transaction token, splices the transaction token, a transaction token generation timestamp and a token bucket ID to obtain an authentication data packet, and sends the authentication data packet to the service system;

step 3, after the service system acquires the authentication data packet, analyzing the data packet to obtain a transaction token, executing subsequent service logic, submitting the authentication data packet to a token management module by the service system after the service logic is processed, analyzing the authentication data packet by the token management module to obtain a transaction token generation time stamp and token bucket ID information, recording the current time of analysis of the authentication data packet as an end time stamp, and naturally taking the time stamp of the service system returned to the authentication data packet as the end time stamp;

step 4, subtracting 1 from the number of the corresponding token buckets according to the token bucket IDs;

and 5, counting the use information of the transaction token by the threshold management module according to the generated time stamp and the ending time stamp obtained in the step 3, and dynamically adjusting the maximum use threshold according to the counting result.

In the above technical solution, when the service system applies for a transaction token, the token management module may separately maintain a token bucket according to a service request type, where the token bucket includes a bucket 1 and a bucket 2, and updates the unique token bucket IDs and token numbers of the bucket 1 and the bucket 2 in a time period, specifically:

defining a time period, assigning a new unique token bucket ID to the bucket 1 in each new time period, setting the token quantity of the bucket 1 to zero, assigning the unique token bucket ID of the previous time period of the bucket 1 and the token quantity stored in the bucket 1 to the bucket 2, and considering the original token quantity in the bucket 2 to be expired and released.

In the above technical solution, the threshold management module:

calculating the generated time stamp and the ending time stamp obtained in the step 3 to obtain the using time of the transaction token; taking a configurable time period as a window, counting the use time of all transaction tokens in the window, comparing with the counting result of the last window data, and when the use time of the tokens in the new window is smaller, considering that the efficiency of the service system for processing corresponding service in the last period is higher, and increasing the maximum use threshold of the current token; when the token usage time in the new window is more, the service system is considered to be less efficient in processing the corresponding service in the last period of time, and the maximum usage threshold of the current token is reduced.

The invention also provides a service request current limiting device:

when the service system receives a new service request, applying for a transaction token to a token management module, wherein the token management module firstly judges whether a corresponding token bucket exists in the type of the current service request, if not, performs resource initialization, generates the token bucket, and sets the maximum use threshold of the transaction token of the initial token bucket;

after receiving a request of a service system for applying a token, the token management module firstly acquires a maximum token use threshold value corresponding to a token bucket of the type of the current service request from the threshold management module, then compares the maximum token use threshold value with the number of currently issued tokens, refuses to issue a new token if the use amount exceeds the threshold value, otherwise generates a transaction token, splices the transaction token, a transaction token generation time stamp and a token bucket ID to obtain an authentication data packet, and sends the authentication data packet to the service system;

after the service system acquires the authentication data packet, analyzing the data packet to obtain a transaction token, executing subsequent service logic, submitting the authentication data packet to a token management module by the service system after the service logic is processed, analyzing the authentication data packet by the token management module to obtain a transaction token generation time stamp and token bucket ID information, and recording the current time of analysis of the authentication data packet as an end time stamp;

subtracting 1 from the number of corresponding token buckets according to the token bucket IDs;

and (3) counting by the threshold management module according to the use information of the generated time stamp and the ending time stamp transaction token obtained in the step (3), and dynamically adjusting the maximum use threshold according to the counting result.

In the above device, when the service system applies for the transaction token, the token management module may separately maintain a token bucket according to the service request type, where the token bucket includes a bucket 1 and a bucket 2, and updates the unique token bucket IDs and token numbers of the bucket 1 and the bucket 2 in a time period, specifically:

In the above device, the threshold management module:

calculating the generated time stamp and the ending time stamp to obtain the using time of the transaction token; taking a configurable time period as a window, counting the use time of all transaction tokens in the window, comparing with the counting result of the last window data, and when the use time of the tokens in the new window is smaller, considering that the efficiency of the service system for processing corresponding service in the last period is higher, and increasing the maximum use threshold of the current token; when the token usage time in the new window is more, the service system is considered to be less efficient in processing the corresponding service in the last period of time, and the maximum usage threshold of the current token is reduced.

The invention also provides a storage medium, and the processor realizes the service request current limiting method when executing the program in the storage medium.

Because the invention adopts the technical means, the invention has the following beneficial effects:

1. when the system behind the flow control program has faults affecting the normal execution of the service, the problems can be found in time according to a feedback mechanism, so that the number of available tokens is controlled to reduce the processing pressure of the system behind, and the purpose of protecting the system is achieved.

2. After the flow control program can sense the system processing condition, the flow control program can analyze the service system processing efficiency in real time, and automatically increase the maximum use quantity of tokens when the system processing efficiency is high so as to fully utilize the system resources; the maximum use quantity of tokens can be reduced when the processing efficiency of the system is reduced or faults occur, the pressure of a rear system is reduced, and the system is given sufficient recovery time.

3. The method can tolerate the problem that the calling party does not return the token because of abnormal reasons such as downtime, and the like, so that the robustness of the system is further enhanced.

4. Based on the automatic sensing and real-time analysis functions of the flow control program, negligible resource consumption is used for replacing the release of the dependence on the third party component, so that the operation and maintenance cost of the system is reduced and the stability of the system is improved.

Drawings

FIG. 1 is a block diagram of the overall structure of the present invention;

FIG. 2 is a token acquisition flow chart;

fig. 3 is a logic block diagram of threshold dynamic adjustment.

Detailed Description

Hereinafter, embodiments of the present invention will be described in detail. While the invention will be described and illustrated in conjunction with certain specific embodiments, it will be understood that it is not intended to limit the invention to these embodiments alone. On the contrary, the invention is intended to cover modifications and equivalent arrangements included within the scope of the appended claims.

In addition, numerous specific details are set forth in the following description in order to provide a better illustration of the invention. It will be understood by those skilled in the art that the present invention may be practiced without these specific details.

1. Integral structure

The whole structure of the invention is shown in figure 1

When the service system receives a new request, the service system applies for a transaction token to the token management module, and the token management module firstly judges whether a corresponding token bucket exists in the current request type or not, and if not, the service system performs resource initialization. After receiving a request for applying a token, the token management module firstly acquires a maximum use threshold value of the token corresponding to the current request type from the threshold management module, then compares the maximum use threshold value with the number of the currently issued tokens, refuses to issue a new token if the use amount exceeds the threshold value, otherwise issues a transaction token to the service system, and when issuing the token, the transaction token generation timestamp and the token bucket ID need to be spliced and packaged to obtain an authentication data packet which is sent to the service system.

After the service system finishes the processing of the request, the service system actively returns the authentication data packet obtained by the application. The token management module releases the token returned by the service system for subsequent requests, and simultaneously counts partial use information of the token for the threshold management module to analyze afterwards.

The whole flow control program is divided into two modules:

the token management module:

token management: and the management work such as token generation, token issuing, timeout release and the like of the flow control are responsible.

And (3) token recovery: is responsible for normal release of the transaction token, partial usage information statistics of the license, etc.

Threshold management module: and managing the maximum use threshold of the tokens of each token bucket, completing analysis work of transaction resource consumption, and supporting data for dynamic adjustment of the token use threshold.

2. Token management

After receiving the new request, the business system may initiate a transaction token application request to the flow control program according to a number of different dimensions. The method can be applied according to the communication dimension as in the prior art scheme, namely: the same token is used for different types of requests received by the service system; the application may also be applied according to the dimension of the request type, namely: the service system receives the request, judges the type of the request, and creates a token bucket according to the type of the request. The service can be better isolated according to the request type application, and the flow of a certain service can be regulated on the premise of not influencing other services.

The generation of the token and the acquisition of the token are schematically shown in fig. 2.

When the service system applies for the transaction token, the token management module independently maintains a token bucket according to the service request type, wherein the token bucket comprises a bucket 1 and a bucket 2, and updates the unique token bucket ID and the token quantity of the bucket 1 and the bucket 2 in a time period, specifically:

defining a time period, assigning a new unique token bucket ID to the bucket 1 in each new time period, setting the token quantity of the bucket 1 to zero, assigning the unique token bucket TD of the last time period of the bucket 1 and the token quantity stored in the bucket 1 to the bucket 2, and determining the original token quantity in the bucket 2 to be expired and released, wherein the maximum capacity of the bucket 1 is the maximum token usage threshold minus the token quantity stored in the bucket 2.

By utilizing the cooperation of the two barrels to store the token information, the token can still be released when the system fails or is down, and the token is ensured not to be lost.

3. Token recovery

And after the service system applies for the token, normally executing service logic, and actively returning the token after completion. After the token management module receives the token returned by the service system, the token is subjected to cancellation release processing, and a release result is returned to the service system, so that the service system continues to execute the subsequent flow. When releasing tokens, the number of tokens in the corresponding bucket is decremented by one.

After the token is released, the system starts counting the service condition of the token, mainly the service duration of the token. And reporting the counted use information to a threshold management module for standby.

If the service system fails to return the token normally due to downtime, the token will remain in the bucket 1 until the bucket 2 is taken over after reaching the timeout release period. When bucket 2 again reaches the timeout release period, the token is considered to be an expired release for use by subsequent requests.

4. Threshold management

Each token bucket has a maximum usage threshold, which is the maximum concurrency of the traffic corresponding to the token. When a token is created, a corresponding initial threshold value is synchronously maintained in the module, a task is created, data fed back by the token management module is analyzed, and the threshold value is dynamically adjusted according to a set rule. The initial threshold supports customization.

The module can analyze the data fed back by the token management module according to two dimensions of time and number:

in the aspect of time, a configurable time period is taken as a window, all data in the window are counted, and then the counted data is compared with the counted result of the data in the last window. When the service time of the token in the new window is smaller, the service system can be considered to have higher efficiency of processing the corresponding service in the last period of time, and then the maximum service threshold of the current token can be properly increased; when the token usage time in the new window is more, the service system can be considered to be less efficient in processing the corresponding service in the last period of time, and then the maximum usage threshold of the current token can be appropriately reduced.

Alternatively, a sample reference value may be set in terms of number, and the statistics of the sample data are only performed when the number of samples fed back by the token management module reaches the reference value, and then the statistics of the sample data are compared with the statistics of the last sample data. The comparison logic is the same as the comparison logic of the time dimension.

The data analysis algorithm supports extended changes. The threshold adjustment is schematically shown in fig. 3.

In addition to the automatic adjustment scheme described above, the module also supports manual adjustment through the interface. The manual adjustment scheme supports immediate adjustment, timing adjustment, periodic adjustment, etc. for selection. When the manual adjustment scheme is executed, the program is automatically switched to the automatic adjustment scheme.

And (3) immediately adjusting: the specified token maximum use threshold is immediately adjusted to a given value.

And (3) timing adjustment: at a specified point in time, a specified token maximum use threshold is adjusted to a given value.

And (3) period adjustment: setting a periodic time point, the system will adjust the specified token maximum use threshold to a given value at a specified time point within each period.

In addition, the threshold management module also provides data for the period value of the bucket 2 timeout release. The system defaults to using the longest usage time of tokens in valid samples as the timeout release period for bucket 2 and supports modification.

Claims

1. A method for limiting a service request, comprising the steps of:

step 3, after the service system acquires the authentication data packet, analyzing the data packet to obtain a transaction token, executing subsequent service logic, submitting the authentication data packet to a token management module by the service system after the service logic is processed, analyzing the authentication data packet by the token management module to obtain a transaction token generation time stamp and token bucket ID information, and recording the current time of analysis of the authentication data packet as an end time stamp;

step 5, the threshold management module counts the use information of the transaction token according to the transaction token generation time stamp and the end time stamp obtained in the step 3, and dynamically adjusts the maximum use threshold according to the statistics result;

2. The method of claim 1, wherein the token management module maintains token buckets individually according to service request types when the service system applies for transaction tokens, wherein the token buckets include "bucket 1" and "bucket 2", and updates the unique token bucket IDs and token numbers of bucket 1 and bucket 2 in a time period, specifically:

defining a time period, assigning a new unique token bucket ID to the bucket 1 in each new time period, setting the token quantity of the bucket 1 to zero, assigning the unique token bucket ID of the previous time period of the bucket 1 and the token quantity stored in the bucket 1 to the bucket 2, and considering the original token quantity in the bucket 2 to be expired and released, wherein the maximum capacity of the bucket 1 is the maximum token usage threshold minus the token quantity stored in the bucket 2.

3. A service request flow limiting device, characterized in that:

the threshold management module counts the use information of the transaction token according to the obtained generation time stamp and the obtained ending time stamp, and dynamically adjusts the maximum use threshold according to the statistics result;

threshold management module:

4. A service request flow limiting device according to claim 3, wherein the token management module maintains token buckets individually according to service request types when the service system applies for transaction tokens, the token buckets include "bucket 1" and "bucket 2", and the time period update is performed on the unique token bucket IDs and token numbers of bucket 1 and bucket 2, specifically:

5. A storage medium, characterized in that a processor implements a service request throttling method according to any of claims 1-2 when executing a program in the storage medium.