CN111371697A - Accurate flow control method and device for intersystem access - Google Patents

Accurate flow control method and device for intersystem access Download PDF

Info

Publication number
CN111371697A
CN111371697A CN202010180204.3A CN202010180204A CN111371697A CN 111371697 A CN111371697 A CN 111371697A CN 202010180204 A CN202010180204 A CN 202010180204A CN 111371697 A CN111371697 A CN 111371697A
Authority
CN
China
Prior art keywords
flow control
access
token
time
rule
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010180204.3A
Other languages
Chinese (zh)
Inventor
周侃
陈宏鸿
范兴泽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Digital China Financial Software Co ltd
Original Assignee
Digital China Financial Software Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Digital China Financial Software Co ltd filed Critical Digital China Financial Software Co ltd
Priority to CN202010180204.3A priority Critical patent/CN111371697A/en
Publication of CN111371697A publication Critical patent/CN111371697A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/215Flow control; Congestion control using token-bucket

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The invention provides an accurate flow control method and device for intersystem access. The method comprises the following steps: judging whether a custom flow control rule exists or not; if the rule does not exist, initializing flow control tokens, initially placing the number of the tokens, and setting expiration time, wherein the setting value of the expiration time is greater than the flow control period; if the rule exists, comparing the current request time with the last token placing time of the current rule, if the interval between the current request time and the last token placing time exceeds the flow control period, placing a flow control token into the distributed cache according to the maximum flow control rule, if the interval does not exceed the period, placing the flow control token into the distributed cache according to the self-defined flow control rule, and resetting the last token placing time; and successfully acquiring the flow control token and executing the request. The inter-system access-oriented accurate flow control method and device provided by the invention are based on the algorithm of the token bucket realized by distributed cache, and more efficient and accurate flow control is realized.

Description

Accurate flow control method and device for intersystem access
Technical Field
The invention relates to the technical field of distributed data processing, in particular to an accurate flow control method and device for intersystem access.
Background
In chinese patent application publication No. CN102404796B, it is mentioned that the performance of network hardware devices and software is continuously improved in order to cope with the rapid increase of the network user scale. However, a service satisfactory to the user is still not provided. The method mainly researches how to apply communication agent technology, database connection pool and cluster technology to improve the performance of the database server under the condition that a large number of concurrent users exist, and provides normal network service for the users. The access control technology of a large number of concurrent users is deeply explored and researched, and the adopted technology mainly comprises a communication agent technology, a database connection pool establishment technology, a program optimization technology and a clustering technology. When a client logs in a server, the client not only needs to establish connection with a database and transmit various information to the database, but also needs to maintain the connection after the connection of the database is established, which consumes a large amount of server resources. When the number of users is large, the resources of the database server are easily occupied too much, the overall performance of the database server is reduced, the communication proxy server is responsible for monitoring data transmitted on a network, the data for accessing the database is collected, analyzed and filtered, the processed data is put into a database connection pool queue, the connection of the database is realized through a small number of connections, meanwhile, a storage process is provided to improve the working efficiency of database developers and the efficiency of program execution, and the essence is to provide a means for the developers to store common SQL sentences in the database so as to be reused.
In the chinese invention patent application with publication number CN106817424A, a method and system for controlling access traffic is proposed. The method comprises the following steps: receiving a service request of a peripheral system; judging whether the access quantity of the peripheral system exceeds a multi-level access quantity threshold value according to the service request time of the peripheral system, wherein the multi-level access quantity threshold value sets different access quantity threshold values according to different service request times; and if the access amount of the peripheral system does not exceed the multi-level access amount threshold, processing the service request, and controlling the access flow, so that the stress on a core system caused by abnormal flow access of the peripheral system can be prevented.
The two flow control methods have the problems of single control mode and inaccurate flow limiting operation.
Disclosure of Invention
The technical problem to be solved by the invention is to provide an accurate flow control method and device for intersystem access, which realize more efficient and accurate flow control based on a token bucket algorithm realized by distributed cache.
In order to solve the above technical problem, the present invention provides an accurate flow control method for inter-system access, where the method includes: judging whether a custom flow control rule exists or not; if the rule does not exist, initializing flow control tokens, initially placing the number of the tokens, and setting expiration time, wherein the setting value of the expiration time is greater than the flow control period; if the rule exists, comparing the current request time with the last token placing time of the current rule, if the interval between the current request time and the last token placing time exceeds the flow control period, placing a flow control token into the distributed cache according to the maximum flow control rule, if the interval does not exceed the period, placing the flow control token into the distributed cache according to the self-defined algorithm rule, and resetting the last token placing time; and successfully acquiring the flow control token and executing the request.
In some embodiments, when the flow control tokens are placed according to the maximum flow control rule, the number of the flow control tokens placed in a unit time is the maximum.
In some embodiments, when the flow control tokens are placed according to the custom algorithm rule, the number of placed flow control tokens is calculated according to the following formula:
n=delta/(1/qps)
where n is the number of placed flow control tokens, delta is the time interval between the current request time and the last token placement time of the current rule, and qps is the number of placed tokens per unit time.
In some embodiments, the expiration time is set at twice the flow control period.
In some embodiments, further comprising: and after the flow control token is placed in the distributed cache, the flow control token is failed to be obtained, and the request is rejected.
In some embodiments, further comprising: and controlling the access flow according to the access control list, wherein before the flow control, the token initialization and the loading of the access control list are ensured to be successful.
In some embodiments, controlling access traffic according to the access control list comprises: and controlling the access flow according to the standard access control list or the extended access control list.
In some embodiments, controlling access traffic according to a standard access control list comprises: and filtering the data packet according to the source IP address and the user-defined rule of the data packet.
In some embodiments, controlling access traffic according to the extended access control list comprises: and defining rules according to the source IP address, the destination IP address, the source port number, the destination port number and the protocol of the data packet and other self-defined rules so as to filter the data packet.
In addition, the present invention also provides an accurate flow control apparatus for inter-system access, the apparatus comprising: one or more processors; a storage device for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method for accurate flow control for inter-system access according to the foregoing description.
After adopting such design, the invention has at least the following advantages:
1) accurate flow control is realized, and stable operation of access among systems under large concurrent requests is ensured;
2) the support of various flow control algorithms, wherein the processing is more efficient for the algorithm of the token bucket realized by the distributed cache;
3) the configuration of the multidimensional system access flow control can be based on certain rules, such as account number, IP, and flow limitation of system call logic level.
Drawings
The foregoing is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and the detailed description.
FIG. 1 is a schematic diagram of a prior art leaky bucket algorithm;
FIG. 2 is a schematic diagram of the token bucket algorithm provided by the prior art;
FIG. 3 is a flow chart of a token bucket implemented based on a distributed cache according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of current limiting according to a certain rule, such as an account number, an IP, and a system call logic level, according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a precise flow control device for inter-system access according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described in conjunction with the accompanying drawings, and it will be understood that they are described herein for the purpose of illustration and explanation and not limitation.
The invention mainly emphasizes flexible multidimensional flow configuration and accurate flow control method of intersystem access, ensures reasonable access pressure and controls access flow among systems, and mainly relates to the following steps:
the current limiting algorithm in the present invention mainly includes: a leaky bucket algorithm and a token bucket algorithm.
1) Leaky bucket algorithm
The algorithm idea of the Leaky Bucket (leak Bucket) is simple, water (request) enters the Leaky Bucket firstly, the Leaky Bucket discharges water at a certain speed (the interface has response speed), when the water inflow speed is too high, the water can directly overflow (the access frequency exceeds the interface response speed), then the request is rejected, and the algorithm of the Leaky Bucket can be seen to forcibly limit the transmission speed of data. Please refer to fig. 1 for a schematic diagram.
It can be seen that there are two variables, one is the size of the bucket, which supports how much water can be stored when the flow is suddenly increased (burst), and the other is the size of the bucket leak (rate). Since the leakage rate of leaky buckets is a fixed parameter, even if there is no resource conflict in the network (no congestion occurs). The leaky bucket algorithm also cannot burst (burst) the flow to the port rate. Thus, the leaky bucket algorithm lacks efficiency for traffic that has bursty characteristics.
2) Token bucket algorithm
The Token Bucket algorithm (Token Bucket) and the leak Bucket algorithm have the same effect but opposite directions, and are easier to understand. As time passes, the system will add Token to the bucket at a constant 1/QPS interval (10 ms if QPS is 100) and not if the bucket is full. When a new request comes, one Token is taken each and if no Token is available, the service is blocked or denied.
Another benefit of token buckets is the ease with which the speed of change can be achieved. Once the rate needs to be increased, the rate of tokens put into the bucket is increased as needed. A certain number of tokens are typically added to the bucket at regular intervals (e.g., 1000 milliseconds), and some variations calculate the number of tokens to be added in real time.
a. Releasing tokens to a token bucket at a specific rate;
b. classifying the messages according to a preset matching rule, wherein the messages which do not conform to the matching rule are directly sent without being processed by a token bucket;
c. and if the message conforms to the matching rule, the token bucket is required to process the message. When enough tokens are in the bucket, the message can be continuously sent, and meanwhile, the token quantity in the token bucket is correspondingly reduced according to the length of the message;
d. when the token in the token bucket is insufficient, the message cannot be sent, and the message can be sent only when a new token is generated in the bucket. Therefore, the flow of the message can be limited only to be less than or equal to the token generation speed, and the purpose of limiting the flow is achieved. The operation process of the token bucket algorithm is shown in fig. 2.
For many application scenarios, in addition to being able to limit the average transfer rate of data, it is also desirable to allow some degree of bursty transfer. At this time, the leaky bucket algorithm may be inappropriate, and the token bucket algorithm is more appropriate.
3) Token bucket based on distributed cache implementation
Key presence or absence. Because the flow chart is based on cluster flow limitation by distributed cache, statistics needs to be carried out according to different keys, and the initialized keys are accessed for the first time.
2. If the key does not exist, the token bucket is initialized, the initial token count is prevented, and the key expiration time is set to interval 2. The initial token number here may be set to a throttling threshold such as throttling 10qps in general, and the initial value may be set to 10 to handle the initial traffic. interval is an interval time, such as the current limit threshold 10qps, with interval set to 1 s. The expiration time is the time of the key in the cache, and interval x 2 is to prevent the key from being expired and not intercepting the traffic.
3. If a key exists, the current request time is compared to the last token placed time for the current key. And if the interval exceeds the interval, entering the step 4, and if the interval does not exceed the interval, entering the step 5.
4. The interval has exceeded 1s and tokens are placed directly to the maximum number.
5. The interval does not exceed 1s, delta is defined as the time difference, and the number of tokens placed is delta/(1/qps). The number of tokens is guaranteed not to exceed the capacity of the bucket when the tokens are put in. At the same time, the time to put the token is reset.
6. Obtaining a token from the bucket, and executing the request after the token is successfully obtained; obtaining the token time and rejecting the request.
The operation of the above steps 1 to 6 is shown in fig. 3.
In addition, the flow limitation can be performed according to certain rules, such as account number, IP, and flow limitation of a system call logic level.
The access control list is a technique used to identify and filter data traffic sent by or to certain networks that meets the conditions specified by us in networks using routing technology to determine whether the data traffic should be forwarded or dropped.
In the face of increasingly complex network environments, network administrators must deny undesirable connections while allowing legitimate access because of the dangers associated with the critical devices and data of these connections. While some approaches may address such challenges, such as encryption techniques, callback techniques, etc., these approaches do not provide accurate, flexible control of data traffic. The access control list can filter harmful data packets through controlling network data flow, so as to achieve the purpose of executing the security policy. By applying the access control list correctly, the network administrator can do almost any security policy he wants to get. Due to such features, the access control list becomes an important means for implementing the firewall.
Access control lists are basically divided into two main categories: a standard access control list and an extended access control list. And the standard access control list filters the data packet according to the source IP address and the user-defined rule of the data packet. The extended access control list defines rules according to the source IP address, the destination IP address, the source port number, the destination port number, and the protocol of the packet, and other customized rules, so as to filter the packet.
When a request is received, it is first determined whether the IP/custom rule applies, and if not, the packet is routed normally. If so, processing, from the beginning of the process, compares the conditions with the packet contents. If there is no match, the next statement in the list is processed, and if there is a match, the permit or deny operation is performed. If no matching rule is found in the entire list, the request is discarded and rejected according to the custom configuration rule.
The control process of the above-described current limit adjustment is illustrated by fig. 4.
Fig. 5 shows a typical structure of a precise flow control apparatus facing an inter-system access. For example, the precision flow control apparatus 500 facing inter-system access may be used for a storage device serving as a storage device or the like. As described herein, the precision flow control device 500 oriented to inter-system access may be used to implement the function of precision flow control in a distributed system. The precision flow control device 500 for inter-system access may be implemented in a single node, or the functions of the precision flow control device 500 for inter-system access may be implemented in multiple nodes in a network. Those skilled in the art will appreciate that the term inter-system access oriented precision flow control device includes a broad sense of apparatus and that the inter-system access oriented precision flow control device 500 shown in fig. 5 is only one example. The inclusion of the inter-system access-oriented precision flow control device 500 is for clarity and is not intended to limit the application of the present invention to a particular inter-system access-oriented precision flow control device embodiment or to a class of inter-system access-oriented precision flow control device embodiments. At least some of the features/methods described herein may be implemented in a network device or component, such as, for example, an inter-system access oriented precision flow control device 500. For example, the features/methods of the present invention may be implemented in hardware, firmware, and/or software running installed on hardware. The precise flow control device 500 for inter-system access may be any device that processes, stores, and/or forwards data frames through a network, such as a server, a client, a data source, and the like. As shown in fig. 5, the precise flow control device 500 facing inter-system access may include a transceiver (Tx/Rx)510, which may be a transmitter, a receiver, or a combination thereof. Tx/Rx 510 may be coupled to a plurality of ports 550 (e.g., an uplink interface and/or a downlink interface) for transmitting and/or receiving frames from other nodes. Processor 530 may be coupled to Tx/Rx 510 to process frames and/or determine to which nodes to send frames. Processor 530 may include one or more multi-core processors and/or memory devices 532, which may serve as data stores, buffers, and the like. Processor 530 may be implemented as a general-purpose processor or may be part of one or more Application Specific Integrated Circuits (ASICs) and/or Digital Signal Processors (DSPs).
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the present invention in any way, and it will be apparent to those skilled in the art that the above description of the present invention can be applied to various modifications, equivalent variations or modifications without departing from the spirit and scope of the present invention.

Claims (10)

1. An accurate flow control method facing inter-system access is characterized by comprising the following steps:
judging whether a custom flow control rule exists or not;
if the rule does not exist, initializing flow control tokens, initially placing the number of the tokens, and setting expiration time, wherein the setting value of the expiration time is greater than the flow control period;
if the rule exists, comparing the current request time with the last token placing time of the current rule, if the interval between the current request time and the last token placing time exceeds the flow control period, placing a flow control token into the distributed cache according to the maximum flow control rule, if the interval does not exceed the period, placing the flow control token into the distributed cache according to the self-defined flow control rule, and resetting the last token placing time;
and successfully acquiring the flow control token and executing the request.
2. The method according to claim 1, wherein when the flow control tokens are placed according to a maximum flow control rule, the number of the flow control tokens placed in a unit time is a maximum value.
3. The inter-system access-oriented accurate flow control method according to claim 1, wherein when the flow control tokens are placed according to a custom algorithm rule, the number of the placed flow control tokens is calculated according to the following formula:
n=delta/(1/qps)
where n is the number of placed flow control tokens, delta is the time interval between the current request time and the last token placement time of the current rule, and qps is the number of placed tokens per unit time.
4. The method of claim 1, wherein the expiration time is set to be twice the flow control period.
5. The method for accurate flow control of intersystem access according to claim 1, further comprising:
and after the flow control token is placed in the distributed cache, the flow control token is failed to be obtained, and the request is rejected.
6. The method for accurate flow control of intersystem access according to claim 1, further comprising:
and controlling the access flow according to the access control list, wherein before the flow control, the token initialization and the loading of the access control list are ensured to be successful.
7. The method of claim 6, wherein the controlling the access flow according to the access control list comprises:
and controlling the access flow according to the standard access control list or the extended access control list.
8. The method of claim 7, wherein the controlling the access flow according to the standard access control list comprises:
and filtering the data packet according to the source IP address of the data packet.
9. The method of claim 7, wherein the controlling the access traffic according to the extended access control list comprises:
and defining rules according to the source IP address, the destination IP address, the source port number, the destination port number and the protocol of the data packet so as to filter the data packet.
10. An inter-system access-oriented precise flow control device, comprising:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of accurate flow control for inter-system access according to any one of claims 1 to 9.
CN202010180204.3A 2020-03-16 2020-03-16 Accurate flow control method and device for intersystem access Pending CN111371697A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010180204.3A CN111371697A (en) 2020-03-16 2020-03-16 Accurate flow control method and device for intersystem access

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010180204.3A CN111371697A (en) 2020-03-16 2020-03-16 Accurate flow control method and device for intersystem access

Publications (1)

Publication Number Publication Date
CN111371697A true CN111371697A (en) 2020-07-03

Family

ID=71211867

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010180204.3A Pending CN111371697A (en) 2020-03-16 2020-03-16 Accurate flow control method and device for intersystem access

Country Status (1)

Country Link
CN (1) CN111371697A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112929291A (en) * 2021-02-18 2021-06-08 欧冶云商股份有限公司 Distributed current limiting method based on redis, computer equipment and storage medium
CN113438183A (en) * 2021-06-29 2021-09-24 软通动力信息技术(集团)股份有限公司 Outgoing flow control method, device, equipment and storage medium of network framework
CN115473852A (en) * 2022-07-26 2022-12-13 青岛海尔科技有限公司 Current limiting method for user side input request, storage medium and electronic device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102387082A (en) * 2011-11-25 2012-03-21 西安电子科技大学 Flow-classification-based grouping flow control system and control method
CN106656850A (en) * 2016-10-27 2017-05-10 盛科网络(苏州)有限公司 Chip realizing method for automatically identifying network traffic and making speed limit
CN109194584A (en) * 2018-08-13 2019-01-11 中国平安人寿保险股份有限公司 A kind of flux monitoring method, device, computer equipment and storage medium
CN110380986A (en) * 2019-07-23 2019-10-25 中南民族大学 Flow limitation method, device, equipment and storage medium based on Zuul

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102387082A (en) * 2011-11-25 2012-03-21 西安电子科技大学 Flow-classification-based grouping flow control system and control method
CN106656850A (en) * 2016-10-27 2017-05-10 盛科网络(苏州)有限公司 Chip realizing method for automatically identifying network traffic and making speed limit
CN109194584A (en) * 2018-08-13 2019-01-11 中国平安人寿保险股份有限公司 A kind of flux monitoring method, device, computer equipment and storage medium
CN110380986A (en) * 2019-07-23 2019-10-25 中南民族大学 Flow limitation method, device, equipment and storage medium based on Zuul

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112929291A (en) * 2021-02-18 2021-06-08 欧冶云商股份有限公司 Distributed current limiting method based on redis, computer equipment and storage medium
CN113438183A (en) * 2021-06-29 2021-09-24 软通动力信息技术(集团)股份有限公司 Outgoing flow control method, device, equipment and storage medium of network framework
CN115473852A (en) * 2022-07-26 2022-12-13 青岛海尔科技有限公司 Current limiting method for user side input request, storage medium and electronic device

Similar Documents

Publication Publication Date Title
US11671402B2 (en) Service resource scheduling method and apparatus
CN111371697A (en) Accurate flow control method and device for intersystem access
EP3516833B1 (en) Methods, systems, and computer readable media for discarding messages during a congestion event
US10476629B2 (en) Performing upper layer inspection of a flow based on a sampling rate
JP2004364306A (en) System for controlling client-server connection request
US20220021701A1 (en) Method and System for Providing Edge Service, and Computing Device
US7680062B2 (en) Apparatus and method for controlling abnormal traffic
US20120324573A1 (en) Method for determining whether or not specific network session is under denial-of-service attack and method for the same
US10313238B2 (en) Communication system, communication method, and non-transitiory computer readable medium storing program
JP7462757B2 (en) Network security protection method and protection device
CN104796406A (en) Method and device for identifying application
EP2929670B1 (en) System to protect a mobile network
KR20060121647A (en) Data communication coordination with sequence numbers
WO2019192318A1 (en) Traffic smoothing method, server, and forwarding device
CN110661722B (en) Flow control method and device
KR20190018947A (en) Apparatus and method for handling a network attack in a software defined network
CN113821410A (en) Log processing method and device
US20180255061A1 (en) Device Management Across Multiple Operator Networks
US20210004308A1 (en) Data processing method and system
CN110166359B (en) Message forwarding method and device
EP4181554A1 (en) Traffic control server and method
EP4366236A1 (en) Method and apparatus for identifying source address of message
CN113630388B (en) Unidirectional transmission method, unidirectional transmission device, computer equipment and readable storage medium
CN111327604B (en) Data processing system and method thereof
KR20180041976A (en) SDN for preventing malicious application and Determination apparatus comprising the same

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200703

RJ01 Rejection of invention patent application after publication