CN113556245A - Bandwidth speed limiting method, system, device and storage medium for Rsocket library - Google Patents
Bandwidth speed limiting method, system, device and storage medium for Rsocket library Download PDFInfo
- Publication number
- CN113556245A CN113556245A CN202110603061.7A CN202110603061A CN113556245A CN 113556245 A CN113556245 A CN 113556245A CN 202110603061 A CN202110603061 A CN 202110603061A CN 113556245 A CN113556245 A CN 113556245A
- Authority
- CN
- China
- Prior art keywords
- bandwidth
- speed
- limiting
- rsocket
- library
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/08—Configuration management of networks or network elements
- H04L41/0896—Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/16—Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
- H04L69/161—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields
- H04L69/162—Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields involving adaptations of sockets based mechanisms
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The application relates to the technical field of flow control, and discloses a bandwidth speed limiting method, a system, a device and a storage medium for an Rsocket library, wherein the method comprises the following steps: establishing shared memory communication with a Helper process, and acquiring bandwidth speed-limiting information from the Helper process, wherein the bandwidth speed-limiting information comprises expected bandwidth; sending a speed limit request to a speed limiter module, wherein the speed limit request comprises the bandwidth speed limit information, and the speed limiter module limits the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth. The invention can achieve the purpose of limiting the speed of the outlet flow bandwidth and the inlet flow bandwidth of the Rsocket library.
Description
Technical Field
The application relates to the technical field of flow control, in particular to a bandwidth speed limiting method, a system, a device and a storage medium for an Rsocket library.
Background
The Rsocket library (RDMAsocket API, RDMA socket application Call interface) is a protocol for providing socket interfaces over RDMA networks, is part of the library of librdmcam provided by the linux-RDMA group, and is also part of the OFED library. The main work of Rsockets is to implement semantic adaptation of all socket calls over the interface Verbs provided by RDMA in order to use the RDMA network.
When inter-process communication is performed in the same host based on the Rsocket library, in order to balance the use of the corresponding bandwidth among the processes, it is necessary to limit the speed of the egress traffic bandwidth and the ingress traffic bandwidth of the Rsocket library.
However, in the prior art, a data center generally manages its own containers through container arrangement such as kubernets (a container arrangement engine of Google open source), different containers have different bandwidth speed limiting policies, however, Kurbenetes cannot implement an effective bandwidth speed limiting policy for containers transmitted using RDMA network, while RDMA transmission is still used at the bottom layer of Rsocket, and therefore kubernets cannot implement a bandwidth speed limiting policy for Rsocket either. At present, no proper bandwidth speed limiting mechanism is available for limiting the outlet traffic bandwidth and the inlet traffic bandwidth of the Rsocket library.
Disclosure of Invention
An object of the embodiment of the present application is to provide a bandwidth speed limiting method, system, device and storage medium for an Rsocket library, which can achieve the purpose of speed limiting of an egress traffic bandwidth and an ingress traffic bandwidth of the Rsocket library.
The invention adopts the following technical scheme:
the invention provides a bandwidth speed limiting method facing an Rsocket library in a first aspect, which comprises the following steps:
establishing shared memory communication with a Helper process, and acquiring bandwidth speed-limiting information from the Helper process, wherein the bandwidth speed-limiting information comprises expected bandwidth;
sending a speed limit request to a speed limiter module, wherein the speed limit request comprises the bandwidth speed limit information, and the speed limiter module limits the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
According to a manner that can be realized in the first aspect of the present invention, the expected bandwidth includes an expected bandwidth of egress traffic, and the speed limiter module limits the bandwidth of the Rsocket library by using a token bucket algorithm according to the expected bandwidth, including: and controlling the difference value between the outlet flow bandwidth of the Rsocket library and the outlet flow expected bandwidth to be smaller than a first preset difference value threshold.
According to a manner that can be realized in the first aspect of the present invention, the expected bandwidth includes an expected bandwidth of ingress traffic, and the speed limiter module limits the bandwidth of an Rsocket library by using a token bucket algorithm according to the expected bandwidth, including: and controlling the difference value between the inlet traffic bandwidth of the Rsocket library and the inlet traffic expected bandwidth to be smaller than a second preset difference value threshold.
According to an implementation manner of the first aspect of the present invention, the bandwidth speed-limiting information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed-limiting the bandwidth of the Rsocket library by using a token bucket algorithm includes:
tokens are placed in the bucket according to the token generation time gap.
According to an implementation manner of the first aspect of the present invention, the speed limit request further includes a data cache length of a socket application, and the speed limit of the bandwidth of the Rsocket library by using the token bucket algorithm includes:
setting the number of tokens to be taken from the bucket to be the data cache length of the socket application.
According to an implementation manner of the first aspect of the present invention, the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the method further includes:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application, wherein the data transmission request comprises data priority, data cache length and expected time;
determining the bandwidth required by data processing according to the data caching length and the expected time;
and sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
According to a manner that can be realized in the first aspect of the present invention, the adaptive standard speed-limiting bandwidth is selected from a list of standard speed-limiting bandwidths as the expected bandwidth, and specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
According to another implementation manner of the first aspect of the present invention, each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and the selecting an adaptive standard speed-limiting bandwidth from the standard speed-limiting bandwidth list as the expected bandwidth specifically includes:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
According to an enabling aspect of the first aspect of the invention, the method further comprises:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application;
and when the data transmission request carries information that the marked data is important data, the step of sending the speed limit request to the speed limiter module is not executed.
The second aspect of the present invention provides a bandwidth speed limiting system for an Rsocket library, where the system includes:
the device comprises a bandwidth speed limit information acquisition module, a Helper process and a bandwidth speed limit information acquisition module, wherein the bandwidth speed limit information acquisition module is used for establishing shared memory communication with the Helper process and acquiring bandwidth speed limit information from the Helper process, and the bandwidth speed limit information comprises expected bandwidth;
the request sending module is used for sending a message speed limit request to the speed limiter module, wherein the message speed limit request comprises the bandwidth speed limit information;
and the speed limiter module is used for limiting the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
According to a manner that can be realized in the second aspect of the present invention, the expected bandwidth includes an egress traffic expected bandwidth, and the speed limiter module includes an egress traffic speed limiting unit, where the egress traffic speed limiting unit is configured to control a difference between an egress traffic bandwidth of the Rsocket repository and the egress traffic expected bandwidth to be smaller than a first preset difference threshold.
According to a manner that can be realized in the second aspect of the present invention, the expected bandwidth includes an ingress traffic expected bandwidth, and the speed limiter module includes an ingress traffic speed limiting unit, where the ingress traffic speed limiting unit is configured to control a difference between an ingress traffic bandwidth of the Rsocket repository and the ingress traffic expected bandwidth to be smaller than a second preset difference threshold.
According to an implementation manner of the second aspect of the present invention, the bandwidth speed limit information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed limiter module further includes a token bucket algorithm execution unit, where the token bucket algorithm execution unit is configured to place a token in a bucket according to the token generation time gap.
According to a manner that can be realized by the second aspect of the present invention, the message speed limit request further includes a data cache length of a socket application, and the token bucket algorithm execution unit is further configured to set the number of tokens to be taken from the bucket as the data cache length of the socket application.
According to an implementation manner of the second aspect of the present invention, the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the system further includes:
the first receiving module is used for receiving a data transmission request sent by a socket application before establishing shared memory communication with a Helper process, wherein the data transmission request comprises data priority, data cache length and expected time;
the determining module is used for determining the bandwidth required by data processing according to the data caching length and the expected time;
and the information sending module is used for sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
According to a manner that can be realized in the second aspect of the present invention, the adaptive standard speed-limiting bandwidth is selected from a list of standard speed-limiting bandwidths as the expected bandwidth, specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
According to another implementation manner of the second aspect of the present invention, each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and the selecting an adaptive standard speed-limiting bandwidth from the standard speed-limiting bandwidth list as the expected bandwidth specifically includes:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
According to an enabling manner of the second aspect of the invention, the system further comprises:
the second receiving module is used for receiving a data transmission request sent by the socket application before establishing the shared memory communication with the Helper process;
and the control module is used for controlling the request sending module not to execute the step of sending the message speed limit request to the speed limiter module when the data transmission request carries information with marked data as important data.
In an embodiment of a third aspect of the present invention, a bandwidth speed limiting device facing an Rsocket library is provided, where the device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the bandwidth speed limiting device implements a bandwidth speed limiting method facing an Rsocket library as described in any one of the above embodiments.
A fourth aspect of the present invention provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed, the method for limiting bandwidth for a Rsocket library as described in any of the above embodiments is implemented.
The invention can achieve the purpose of limiting the speed of the outlet flow bandwidth and the inlet flow bandwidth of the Rsocket library.
Drawings
Fig. 1 is a schematic flow chart of a preferred embodiment of a bandwidth speed limiting method for an Rsocket library provided in the present invention;
fig. 2 is a schematic structural diagram of a preferred embodiment of the bandwidth speed limiting system facing to the Rsocket library provided in the present invention.
Reference numerals:
the device comprises a bandwidth speed limit information acquisition module 1, a request sending module 2 and a speed limiter module 3.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
To facilitate an understanding of the embodiments of the present invention, a few elements that will be introduced in the description of the embodiments of the present invention are first introduced here.
Token bucket algorithm: is a bucket that holds tokens of a fixed capacity and adds tokens to the bucket at a fixed rate. The number of tokens held in the bucket has a maximum upper limit and is discarded or rejected after exceeding. When traffic or network requests arrive, each request gets a token, if it can be got, it is processed directly, and the token bucket deletes a token. If the acquisition is different, the request is either directly discarded or awaits in a buffer.
And (4) process: the method is a running activity of a program in a computer on a certain data set, is a basic unit for resource allocation and scheduling of a system, and is the basis of an operating system structure.
Sharing the memory: refers to a large capacity memory that can be accessed by different central processing units in a multi-processor computer system.
Fig. 1 is a schematic flow diagram illustrating a preferred embodiment of a bandwidth speed limiting method for an Rsocket library according to the present invention.
As shown in fig. 1, the method includes:
s1, establishing shared memory communication with the Helper process, and acquiring bandwidth speed-limiting information from the Helper process, wherein the bandwidth speed-limiting information comprises expected bandwidth;
s2 sending a speed limit request to a speed limiter module, wherein the speed limit request comprises the bandwidth speed limit information, and the speed limiter module limits the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
When establishing the shared memory communication with the Helper process, the following procedures can be specifically executed:
the socket application calls a socket function of the Linux system, an Rsocket library is used for intercepting the socket function, an Rsocket function in the Rsocket library is used for replacing the socket function, and the Rsocket function calls a Helper shared memory to establish a function;
the Helper shared memory establishing function sends a shared memory establishing request to a Helper process;
after receiving the shared memory establishing request, the Helper process calculates a shared memory key and returns the shared memory key to the Helper shared memory establishing function;
the Helper shared memory establishing function receives the shared memory secret key of the Helper process, and establishes a shared memory with the Helper process by using the shmget function provided by the Linux system and the received shared memory secret key.
The speed limit request is sent to the speed limiter module, and the speed limit request can be specifically executed according to the following procedures:
the socket application calls a send function of the Linux system, an Rsocket library is used for intercepting the send function, and an rsend function in the Rsocket library is adopted for replacing the send function;
and calling the speed limiter function by the rsend function, so that the speed limiter function transmits a speed limiting request to the speed limiter module.
The embodiment of the invention realizes the purpose of limiting the speed of the outlet flow bandwidth and the inlet flow bandwidth of the Rsocket library.
The method for limiting the bandwidth of the Rsocket library by the speed limiter module by adopting a token bucket algorithm according to the expected bandwidth comprises the following steps: and controlling the difference value between the outlet flow bandwidth of the Rsocket library and the outlet flow expected bandwidth to be smaller than a first preset difference value threshold.
Further, the expected bandwidth includes an expected bandwidth of an ingress traffic, and the speed limiter module limits the bandwidth of the Rsocket library by using a token bucket algorithm according to the expected bandwidth, including: and controlling the difference value between the inlet traffic bandwidth of the Rsocket library and the inlet traffic expected bandwidth to be smaller than a second preset difference value threshold.
In one embodiment, the token bucket algorithm is based on the principle of implementation of an open source rate limiter (Guava ratelimiter) in java.
In another embodiment, the bandwidth speed limit information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed limiting the bandwidth of the Rsocket library by using a token bucket algorithm includes:
tokens are placed in the bucket according to the token generation time gap.
Further, the speed limit request further includes a data cache length of the socket application, and the speed limit of the bandwidth of the Rsocket library by using the token bucket algorithm includes:
setting the number of tokens to be taken from the bucket to be the data cache length of the socket application, the length being in bytes.
The following describes a specific implementation procedure for speed limiting based on the token bucket algorithm of the open source rate limiter.
When the speed limit is carried out based on the token bucket algorithm of the open-source speed limiter, the related parameters of the token bucket need to be initialized. When the expected bandwidth contains both the ingress traffic expected bandwidth and the ingress traffic expected bandwidth, the initialization process may proceed as follows:
recording the expected bandwidth of the egress traffic as Nout, recording the expected bandwidth of the ingress traffic as Nin, wherein the unit of the expected bandwidth is Gbps, and the Helper process calculates the time gap of the egress traffic generation as Nout/1000000000, and records as T31; calculating the time gap for generating the inlet flow as Nin/1000000000, and recording as T32;
the rsocket function calls an outlet flow initialization token bucket function; the egress traffic initialization token bucket function sets the following parameters:
setting the size of a token bucket, and recording as Nmax 1; setting the number of the pre-consumption tokens as 0 and recording as Npend 1; setting the number of the remaining tokens as 0 and recording as Nleft 1; setting the number of generated tokens as 0 and recording as Ndate 1; setting the time when the secondary token is generated as 0, and marking as Tcurrent 1; setting the next production token time as 0, marking as Tnext1, and setting the delay waiting time as 0, marking as T41;
the rsocket function calls an ingress traffic initialization token bucket function, which sets the following parameters:
setting the size of a token bucket, and recording as Nmax 2; setting the number of the pre-consumption tokens as 0 and recording as Npend 2; setting the number of the remaining tokens as 0 and recording as Nleft 2; setting the number of generated tokens as 0 and recording as Ndate 2; setting the time when the secondary token is generated as 0, and marking as Tcurrent 2; the next production token time is set to be 0, denoted as Tnext2, and the delay wait time is set to be 0, denoted as T42.
On one hand, when the token bucket algorithm based on the open source speed limiter is adopted to limit the bandwidth of the outlet traffic of the socket application, the following procedures can be referred to for execution:
1.1, the speed limiter module obtains the receiving buffer length of the socket application and generates a time gap T31 from the speed limit request sent by the speed limiter function;
1.2 the speed limiter module obtains a current time stamp through a gettimeaddress function of a Linux system, and the current time stamp is marked as T11, and the number of the taken tokens is the length of a socket receiving cache and is marked as N1;
1.3 if the current timestamp T11 is greater than the next generation token time Tnext1, then the calculation generation token number Ncreate1 is (T11-Tnext 1)/T31. Ncreate1 indicates the number of tokens that should be generated between T11 and Tnext 1. 1.4, otherwise, 1.6;
1.4 remaining token number Nleft1 ═ min (Nmax1, Nleft1+ Ncreate 1); min (Nmax1, Nleft1+ Ndate 1) represents taking the lesser of Nmax1 and Nleft1+ Ndate 1;
1.5 the next generation token time Tnext1 ═ T11;
1.6 pre-consumption token number Nspend1 ═ N1-min (N1, Nleft1), min (N1, Nleft1) means taking the smaller between N1 and Nleft 1;
1.7, calculating the waiting delay time T41 ═ Nspend1 × T31;
1.8 when the secondary token generation time Tcurrent1 is Tnext 1;
1.9 next generation token time Tnext1 ═ Tnext1+ T41;
1.10 remaining token capacity Nleft1 ═ Nleft1-min (N1, Nleft 1);
1.11 if Tcurrent1-T11 equals zero, then turn 1.12; if not, then the time to wait T11-Tcurrent1 needs to be delayed, 1.12 is turned;
1.12 returns success;
1.13 end.
On the other hand, when the token bucket algorithm based on the open source rate limiter is adopted to limit the bandwidth of the inlet traffic of the socket application, the following procedures can be referred to for execution:
and 2.1, the speed limiter module acquires the sending buffer length of the socket application and the generation time gap T32 from the speed limit request sent by the speed limiter function.
2.2 the speed limiter module obtains the current time stamp through gettimeaddress function of Linux system, and records it as T12, and the token number is the length of the socket sending buffer, and records it as N2;
2.3 if the current timestamp T12 is greater than the next generation token time Tnext2, then calculate the generation token number Ncreate2 ═ T12-Tnext2)/T32, Ncreate2 represents the number of tokens that should be generated between T12 and Tnext2, turn 2.4, otherwise turn 2.6;
2.4 remaining token number Nleft2 ═ min (Nmax2, Nleft2+ Ncreate2), min (Nmax2, Nleft2+ Ncreate2) means taking the smaller of Nmax2 and Nleft2+ Ncreate 2;
2.5 the next generation token time Tnext2 ═ T12;
2.6 pre-consumption token number Nspend2 — N2-min (N2, Nleft2), min (N2, Nleft2) indicates taking the smaller of N2 and Nleft 2;
2.7 calculating the waiting delay time T42 ═ Nspend2 × T32;
2.8 when the secondary token generation time Tcurrent2 is Tnext 2;
2.9 next generation token time Tnext2 ═ Tnext2+ T42;
2.10 remaining token capacity Nleft2 ═ Nleft2-min (N2, Nleft 2);
2.11 if Tcurrent2-T12 equals zero, then go to 2.12; if not, then the wait T12-Tcurrent2 is delayed, 2.12.
2.12 returns success.
2.13 end.
Further, obtaining the expected bandwidth according to a configuration file built in the Helper process, wherein the configuration file stores the information of the expected bandwidth.
The embodiment of the invention limits the flow bandwidth of the Rsocket library through the fixed expected bandwidth.
In another embodiment, the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the method further includes:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application, wherein the data transmission request comprises data priority, data cache length and expected time;
determining the bandwidth required by data processing according to the data caching length and the expected time;
and sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
And the value of the bandwidth required by the data processing is the value obtained by dividing the data cache length by the expected time.
For two adjacent data priorities, the minimum standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponding to the larger data priority is not less than the maximum standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponding to the smaller data priority.
In one embodiment, the adaptive standard speed-limit bandwidth is selected from a list of standard speed-limit bandwidths as the expected bandwidth, specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
In another embodiment, each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and the adaptive standard speed-limiting bandwidth is selected from the standard speed-limiting bandwidth list as the expected bandwidth, specifically:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
In the above embodiment of the present invention, the standard speed-limiting bandwidth is selected according to the data priority, and then the adaptive standard speed-limiting bandwidth is selected from the standard speed-limiting bandwidth list according to the bandwidth required by data processing as the expected bandwidth, so that the adaptive expected bandwidth can be selected according to the actual data traffic and the data priority to perform more accurate speed limitation on data transmission.
Further, the method further comprises:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application;
and when the data transmission request carries information that the marked data is important data, the step of sending the speed limit request to the speed limiter module is not executed.
The embodiment does not limit the speed when transmitting the important data, and avoids discarding the important data due to the speed limit.
The method of the invention can be suitable for the inter-process communication process in the same host, and also can be applied to the inter-process communication process between the hosts.
The embodiment of the second aspect of the invention provides a bandwidth speed limiting system facing an Rsocket library.
Fig. 2 is a schematic structural diagram of an preferred embodiment of the bandwidth speed limiting system for an Rsocket library provided in the present invention, where the system is capable of implementing all processes of the bandwidth speed limiting method for an Rsocket library described in any of the above embodiments.
As shown in fig. 2, the system includes:
the bandwidth speed limit information acquisition module 1 is used for establishing shared memory communication with a Helper process and acquiring bandwidth speed limit information from the Helper process, wherein the bandwidth speed limit information comprises expected bandwidth;
the request sending module 2 is used for sending a message speed limit request to the speed limiter module, wherein the message speed limit request comprises the bandwidth speed limit information;
and the speed limiter module 3 is used for limiting the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
According to a manner that can be realized according to the embodiment of the second aspect of the present invention, the expected bandwidth includes an egress traffic expected bandwidth, the speed limiter module includes an egress traffic speed limiting unit, and the egress traffic speed limiting unit is configured to control a difference value between an egress traffic bandwidth of the Rsocket library and the egress traffic expected bandwidth to be smaller than a first preset difference threshold.
According to a manner that can be realized according to the embodiment of the second aspect of the present invention, the expected bandwidth includes an ingress traffic expected bandwidth, and the speed limiter module includes an ingress traffic speed limiting unit, where the ingress traffic speed limiting unit is configured to control a difference between an ingress traffic bandwidth of the Rsocket library and the ingress traffic expected bandwidth to be smaller than a second preset difference threshold.
Further, the bandwidth speed limit information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed limiter module further includes a token bucket algorithm execution unit, where the token bucket algorithm execution unit is configured to place a token in a bucket according to the token generation time gap.
Further, the message speed limit request further includes a data cache length of a socket application, and the token bucket algorithm execution unit is further configured to set the number of tokens to be taken from the bucket to the data cache length of the socket application.
Further, the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the system further includes:
the first receiving module is used for receiving a data transmission request sent by a socket application before establishing shared memory communication with a Helper process, wherein the data transmission request comprises data priority, data cache length and expected time;
the determining module is used for determining the bandwidth required by data processing according to the data caching length and the expected time;
and the information sending module is used for sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
According to a manner that can be realized in the second aspect of the present invention, the adaptive standard speed-limiting bandwidth is selected from a list of standard speed-limiting bandwidths as the expected bandwidth, specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
In another implementation manner, each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and the selecting an adaptive standard speed-limiting bandwidth from the standard speed-limiting bandwidth list as the expected bandwidth specifically includes:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
According to an enabling manner of the second aspect of the invention, the system further comprises:
the second receiving module is used for receiving a data transmission request sent by the socket application before establishing the shared memory communication with the Helper process;
and the control module is used for controlling the request sending module not to execute the step of sending the message speed limit request to the speed limiter module when the data transmission request carries information with marked data as important data.
The functions and implementation manners of the modules in the embodiment of the system of the present invention are the same as those in the embodiment of the bandwidth speed limiting method for the Rsocket library, and specific analysis may refer to the embodiment of the bandwidth speed limiting method for the Rsocket library, and are not described herein again to avoid repetition.
The invention also provides a bandwidth speed limiting device facing the Rsocket library, which comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to realize the bandwidth speed limiting method facing the Rsocket library according to any one of the embodiments.
The invention also provides a computer readable storage medium, wherein a computer program is stored in the computer readable storage medium, and when the computer program is executed, the bandwidth speed limiting method facing the Rsocket library is realized.
The Processor may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. The general processor can be a microprocessor or the processor can also be any conventional processor and the like, the processor is a control center of the bandwidth speed limiting device facing the Rsocket library, and various interfaces and lines are utilized to connect all parts of the whole bandwidth speed limiting device facing the Rsocket library.
The memory can be used for storing the computer program and/or the module, and the processor realizes various functions of the bandwidth speed limiting device facing the Rsocket library by running or executing the computer program and/or the module stored in the memory and calling data stored in the memory. The memory may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
If the modules/units integrated with the bandwidth speed limiting device facing the Rsocket library are realized in the form of software functional units and sold or used as independent products, the modules/units can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
The foregoing is a preferred embodiment of the present application, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present application, and these modifications and decorations are also regarded as the protection scope of the present application.
Claims (20)
1. The bandwidth speed limiting method facing the Rsocket library is characterized by comprising the following steps:
establishing shared memory communication with a Helper process, and acquiring bandwidth speed-limiting information from the Helper process, wherein the bandwidth speed-limiting information comprises expected bandwidth;
sending a speed limit request to a speed limiter module, wherein the speed limit request comprises the bandwidth speed limit information, and the speed limiter module limits the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
2. The method for limiting the bandwidth of the Rsocket library according to claim 1, wherein the expected bandwidth comprises an expected bandwidth of an egress traffic, and the speed limiter module limits the bandwidth of the Rsocket library according to the expected bandwidth by using a token bucket algorithm, and the method comprises the following steps: and controlling the difference value between the outlet flow bandwidth of the Rsocket library and the outlet flow expected bandwidth to be smaller than a first preset difference value threshold.
3. The method for limiting the bandwidth of the Rsocket library according to claim 1, wherein the expected bandwidth comprises an expected bandwidth of an inlet flow, and the speed limiter module limits the bandwidth of the Rsocket library according to the expected bandwidth by using a token bucket algorithm, and the method comprises the following steps: and controlling the difference value between the inlet traffic bandwidth of the Rsocket library and the inlet traffic expected bandwidth to be smaller than a second preset difference value threshold.
4. The method for limiting the bandwidth speed facing the Rsocket library according to claim 1, wherein the bandwidth speed limiting information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed limiting of the bandwidth of the Rsocket library by using a token bucket algorithm includes:
tokens are placed in the bucket according to the token generation time gap.
5. The method for limiting the bandwidth speed facing the Rsocket library according to claim 4, wherein the speed-limiting request further includes a data cache length of a socket application, and the speed-limiting the bandwidth of the Rsocket library by using a token bucket algorithm includes:
setting the number of tokens to be taken from the bucket to be the data cache length of the socket application.
6. The Rsocket library-oriented bandwidth speed limiting method according to claim 1, wherein the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the method further comprises:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application, wherein the data transmission request comprises data priority, data cache length and expected time;
determining the bandwidth required by data processing according to the data caching length and the expected time;
and sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
7. The bandwidth speed-limiting method for the Rsocket library as claimed in claim 6, wherein the adaptive standard speed-limiting bandwidth is selected from a standard speed-limiting bandwidth list as the expected bandwidth, and specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
8. The bandwidth speed-limiting method for the Rsocket library according to claim 6, wherein each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and an adaptive standard speed-limiting bandwidth is selected from the standard speed-limiting bandwidth list as the expected bandwidth, specifically:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
9. The bandwidth speed limiting method facing the Rsocket library according to claim 1, wherein the method further comprises:
before establishing shared memory communication with a Helper process, receiving a data transmission request sent by a socket application;
and when the data transmission request carries information that the marked data is important data, the step of sending the speed limit request to the speed limiter module is not executed.
10. The bandwidth speed-limiting system facing the Rsocket library is characterized by comprising the following components:
the device comprises a bandwidth speed limit information acquisition module, a Helper process and a bandwidth speed limit information acquisition module, wherein the bandwidth speed limit information acquisition module is used for establishing shared memory communication with the Helper process and acquiring bandwidth speed limit information from the Helper process, and the bandwidth speed limit information comprises expected bandwidth;
the request sending module is used for sending a message speed limit request to the speed limiter module, wherein the message speed limit request comprises the bandwidth speed limit information;
and the speed limiter module is used for limiting the bandwidth of the Rsocket library by adopting a token bucket algorithm according to the expected bandwidth.
11. The bandwidth speed-limiting system facing the Rsocket library of claim 10, wherein the expected bandwidth comprises an outlet traffic expected bandwidth, the speed limiter module comprises an outlet traffic speed-limiting unit, and the outlet traffic speed-limiting unit is configured to control a difference value between the outlet traffic bandwidth of the Rsocket library and the outlet traffic expected bandwidth to be smaller than a first preset difference threshold.
12. The bandwidth speed-limiting system facing the Rsocket library of claim 10, wherein the expected bandwidth comprises an inlet traffic expected bandwidth, the speed limiter module comprises an inlet traffic speed-limiting unit, and the inlet traffic speed-limiting unit is configured to control a difference value between an inlet traffic bandwidth of the Rsocket library and the inlet traffic expected bandwidth to be smaller than a second preset difference threshold.
13. The Rsocket-library-oriented bandwidth speed limiting system according to claim 10, wherein the bandwidth speed limiting information further includes a token generation time gap calculated by the Helper process according to the expected bandwidth, and the speed limiter module further includes a token bucket algorithm execution unit, and the token bucket algorithm execution unit is configured to place a token in a bucket according to the token generation time gap.
14. The Rsocket library-oriented bandwidth rate limiting system of claim 13, wherein the message rate limit request further comprises a data cache length of a socket application, and the token bucket algorithm execution unit is further configured to set the number of tokens to be taken from the bucket to the data cache length of the socket application.
15. The Rsocket library-oriented bandwidth speed limiting system according to claim 10, wherein the Helper process stores a standard speed-limiting bandwidth list corresponding to each preset data priority, and the system further comprises:
the first receiving module is used for receiving a data transmission request sent by a socket application before establishing shared memory communication with a Helper process, wherein the data transmission request comprises data priority, data cache length and expected time;
the determining module is used for determining the bandwidth required by data processing according to the data caching length and the expected time;
and the information sending module is used for sending the bandwidth and the data priority required by the data processing to the Helper process, so that the Helper process selects the adaptive standard speed-limiting bandwidth as the expected bandwidth according to the standard speed-limiting bandwidth list corresponding to the data priority.
16. The bandwidth speed-limiting system facing the Rsocket library according to claim 15, wherein an adapted standard speed-limiting bandwidth is selected from a standard speed-limiting bandwidth list as the expected bandwidth, specifically:
judging whether the bandwidth required by the data processing is less than or equal to the minimum standard speed-limiting bandwidth in a standard speed-limiting bandwidth list;
if so, selecting the minimum standard speed limit bandwidth as the expected bandwidth;
if not, selecting the maximum standard speed limit bandwidth as the expected bandwidth from the standard speed limit bandwidths smaller than the bandwidth required by the data processing in the standard speed limit bandwidth list.
17. The Rsocket-library-oriented bandwidth speed-limiting system according to claim 15, wherein each standard speed-limiting bandwidth in the standard speed-limiting bandwidth list corresponds to an adaptive bandwidth range, and an adaptive standard speed-limiting bandwidth is selected from the standard speed-limiting bandwidth list as the expected bandwidth, specifically:
and when the bandwidth required by the data processing is in an adaptive bandwidth range corresponding to a standard speed limit bandwidth, taking the standard speed limit bandwidth as the expected bandwidth.
18. The Rsocket library-oriented bandwidth speed limiting system of claim 10, wherein the system further comprises:
the second receiving module is used for receiving a data transmission request sent by the socket application before establishing the shared memory communication with the Helper process;
and the control module is used for controlling the request sending module not to execute the step of sending the message speed limit request to the speed limiter module when the data transmission request carries information with marked data as important data.
19. The bandwidth speed limiting device facing the Rsocket library is characterized by comprising a processor, a memory and a computer program which is stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to realize the bandwidth speed limiting method facing the Rsocket library as claimed in any one of claims 1 to 9.
20. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method for limiting bandwidth according to any one of claims 1 to 9 is implemented.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110603061.7A CN113556245A (en) | 2021-05-31 | 2021-05-31 | Bandwidth speed limiting method, system, device and storage medium for Rsocket library |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110603061.7A CN113556245A (en) | 2021-05-31 | 2021-05-31 | Bandwidth speed limiting method, system, device and storage medium for Rsocket library |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113556245A true CN113556245A (en) | 2021-10-26 |
Family
ID=78101920
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110603061.7A Withdrawn CN113556245A (en) | 2021-05-31 | 2021-05-31 | Bandwidth speed limiting method, system, device and storage medium for Rsocket library |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113556245A (en) |
-
2021
- 2021-05-31 CN CN202110603061.7A patent/CN113556245A/en not_active Withdrawn
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110958281B (en) | Data transmission method and communication device based on Internet of things | |
| WO2020052605A1 (en) | Network slice selection method and device | |
| EP3720052B1 (en) | Service identification method and device, and network device | |
| KR101240143B1 (en) | Non-blocking admission control | |
| WO2021093501A1 (en) | Task processing method and network device | |
| US11258717B2 (en) | Method for sending service packet, network device, and system | |
| CN106571978B (en) | Data packet capturing method and device | |
| JP7487316B2 (en) | Service level configuration method and apparatus | |
| US11010165B2 (en) | Buffer allocation with memory-based configuration | |
| CN111459690A (en) | Data transmitting and receiving control method and device, mobile terminal and storage medium | |
| JP7462757B2 (en) | Network security protection method and protection device | |
| CN110830388B (en) | Data scheduling method, device, network equipment and computer storage medium | |
| US20150296406A1 (en) | Traffic distribution method, device, and system | |
| CN112948133A (en) | Resource scheduling management method and device and cloud platform | |
| CN114466226B (en) | Bandwidth duration duty cycle determination method, device, equipment and computer readable medium | |
| US11316916B2 (en) | Packet processing method, related device, and computer storage medium | |
| CN112306693B (en) | Data packet processing method and device | |
| US20180081746A1 (en) | Application message processing system, method, and application device | |
| CN114697387A (en) | Data packet transmission method, device and storage medium | |
| US10887236B2 (en) | Method, network interface card, and computer program product for load balance | |
| CN113556245A (en) | Bandwidth speed limiting method, system, device and storage medium for Rsocket library | |
| CN107124353B (en) | Message processing method and device, computer device and storage medium | |
| CN116244231A (en) | Data transmission method, device and system, electronic equipment and storage medium | |
| CN113453285B (en) | Resource adjusting method, device and storage medium | |
| CN113626221B (en) | Message enqueuing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20211026 |
|
| WW01 | Invention patent application withdrawn after publication |