WO2022257425A1 - 传输速率控制方法、装置、电子设备和可读存储介质 - Google Patents

传输速率控制方法、装置、电子设备和可读存储介质 Download PDF

Info

Publication number
WO2022257425A1
WO2022257425A1 PCT/CN2021/140863 CN2021140863W WO2022257425A1 WO 2022257425 A1 WO2022257425 A1 WO 2022257425A1 CN 2021140863 W CN2021140863 W CN 2021140863W WO 2022257425 A1 WO2022257425 A1 WO 2022257425A1
Authority
WO
WIPO (PCT)
Prior art keywords
rate
burst
transmission rate
read
bucket
Prior art date
Application number
PCT/CN2021/140863
Other languages
English (en)
French (fr)
Inventor
万文聪
刘宽
段江南
黄景平
Original Assignee
天翼云科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 天翼云科技有限公司 filed Critical 天翼云科技有限公司
Publication of WO2022257425A1 publication Critical patent/WO2022257425A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/215—Flow control; Congestion control using token-bucket
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5083—Techniques for rebalancing the load in a distributed system
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/08—Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
    • H04L43/0876—Network utilisation, e.g. volume of load or congestion level
    • H04L43/0894—Packet rate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the technical field of distributed storage, and in particular to a transmission rate control method, device, electronic device and computer-readable storage medium of a distributed storage system.
  • Ceph distributed File System
  • Ceph restricts the read and write operations of block storage through the token bucket algorithm to ensure Qos (Quality of Service, service quality).
  • Qos Quality of Service, service quality
  • Ceph has a certain burst capability (burst capability refers to allowing the read and write speed to exceed the limit within a certain period of time to deal with sudden arrivals.
  • the purpose of the present application is to provide a transmission rate control method, device, electronic device and computer-readable storage medium of a distributed storage system, at least to a certain extent, to overcome the long alarm correlation analysis period and slow alarm output in related technologies. computationally intensive problem.
  • a transmission rate control method of a distributed storage system including: configuring a first number of tokens in a token bucket, and setting a leaky bucket at the entrance of the token bucket, and The leaky bucket is configured with the second number of tokens; in response to the read-write request initiated by the client, determining the request transmission rate corresponding to the read-write operation; when it is detected that the request transmission rate is greater than the When the burst rate of the token bucket is determined, the burst rate is determined as the actual transmission rate of the read and write operations, and the leaky bucket is triggered to flow tokens into the token bucket, wherein the burst The rate is less than or equal to the first amount.
  • the transmission rate control method before determining the requested transmission rate corresponding to the read and write operation in response to the read and write request initiated by the client, the transmission rate control method further includes: according to the burst rate And the preset burst duration configures the second number, the second number is greater than the first number; when it is detected that the request transmission rate is greater than the burst rate of the token bucket, the The burst rate is determined as the actual transmission rate of the read and write operations, and triggering the leaky bucket to flow tokens into the token bucket also includes: triggering the leaky bucket to flow tokens into the token bucket based on the burst rate The token bucket transmits the token to execute the read/write operation based on the burst rate, wherein the duration for executing the read/write operation based on the burst rate is less than or equal to the burst duration.
  • the triggering the leaky bucket to transmit the token to the token bucket based on the burst rate further includes: transmitting the token into the leaky bucket based on a limited rate cards, wherein the limit rate is less than the burst rate.
  • it further includes: when it is detected that the time for performing the read and write operations based on the actual transmission rate reaches the burst duration, the read and write operations are not completed, and setting the limit rate to Switching to the actual transmission rate of the read and write operations, so as to continue to execute the read and write operations based on the limited rate.
  • the configuring the second number according to the burst rate and the preset burst duration specifically includes: calculating a rate difference between the burst rate and the limited rate ; calculating the product of the rate difference and the burst duration as the intermediate quantity; calculating the difference between the intermediate quantity and the first quantity as the second quantity.
  • it further includes: adjusting the second quantity, so as to adjust the burst duration based on the adjusted second quantity; and/or adjusting the burst duration according to the demand parameter of the read and write request.
  • the burst duration is configured to configure the second quantity based on the adjusted burst duration.
  • the method further includes: when detecting that the requested transmission rate is less than or equal to the burst rate, setting the requested transmission rate as the actual transmission rate of the read and write operations.
  • a transmission rate control device for a distributed storage system, including: a configuration module configured to configure a first number of tokens in a token bucket, and A leaky bucket is set at the leaky bucket, and the second number of tokens are configured in the leaky bucket; a determination module is configured to respond to the read-write request initiated by the client, and determine the request transmission rate corresponding to the read-write operation; A control module, configured to determine the burst rate as the actual transmission rate of the read and write operations when detecting that the request transmission rate is greater than the burst rate of the token bucket, and trigger the leaky bucket to Tokens flow into the token bucket, wherein the burst rate is less than or equal to the first quantity.
  • an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above by executing the executable instructions A transmission rate control method for a distributed storage system.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for controlling the transmission rate of any one of the above-mentioned distributed storage systems is implemented.
  • the transmission rate control scheme of the distributed storage system provided by the embodiment of this application, by connecting the leaky bucket and the token bucket in series, pre-configures the first number of tokens in the token bucket, and limits the burst of the token bucket rate, so that when receiving a read-write request from the client, based on the relationship between the request transfer rate and the burst rate, the actual transfer rate of the read-write request can be determined. Specifically, when the request transfer rate is detected to be greater than the token bucket When the burst rate is used, the burst rate is directly used as the actual transmission rate, and the read and write operations are performed, so that while ensuring the read and write efficiency, the impact of the read and write operations of the client on the read and write capabilities obtained by other clients can be reduced .
  • FIG. 1 shows a flowchart of a transmission rate control method of a distributed storage system in an embodiment of the present application
  • FIG. 2 shows a flow chart of another transmission rate control method of a distributed storage system in an embodiment of the present application
  • Fig. 3 shows the flow chart of another kind of transmission rate control method of distributed storage system in the embodiment of the present application
  • FIG. 4 shows a flowchart of another transmission rate control method of a distributed storage system in an embodiment of the present application
  • FIG. 5 shows a flowchart of another transmission rate control method of a distributed storage system in an embodiment of the present application
  • FIG. 6 shows a flowchart of another transmission rate control method of a distributed storage system in an embodiment of the present application
  • FIG. 7 shows a schematic diagram of a transmission rate control device of a distributed storage system in an embodiment of the present application
  • FIG. 8 shows a schematic diagram of an electronic device in an embodiment of the present application.
  • Fig. 9 shows a schematic diagram of a computer-readable storage medium in an embodiment of the present application.
  • Example embodiments will now be described more fully with reference to the accompanying drawings.
  • Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the concepts of example embodiments to those skilled in the art.
  • the described features, structures or properties may be combined in any suitable manner in one or more embodiments.
  • numerous specific details are provided in order to give a thorough understanding of embodiments of the present application.
  • those skilled in the art will appreciate that the technical solutions of the present application can be practiced without one or more of the specified details, or other components, devices, steps, etc. can be adopted.
  • well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the application.
  • the distributed storage system Ceph includes a server and multiple clients. Multiple clients communicate with the server.
  • the client accesses the cluster through the interface.
  • the access includes two types of interfaces: the overall access to the cluster and the access to objects.
  • the interface passes through The network realizes the access to the Ceph cluster, so as to realize the virtual machine startup, file copy, file download and other operations of the client.
  • the server executes the transmission rate control method of the distributed storage system, including the following steps:
  • Step S102 configuring a first number of tokens in the token bucket, and setting a leaky bucket at the entrance of the token bucket, and configuring a second number of tokens in the leaky bucket.
  • each token may represent a byte.
  • the leaky bucket algorithm means that the traffic first enters the leaky bucket, and the leaky bucket sends out traffic at a certain speed. When the receiving speed of the traffic is too high, it will overflow directly, so that the leaky bucket algorithm can forcibly limit the data transmission rate.
  • the principle of the token bucket algorithm is that the distributed storage system puts tokens into the bucket at a constant speed, and if the request needs to be processed, it needs to obtain a token from the bucket first, when there is no token available in the bucket , the service is denied.
  • Each token in the token bucket represents a byte. If the token exists in the token bucket, the traffic is allowed; if there is no token in the token bucket, the traffic is not allowed. Therefore, if the burst threshold is properly configured and there are enough tokens in the token bucket, traffic can be sent at the peak rate.
  • the token bucket algorithm allows burst transmission. By combining the leaky bucket algorithm with the token bucket algorithm and setting the first number and the second number reasonably, a controllable duration of burst transfers.
  • the number of leaky buckets may be one or multiple, and the number of token buckets may be one or multiple.
  • Step S104 in response to the read-write request initiated by the client, determine the requested transmission rate corresponding to the read-write operation.
  • the requested transmission rate of the read and write request can also be obtained to determine whether the client can read and write based on the requested transmission rate based on the current read and write capabilities. Action request.
  • Step S106 when it is detected that the request transmission rate is greater than the burst rate of the token bucket, determine the burst rate as the actual transmission rate of the read and write operations, and trigger the leaky bucket to flow tokens into the token bucket, wherein the burst The rate is less than or equal to the first amount.
  • the burst rate of the token bucket refers to the rate at which tokens are transferred from the leaky bucket to the token bucket, that is, the rate at which tokens leak out of the token bucket.
  • the burst rate is the same as the first number to achieve a maximum burst rate.
  • the first number of tokens are pre-configured in the token bucket, and the burst rate of the token bucket is limited, so that the read and write requests of the client can be received
  • determine the actual transmission rate of the read and write requests based on the relationship between the requested transmission rate and the burst rate. Specifically, when it is detected that the requested transmission rate is greater than the burst rate of the token bucket, the burst rate is directly used as The actual transmission rate is used to perform read and write operations, so as to ensure the read and write efficiency while reducing the impact of the client's read and write operations on the read and write capabilities obtained by other clients.
  • this method can be widely applied to various systems that require burst control, such as allowing cloud computers and cloud servers to continuously execute IO at a burst rate in scenarios such as startup, file copy, and file download. Operation to ensure the efficiency of reading and writing, which is conducive to improving the user experience.
  • the server executes the transmission rate control method of the distributed storage system, including the following steps:
  • Step S202 configure the first number of tokens in the token bucket, and set a leaky bucket at the entrance of the token bucket, configure the second number of tokens in the leaky bucket, according to the burst rate and the preset burst The duration configures the second quantity, and the second quantity is greater than the first quantity.
  • the capacity of the leaky bucket is set to be much larger than the capacity of the token bucket, and a certain amount of Tokens to effectively extend the burst duration.
  • the burst duration may be determined first, and then the second quantity may be determined based on the burst duration, thereby realizing controllable configuration of the burst duration.
  • Step S204 in response to the read-write request initiated by the client, determine the requested transmission rate corresponding to the read-write operation.
  • Step S206 when it is detected that the request transmission rate is greater than the burst rate of the token bucket, the burst rate is determined as the actual transmission rate of the read and write operations, and the leaky bucket is triggered to transmit tokens to the token bucket based on the burst rate, based on The read and write operations are performed at the burst rate, wherein the duration of the read and write operations based on the burst rate is less than or equal to the burst duration.
  • the duration of the read and write operations performed based on the burst rate may be further limited according to the burst duration, so as to ensure the controllability of the duration of the read and write operations performed at the burst rate.
  • the duration for performing read and write operations based on the burst rate is equal to the burst duration.
  • the first number of tokens is pre-configured in the token bucket, and the second number of tokens is configured in the leaky bucket.
  • the second number can be based on the required Configure the burst duration.
  • the server executes the transmission rate control method of the distributed storage system, including the following steps:
  • Step S302 configuring a first number of tokens in the token bucket, and setting a leaky bucket at the entrance of the token bucket, configuring a second number of tokens in the leaky bucket, according to the burst rate and the preset burst The duration configures the second quantity, and the second quantity is greater than the first quantity.
  • Step S304 in response to the read-write request initiated by the client, determine the requested transmission rate corresponding to the read-write operation.
  • Step S306 when it is detected that the requested transmission rate is greater than the burst rate, determine the burst rate as the actual transmission rate of the read and write operations, trigger the leaky bucket to transmit tokens to the token bucket based on the burst rate, and execute based on the burst rate read and write operations. as well as
  • Step S308 transmitting tokens into the leaky bucket based on a limited rate, wherein the limited rate is less than the burst rate, and the duration for performing read and write operations based on the burst rate is less than or equal to the burst duration.
  • tokens are derived from the leaky bucket based on the burst rate
  • tokens are imported into the leaky bucket based on a limited rate.
  • it can meet the requirement of extending the burst duration; After the tokens in the bucket and the tokens in the token bucket are used up, read and write operations can continue to be performed based on the limited rate as the actual transmission rate.
  • Step S310 when it is detected that the time for performing read and write operations based on the actual transmission rate reaches the burst duration, the read and write operations are not completed, and the limited rate is switched to the actual transmission rate of the read and write operations, so as to continue to perform read and write operations based on the limited rate .
  • a specific implementation of configuring the second number according to the burst rate and the preset burst duration includes:
  • Step S402 calculating the rate difference between the burst rate and the limited rate.
  • Step S404 calculating the product of the rate difference and the burst duration as an intermediate quantity.
  • Step S406 calculating the difference between the intermediate amount and the first amount as the second amount.
  • the second number is configured through the burst rate and the limited rate, where the first number is the capacity of the token bucket, the second number is the capacity of the leaky bucket, and the maximum accumulated token The number is the sum of the capacity of the token bucket and the capacity of the leaky bucket.
  • Tokens consumed in a continuous burst maximum accumulative number of tokens + newly generated tokens, as shown in formula (1), and the corresponding second quantity is shown in formula (2).
  • the client when tokens are accumulated to fill up the large leaky bucket and the small token bucket, the client can execute IO requests at a burst rate within a set time range.
  • it also includes: adjusting the second quantity, so as to adjust the burst duration based on the adjusted second quantity; and/or adjusting the burst duration according to the demand parameters of the read and write requests, so as to adjust the burst duration based on the adjusted The burst duration configures the second quantity.
  • the capacity of the leaky bucket is set to be adjustable, so when the burst duration needs to be adjusted, it can be realized by adjusting the capacity of the leaky bucket, that is, the second number.
  • the demand parameters of the read and write requests such as the required time for starting up the cloud computer, the required time for downloading the file, and the required time for uploading the file, etc., and determine the corresponding burst duration based on these demand parameters.
  • the burst duration is further configured with the second quantity, so as to realize adaptation and adjustment of read and write capabilities of different read and write operations.
  • the method further includes: when detecting that the requested transmission rate is less than or equal to the burst rate, converting the requested transmission rate to the actual transmission rate of the read and write operations.
  • a large leaky bucket is added to the original solution, and tokens in the leaky bucket leak to the token bucket at a burst rate, and continuous burst capability is obtained by adjusting the capacity of the leaky bucket. Specifically, when the IO request speed is lower than the token generation speed, the tokens will continue to accumulate. The larger the capacity of the bucket, the more tokens can be saved, and the longer the sustainable burst time will be.
  • the IO is limited to realize the transmission rate control of the distributed storage system, including:
  • Step S502 setting the limited rate L, the burst rate B and the burst duration S.
  • Step S504 put tokens into the leaky bucket according to the limited rate L.
  • Step S506 put tokens into the token bucket according to the burst rate B.
  • Step S508 read and write operations.
  • the client when the client initiates an IO request, it first judges whether it can apply for enough tokens from the small token bucket. If it can, it consumes the corresponding number of tokens in the bucket and releases it. to enough tokens.
  • step S510 the token application is successful.
  • Step S512 waiting for blocking operation.
  • Step S514, continue to execute the operation.
  • IO operations can always be performed at a burst rate, with continuous burst Capacity, large leaky bucket plus small token bucket, the total number of tokens that can be stored is the burst duration * (burst rate - limited rate), plus tokens replenished at a limited rate per second, when the bucket is full , IO operations can be continuously performed at a burst rate within the burst duration range, and the burst duration is controllable.
  • the transmission rate control method of the distributed storage system includes:
  • Step S602 configuring a first number of tokens in the token bucket, and setting a leaky bucket at the entrance of the token bucket, and configuring a second number of tokens in the leaky bucket, the second number being greater than the first number.
  • Step S604 in response to the read and write request initiated by the client, detecting the relationship between the requested transmission rate and the burst rate corresponding to the read and write operations.
  • Step S606 when it is detected that the requested transmission rate is greater than the burst rate of the token bucket, determine the burst rate as the actual transmission rate of the read and write operations.
  • Step S608 trigger the leaky bucket to transmit tokens to the token bucket based on the burst rate, and transmit tokens to the leaky bucket based on the limited rate.
  • Step S610 when it is detected that the time for performing read and write operations based on the actual transmission rate reaches the burst duration, the read and write operations are not completed, and the limited rate is switched to the actual transmission rate of the read and write operations, so as to continue to perform read and write operations based on the limited rate .
  • Step S612 when it is detected that the requested transmission rate is less than or equal to the burst rate, the requested transmission rate is set to the actual transmission rate of the read and write operations.
  • the capacity of the small token bucket is fixed at the burst rate value, tokens flow into the large leaky bucket at a limited rate, and the large leaky bucket Tokens in the flow to the small token bucket at a burst rate.
  • the client can always perform read and write requests at a burst rate. If the request rate is greater than the token replenishment rate, when the tokens in both the large leaky bucket and the small token bucket are used up After that, the client's IO request execution rate is consistent with the token replenishment rate, and the burst duration is controlled by adjusting the capacity of the large leaky bucket.
  • the transmission rate control device 700 of the distributed storage system according to this embodiment of the present disclosure will be described below with reference to FIG. 7 .
  • the transmission rate control device 700 of the distributed storage system shown in FIG. 7 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.
  • the transmission rate control device 700 of the distributed storage system is expressed in the form of a hardware module.
  • the components of the transmission rate control device 700 of the distributed storage system may include but not limited to: a configuration module 702 configured to configure a first number of tokens in the token bucket, and set a leaky bucket at the entrance of the token bucket.
  • the leaky bucket is configured with a second number of tokens; the determination module 704 is used to determine the request transmission rate corresponding to the read and write operation in response to the read and write request initiated by the client; the control module 706 is used to detect that the request transmission rate is greater than For the burst rate, the burst rate is determined as the actual transmission rate of the read and write operations, and the leaky bucket is triggered to flow tokens into the token bucket, wherein the burst rate is less than or equal to the first quantity.
  • the configuration module 702 is also used to: configure the second number according to the burst rate and the preset burst duration, the second number is greater than the first number; the control module 706 is also used to: trigger the leaky bucket The token is transmitted to the token bucket based on the burst rate, and the read and write operations are performed based on the burst rate, wherein the time length for performing the read and write operations based on the burst rate is less than or equal to the burst duration.
  • control module 706 is further configured to: transmit tokens into the leaky bucket based on a limited rate, wherein the limited rate is smaller than the burst rate.
  • a switching module 708, which is used to switch the limited rate to read and write when it is detected that the time for performing read and write operations based on the actual transmission rate reaches the burst duration, and the read and write operations are not completed.
  • the actual transfer rate of the operation to continue reading and writing operations based on the limited rate.
  • the configuration module 702 is also used to: calculate the rate difference between the burst rate and the limited rate; calculate the product of the rate difference and the burst duration as an intermediate quantity; calculate the intermediate quantity and the second The difference between the first quantities, as the second quantity.
  • an adjustment module 710 configured to adjust the second quantity, so as to adjust the burst duration based on the adjusted second quantity; and/or adjust the burst duration according to the demand parameters of the read and write requests to configure the second number based on the adjusted burst duration.
  • control module 706 is further configured to: when detecting that the requested transmission rate is less than or equal to the burst rate, set the requested transmission rate to the actual transmission rate of the read and write operations.
  • FIG. 8 An electronic device 800 according to this embodiment of the present disclosure is described below with reference to FIG. 8 .
  • the electronic device 800 shown in FIG. 8 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.
  • electronic device 800 takes the form of a general-purpose computing device.
  • Components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).
  • the storage unit stores program codes, and the program codes can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of this specification.
  • the processing unit 810 may execute steps S202, S204, and S206 as shown in FIG. 1, and other steps defined in the transmission rate control method for a distributed storage system of the present application.
  • the storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 8201 and/or a cache storage unit 8202 , and may further include a read-only storage unit (ROM) 8203 .
  • RAM random access storage unit
  • ROM read-only storage unit
  • Storage unit 820 may also include programs/utilities 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, Implementations of networked environments may be included in each or some combination of these examples.
  • Bus 830 may represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local area using any of a variety of bus structures. bus.
  • Electronic device 800 may also communicate with one or more external devices 870 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device, and/or communicate with devices that enable The electronic device 800 is capable of communicating with any device (eg, router, modem, etc.) that communicates with one or more other computing devices. Such communication may occur through input/output (I/O) interface 850 .
  • the electronic device 800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet) through the network adapter 860 .
  • networks such as a local area network (LAN), a wide area network (WAN) and/or a public network such as the Internet
  • the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830 .
  • other hardware and/or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and Data backup storage system, etc.
  • the technical solutions according to the embodiments of the present application can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, server, terminal device, or network device, etc.) execute the method according to the embodiment of the present application.
  • a non-volatile storage medium which can be CD-ROM, U disk, mobile hard disk, etc.
  • a computing device which may be a personal computer, server, terminal device, or network device, etc.
  • a computer-readable storage medium is also provided, on which a program product capable of implementing the above-mentioned method in this specification is stored.
  • various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code.
  • the program product runs on the terminal device, the program code is used to make the terminal device execute the above-mentioned Steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section.
  • a program product 1400 for implementing the above method according to an embodiment of the present disclosure is described, which may adopt a portable compact disk read-only memory (CD-ROM) and include program codes, and may be used in a terminal device, For example running on a personal computer.
  • CD-ROM portable compact disk read-only memory
  • the program product of the present disclosure is not limited thereto.
  • a readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus or device.
  • a computer readable signal medium may include a data signal carrying readable program code in baseband or as part of a carrier wave. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural Programming language - such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.
  • the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider). business to connect via the Internet).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., a wide area network
  • the technical solutions according to the embodiments of the present application can be embodied in the form of software products, which can be stored in a non-volatile storage medium (which can be CD-ROM, U disk, mobile hard disk, etc.) or on the network , including several instructions to make a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the method according to the embodiment of the present application.
  • a non-volatile storage medium which can be CD-ROM, U disk, mobile hard disk, etc.
  • a computing device which may be a personal computer, a server, a mobile terminal, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Environmental & Geological Engineering (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本申请提供了一种传输速率控制方法、装置、电子设备和计算机可读存储介质，涉及分布式存储技术领域。其中，传输速率控制方法包括：在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌；响应于客户端发起的读写请求，确定读写操作对应的请求传输速率；在检测到请求传输速率大于令牌桶的突发速率时，将突发速率确定为读写操作的实际传输速率，并触发漏桶向令牌桶内流入令牌。通过本申请的技术方案，在检测到请求传输速率大于令牌桶的突发速率时，则直接将突发速率作为实际传输速率，执行读写操作，从而能够在保证读写效率的同时，降低该客户端的读写操作对其它客户端获取到的读写能力的影响。

Description

传输速率控制方法、装置、电子设备和可读存储介质
本申请要求在2021年6月11日提交中国专利局、申请号为202110652189.2、发明名称为“传输速率控制方法、装置、电子设备和可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及分布式存储技术领域，尤其涉及一种分布式存储系统的传输速率控制方法、装置、电子设备和计算机可读存储介质。
背景技术
Ceph(分布式文件系统)是集块存储、文件系统、对象存储于一体的分布式统一存储系统。
相关技术中，为了防止某些客户端占用大量读写资源而导致其它客户端出现饥饿的现象，Ceph通过令牌桶算法对块存储的读写操作进行限制，以保证Qos(Quality of Service，服务质量)。具体地，如图1所示，通过设置burst参数控制令牌桶的容量，使Ceph具备了一定的突发能力(突发能力是指允许读写速度在一定时间内超出限制以应对突然到来的较大量的读写请求)，但是在实际应用中，由于使用Ceph块存储的云服务器在开机时的读写较多，开机后读写较少，因此如果burst设置较低，令牌很快就会消耗完从而导致开机时间过长，如果把burst设置较大就无法保证不对其他客户端造成较大影响，失去了限制的意义。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本申请的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本申请的目的在于提供一种分布式存储系统的传输速率控制方法、装置、电子设备和计算机可读存储介质，至少在一定程度上克服由于相关技术中告警关联分析周期长，告警产出慢，计算量大的问题。
本申请的其他特性和优点将通过下面的详细描述变得显然，或部分地 通过本申请的实践而习得。
根据本申请的一个方面，提供一种分布式存储系统的传输速率控制方法，包括：在令牌桶内配置第一数量的令牌，并在所述令牌桶的入口处设置漏桶，在所述漏桶内配置所述第二数量的令牌；响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率；在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，其中，所述突发速率小于或等于所述第一数量。
在本申请的一个实施例中，在响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率之前，所述传输速率控制方法还包括：根据所述突发速率以及预设的突发时长配置所述第二数量，所述第二数量大于所述第一数量；所述在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，还包括：触发所述漏桶基于所述突发速率向所述令牌桶传输所述令牌，以基于所述突发速率执行所述读写操作，其中，基于所述突发速率执行所述读写操作的时长小于或等于所述突发时长。
在本申请的一个实施例中，所述触发所述漏桶基于所述突发速率向所述令牌桶传输所述令牌，还包括：基于限定速率向所述漏桶内传输所述令牌，其中，所述限定速率小于所述突发速率。
在本申请的一个实施例中，还包括：在检测到基于所述实际传输速率执行所述读写操作的时长达到所述突发时长时，所述读写操作未完成，将所述限定速率切换为所述读写操作的实际传输速率，以基于所述限定速率继续执行所述读写操作。
在本申请的一个实施例中，所述根据所述突发速率以及预设的突发时长配置所述第二数量，具体包括：计算所述突发速率和所述限定速率之间的速率差；计算所述速率差和所述突发时长之间的乘积，作为所述中间量；计算所述中间量和所述第一数量之间的差值，作为所述第二数量。
在本申请的一个实施例中，还包括：调节所述第二数量，以基于所述调节后的第二数量调节所述突发时长；和/或根据所述读写请求的需求参数调节突发时长，以基于调节后的所述突发时长配置所述第二数量。
在本申请的一个实施例中，还包括：在检测到所述请求传输速率小于或等于所述突发速率时，将所述请求传输速率为所述读写操作的实际传输速率。
根据本申请的另一个方面，提供一种分布式存储系统的传输速率控制装置，包括：配置模块，用于在令牌桶内配置第一数量的令牌，并在所述令牌桶的入口处设置漏桶，在所述漏桶内配置所述第二数量的令牌；确定模块，用于响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率；控制模块，用于在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，其中，所述突发速率小于或等于所述第一数量。
根据本申请的再一个方面，提供一种电子设备，包括：处理器；以及存储器，用于存储处理器的可执行指令；其中，处理器配置为经由执行可执行指令来执行上述任意一项的分布式存储系统的传输速率控制方法。
根据本申请的又一个方面，提供一种计算机可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现上述任意一项的分布式存储系统的传输速率控制方法。
本申请的实施例所提供的分布式存储系统的传输速率控制方案，通过串接漏桶和令牌桶，预先在令牌桶中配置第一数量的令牌，并限定令牌桶的突发速率，从而能够在接收到客户端的读写请求时，基于请求传输速率和突发速率之间的关系，确定读写请求的实际传输速率，具体地，在检测到请求传输速率大于令牌桶的突发速率时，则直接将突发速率作为实际传输速率，执行读写操作，从而能够在保证读写效率的同时，降低该客户端的读写操作对其它客户端获取到的读写能力的影响。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本申请的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1示出本申请实施例中一种分布式存储系统的传输速率控制方法的流程图；
图2示出本申请实施例中另一种分布式存储系统的传输速率控制方法的流程图；
图3示出本申请实施例中再一种分布式存储系统的传输速率控制方 法的流程图；
图4示出本申请实施例中又一种分布式存储系统的传输速率控制方法的流程图；
图5示出本申请实施例中又一种分布式存储系统的传输速率控制方法的流程图；
图6示出本申请实施例中又一种分布式存储系统的传输速率控制方法的流程图；
图7示出本申请实施例中一种分布式存储系统的传输速率控制装置的示意图；
图8示出本申请实施例中一种电子设备的示意图；和
图9示出本申请实施例中一种计算机可读存储介质的示意图。
具体实施方式
现在将参考附图更全面地描述示例实施方式。然而，示例实施方式能够以多种形式实施，且不应被理解为限于在此阐述的范例；相反，提供这些实施方式使得本申请将更加全面和完整，并将示例实施方式的构思全面地传达给本领域的技术人员。所描述的征、结构或性可以以任何合适的方式结合在一个或更多实施方式中。在下面的描述中，提供许多具体细节从而给出对本申请的实施方式的充分理解。然而，本领域技术人员将意识到，可以实践本申请的技术方案而省略所述定细节中的一个或更多，或者可以采用其它的、组元、装置、步骤等。在其它情况下，不详细示出或描述公知技术方案以避免喧宾夺主而使得本申请的各方面变得模糊。
此外，附图仅为本申请的示意性图解，图中相同的附图标记表示相同或类似的分，因而将省略对它们的重复描述。附图中所示的一些方框图是功能实体，不一定必须与物理或逻辑上独立的实体相对应。可以采用软件形式来实现这些功能实体，或在一个或多个硬件模块或集成电路中实现这些功能实体，或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。
下面结合附图对本申请示例实施方式进行详细说明。
分布式存储系统Ceph包括服务端和多个客户端，多个客户端与服务端通信连接，客户端通过接口进行集群的访问，访问包括对集群的整体访问和对象的访问两类接口，接口通过网络实现对Ceph集群的访问，以实现客户端的虚拟机启动、文件拷贝、文件下载等操作。
如图1所示，服务端执行分布式存储系统的传输速率控制方法，包括 以下步骤：
步骤S102，在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌。
其中，每一个令牌可以代表一个字节。
漏桶算法指流量先进入到漏桶里，漏桶以一定的速度向外发送流量，当流量的接收速度过大会直接溢出，使漏桶算法能强行限制数据的传输速率。
令牌桶算法的原理是分布式存储系统以一个恒定的速度往桶里放入令牌，而如果请求需要被处理，则需要先从桶里获取一个令牌，当桶里没有令牌可取时，则拒绝服务。
令牌桶中的每一个令牌都代表一个字节。如果令牌桶中存在令牌，则允许发送流量；而如果令牌桶中不存在令牌，则不允许发送流量。因此，如果突发门限被合理地配置并且令牌桶中有足够的令牌，那么流量就可以以峰值速率发送。
由于漏桶算法可以强行限制数据的传输速率，令牌桶算法允许突发传输，通过将漏桶算法和令牌桶算法进行结合，并合理设置第一数量和第二数量，能够实现具有可控时长的突发传输。
另外，本领域的技术人员还能够理解的是，漏桶的数量可以为一个，也可以为多个，令牌桶的数量可以为一个，也可以为多个。
步骤S104，响应于客户端发起的读写请求，确定读写操作对应的请求传输速率。
其中，在获取到客户端的读写请求，即IO请求时，则同样可以获取该读写请求的请求传输速率，以基于当前的读写能力确定是否可以满足客户端基于该请求传输速率进行读写操作的请求。
步骤S106，在检测到请求传输速率大于令牌桶的突发速率时，将突发速率确定为读写操作的实际传输速率，并触发漏桶向令牌桶内流入令牌，其中，突发速率小于或等于第一数量。
其中，令牌桶的突发速率指从漏桶向令牌桶传输令牌的速率，也即令牌桶漏出令牌的速率。
可选地，突发速率与第一数量相同，以实现最大的突发速率。
在该实施例中，通过串接漏桶和令牌桶，预先在令牌桶中配置第一数量的令牌，并限定令牌桶的突发速率，从而能够在接收到客户端的读写请求时，基于请求传输速率和突发速率之间的关系，确定读写请求的实际传输速率，具体地，在检测到请求传输速率大于令牌桶的突发速率时，则直 接将突发速率作为实际传输速率，执行读写操作，从而能够在保证读写效率的同时，降低该客户端的读写操作对其它客户端获取到的读写能力的影响。
具体地，在应用过程中，该方法可广泛应用于各种需要进行突发控制的系统，比如允许在云电脑、云服务器在开机、文件拷贝、文件下载等场景中以突发速率持续执行IO操作，以保证读写效率，进而有利于提升用户的使用体验。
如图2所示，服务端执行分布式存储系统的传输速率控制方法，包括以下步骤：
步骤S202，在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌，根据突发速率以及预设的突发时长配置第二数量，第二数量大于第一数量。
其中，本领域的技术人员能够理解的是，为了保证令牌桶能够具有持续突发能力，将漏桶的容量设置为远远大于令牌桶的容量，并在漏桶中预先配置一定量的令牌，以有效延长突发时长。
具体地，可以先确定突发时长，然后基于突发时长确定第二数量，从而实现突发时长的可控配置。
步骤S204，响应于客户端发起的读写请求，确定读写操作对应的请求传输速率。
步骤S206，在检测到请求传输速率大于令牌桶的突发速率时，将突发速率确定为读写操作的实际传输速率，触发漏桶基于突发速率向令牌桶传输令牌，以基于突发速率执行读写操作，其中，基于突发速率执行读写操作的时长小于或等于突发时长。
其中，在确定突发时长后，则可以进一步将基于突发速率执行的读写操作的时长根据突发时长进行限定，以保证以突发速率执行的读写操作的时长的可控性。
可选地，基于突发速率执行读写操作的时长等于突发时长。
在该实施例中，通过串接漏桶和令牌桶，预先在令牌桶中配置第一数量的令牌，在漏桶中配置第二数量的令牌，第二数量可以基于所需的突发时长进行配置，在检测到请求传输速率大于令牌桶的突发速率时，则直接将突发速率作为实际传输速率，并通过突发时长确定以突发速率执行的读写操作的时长，以执行读写操作，从而能够在保证读写效率的同时，降低该客户端的读写操作对其它客户端获取到的读写能力的影响，并能够对突发时长进行控制，防止少量的客户端占用过多的系统资源，有效地保障服 务质量。
如图3所示，服务端执行分布式存储系统的传输速率控制方法，包括以下步骤：
步骤S302，在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌，根据突发速率以及预设的突发时长配置第二数量，第二数量大于第一数量。
步骤S304，响应于客户端发起的读写请求，确定读写操作对应的请求传输速率。
步骤S306，在检测到请求传输速率大于突发速率时，将突发速率确定为读写操作的实际传输速率，触发漏桶基于突发速率向令牌桶传输令牌，以基于突发速率执行读写操作。以及
步骤S308，基于限定速率向漏桶内传输令牌，其中，限定速率小于突发速率，基于突发速率执行读写操作的时长小于或等于突发时长。
在该实施例中，在基于突发速率从漏桶中导出令牌的同时，基于限定速率向漏桶内导入令牌，一方面，能够满足延长突发时长的需求，另一方面，在漏桶中的令牌和令牌桶中的令牌都用完后，可以继续基于限定速率作为实际传输速率执行读写操作。
如图3所示，在本申请的一个实施例中，还包括：
步骤S310，在检测到基于实际传输速率执行读写操作的时长达到突发时长时，读写操作未完成，将限定速率切换为读写操作的实际传输速率，以基于限定速率继续执行读写操作。
在该实施例中，在漏桶中的令牌和令牌桶中的令牌都用完后，通过将突发速率切换为限定速率，继续基于限定速率作为实际传输速率执行读写操作，有利于保证读写操作的正常执行。
如图4所示，在本申请的一个实施例中，根据突发速率以及预设的突发时长配置第二数量的一种具体实现方式，包括：
步骤S402，计算突发速率和限定速率之间的速率差。
步骤S404，计算速率差和突发时长之间的乘积，作为中间量。
步骤S406，计算中间量和第一数量之间的差值，作为第二数量。
具体地，在确定突发时长的前提下，通过突发速率和限定速率配置第二数量，其中，第一数量为令牌桶的容量，第二数量为漏桶的容量，最大可累积令牌数即为令牌桶的容量和漏桶的容量之和。
具体地，最大可累积令牌数＝大漏桶容量(T)+小令牌桶容量(B)
持续突发消耗的令牌＝最大可累积令牌数+新生成的令牌，如式 (1)所示，对应得到的第二数量如式(2)所示。
B×S＝T+B+L×S   (1)
T＝S×(B-L)-B  (2)
在该实施例中，在令牌累积满大漏桶和小令牌桶的情况下，客户端在设定的时长范围内都可以按突发速率执行IO请求。
在本申请的一个实施例中，还包括：调节第二数量，以基于调节后的第二数量调节突发时长；和/或根据读写请求的需求参数调节突发时长，以基于调节后的突发时长配置第二数量。
在该实施例中，将漏桶的容量设置为可调，因此在需要调整突发时长时，则可以通过调节漏桶的容量，即第二数量实现。
另外，也可以进一步通过获取读写请求的需求参数，比如对于云电脑的开机需求时间，文件的下载需求时间，以及文件的上传需求时间等，通过这些需求参数确定对应的突发时长，以基于突发时长进一步配置第二数量，从而实现了不同的读写操作的读写能力的适配调整。
在本申请的一个实施例中，还包括：在检测到请求传输速率小于或等于突发速率时，将请求传输速率为读写操作的实际传输速率。
在该实施例中，在原有方案增加了一个大漏桶，漏桶里的令牌以突发速率向令牌桶泄漏，通过调节漏桶的容量获得持续突发能力。具体地，当IO请求速度小于令牌产生的速度时，令牌会不断的累积，桶的容量越大，能保存的令牌数就越多，可持续突发的时间越长。
如图5所示，通过大漏桶+小令牌桶相结合对IO进行限制，实现分布式存储系统的传输速率控制，具体包括：
步骤S502，设置限定速率L、突发速率B和突发时长S。
步骤S504，根据限定速率L向漏桶中投放令牌。
步骤S506，根据突发速率B向令牌桶中投放令牌。
步骤S508，读写操作。、
具体地，客户端发起IO请求时，首先判断能否从小令牌桶中申请到足够令牌，若能够则消耗桶中相应个数的令牌并放行，反之则将操作挂起，直到能够获取到足够的令牌。
步骤S510，申请令牌成功。
步骤S512，阻塞操作等待。
步骤S514，继续执行操作。
在该实施例中，由于小令牌桶的容量为B，且限定速率不超过B，在大漏桶中还有令牌的情况下，可以一直以突发速率执行IO操作，具备持 续突发能力，大漏桶加小令牌桶，一共可以存储令牌的数量为突发时长*(突发速率-限定速率)，加上每秒以限定速率补充的令牌，在桶满的情况下，可以在突发时长范围内以突发速率持续执行IO操作，突发时长可控。
如图6所示，根据本申请的实施例的分布式存储系统的传输速率控制方法，包括：
步骤S602，在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌，第二数量大于第一数量。
步骤S604，响应于客户端发起的读写请求，检测读写操作对应的请求传输速率和突发速率之间的关系。
步骤S606，在检测到请求传输速率大于令牌桶的突发速率时，将突发速率确定为读写操作的实际传输速率。
步骤S608，触发漏桶基于突发速率向令牌桶传输令牌，基于限定速率向漏桶内传输令牌。
步骤S610，在检测到基于实际传输速率执行读写操作的时长达到突发时长时，读写操作未完成，将限定速率切换为读写操作的实际传输速率，以基于限定速率继续执行读写操作。
步骤S612，在检测到请求传输速率小于或等于突发速率时，将请求传输速率为读写操作的实际传输速率。
在该实施例中，通过采用大漏桶+小令牌桶结合的令牌消耗方式，小令牌桶的容量固定为突发速率值，令牌以限定速率流入大漏桶中，大漏桶中的令牌以突发速率流向小令牌桶。当大漏桶中有剩余令牌时，客户端可以一直以突发速率执行读写请求，若请求速率大于令牌补充的速率，当大漏桶和小令牌桶中的令牌都用光了以后，客户端的IO请求执行速率与令牌补充的速率一致，通过调节大漏桶的容量实现突发时长的控制。
需要注意的是，上述附图仅是根据本公开示例性实施例的方法所包括的处理的示意性说明，而不是限制目的。易于理解，上述附图所示的处理并不表明或限制这些处理的时间顺序。另外，也易于理解，这些处理可以是例如在多个模块中同步或异步执行的。
所属技术领域的技术人员能够理解，本公开的各个方面可以实现为系统、方法或程序产品。因此，本公开的各个方面可以具体实现为以下形式，即：完全的硬件实施方式、完全的软件实施方式(包括固件、微代码等)，或硬件和软件方面结合的实施方式，这里可以统称为“电路”、“模块”或“系统”。
下面参照图7来描述根据本公开的这种实施方式的分布式存储系统的传输速率控制装置700。图7所示的分布式存储系统的传输速率控制装置700仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
分布式存储系统的传输速率控制装置700以硬件模块的形式表现。分布式存储系统的传输速率控制装置700的组件可以包括但不限于：配置模块702，用于在令牌桶内配置第一数量的令牌，并在令牌桶的入口处设置漏桶，在漏桶内配置第二数量的令牌；确定模块704，用于响应于客户端发起的读写请求，确定读写操作对应的请求传输速率；控制模块706，用于在检测到请求传输速率大于突发速率时，将突发速率确定为读写操作的实际传输速率，并触发漏桶向令牌桶内流入令牌，其中，突发速率小于或等于第一数量。
在本申请的一个实施例中，配置模块702还用于：根据突发速率以及预设的突发时长配置第二数量，第二数量大于第一数量；控制模块706还用于：触发漏桶基于突发速率向令牌桶传输令牌，以基于突发速率执行读写操作，其中，基于突发速率执行读写操作的时长小于或等于突发时长。
在本申请的一个实施例中，控制模块706还用于：基于限定速率向漏桶内传输令牌，其中，限定速率小于突发速率。
在本申请的一个实施例中，还包括：切换模块708，用于在检测到基于实际传输速率执行读写操作的时长达到突发时长时，读写操作未完成，将限定速率切换为读写操作的实际传输速率，以基于限定速率继续执行读写操作。
在本申请的一个实施例中，配置模块702还用于：计算突发速率和限定速率之间的速率差；计算速率差和突发时长之间的乘积，作为中间量；计算中间量和第一数量之间的差值，作为第二数量。
在本申请的一个实施例中，还包括：调节模块710，用于调节第二数量，以基于调节后的第二数量调节突发时长；和/或根据读写请求的需求参数调节突发时长，以基于调节后的突发时长配置第二数量。
在本申请的一个实施例中，控制模块706还用于：在检测到请求传输速率小于或等于突发速率时，将请求传输速率为读写操作的实际传输速率。
下面参照图8来描述根据本公开的这种实施方式的电子设备800。图8显示的电子设备800仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备800以通用计算设备的形式表现。电子设备 800的组件可以包括但不限于：上述至少一个处理单元810、上述至少一个存储单元820、连接不同系统组件(包括存储单元820和处理单元810)的总线830。
其中，存储单元存储有程序代码，程序代码可以被处理单元810执行，使得处理单元810执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。例如，处理单元810可以执行如图1中所示的步骤S202、S204和S206，以及本申请的分布式存储系统的传输速率控制方法中限定的其他步骤。
存储单元820可以包括易失性存储单元形式的可读介质，例如随机存取存储单元(RAM)8201和/或高速缓存存储单元8202，还可以进一步包括只读存储单元(ROM)8203。
存储单元820还可以包括具有一组(至少一个)程序模块8205的程序/实用工具8204，这样的程序模块8205包括但不限于：操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。
总线830可以为表示几类总线结构中的一种或多种，包括存储单元总线或者存储单元控制器、外围总线、图形加速端口、处理单元或者使用多种总线结构中的任意总线结构的局域总线。
电子设备800也可以与一个或多个外部设备870(例如键盘、指向设备、蓝牙设备等)通信，还可与一个或者多个使得用户能与该电子设备交互的设备通信，和/或与使得该电子设备800能与一个或多个其它计算设备进行通信的任何设备(例如路由器、调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口850进行。并且，电子设备800还可以通过网络适配器860与一个或者多个网络(例如局域网(LAN)，广域网(WAN)和/或公共网络，例如因特网)通信。如图所示，网络适配器860通过总线830与电子设备800的其它模块通信。应当明白，尽管图中未示出，可以结合电子设备使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本申请实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计 算设备(可以是个人计算机、服务器、终端装置、或者网络设备等)执行根据本申请实施方式的方法。
在本申请的示例性实施例中，还提供了一种计算机可读存储介质，其上存储有能够实现本说明书上述方法的程序产品。在一些可能的实施方式中，本公开的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当程序产品在终端设备上运行时，程序代码用于使终端设备执行本说明书上述“示例性方法”部分中描述的根据本公开各种示例性实施方式的步骤。
参考图14所示，描述了根据本公开的实施方式的用于实现上述方法的程序产品1400，其可以采用便携式紧凑盘只读存储器(CD-ROM)并包括程序代码，并可以在终端设备，例如个人电脑上运行。然而，本公开的程序产品不限于此，在本文件中，可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了可读程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。可读信号介质还可以是可读存储介质以外的任何可读介质，该可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于无线、有线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言的任意组合来编写用于执行本公开操作的程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、C++等，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。在涉及远程计算设备的情形中，远程计算设备可以通过任意种类的网络，包括局域网(LAN)或广域网(WAN)，连接到用户计算设备，或者，可以连接到外部计算设备(例如利用因特网服务提供商来通过因特网连接)。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本申请的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块 或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
此外，尽管在附图中以特定顺序描述了本申请中方法的各个步骤，但是，这并非要求或者暗示必须按照该特定顺序来执行这些步骤，或是必须执行全部所示的步骤才能实现期望的结果。附加的或备选的，可以省略某些步骤，将多个步骤合并为一个步骤执行，以及/或者将一个步骤分解为多个步骤执行等。
通过以上的实施方式的描述，本领域的技术人员易于理解，这里描述的示例实施方式可以通过软件实现，也可以通过软件结合必要的硬件的方式来实现。因此，根据本申请实施方式的技术方案可以以软件产品的形式体现出来，该软件产品可以存储在一个非易失性存储介质(可以是CD-ROM，U盘，移动硬盘等)中或网络上，包括若干指令以使得一台计算设备(可以是个人计算机、服务器、移动终端、或者网络设备等)执行根据本申请实施方式的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本申请的其它实施方案。本申请旨在涵盖本申请的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本申请的一般性原理并包括本申请未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本申请的真正范围和精神由所附的权利要求指出。

Claims (10)

  1. 一种分布式存储系统的传输速率控制方法，应用于所述分布式存储系统的服务端，其特征在于，所述分布式存储系统还包括多个客户端，所述多个客户端与所述服务端通信连接，所述传输速率控制方法包括：
    在令牌桶内配置第一数量的令牌，并在所述令牌桶的入口处设置漏桶，在所述漏桶内配置所述第二数量的令牌；
    响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率；
    在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，
    其中，所述突发速率小于或等于所述第一数量。
  2. 根据权利要求1所述的分布式存储系统的传输速率控制方法，其特征在于，在响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率之前，所述传输速率控制方法还包括：
    根据所述突发速率以及预设的突发时长配置所述第二数量，所述第二数量大于所述第一数量；
    所述在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，还包括：
    触发所述漏桶基于所述突发速率向所述令牌桶传输所述令牌，以基于所述突发速率执行所述读写操作，
    其中，基于所述突发速率执行所述读写操作的时长小于或等于所述突发时长。
  3. 根据权利要求2所述的分布式存储系统的传输速率控制方法，其特征在于，所述触发所述漏桶基于所述突发速率向所述令牌桶传输所述令牌，还包括：
    基于限定速率向所述漏桶内传输所述令牌，
    其中，所述限定速率小于所述突发速率。
  4. 根据权利要求3所述的分布式存储系统的传输速率控制方法，其特征在于，还包括：
    在检测到基于所述实际传输速率执行所述读写操作的时长达到所述突发时长时，所述读写操作未完成，将所述限定速率切换为所述读写操作 的实际传输速率，以基于所述限定速率继续执行所述读写操作。
  5. 根据权利要求3所述的分布式存储系统的传输速率控制方法，其特征在于，所述根据所述突发速率以及预设的突发时长配置所述第二数量，具体包括：
    计算所述突发速率和所述限定速率之间的速率差；
    计算所述速率差和所述突发时长之间的乘积，作为所述中间量；
    计算所述中间量和所述第一数量之间的差值，作为所述第二数量。
  6. 根据权利要求5所述的分布式存储系统的传输速率控制方法，其特征在于，还包括：
    调节所述第二数量，以基于所述调节后的第二数量调节所述突发时长；和/或
    根据所述读写请求的需求参数调节突发时长，以基于调节后的所述突发时长配置所述第二数量。
  7. 根据权利要求1至6中任一项所述的分布式存储系统的传输速率控制方法，其特征在于，还包括：
    在检测到所述请求传输速率小于或等于所述突发速率时，将所述请求传输速率为所述读写操作的实际传输速率。
  8. 一种分布式存储系统的传输速率控制装置，应用于所述分布式存储系统的服务端，其特征在于，所述分布式存储系统还包括多个客户端，所述多个客户端与所述服务端通信连接，所述传输速率控制装置包括：
    配置模块，用于在令牌桶内配置第一数量的令牌，并在所述令牌桶的入口处设置漏桶，在所述漏桶内配置所述第二数量的令牌；
    确定模块，用于响应于所述客户端发起的读写请求，确定所述读写操作对应的请求传输速率；
    控制模块，用于在检测到所述请求传输速率大于所述令牌桶的突发速率时，将所述突发速率确定为所述读写操作的实际传输速率，并触发所述漏桶向所述令牌桶内流入令牌，
    其中，所述突发速率小于或等于所述第一数量。
  9. 一种电子设备，其特征在于，包括：
    处理器；以及
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器配置为经由执行所述可执行指令来执行权利要求1至7中任意一项所述的分布式存储系统的传输速率控制方法。
  10. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于， 所述计算机程序被处理器执行时实现权利要求1至7中任意一项所述的分布式存储系统的传输速率控制方法。
PCT/CN2021/140863 2021-06-11 2021-12-23 传输速率控制方法、装置、电子设备和可读存储介质 WO2022257425A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110652189.2A CN113645150B (zh) 2021-06-11 2021-06-11 传输速率控制方法、装置、电子设备和可读存储介质
CN202110652189.2 2021-06-11

Publications (1)

Publication Number Publication Date
WO2022257425A1 true WO2022257425A1 (zh) 2022-12-15

Family

ID=78416019

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/140863 WO2022257425A1 (zh) 2021-06-11 2021-12-23 传输速率控制方法、装置、电子设备和可读存储介质

Country Status (2)

Country Link
CN (1) CN113645150B (zh)
WO (1) WO2022257425A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116708315A (zh) * 2023-07-17 2023-09-05 中移(苏州)软件技术有限公司 流量限速方法、装置、电子设备及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113645150B (zh) * 2021-06-11 2023-06-27 天翼云科技有限公司 传输速率控制方法、装置、电子设备和可读存储介质
CN114785739A (zh) * 2022-04-22 2022-07-22 天津中科曙光存储科技有限公司 逻辑卷服务质量的控制方法、装置、设备及介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060268719A1 (en) * 2005-05-26 2006-11-30 Hitachi Communication Technologies, Ltd. Packet forwarding apparatus using token bucket algorithm and leaky bucket algorithm
CN107579926A (zh) * 2017-10-20 2018-01-12 南京易捷思达软件科技有限公司 基于令牌桶算法的Ceph云存储系统的QoS设置方法
CN108804043A (zh) * 2018-06-26 2018-11-13 郑州云海信息技术有限公司 分布式块存储系统带宽流量控制方法、装置、设备及介质
CN113645150A (zh) * 2021-06-11 2021-11-12 中国电信股份有限公司 传输速率控制方法、装置、电子设备和可读存储介质

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100384156C (zh) * 2006-03-24 2008-04-23 华为技术有限公司 一种剩余带宽复用的方法及网络设备
CN101267382A (zh) * 2007-03-13 2008-09-17 大唐移动通信设备有限公司 确定数据传输通道拥塞状态的方法及装置
JP4612713B2 (ja) * 2008-08-12 2011-01-12 株式会社日立製作所 パケット転送装置及びネットワークシステム
EP2234346A1 (en) * 2009-03-26 2010-09-29 BRITISH TELECOMMUNICATIONS public limited company Policing in data networks
CN104125006B (zh) * 2014-07-28 2017-11-03 大连大学 卫星通信网络带宽分配方法
CN106375238A (zh) * 2015-07-21 2017-02-01 深圳市中兴微电子技术有限公司 一种流量监管方法及装置
CN107276827B (zh) * 2017-07-25 2021-04-23 郑州云海信息技术有限公司 一种分布式存储系统中Qos的实现方法及装置
CN108848038B (zh) * 2018-08-30 2021-01-29 华为技术有限公司 基于令牌桶的流量管理方法和令牌桶节点
CN112423346A (zh) * 2020-10-27 2021-02-26 清华大学 一种中继多址接入调度方法、装置、电子设备及存储介质
CN112463066B (zh) * 2020-12-11 2023-01-06 苏州浪潮智能科技有限公司 一种控制分布式块存储逻辑卷突发服务质量的方法和设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060268719A1 (en) * 2005-05-26 2006-11-30 Hitachi Communication Technologies, Ltd. Packet forwarding apparatus using token bucket algorithm and leaky bucket algorithm
CN107579926A (zh) * 2017-10-20 2018-01-12 南京易捷思达软件科技有限公司 基于令牌桶算法的Ceph云存储系统的QoS设置方法
CN108804043A (zh) * 2018-06-26 2018-11-13 郑州云海信息技术有限公司 分布式块存储系统带宽流量控制方法、装置、设备及介质
CN113645150A (zh) * 2021-06-11 2021-11-12 中国电信股份有限公司 传输速率控制方法、装置、电子设备和可读存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116708315A (zh) * 2023-07-17 2023-09-05 中移(苏州)软件技术有限公司 流量限速方法、装置、电子设备及存储介质
CN116708315B (zh) * 2023-07-17 2023-11-03 中移(苏州)软件技术有限公司 流量限速方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN113645150A (zh) 2021-11-12
CN113645150B (zh) 2023-06-27

Similar Documents

Publication Publication Date Title
WO2022257425A1 (zh) 传输速率控制方法、装置、电子设备和可读存储介质
US11321271B2 (en) Host based non-volatile memory clustering mechanism using network mapped storage
WO2020221190A1 (zh) 小程序状态同步方法、设备和计算机存储介质
US10298721B2 (en) Method and system to determine a work distribution model for an application deployed on a cloud
US9485184B2 (en) Congestion control for delay sensitive applications
US8619809B2 (en) Method, system, and program for managing a speed at which data is transmitted between network adaptors
US9483188B2 (en) Multiple I/O request processing in a storage system
US8775497B2 (en) Terminal device, information processing system, request target selection method and program
JP2005527007A (ja) コンピュータネットワーク内のブロックデータ保存
CN103428198B (zh) 网络设备、系统和方法
US20080267067A1 (en) Controlling the flow of data updates between a receiving station and a sending station
US11403253B2 (en) Transport protocol and interface for efficient data transfer over RDMA fabric
WO2015192685A1 (zh) 一种存储数据的方法及网络接口卡
EP4287591A1 (en) Data transmission method and apparatus, and server, storage medium and program product
WO2021212906A1 (zh) 网络端口分配方法、装置、电子设备和计算机可用介质
CN113849271B (zh) 云桌面展示方法、装置、设备、系统和存储介质
CN112600761B (zh) 一种资源分配的方法、装置及存储介质
WO2021012795A1 (zh) 网络节点的调度方法、装置、电子设备和存储介质
US10523741B2 (en) System and method for avoiding proxy connection latency
US20080005289A1 (en) Performance profiling for improved data throughput
WO2020134949A1 (zh) 会话请求发送方法、装置、电子设备和存储介质
US7826356B2 (en) Method and system for controlling flow in an asymmetric communication channel
US20190369706A1 (en) Burst-responsive wireless download packet management for reducing processing workload latency and power consumption
WO2021139225A1 (zh) 用于存储系统的流量控制方法、装置、介质及电子设备
CN106060158B (zh) 一种判断目标设备的状态的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21944921

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE