CN109756429B - Bandwidth allocation method and device - Google Patents
Bandwidth allocation method and device Download PDFInfo
- Publication number
- CN109756429B CN109756429B CN201711078855.6A CN201711078855A CN109756429B CN 109756429 B CN109756429 B CN 109756429B CN 201711078855 A CN201711078855 A CN 201711078855A CN 109756429 B CN109756429 B CN 109756429B
- Authority
- CN
- China
- Prior art keywords
- transmission channel
- bandwidth
- data
- transmission
- quota
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Data Exchanges In Wide-Area Networks (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application provides a bandwidth allocation scheme, which detects the activity state of other transmission channels when the current transmission channel needs to transmit data, determines an inactive transmission channel from the other transmission channels, then recovers the bandwidth quota of the inactive transmission channel, accumulates the bandwidth quota into a global available bandwidth, and allocates the bandwidth quota from the global available bandwidth to the current transmission channel. The inactive transmission channels refer to transmission channels which do not transmit data within a preset time period, when any one transmission channel needs to transmit data, the processing process can be repeated continuously, the recovery and the distribution of the bandwidths of the transmission channels are dynamic, locking is not needed, and the method is suitable for high concurrency scenes.
Description
Technical Field
The present disclosure relates to the field of information technologies, and in particular, to a method and apparatus for bandwidth allocation.
Background
Currently existing systems generally use a TC (traffic control) framework for flow control to achieve bandwidth allocation. Because the processing timing of the TC framework is at a position in the protocol stack close to the network card driver, that is, close to the physical layer, and at a position further back in the data transmission path, since the data from the upper layer cannot be controlled, a buffer zone in the transmission path may be occupied or occupied in a large amount, which may cause a problem of buffer burst (buffer overfill). And the processing of the TC framework is based on a queue, and locking is needed in the processing process, so that the concurrent processing performance is poor. Furthermore, the TC framework is limited by the bit width of the protocol field (typically supporting 128 classifications at most), and cannot support large-scale fine-grained bandwidth allocation, such as connection-level bandwidth allocation.
Content of the application
An object of the present application is to provide a bandwidth allocation scheme, which is used to solve the problem of poor concurrent processing performance in the prior art.
To achieve the above object, the present application provides a bandwidth allocation method, including:
when the current transmission channel needs to transmit data, detecting the active state of other transmission channels, and determining an inactive transmission channel from the other transmission channels, wherein the inactive transmission channel is: a transmission channel for data transmission is not performed within a preset duration;
recovering the bandwidth quota of the inactive transmission channel, and accumulating the bandwidth quota into the global available bandwidth;
and allocating bandwidth quota for the current transmission channel from the globally available bandwidth.
Further, detecting an active state of the other transmission channels and determining an inactive transmission channel from the other transmission channels, including:
detecting the active states of a preset number of other transmission channels, and determining the inactive transmission channels in the preset number of other transmission channels.
Further, detecting an active state of the other transmission channels and determining an inactive transmission channel from the other transmission channels, including:
Acquiring time stamp information corresponding to other transmission channels, wherein the time stamp information is updated when the corresponding transmission channels perform data transmission;
comparing the timestamp information of other transmission channels with the current time to obtain a comparison result;
and if the comparison result exceeds the preset duration, determining the corresponding transmission channel as an inactive transmission channel.
Further, the method further comprises:
when a transmission channel is created, writing context information of the transmission channel into one unoccupied data item of a data structure, wherein each occupied data item in the data structure corresponds to one transmission channel, and the context information comprises timestamp information and marking information of the corresponding transmission channel, and the marking information is used for marking the state of the data item as occupied.
Further, the context information further includes a guaranteed bandwidth value of the transmission channel, where the guaranteed bandwidth value is at least an allocated bandwidth quota when the transmission channel transmits data.
Further, the method further comprises:
and when the transmission channel is released, recovering the bandwidth quota of the transmission channel, accumulating the bandwidth quota into the global available bandwidth, deleting the content of the transmission channel in the corresponding data item, and updating the state of the data item to be unoccupied.
Further, the data items of the data structure also store bandwidth quota corresponding to the transmission channel;
recovering the bandwidth quota of the inactive transmission channel, comprising:
resetting the bandwidth quota of the data item corresponding to the inactive transmission channel;
allocating bandwidth quota for the current transmission channel, including:
writing the bandwidth quota of the data item corresponding to the current transmission channel.
Further, after allocating the bandwidth quota for the current transmission channel, the method includes:
reading the bandwidth quota of the current transmission channel from the data item of the current transmission channel in the data structure;
acquiring the sum of the flow of data to be transmitted and the bandwidth occupation of a current transmission channel as a target flow;
and transmitting the data when the target flow does not exceed the bandwidth quota.
Based on another aspect of the present application, there is also provided a bandwidth allocation apparatus, including:
the state detection device is used for detecting the active states of other transmission channels when the current transmission channel needs to transmit data, and determining an inactive transmission channel from the other transmission channels, wherein the inactive transmission channel is: a transmission channel for data transmission is not performed within a preset duration;
The bandwidth recycling device is used for recycling the bandwidth quota of the inactive transmission channel and accumulating the bandwidth quota into the global available bandwidth;
and the bandwidth allocation device is used for allocating bandwidth quota for the current transmission channel from the globally available bandwidth.
Further, the state detection device is configured to detect an active state of a preset number of other transmission channels, and determine an inactive transmission channel among the preset number of other transmission channels.
Further, the state detection device is configured to obtain timestamp information corresponding to other transmission channels, where the timestamp information is updated when the corresponding transmission channel performs data transmission; comparing the timestamp information of other transmission channels with the current time to obtain a comparison result; and if the comparison result exceeds the preset duration, determining the corresponding transmission channel as an inactive transmission channel.
Further, the apparatus further comprises:
and the control device is used for writing the context information of the transmission channel into one unoccupied data item of the data structure when the transmission channel is created, wherein each occupied data item in the data structure corresponds to one transmission channel, the context information comprises timestamp information and marking information of the corresponding transmission channel, and the marking information is used for marking the state of the data item as occupied.
Further, the context information further includes a guaranteed bandwidth value of the transmission channel, where the guaranteed bandwidth value is at least an allocated bandwidth quota when the transmission channel transmits data.
Further, the bandwidth recycling device is further configured to recycle the bandwidth quota of the transmission channel and accumulate the bandwidth quota into the global available bandwidth when the transmission channel is released;
the control device is further configured to delete content of the transmission channel in a corresponding data item when the transmission channel is released, and update a state of the data item to be unoccupied.
Further, the data items of the data structure also store bandwidth quota corresponding to the transmission channel;
the control device is further configured to clear the bandwidth quota of the data item corresponding to the inactive transmission channel, and write the bandwidth quota of the data item corresponding to the current transmission channel.
Further, the apparatus further comprises:
the data transmission device is used for reading the bandwidth quota of the current transmission channel from the data item of the current transmission channel in the data structure after the bandwidth quota is allocated to the current transmission channel, and obtaining the sum of the flow of the data to be transmitted and the bandwidth occupation of the current transmission channel as a target flow; and transmitting the data when the target traffic does not exceed the bandwidth quota.
In addition, the application further provides a bandwidth allocation device, which comprises:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method as described above.
In the scheme provided by the application, when the current transmission channel needs to transmit data, the active states of other transmission channels are detected, an inactive transmission channel is determined from the other transmission channels, then the bandwidth quota of the inactive transmission channel is recovered, accumulated into the global available bandwidth, and the bandwidth quota is allocated to the current transmission channel from the global available bandwidth. The inactive transmission channels refer to transmission channels which do not transmit data within a preset time period, when any one transmission channel needs to transmit data, the processing process can be repeated continuously, the recovery and the distribution of the bandwidths of the transmission channels are dynamic, locking is not needed, and the method is suitable for high concurrency scenes.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:
Fig. 1 is a process flow diagram of a bandwidth allocation method according to an embodiment of the present application;
FIG. 2 is a flowchart illustrating a process for creating a transmission channel according to an embodiment of the present application;
fig. 3 is a flowchart of a process when a transmission channel is released in the embodiment of the present application;
fig. 4 is a process flow chart when data is transmitted by adopting the bandwidth allocation method provided in the embodiment of the present application;
FIG. 5 is a schematic diagram of an array defined in an embodiment of the present application;
fig. 6 is a schematic structural diagram of a bandwidth allocation device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of another bandwidth allocation apparatus according to an embodiment of the present application;
the same or similar reference numbers in the drawings refer to the same or similar parts.
Detailed Description
The present application is described in further detail below with reference to the accompanying drawings.
In a typical configuration of the present application, the terminals, the devices of the services network each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium which can be used to store information that can be accessed by a computing device.
The embodiment of the application provides a bandwidth allocation method, which is used for allocating proper bandwidth quota to each transmission channel so as to control the flow of the transmission channel, and the processing flow of the method is as shown in fig. 1, and comprises the following steps:
in step S101, when the current transmission channel needs to transmit data, the active state of the other transmission channels is detected, and the inactive transmission channel is determined from the other transmission channels. The transmission channel is a channel for data transceiving, and not only can be used for distributing network bandwidth, such as network data transmission, but also can be used for distributing bandwidth of data transmission in other modes, such as data transmission among processes or distributing storage bandwidth in an I/O (input/output) scene. The transmission channel may be an established connection between two devices when used for transmission of network data, a message queue or the like when used for transmission of data between processes, and an I/O channel (e.g., stream, file, etc.) when reading and writing data in an I/O scenario. Bandwidth refers to the amount of data that can be transmitted per unit time.
When the scheme is applied to network bandwidth allocation, the scheme can be suitable for high-performance networks adopting RDMA (Remote Direct Memory Access ), infiniBand (InfiniBand) and other technologies, and can also be suitable for the environments of traditional Ethernet (such as TCP/IP networks) or various virtualized networks.
When data is transmitted, the method provided by the embodiment of the application can be realized in a kernel mode or a user mode according to the corresponding transmission protocol, and the distribution of network bandwidth can be completed. For example, in a high performance network using RDMA, a transmitting end device directly transmits data to a storage area of a receiving end device through the network, and in this process, a CPU of the receiving end device only needs to perform processing once at the end of initialization and transmission, without intervention in the transmission process, and most of the processing is completed in a user mode. In the TCP/IP network, when transmitting data, the CPU is required to participate in the operation of directly transmitting the data to the designated storage area of the receiving device, so most of the processing is completed in the kernel mode. Therefore, the bandwidth allocation scheme provided by the embodiment of the application is not limited to be realized in a kernel mode or a user mode, and can be realized in the kernel mode or the user mode according to the scene selection of actual application.
The allocation granularity can be set according to actual requirements. For example, in the network bandwidth allocation scenario, the allocation of the connection level with fine granularity may be selected according to the granularity of allocation, or the allocation of the process level and the process group level with coarser granularity may be selected. Wherein each process group may contain at least one process, and each process may correspond to one or more connections. When the scheme is implemented, the transmission channel is defined as a connection, a process or a process group, so that bandwidth allocation with corresponding granularity can be realized.
The current transmission channel is a transmission channel which needs to send data at present, and needs to allocate bandwidth quota for the current transmission channel as the upper limit value of bandwidth for transmitting data, so as to control flow and realize bandwidth allocation. The other transmission channels correspond to the current transmission channel, and refer to one or more transmission channels other than the current transmission channel, which may be currently transmitting data, may have just completed data transmission, or may have not transmitted data for a longer time. The inactive transmission channel refers to a transmission channel that has not been subjected to data transmission within a preset time period, for example, the preset time period is set to 5 seconds, and then the inactive transmission channel is a transmission channel that has not been subjected to data transmission within 5 seconds.
In one embodiment of the present application, a piece of timestamp information may be maintained for each transmission channel, where the timestamp information is updated when the corresponding transmission channel performs data transmission, for example, when connection 1 sends data at 00:15:11, the corresponding timestamp information is updated to 00:15:11, and then if data is sent again at 00:15:14, the corresponding timestamp information is updated to 00:15:14 again.
Therefore, when the active state of other transmission channels is detected and the inactive transmission channel is determined from the other transmission channels, the timestamp information corresponding to the other transmission channels can be acquired first, then the timestamp information of the other transmission channels and the current time are compared, a comparison result is acquired, and if the comparison result exceeds the preset duration, the corresponding transmission channel is determined to be the inactive transmission channel. Taking the connection 1 as an example, when the active state of the connection 1 is detected, if the current time is 00:15:20, the comparison result is 6 seconds, which indicates that the connection 1 has 6 seconds of data which have not been transmitted, and if the preset time period is 5 seconds, it can be determined that the connection 1 is an inactive connection.
Step S102, recovering the bandwidth quota of the inactive transmission channel, and accumulating the bandwidth quota into the global available bandwidth. The bandwidth quota refers to the bandwidth currently occupied by a transmission channel, the global available bandwidth refers to the bandwidth unoccupied in the upper bandwidth limit, the bandwidth can be allocated to the transmission channel currently needing to transmit data, for example, the upper bandwidth limit is 10MB, 3 connections currently transmitting data exist, the bandwidth quota allocated to the connections is respectively 1MB, 2MB and 3MB, the global available bandwidth at the moment is 4MB, if the connection with the bandwidth of 1MB is transmitted and enters the inactive state, the bandwidth quota can be recovered and accumulated into the global available bandwidth, and the global available bandwidth at the moment becomes 5MB.
By detecting the active state of other transmission channels, inactive transmission channels can be determined from the other transmission channels, and since the inactive transmission channels have not transmitted data for a longer time, the transmission channels can be considered to have a higher possibility of not transmitting data for a short time, so that the bandwidth quota can be recovered, cleared and accumulated into the global available bandwidth.
In an actual scenario, a preset number of other transmission channels may be selected for detection, and among the preset number of other transmission channels, an inactive transmission channel is determined, where the other transmission channels selected for detection may be in a random manner, or may be detected according to a set selection policy, for example, by randomly selecting 64 connections, or selecting N% of the total number of transmission channels.
Step S103, allocating bandwidth quota for the current transmission channel from the globally available bandwidth. In an actual scenario, the bandwidth quota allocated to the current transmission channel may be all the current globally available bandwidths, or may be a part of the globally available bandwidths, for example, a bandwidth value may be set by a user, an bandwidth quota of the value may be allocated to the transmission channel, or an appropriate bandwidth value may be estimated according to the historical flow of the transmission channel when data is transmitted, and an bandwidth quota may be allocated to the transmission channel.
In an embodiment of the present application, before implementing the bandwidth allocation scheme of the present application, the determination may be made based on the global available bandwidth and the bandwidth quota that needs to be allocated to the current transmission channel, and if the global available bandwidth can be greater than or equal to the bandwidth quota that needs to be allocated to the current transmission channel, the bandwidth allocation quota may be allocated to the current transmission channel directly by executing step S103 without executing steps S101 and S102 to perform bandwidth reclamation.
For example, the upper bandwidth limit is 10MB, at the present moment, there are already 3 connections transmitting data, and the allocated bandwidth quota for these connections is 1MB, 2MB and 3MB, respectively, and the globally available bandwidth at this time is 4MB. If the bandwidth quota required by the new connection to be created is only 2MB, the current globally available bandwidth can meet the bandwidth allocation of the new connection, so that the bandwidth of 2MB can be allocated for the newly created connection directly at the globally available bandwidth of 4MB.
After the corresponding bandwidth quota is allocated to the current transmission channel, the bandwidth quota can be used as the upper limit value of the flow of the data transmission. When data is transmitted, the sum of the flow of the data to be transmitted and the bandwidth occupation of the current transmission channel is obtained and used as a target flow, then the target flow and the allocated bandwidth quota are compared, if the target flow does not exceed the bandwidth quota, the data can be transmitted, otherwise, retry, deferred transmission or discarding and the like are selected according to a transmission strategy.
The scheme of the application can be realized on a transmitting side or a receiving side, and when the scheme is realized on the transmitting side, the data is transmitted through the current transmission channel, namely the data transmitting process, so that when the target flow to be transmitted exceeds the bandwidth quota, the processing such as retrying, deferring to transmit or discarding the data is selected according to the transmitting strategy. When the receiving side is realized, the data is transmitted through the current transmission channel, namely the receiving process of the data, and when the target flow to be received exceeds the bandwidth quota, the processing such as discarding the data, requesting the retransmission of the sending end and the like is selected according to the receiving strategy.
When any one transmission channel needs to transmit data, the processing process can be repeated continuously, and the recovery and the distribution of the bandwidths of all the transmission channels are dynamic, so that locking is not needed, and the method is suitable for high concurrency scenes. Different from the TC framework, when the scheme of the embodiment of the present application is applied to the transmitting side, the scheme may be implemented in a socket (socket) of a protocol stack, which is located at a position relatively forward in a data transmission path, so that occurrence of buffer burst can be suppressed.
In order to make the process of bandwidth allocation more efficient, the management costs of the individual transport channels can be passed on to the control plane (creation and release of transport channels) to ensure a simple and efficient data plane (data transmission). Thereby, a data structure is defined for managing the context information of the transmission channel, which context information comprises at least the time stamp information and the tag information of the corresponding transmission channel. The context information of one transmission channel is stored in each data item in the data structure, when the context information is written in one data item, the state of the data item is occupied, otherwise, the state of the data item can be occupied, and the marking information can be used for marking the state of the data item as occupied, so that the bandwidth allocation of each transmission channel can be independently ensured.
When the transmission channel is created, the context information of the transmission channel is written into an unoccupied data item of the data structure. The data structures employed in the actual scenario may be arrays, but any other feasible data structures may be used, such as matrices, data sequences, etc. Each array item of the array is used for storing the context information of one transmission channel, in order to better support the large-scale bandwidth allocation with fine granularity, a multidimensional array can be adopted, and the searching of the corresponding array item is optimized by combining a bitmap algorithm. Because the time stamp information is also stored in the array, the array can be accessed to operate when the time stamp information needs to be updated or the time stamp information needs to be read. Fig. 2 shows a schematic diagram when a transmission channel is created.
In this case, a predetermined number of other transmission channels are selected for detection, and when determining an inactive transmission channel, a fixed number of array items may be scanned each time, starting from the array item corresponding to the current transmission channel. For example, if the index of the array item corresponding to the current transmission channel is 100, 64 array items with index of 101-164 can be scanned, and the timestamp information in the 64 array items can be obtained to detect the activity state.
In addition, the context information may further include a guaranteed bandwidth value of the transmission channel, where the guaranteed bandwidth value refers to at least a bandwidth quota that needs to be allocated by the transmission channel when transmitting data. For example, if a connection needs to transmit more important information, at least its 256KB bandwidth quota needs to be guaranteed when it needs to transmit data.
When the transmission channel is released, the bandwidth quota of the transmission channel is recovered and accumulated into the global available bandwidth, the content of the transmission channel in the corresponding data item is deleted, and the state of the data item is updated to be unoccupied, so that a new transmission channel which is created later can use the data item. In addition, the data items of the data structure can also store the bandwidth quota of the corresponding transmission channel, if the information needs to be read or modified, the data items in the data structure can be directly accessed, for example, when the bandwidth quota of the inactive transmission channel is recovered, the corresponding data items can be accessed, and the bandwidth quota of the data items corresponding to the inactive transmission channel is cleared. And when the bandwidth quota is allocated to the current transmission channel, the data item can be directly accessed, and the bandwidth quota is written into the data item corresponding to the current transmission channel. The current number of transmission channels is updated after each creation or release of a transmission channel. Fig. 3 shows a schematic diagram when a transmission channel is created.
And when data is transmitted, the bandwidth quota of the data is read from the data item of the current transmission channel in the data structure, then the sum of the flow of the data to be transmitted and the bandwidth occupation of the current transmission channel is obtained and used as a target flow, then the target flow and the allocated bandwidth quota are compared, if the target flow does not exceed the bandwidth quota, the data can be transmitted, otherwise, retry, deferring sending or discarding and the like are selected according to the transmission strategy. Therefore, during data transmission, only the bandwidth quota of the corresponding transmission channel needs to be read from the data structure, and the management of the transmission channel is mainly in the control plane, namely, during the creation and release of the transmission channel, the mode ensures the simplicity and the high efficiency of the data plane, and bandwidth can be allocated at the connection level.
Fig. 4 is a process flow diagram of implementing data transmission by using the method provided in the embodiment of the present application, where the data structure for implementing bandwidth allocation at a connection level and storing connection context information uses an array, and each array item (slot) stores context information of a connection, including timestamp information, a guaranteed-bottom bandwidth value of the connection, and a flag information, as shown in fig. 5. When a connection is created, its context information is written into an unoccupied slot, which is marked as occupied, so that the slot corresponds to the connection, and in addition, the bandwidth quota (already occupied bandwidth) of the connection can be written into the slot, and the slot can be directly accessed to acquire or update information about the connection.
The processing flow for implementing data transmission in this embodiment includes:
in step S401, when data is transmitted through the connection, the current bandwidth occupation of the connection is calculated first, and then the data amount of the data to be transmitted this time is added as the target flow for determination. The current bandwidth occupancy may be calculated using a sliding window algorithm, a token bucket algorithm, or other similar algorithm.
In addition, other manners may be used to calculate the target flow, for example, the target flow at a certain moment in the future may be estimated at the current moment, so that the determination in step S402 may be performed in advance, and the corresponding transmission policy may be selected. The manner in which future likely target flows are estimated may include, but is not limited to, the following: for example, by calculating an average value based on the bandwidth occupancy of the connection for the previous N seconds, or by transmitting an average bandwidth occupancy of some type of data in the history data.
Step S402, compares the target traffic with the bandwidth quota allocated to the connection to see if the bandwidth quota is exceeded. Wherein the bandwidth quota for the connection is determined based on the method shown in fig. 1.
Step S403, if the target flow exceeds the bandwidth quota, the method directly returns, and retries or delays sending according to the policy selection.
Step S404, if the target traffic does not exceed the bandwidth quota, updating the current bandwidth occupancy of the connection.
Step S405, data is transmitted through the connection.
Based on the same inventive concept, the embodiments of the present application also provide an apparatus for implementing the above bandwidth allocation method, where the corresponding method is the method in the foregoing embodiment, and the principle of solving the problem is similar to that of the method.
The bandwidth allocation device is used for allocating proper bandwidth quota to each transmission channel, so that the flow of the transmission channel is controlled. The configuration of the apparatus is as shown in fig. 6, and includes at least a state detecting means 610, a bandwidth reclaiming means 620 and a bandwidth allocating means 630. The state detecting device 610 may detect an active state of the other transmission channels when the current transmission channel needs to transmit data, and determine an inactive transmission channel from the other transmission channels. The transmission channel is a channel for data transceiving, and not only can be used for distributing network bandwidth, such as network data transmission, but also can be used for distributing bandwidth of data transmission in other modes, such as data transmission among processes or distributing storage bandwidth in an I/O (input/output) scene. The transmission channel may be an established connection between two devices when used for transmission of network data, a message queue or the like when used for transmission of data between processes, and an I/O channel (e.g., stream, file, etc.) when reading and writing data in an I/O scenario. Bandwidth refers to the amount of data that can be transmitted per unit time.
When the scheme is applied to network bandwidth allocation, the scheme can be suitable for high-performance networks adopting RDMA (Remote Direct Memory Access ), infiniBand (InfiniBand) and other technologies, and can also be suitable for the environments of traditional Ethernet (such as TCP/IP networks) or various virtualized networks.
When data is transmitted, the method provided by the embodiment of the application can be realized in a kernel mode or a user mode according to the corresponding transmission protocol, and the distribution of network bandwidth can be completed. For example, in a high performance network using RDMA, a transmitting end device directly transmits data to a storage area of a receiving end device through the network, and in this process, a CPU of the receiving end device only needs to perform processing once at the end of initialization and transmission, without intervention in the transmission process, and most of the processing is completed in a user mode. In the TCP/IP network, when transmitting data, the CPU is required to participate in the operation of directly transmitting the data to the designated storage area of the receiving device, so most of the processing is completed in the kernel mode. Therefore, the bandwidth allocation scheme provided by the embodiment of the application is not limited to be realized in a kernel mode or a user mode, and can be realized in the kernel mode or the user mode according to the scene selection of actual application.
The allocation granularity can be set according to actual requirements. For example, in the network bandwidth allocation scenario, the allocation of the connection level with fine granularity may be selected according to the granularity of allocation, or the allocation of the process level and the process group level with coarser granularity may be selected. Wherein each process group may contain at least one process, and each process may correspond to one or more connections. When the scheme is implemented, the transmission channel is defined as a connection, a process or a process group, so that bandwidth allocation with corresponding granularity can be realized.
The current transmission channel is a transmission channel which needs to send data at present, and needs to allocate bandwidth quota for the current transmission channel as the upper limit value of bandwidth for transmitting data, so as to control flow and realize bandwidth allocation. The other transmission channels correspond to the current transmission channel, and refer to one or more transmission channels other than the current transmission channel, which may be currently transmitting data, may have just completed data transmission, or may have not transmitted data for a longer time. The inactive transmission channel refers to a transmission channel that has not been subjected to data transmission within a preset time period, for example, the preset time period is set to 5 seconds, and then the inactive transmission channel is a transmission channel that has not been subjected to data transmission within 5 seconds.
In one embodiment of the present application, a piece of timestamp information may be maintained for each transmission channel, where the timestamp information is updated when the corresponding transmission channel performs data transmission, for example, when connection 1 sends data at 00:15:11, the corresponding timestamp information is updated to 00:15:11, and then if data is sent again at 00:15:14, the corresponding timestamp information is updated to 00:15:14 again.
Thus, when detecting the active states of the other transmission channels and determining the inactive transmission channel from the other transmission channels, the state detecting device 610 may first obtain timestamp information corresponding to the other transmission channels, then compare the timestamp information of the other transmission channels with the current time, obtain a comparison result, and determine the corresponding transmission channel as the inactive transmission channel if the comparison result exceeds the preset duration. Taking the connection 1 as an example, when the active state of the connection 1 is detected, if the current time is 00:15:20, the comparison result is 6 seconds, which indicates that the connection 1 has 6 seconds of data which have not been transmitted, and if the preset time period is 5 seconds, it can be determined that the connection 1 is an inactive connection.
The bandwidth reclamation apparatus 620 is configured to reclaim the bandwidth quota of the inactive transmission channel and accumulate the bandwidth quota into the globally available bandwidth. The bandwidth quota refers to the bandwidth currently occupied by a transmission channel, the global available bandwidth refers to the bandwidth unoccupied in the upper bandwidth limit, the bandwidth can be allocated to the transmission channel currently needing to transmit data, for example, the upper bandwidth limit is 10MB, 3 connections currently transmitting data exist, the bandwidth quota allocated to the connections is respectively 1MB, 2MB and 3MB, the global available bandwidth at the moment is 4MB, if the connection with the bandwidth of 1MB is transmitted and enters the inactive state, the bandwidth quota can be recovered and accumulated into the global available bandwidth, and the global available bandwidth at the moment becomes 5MB.
The state detecting means 610 may determine an inactive transmission channel from the other transmission channels by detecting the active state of the other transmission channels, and since the inactive transmission channel has not transmitted data for a long time, it may be considered that the transmission channels are not transmitted data for a short time, so that the bandwidth recycling means 620 may recycle its bandwidth quota, clear its bandwidth quota, and accumulate it into the globally available bandwidth.
In an actual scenario, the state detection device may select a preset number of other transmission channels to detect, and determine an inactive transmission channel among the preset number of other transmission channels, where the other transmission channels selected to detect may be in a random manner, or may select 64 connections to detect according to a set selection policy, for example, randomly, or select N% of the total number of transmission channels.
Bandwidth allocation means 630 is arranged to allocate a bandwidth quota for the current transmission channel from the globally available bandwidth. In an actual scenario, the bandwidth quota allocated by the bandwidth allocation device for the current transmission channel may be all the current globally available bandwidths, or may be a part of the globally available bandwidths, for example, a bandwidth value may be set by a user, an bandwidth quota of the value may be allocated for the transmission channel, or an appropriate bandwidth value may be estimated according to the historical flow of the transmission channel when data is transmitted, and an bandwidth quota may be allocated for the transmission channel.
In one embodiment of the present application, before implementing the bandwidth allocation scheme of the present application, the bandwidth allocation device 630 may determine, based on the globally available bandwidth and the bandwidth quota that needs to be allocated for the current transmission channel, and if the globally available bandwidth can be greater than or equal to the bandwidth quota that needs to be allocated for the current transmission channel, the bandwidth quota may be directly allocated for the current transmission channel by the bandwidth allocation device 630 without detection and bandwidth reclamation by the state detection device 610 and the bandwidth reclamation device 620.
For example, the upper bandwidth limit is 10MB, at the present moment, there are already 3 connections transmitting data, and the allocated bandwidth quota for these connections is 1MB, 2MB and 3MB, respectively, and the globally available bandwidth at this time is 4MB. If the bandwidth quota required by the new connection to be created is only 2MB, the current globally available bandwidth can meet the bandwidth allocation of the new connection, so that the bandwidth of 2MB can be allocated for the newly created connection directly at the globally available bandwidth of 4MB.
After the corresponding bandwidth quota is allocated to the current transmission channel, the bandwidth quota can be used as the upper limit value of the flow of the data transmission. When transmitting data, the data transmission device of the bandwidth allocation device can acquire the sum of the flow of the data to be transmitted (namely the data quantity to be transmitted) and the bandwidth occupation of the current transmission channel as a target flow, then compare the target flow with the allocated bandwidth quota, if the target flow does not exceed the bandwidth quota, the data can be transmitted, otherwise, retry, delay transmission or discard are selected according to the transmission strategy.
The scheme of the application can be realized on a transmitting side or a receiving side, and when the scheme is realized on the transmitting side, the data is transmitted through the current transmission channel, namely the data transmitting process, so that when the target flow to be transmitted exceeds the bandwidth quota, the processing such as retrying, deferring to transmit or discarding the data is selected according to the transmitting strategy. When the receiving side is realized, the data is transmitted through the current transmission channel, namely the receiving process of the data, and when the target flow to be received exceeds the bandwidth quota, the processing such as discarding the data, requesting the retransmission of the sending end and the like is selected according to the receiving strategy.
When any one transmission channel needs to transmit data, the processing process can be repeated continuously, and the recovery and the distribution of the bandwidths of all the transmission channels are dynamic, so that locking is not needed, and the method is suitable for high concurrency scenes. Different from the TC framework, when the scheme of the embodiment of the present application is applied to the transmitting side, the scheme may be implemented in a socket (socket) of a protocol stack, which is located at a position relatively forward in a data transmission path, so that occurrence of buffer burst can be suppressed.
In order to make the process of bandwidth allocation more efficient, the management costs of the individual transport channels can be passed on to the control plane (creation and release of transport channels) to ensure a simple and efficient data plane (data transmission). Thereby, a data structure is defined for managing the context information of the transmission channel, which context information comprises at least the time stamp information and the tag information of the corresponding transmission channel. The context information of one transmission channel is stored in each data item in the data structure, when the context information is written in one data item, the state of the data item is occupied, otherwise, the state of the data item can be occupied, and the marking information can be used for marking the state of the data item as occupied, so that the bandwidth allocation of each transmission channel can be independently ensured. Therefore, the bandwidth allocation device provided in the embodiment of the present application may further include a control device, configured to implement a control plane related process.
The control means write the context information of the transmission channel into an unoccupied data item of the data structure when the transmission channel is created. The data structures employed in the actual scenario may be arrays, but any other feasible data structures may be used, such as matrices, data sequences, etc. Each array item of the array is used for storing the context information of one transmission channel, in order to better support the large-scale bandwidth allocation with fine granularity, a multidimensional array can be adopted, and the searching of the corresponding array item is optimized by combining a bitmap algorithm. Because the time stamp information is also stored in the array, the array can be accessed to operate when the time stamp information needs to be updated or the time stamp information needs to be read. Fig. 2 shows a schematic diagram when a transmission channel is created.
In this case, when the state detection device selects the preset number of other transmission channels to detect and determines the transmission channel that is not active, the state detection device may start from the array item corresponding to the current transmission channel and scan a fixed number of array items each time. For example, if the index of the array item corresponding to the current transmission channel is 100, 64 array items with index of 101-164 can be scanned, and the timestamp information in the 64 array items can be obtained to detect the activity state.
In addition, the context information may further include a guaranteed bandwidth value of the transmission channel, where the guaranteed bandwidth value refers to at least a bandwidth quota that needs to be allocated by the transmission channel when transmitting data. For example, if a connection needs to transmit more important information, at least its 256KB bandwidth quota needs to be guaranteed when it needs to transmit data.
When the transmission channel is released, the bandwidth recycling device also recycles the bandwidth quota of the transmission channel and accumulates the bandwidth quota into the global available bandwidth, and meanwhile, the control device deletes the content of the transmission channel in the corresponding data item and updates the state of the data item to be unoccupied, so that a new transmission channel which is created later can use the data item. In addition, the data items of the data structure can also store the bandwidth quota of the corresponding transmission channel, if the information needs to be read or modified, the data items in the data structure can be directly accessed, for example, when the bandwidth quota of the inactive transmission channel is recovered, the corresponding data items can be accessed, and the bandwidth quota of the data items corresponding to the inactive transmission channel is cleared. And when the bandwidth quota is allocated to the current transmission channel, the data item can be directly accessed, and the bandwidth quota is written into the data item corresponding to the current transmission channel. The current number of transmission channels is updated after each creation or release of a transmission channel. Fig. 3 shows a schematic diagram when a transmission channel is created.
When transmitting data, the data transmitting device of the bandwidth control device reads the bandwidth quota from the data item of the current transmission channel in the data structure, then obtains the sum of the flow of the data to be transmitted and the bandwidth occupation of the current transmission channel as a target flow, compares the target flow with the allocated bandwidth quota, and can transmit the data if the target flow does not exceed the bandwidth quota, otherwise, selects retry, delay transmission or discard according to the transmission policy. Therefore, during data transmission, only the bandwidth quota of the corresponding transmission channel needs to be read from the data structure, and the management of the transmission channel is mainly in the control plane, namely, during the creation and release of the transmission channel, the mode ensures the simplicity and the high efficiency of the data plane, and bandwidth can be allocated at the connection level.
Furthermore, portions of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application by way of operation of the computer. Program instructions for invoking the methods of the present application may be stored in fixed or removable recording media and/or transmitted via a data stream in a broadcast or other signal bearing medium and/or stored within a working memory of a computer device operating according to the program instructions. Here, one embodiment according to the present application includes an apparatus as shown in fig. 7, which includes a memory 710 for storing computer program instructions and a processor 720 for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to operate based on the foregoing methods and/or techniques according to embodiments of the present application.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, using Application Specific Integrated Circuits (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application may be executed by a processor to implement the above steps or functions. Likewise, the software programs of the present application (including associated data structures) may be stored on a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. In addition, some steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.
Claims (17)
1. A method of bandwidth allocation, wherein the method comprises:
when the current transmission channel needs to transmit data, detecting the active state of other transmission channels, and determining an inactive transmission channel from the other transmission channels, wherein the inactive transmission channel is: the method comprises the steps that no transmission channel for data transmission is carried out in a preset time period, the current transmission channel is a transmission channel for data transmission currently required, and the other transmission channels are one or more transmission channels except the current transmission channel;
recovering the bandwidth quota of the inactive transmission channel, and accumulating the bandwidth quota into a global available bandwidth, wherein the bandwidth quota of the inactive transmission channel is the bandwidth currently occupied by the inactive transmission channel, and the global available bandwidth is the unoccupied bandwidth in the upper limit of the bandwidth;
and according to the bandwidth quota required to be allocated for transmitting data by the current transmission channel, allocating the bandwidth quota for the current transmission channel from the globally available bandwidth.
2. The method of claim 1, wherein detecting the active state of the other transmission channels and determining the inactive transmission channel from the other transmission channels comprises:
Detecting the active states of a preset number of other transmission channels, and determining the inactive transmission channels in the preset number of other transmission channels.
3. The method of claim 1, wherein detecting the active state of the other transmission channels and determining the inactive transmission channel from the other transmission channels comprises:
acquiring time stamp information corresponding to other transmission channels, wherein the time stamp information is updated when the corresponding transmission channels perform data transmission;
comparing the timestamp information of other transmission channels with the current time to obtain a comparison result;
and if the comparison result exceeds the preset duration, determining the corresponding transmission channel as an inactive transmission channel.
4. A method according to claim 3, wherein the method further comprises:
when a transmission channel is created, writing context information of the transmission channel into one unoccupied data item of a data structure, wherein each occupied data item in the data structure corresponds to one transmission channel, and the context information comprises timestamp information and marking information of the corresponding transmission channel, and the marking information is used for marking the state of the data item as occupied.
5. The method of claim 4, wherein the context information further comprises a guard bandwidth value for the transmission channel, the guard bandwidth value being at least a quota of bandwidth that the transmission channel needs to allocate when transmitting data.
6. The method according to claim 4 or 5, wherein the method further comprises:
and when the transmission channel is released, recovering the bandwidth quota of the transmission channel, accumulating the bandwidth quota into the global available bandwidth, deleting the content of the transmission channel in the corresponding data item, and updating the state of the data item to be unoccupied.
7. The method of claim 4, wherein the data items of the data structure further store bandwidth quotas of corresponding transmission channels;
recovering the bandwidth quota of the inactive transmission channel, comprising:
resetting the bandwidth quota of the data item corresponding to the inactive transmission channel;
allocating bandwidth quota for the current transmission channel, including:
writing the bandwidth quota of the data item corresponding to the current transmission channel.
8. The method of claim 7, wherein after allocating the bandwidth quota for the current transmission channel, further comprising:
Reading the bandwidth quota of the current transmission channel from the data item of the current transmission channel in the data structure;
acquiring the sum of the flow of data to be transmitted and the bandwidth occupation of a current transmission channel as a target flow;
and transmitting the data when the target flow does not exceed the bandwidth quota.
9. A bandwidth allocation apparatus, wherein the apparatus comprises:
the state detection device is used for detecting the active states of other transmission channels when the current transmission channel needs to transmit data, and determining an inactive transmission channel from the other transmission channels, wherein the inactive transmission channel is: the method comprises the steps that no transmission channel for data transmission is carried out in a preset time period, the current transmission channel is a transmission channel for data transmission currently required, and the other transmission channels are one or more transmission channels except the current transmission channel;
the bandwidth recycling device is used for recycling the bandwidth quota of the inactive transmission channel and accumulating the bandwidth quota into the overall available bandwidth, wherein the bandwidth quota of the inactive transmission channel is the bandwidth currently occupied by the inactive transmission channel, and the overall available bandwidth is the unoccupied bandwidth in the upper limit of the bandwidth;
And the bandwidth allocation device is used for allocating the bandwidth quota for the current transmission channel from the globally available bandwidth according to the bandwidth quota required to be allocated for the data transmission of the current transmission channel.
10. The apparatus of claim 9, wherein the status detecting means is configured to detect an active status of a preset number of other transmission channels, and determine an inactive transmission channel among the preset number of other transmission channels.
11. The apparatus of claim 9, wherein the status detecting means is configured to obtain timestamp information corresponding to other transmission channels, where the timestamp information is updated when data is transmitted by the corresponding transmission channel; comparing the timestamp information of other transmission channels with the current time to obtain a comparison result; and if the comparison result exceeds the preset duration, determining the corresponding transmission channel as an inactive transmission channel.
12. The apparatus of claim 11, wherein the apparatus further comprises:
and the control device is used for writing the context information of the transmission channel into one unoccupied data item of the data structure when the transmission channel is created, wherein each occupied data item in the data structure corresponds to one transmission channel, the context information comprises timestamp information and marking information of the corresponding transmission channel, and the marking information is used for marking the state of the data item as occupied.
13. The device of claim 12, wherein the context information further comprises a guard bandwidth value for the transmission channel, the guard bandwidth value being at least a quota of bandwidth that the transmission channel needs to allocate when transmitting data.
14. The apparatus according to claim 12 or 13, wherein the bandwidth reclamation means is further configured to, when a transmission channel is released, reclaim a bandwidth quota of the transmission channel, and accumulate into a globally available bandwidth;
the control device is further configured to delete content of the transmission channel in a corresponding data item when the transmission channel is released, and update a state of the data item to be unoccupied.
15. The device of claim 12, wherein the data items of the data structure further store bandwidth quotas of corresponding transmission channels;
the control device is further configured to clear the bandwidth quota of the data item corresponding to the inactive transmission channel, and write the bandwidth quota of the data item corresponding to the current transmission channel.
16. The apparatus of claim 15, wherein the apparatus further comprises:
the data transmission device is used for reading the bandwidth quota of the current transmission channel from the data item of the current transmission channel in the data structure after the bandwidth quota is allocated to the current transmission channel, and obtaining the sum of the flow of the data to be transmitted and the bandwidth occupation of the current transmission channel as a target flow; and transmitting the data when the target traffic does not exceed the bandwidth quota.
17. A bandwidth allocation apparatus, wherein the apparatus comprises:
a processor; and
a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711078855.6A CN109756429B (en) | 2017-11-06 | 2017-11-06 | Bandwidth allocation method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711078855.6A CN109756429B (en) | 2017-11-06 | 2017-11-06 | Bandwidth allocation method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109756429A CN109756429A (en) | 2019-05-14 |
| CN109756429B true CN109756429B (en) | 2023-05-02 |
Family
ID=66399537
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711078855.6A Active CN109756429B (en) | 2017-11-06 | 2017-11-06 | Bandwidth allocation method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109756429B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110147109A (en) * | 2019-05-21 | 2019-08-20 | 重庆长安汽车股份有限公司 | A kind of archetype development system of automated driving system |
| CN112804084B (en) * | 2020-12-28 | 2022-09-13 | 中金数据（武汉）超算技术有限公司 | Bandwidth management method and device based on network interconnection line |
| CN114598649B (en) * | 2022-03-17 | 2023-08-04 | 苏州浪潮智能科技有限公司 | Kernel bandwidth management method, device, storage equipment and storage medium |
| CN114884823B (en) * | 2022-04-29 | 2024-03-22 | 北京有竹居网络技术有限公司 | Flow congestion control method, device, computer readable medium and electronic equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7257632B2 (en) * | 2001-07-30 | 2007-08-14 | Fujitsu Limited | Method and apparatus for a bandwidth broker in a packet network |
| CN101599908B (en) * | 2009-07-15 | 2011-11-16 | 杭州华三通信技术有限公司 | Method and equipment for automatically distributing bandwidth |
| CN106533982B (en) * | 2016-11-14 | 2019-05-21 | 西安电子科技大学 | The dynamic queue's dispatching device and method borrowed based on bandwidth |
-
2017
- 2017-11-06 CN CN201711078855.6A patent/CN109756429B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109756429A (en) | 2019-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109756429B (en) | Bandwidth allocation method and device | |
| US10257282B2 (en) | Lock management method in cluster, lock server, and client | |
| US7945819B2 (en) | Message state maintenance at a message log | |
| EP3230868B1 (en) | Multiple transaction logs in a distributed storage system | |
| US10838777B2 (en) | Distributed resource allocation method, allocation node, and access node | |
| US5446901A (en) | Fault tolerant distributed garbage collection system and method for collecting network objects | |
| AU2006262163B2 (en) | Managing message queues | |
| CN103493449A (en) | Effective circuits in packet-switched networks | |
| CN103885895A (en) | Write Performance in Fault-Tolerant Clustered Storage Systems | |
| WO2019024910A1 (en) | Storage resource recovery method, apparatus and system | |
| US6886031B2 (en) | Efficient connection and memory management for message passing on a single SMP or a cluster of SMPs | |
| CN101751470B (en) | System for storing and/or retrieving a data-set and method thereof | |
| CN113179327B (en) | High concurrency protocol stack unloading method, equipment and medium based on large-capacity memory | |
| CN110704214A (en) | Inter-process communication method and device | |
| CN111291062A (en) | Data synchronous writing method and device, computer equipment and storage medium | |
| CN106571935B (en) | Resource scheduling method and equipment | |
| CN108958660B (en) | Distributed storage system and data processing method and device thereof | |
| KR101499890B1 (en) | Low Latency Framework System | |
| CN116932273A (en) | Function level reset processing method and device used in RDMA and storage medium | |
| CN111400241B (en) | Data reconstruction method and device | |
| CN112948501B (en) | Data analysis method, device and system | |
| US20150026126A1 (en) | Method of replicating data in asymmetric file system | |
| CN114237489B (en) | Method and device for writing logic resources into SMR disk, electronic equipment and storage medium | |
| CN116827880B (en) | Cache space management method and device | |
| CN109376014B (en) | Distributed lock manager implementation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |