CN114598657A - Data transmission method, system and medium based on two-layer multi-path flow control - Google Patents
Data transmission method, system and medium based on two-layer multi-path flow control Download PDFInfo
- Publication number
- CN114598657A CN114598657A CN202210095713.5A CN202210095713A CN114598657A CN 114598657 A CN114598657 A CN 114598657A CN 202210095713 A CN202210095713 A CN 202210095713A CN 114598657 A CN114598657 A CN 114598657A
- Authority
- CN
- China
- Prior art keywords
- data
- receiving
- quota
- path
- paths
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/20—Traffic policing
Abstract
The invention provides a data transmission method, a system and a medium based on two-layer multipath flow control, wherein the method comprises the following steps: when a plurality of paths exist between the sending device and the receiving device, the receiving device reports the receivable data quota on the available paths to the sending device through the selected available paths; the sending equipment selects one or more paths according to the situation and sends the data within the data quota of the selected available path; after receiving the partial data, the receiving device confirms the received data quantity to the sending device and confirms the new receivable data quota on the selected available path; and after receiving the new receivable data quota on the selected available path, the sending equipment releases the quota with the corresponding size on the path, and performs path selection again and sends the data until all data are sent. The invention realizes the data transmission without packet loss and the maximum application of network bandwidth by controlling and coordinating the transmission rate of the remote zero-copy task on a plurality of paths.
Description
Technical Field
The invention relates to the technical field of data transmission, in particular to a data transmission method, a data transmission system and a data transmission medium based on two-layer multipath flow control.
Background
The remote zero-copy technology can realize the rapid transmission of data among network nodes, the whole transmission process does not need the intervention of an operating system, a CPU (central processing unit) and a network software protocol stack, the low-delay and high-bandwidth data transmission is realized through the direct transmission of memory data, and the computing resources of the nodes are not consumed. A representative technology currently providing Remote zero-copy capability is Remote Direct Memory Access (RDMA). There are three different hardware implementations of RDMA. InfiniBand, iWarp (internet wire Area RDMA Protocol), RoCE (RDMA over converted Ethernet), respectively. Infiniband is a network specially designed for RDMA, and ensures reliable transmission from a hardware level, while RoCE and iWARP are both RDMA technologies based on Ethernet and support corresponding verbs interfaces. Ethernet technology has found wider application in practical deployment due to its lower cost and faster development.
Standard RDMA technology supports only single-path data transmission, and cannot maximally utilize multipath resources in a network environment having a large number of equal-cost multipaths, such as a data center network, to improve transmission performance. There are several schemes for providing multipath support for RDMA, wherein the performance and flexibility of the two-layer multipath scheme should be the highest, because the coupling degree of the network layer and the transport layer of the upper layer in the scheme is the smallest, the scheme can adapt to the most widely applied scenarios, and the scheme can be extended to other non-RDMA remote zero-copy schemes.
In addition to solving the problem of path selection, the transmission of network data also needs to solve the problem of flow control in the transmission process. However, the flow control scheme in the RDMA standard only considers a single-path scenario and does not explicitly specify a flow control mechanism in a multi-path scenario; some RDMA multipath schemes also propose a flow control scheme under a multipath scene, but the multipath scheme is constructed on a transmission layer, so the flow control scheme is not suitable for a two-layer multipath scene.
Two types of flow control schemes used in the prior art are described below.
One type of existing flow control technical scheme uses a credit-based mechanism, a receiving node sends a data receiving credit line to a sending node, and the sending node cannot send data beyond a specified line unless the data is received and the line is released to continue sending. The scheme discusses the specific flow of the data sending node and the receiving node for communication and use of the quota and the specific behavior of the node under different quota conditions in detail.
However, the scheme is mainly designed for a single-path transmission scene, and a multi-path scene is not considered. When a sending node can transmit data to a receiving node through multiple paths, the data is not necessarily distributed evenly on the paths, but the transmission rates on different paths may be different, so the flow control mechanisms on the paths should be related, and must be considered as a whole to achieve the optimal. The solution does not have the capability of comprehensively coordinating the flow control of multiple paths.
And the other type of the RDMA multi-path scheme is provided, and simultaneously, a corresponding flow control scheme is designed. The scheme employs a flow control mechanism based on a congestion control window. This mechanism is very similar to the flow control method of TCP, where a sliding window is maintained at the sending node to control the size of the unacknowledged data sent. The receiving node may send an acknowledgement message to the sending node to feed back the size of the received data to free up space for a partial sliding window to transmit more data. Since the ECN signal in the network is also carried in the acknowledgement, the sending node can thereby adjust the size of the sliding window according to the state of the receiving node and the network, in a manner substantially consistent with the behavior of linear increase multiplicative decrease (AIMD) in TCP.
The multipath scheme proposed by the scheme is built on a network layer and a transmission layer, so that the flow control mechanism which supports the multipath is also very close to a typical transmission layer protocol (TCP protocol). The mechanism relies on feedback with the ECN label of the network layer and is not applicable to two-layer multipath scenarios below the network layer. In addition, the mechanism based on the sliding window is too complex for the two-layer link layer, and the hardware implementation cost is too high, so that the practicability of the scheme is influenced.
Disclosure of Invention
The invention mainly aims to provide a data transmission method, a system and a medium based on two-layer multipath flow control, aiming at realizing data transmission without packet loss and maximum application of network bandwidth by controlling and coordinating the transmission rate of a remote zero-copy task on a plurality of paths.
In order to achieve the above object, the present invention provides a data transmission method based on two-layer multipath flow control, including the following steps:
when a plurality of paths exist between the sending device and the receiving device, the receiving device reports the receivable data quota on the selected available paths to the sending device through all the selected available paths;
the sending equipment selects one or more paths according to the situation and sends the data within the data quota of the selected available path;
after receiving part of data, the receiving device confirms the received data quantity to the sending device and confirms the new receivable data quota on the selected available path;
and after receiving the new receivable data quota on the selected available path, the sending equipment releases the quota with the corresponding size on the path, and performs path selection again and sends data until all data are sent.
A further technical solution of the present invention is that the step of the sending device selecting one or more paths according to circumstances and sending data within the data quota on the selected available path includes:
the sending equipment searches the feedback record from the receiving equipment and determines whether all available paths with non-zero receiving quota still exist;
if yes, the sending equipment allocates the data to be sent to available paths according to the receiving quota of each path.
A further technical solution of the present invention is that, after receiving a part of data, the receiving device confirms the received data amount to the sending device, and confirms a new receivable data quota on the selected available path, including:
after receiving the data, the receiving device respectively records the used receiving quota of each path and stores the data in the memory designated area of the receiving device;
and the receiving equipment redistributes the receiving quota released by the received data to each path according to the processing condition of the data and feeds the receiving quota back to the sending equipment through the corresponding path.
A further technical solution of the present invention is that, after receiving the new receivable data quota on the selected available path, the sending device releases the quota of the corresponding size on the path, and performs path selection again and sends data until all data are sent, including:
and after the sending equipment receives the feedback of the receiving equipment, updating the receiving quota of each available path, and if data still remain to be sent, using the newly released quota to continue sending the data.
A further technical solution of the present invention is that, the step of allocating, by the sending device, data to be sent to available paths based on the receiving quota of each path of the data to be sent includes:
if the sum of the receiving limits of all available paths is less than or equal to the data to be sent, the limits on all available paths are completely used, and the sending is suspended to wait for the confirmation of the receiving node;
and if the sum of the receiving quota of all available paths is larger than the data to be transmitted, selecting the paths according to the sequence of the data quota from large to small to transmit all the data at one time.
The further technical scheme of the invention is that the method also comprises the following steps:
when the receiving equipment receives a network topology change notification returned by a two-layer multi-path detection module and an updated available path list, the receiving equipment compares the updated list with a current list;
if the network state change does not cause the available multipath to change, no action is required;
if the available paths are reduced due to the change of the network state, sending a notice to the sending node through the remaining available paths, and canceling the receiving data quota on the invalid path;
and if the available paths are increased due to the change of the network state, considering the total processing upper limit of the receiving equipment, allocating reasonable receiving data quota for the new paths, and feeding back the quota to the sending equipment through the paths in time.
In order to achieve the above object, the present invention further provides a data transmission system based on two-layer multipath flow control, which includes a memory, a processor, and a data transmission program based on two-layer multipath flow control stored on the processor, where the data transmission program based on two-layer multipath flow control is executed by the processor to perform the steps of the method described above.
In order to achieve the above object, the present invention further provides a computer-readable storage medium, where a data transmission program based on two-layer multi-path flow control is stored, and when executed by a processor, the data transmission program based on two-layer multi-path flow control performs the steps of the method as described above.
The data transmission method, the data transmission system and the data transmission storage medium based on the two-layer multi-path flow control have the advantages that: compared with a multi-path flow control algorithm based on a sliding congestion window, the method has the advantages of low computational complexity, no dependence on specific network layer and transport layer protocols and the like; compared with the traditional credit-based two-layer flow control algorithm, the invention expands the support to multi-path, so that the transmission flow and the speed on a plurality of paths can be coordinated comprehensively between the sending node and the receiving node, and the overall transmission optimization and the most utilization of network resources are achieved.
Drawings
Fig. 1 is a flowchart illustrating a data transmission method based on two-layer multi-path flow control according to a preferred embodiment of the present invention.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The invention provides a data transmission method based on two-layer multipath flow control, which is mainly characterized in that a novel two-layer multipath flow control method is adopted, the method is used for a remote zero-copy transmission scene in an Ethernet environment, and the data transmission without packet loss and the maximum application of network bandwidth are realized by controlling and coordinating the transmission rate of a remote zero-copy task on a plurality of paths.
The technical problem to be solved by the data transmission method based on the two-layer multi-path flow control is that the technical key points mainly comprise the following two aspects:
1) mechanism for allocation of reception quota over multipath: the sending node and the receiving node need to coordinate the transmission capability of the data without packet loss on the transmission path, and the receiving node needs to inform the sending node of the data receiving allowance allowed by the receiving node, so as to avoid the sending node sending the data exceeding the processing capability of the receiving node. Due to the existence of multiple paths, the receiving node needs to consider whether the sum of the receiving capabilities of multiple receiving ports exceeds the data processing capability of the receiving node or not, and the processing capability is reasonably distributed on each path, besides the influence factors such as the size of each receiving port buffer area and the like which limit the receiving capability of the ports;
2) dynamic multipath flow control in data transmission; when there are multiple paths available, the sending node needs to consider which paths to select and how much data to send on each path, so as to achieve the maximum utilization of network resources. Because the receiving quota fed back from different paths may be different, the transmission delay may continuously change due to different states of the intermediate switches on each path, and the sending node needs to control the traffic on each path in real time according to the current situation, so as to achieve the overall optimal transmission process.
Specifically, as shown in fig. 1, the preferred embodiment of the data transmission method based on two-layer multipath flow control of the present invention includes the following steps:
in step S10, when a plurality of paths exist between the sending apparatus and the receiving apparatus, the receiving apparatus reports the receivable data quota on the selected available path to the sending apparatus through all the selected available paths.
And step S20, the sending equipment selects one or more paths according to the situation and sends the data within the data quota of the selected available path.
Step S30, after receiving the partial data, the receiving device confirms the received data amount and confirms the new receivable data quota on the selected available path to the sending device.
Step S40, after receiving the new receivable data quota on the selected available path, the sending device releases the quota with the corresponding size on the path, and performs path selection again and sends data until all data are sent.
Further, the step of the sending device selecting one or more paths according to the situation and sending the data within the data quota of the selected available path includes:
the sending device looks up the feedback record from the receiving device to determine if there are still all available paths with non-zero reception quota.
If yes, the sending equipment allocates the data to be sent to available paths according to the receiving quota of each path.
Further, after receiving the partial data, the receiving device confirms the received data amount to the sending device, and confirms the new receivable data quota on the selected available path, including:
and after receiving the data, the receiving equipment respectively records the used receiving quota of each path and stores the data into the memory designated area of the receiving equipment.
And the receiving equipment redistributes the receiving quota released by the received data to each path according to the processing condition of the data and feeds the receiving quota back to the sending equipment through the corresponding path.
Further, after receiving the new receivable data quota on the selected available path, the sending device releases the quota of the corresponding size on the path, and performs path selection again and sends data until all data are sent, including:
and after receiving the feedback of the receiving equipment, the sending equipment updates the receiving quota of each available path, and if data still remain to be sent, the newly released quota is used for continuously sending the data.
Further, the step of allocating, by the sending device, the data to be sent to available paths according to the receiving quota of each path of the data to be sent includes:
and if the sum of the receiving limits of all available paths is less than or equal to the data to be sent, completely using the limits on all available paths, and suspending the sending to wait for the confirmation of the receiving node.
And if the sum of the receiving quota of all available paths is larger than the data to be transmitted, selecting the paths according to the sequence of the data quota from large to small to transmit all the data at one time.
Further, the preferred embodiment of the data transmission method based on two-layer multipath flow control of the present invention further includes:
and when the receiving equipment receives a network topology change notification returned by the two-layer multi-path detection module and the updated available path list, the receiving equipment compares the updated list with the current list.
If the network state change does not result in a change in the available multipaths, no action is required.
And if the available paths are reduced due to the change of the network state, sending a notice to the sending node through the rest available paths, and canceling the receiving data quota on the invalid paths.
And if the available paths are increased due to the change of the network state, considering the total processing upper limit of the receiving equipment, allocating reasonable receiving data quota for the new paths, and feeding back the quota to the sending equipment through the paths in time.
In summary, the main technical key point of the data transmission method based on two-layer multipath flow control of the present invention is to provide a new flow control scheme under two-layer multipath environment, which includes the following three points:
1. providing a flow control method based on receiving quota for comprehensively and comprehensively coordinating multiple path resources;
2. providing a distribution method for optimizing and configuring an acceptance limit among receiving ports of receiving nodes according to the total upper limit of the processing capacity of the receiving nodes;
3. a method for dynamically selecting and distributing transmission data on a plurality of paths by a sending node according to network and receiving node feedback is provided.
The data transmission method based on the two-layer multi-path flow control can be used for constructing a distributed storage system, effectively improving the bandwidth utilization efficiency of remote zero-copy application to a plurality of paths, improving the overall transmission speed on the premise of not losing packets, constructing a distributed online processing system and realizing high-speed transmission and state synchronization of data among multiple nodes.
The data transmission method based on the two-layer multi-path flow control has the beneficial effects that: compared with a multi-path flow control algorithm based on a sliding congestion window, the method has the advantages of low calculation complexity, no dependence on specific network layer and transport layer protocols and the like; compared with the traditional credit-based two-layer flow control algorithm, the invention expands the support to multi-path, so that the transmission flow and the speed on a plurality of paths can be coordinated comprehensively between the sending node and the receiving node, and the overall transmission optimization and the most utilization of network resources are achieved.
In order to achieve the above object, the present invention further provides a data transmission system based on two-layer multipath flow control, where the system includes a memory, a processor, and a data transmission program based on two-layer multipath flow control stored on the processor, and when the data transmission program based on two-layer multipath flow control is executed by the processor, the steps of the method described in the above embodiments are executed, which are not described herein again.
In order to achieve the above object, the present invention further provides a computer-readable storage medium, where a data transmission program based on two-layer multipath flow control is stored, and when the data transmission program based on two-layer multipath flow control is executed by a processor, the steps of the method are executed, which are not described herein again.
The above description is only for the preferred embodiment of the present invention and is not intended to limit the scope of the present invention, and all equivalent structures or flow transformations made by the present specification and drawings, or applied directly or indirectly to other related arts, are included in the scope of the present invention.
Claims (8)
1. A data transmission method based on two-layer multipath flow control is characterized by comprising the following steps:
when a plurality of paths exist between the sending device and the receiving device, the receiving device reports the receivable data quota on the selected available paths to the sending device through all the selected available paths;
the sending equipment selects one or more paths according to conditions and sends data within the data quota of the selected available path;
after receiving the partial data, the receiving equipment confirms the received data quantity to the sending equipment and confirms the new receivable data quota on the selected available path;
and after receiving the new receivable data quota on the selected available path, the sending equipment releases the quota with the corresponding size on the path, and performs path selection again and sends data until all data are sent.
2. The method as claimed in claim 1, wherein the step of the sending device selecting one or more paths according to the situation and sending the data within the data quota of the selected available path comprises:
the sending equipment searches the feedback record from the receiving equipment and determines whether all available paths with non-zero receiving quota still exist;
if yes, the sending equipment allocates the data to be sent to available paths according to the receiving quota of each path.
3. The method as claimed in claim 2, wherein the step of confirming the received data amount and confirming the new receivable data quota on the selected available path to the sending device after the receiving device receives the partial data comprises:
after receiving the data, the receiving device respectively records the used receiving quota of each path and stores the data in the memory designated area of the receiving device;
and the receiving equipment redistributes the receiving quota released by the received data to each path according to the processing condition of the data and feeds the receiving quota back to the sending equipment through the corresponding path.
4. The data transmission method based on two-layer multipath flow control as claimed in claim 3, wherein the step of releasing the quota of corresponding size on the selected available path after the transmitting device receives the new receivable data quota on the path, and performing path selection again and transmitting data until all data are transmitted includes:
and after receiving the feedback of the receiving equipment, the sending equipment updates the receiving quota of each available path, and if data still remain to be sent, the newly released quota is used for continuously sending the data.
5. The method for data transmission based on two-layer multipath flow control according to claim 2, wherein the step of allocating the receiving quota of each path of the data to be transmitted to the available paths by the transmitting device comprises:
if the sum of the receiving limits of all available paths is less than or equal to the data to be sent, the limits on all available paths are completely used, and the sending is suspended to wait for the confirmation of the receiving node;
and if the sum of the receiving quota of all available paths is larger than the data to be transmitted, selecting the paths according to the sequence of the data quota from large to small to transmit all the data at one time.
6. The method for data transmission based on two-layer multi-path flow control according to claim 1, further comprising:
when the receiving equipment receives a network topology change notification returned by a two-layer multi-path detection module and an updated available path list, the receiving equipment compares the updated list with a current list;
if the network state change does not cause the available multipath to change, no action is required;
if the available paths are reduced due to the change of the network state, sending a notice to the sending node through the remaining available paths, and canceling the receiving data quota on the invalid path;
and if the available paths are increased due to the change of the network state, considering the total processing upper limit of the receiving equipment, allocating reasonable receiving data quota for the new paths, and feeding back the quota to the sending equipment through the paths in time.
7. A data transmission system based on two-layer multipath flow control, the system comprising a memory, a processor, and a data transmission program based on two-layer multipath flow control stored on the processor, wherein the data transmission program based on two-layer multipath flow control is executed by the processor to perform the steps of the method according to any one of claims 1 to 6.
8. A computer-readable storage medium, wherein the computer-readable storage medium stores a data transmission program based on two-layer multipath flow control, and the data transmission program based on two-layer multipath flow control is executed by a processor to perform the steps of the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210095713.5A CN114598657A (en) | 2022-01-26 | 2022-01-26 | Data transmission method, system and medium based on two-layer multi-path flow control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210095713.5A CN114598657A (en) | 2022-01-26 | 2022-01-26 | Data transmission method, system and medium based on two-layer multi-path flow control |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114598657A true CN114598657A (en) | 2022-06-07 |
Family
ID=81806453
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210095713.5A Pending CN114598657A (en) | 2022-01-26 | 2022-01-26 | Data transmission method, system and medium based on two-layer multi-path flow control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114598657A (en) |
-
2022
- 2022-01-26 CN CN202210095713.5A patent/CN114598657A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6526825B2 (en) | Method and apparatus for transmitting transmission control protocol TCP data packets, and system | |
| CN110191065B (en) | High-performance load balancing system and method based on software defined network | |
| US8576875B2 (en) | Systems and methods of improving performance of transport protocols in a multi-path environment | |
| US9088511B2 (en) | Multi-hop error recovery | |
| US5802310A (en) | Systems and methods for data channel queue control in a communications network | |
| CN103476062B (en) | Data flow scheduling method, equipment and system | |
| US20210320820A1 (en) | Fabric control protocol for large-scale multi-stage data center networks | |
| CN110662256B (en) | Multi-path cross-protocol transmission data packet scheduling method and system | |
| CN107396396B (en) | Data transmission management method supporting multi-source multipath | |
| US20210297350A1 (en) | Reliable fabric control protocol extensions for data center networks with unsolicited packet spraying over multiple alternate data paths | |
| US20210297351A1 (en) | Fabric control protocol with congestion control for data center networks | |
| US20070226347A1 (en) | Method and apparatus for dynamically changing the TCP behavior of a network connection | |
| EP3739784A1 (en) | Data packet sending method and related device | |
| US20220104066A1 (en) | Systems and methods for congestion control on mobile edge networks | |
| WO2021244450A1 (en) | Communication method and apparatus | |
| CN104683259A (en) | TCP congestion control method and device | |
| CN113472646A (en) | Data transmission method, node, network manager and system | |
| CN111194090B (en) | Edge calculation-oriented multi-strategy channel allocation method | |
| CN115134304B (en) | Self-adaptive load balancing method for avoiding data packet disorder of cloud computing data center | |
| CN114598657A (en) | Data transmission method, system and medium based on two-layer multi-path flow control | |
| US20210297343A1 (en) | Reliable fabric control protocol extensions for data center networks with failure resilience | |
| JP2933905B1 (en) | Multicast communication system | |
| CN115733799A (en) | Network congestion control method and related device | |
| CN113612698A (en) | Data packet sending method and device | |
| US20200145478A1 (en) | Method, electronic device, and computer program product for handling congestion of data transmission |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication |