CN117891626A - Remote direct data access method and related equipment - Google Patents

Abstract

The embodiment of the disclosure provides a remote direct data access method, a remote direct data access device, computer equipment and a readable storage medium, and relates to the technical field of computers. The method comprises the following steps: constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and adopting different strategies to the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels. The method provided by the embodiment of the disclosure can realize the fine control of remote direct data access.

Description

Remote direct data access method and related equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a remote direct data access method, a remote direct data access device, a computer device, and a readable storage medium.

Background

Currently, the importance of efficient network operations is increasing when running large numbers of data-intensive applications (such as big data analysis, machine learning, and scientific simulation) in a data center or high performance computing (HPC, high Performance Computing) environment. In a data center network (DCN, DATA CENTER network), remote Direct Memory Access (RDMA) is one of the most promising network technologies, applicable to data-intensive applications requiring high bandwidth and ultra-low latency. RDMA supports zero-copy read/write operations by implementing transfer logic in a hardware Network Interface Card (NIC) that transfers data directly from the memory of one compute node to the memory of another compute node. By eliminating the involvement of the operating system, RDMA can achieve high throughput and low latency. Modern DCNs typically utilize multipath transmissions to enable data center applications to achieve high network throughput and to improve reliability against sudden network changes such as congestion or link failure. RDMA providers rarely consider the approach of utilizing multipath network resources in DCNs because RDMA was originally designed based on a single-path transmission to deliver data along a single network path. This results in applications on RDMA often suffering significant performance degradation and being unable to respond quickly to dynamic network state changes. Currently, there is a multi-path transmission method MP (multi path) -RDMA that can manage multiple virtual paths and forward each packet through the virtual paths, but it has compatibility limitations and requires modification of the hardware NIC. The multipath RDMA method implemented by the user layer can support commercial RDMA NICs, so that the multipath method of the user layer becomes a promising and easy-to-implement multipath RDMA transmission method.

Disclosure of Invention

The embodiment of the disclosure provides a remote direct data access method, a remote direct data access device, computer equipment and a readable storage medium, and relates to the technical field of computers.

The embodiment of the disclosure provides a remote direct data access method, which comprises the following steps: constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and adopting different strategies to the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels.

In one embodiment, separating the streams transmitted between the local host and the remote host into large and small streams includes: setting a first quantity threshold of data packets; dividing a stream, in which the number of data packets included in the stream is greater than the first number threshold, into the large stream; and dividing the stream with the number of the data packets less than or equal to the first number threshold into the small streams.

In one embodiment, taking the large flow and the small flow into different policies to complete the transfer between the local host and the remote host through the N queue pair channels includes: transmitting each of the streamlets through a priority path; and transmitting the large stream through M paths in the priority path and the common path, wherein M is less than or equal to N.

In one embodiment, transmitting the large stream through M paths of the priority path and the normal path includes: dividing the large stream into M substreams; and completing transmission between the local host and the remote host through the M paths by the M sub-streams.

In one embodiment, completing the transmission between the local host and the remote host over the M paths includes: acquiring the transmission rate of each sub-stream in the M paths; acquiring the residual transmission time of each sub-stream according to the transmission rate of each sub-stream and the residual transmission data of each sub-stream; all or part of the data of the sub-stream with more remaining time is distributed to the paths with completed transmission or less remaining transmission time for transmission.

In one embodiment, the method further comprises: when the transmission rate of one sub-stream is zero, all the data of the one sub-stream is distributed to paths with completed transmission or less residual transmission time for transmission.

In one embodiment, the method further comprises: reducing the first number threshold when the ratio of the number of small flows to the number of large flows obtained by dividing according to the first number threshold is greater than a first ratio, so that the ratio of the number of small flows to the number of large flows is equal to the first ratio; the first number threshold is dynamically updated in subsequent transmissions.

The disclosed embodiments provide a remote direct data access apparatus, comprising: the system comprises a construction unit, a control unit and a control unit, wherein the construction unit is used for constructing N queue pair channels between a local host and a remote host, N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; a classification unit for dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and the strategy unit is used for adopting different strategies to the large flow and the small flow to complete transmission between the local host and the remote host through the N queue pair channels.

The embodiment of the disclosure provides a computer device, which comprises a processor, a memory and an input-output interface; the processor is respectively connected with the memory and the input/output interface, wherein the input/output interface is used for receiving data and outputting data, the memory is used for storing a computer program, and the processor is used for calling the computer program so that the computer device executes the method according to any one of the embodiments.

Embodiments of the present disclosure provide a computer readable storage medium storing a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of the above embodiments.

The remote direct data access method comprises the steps of constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; dividing a flow (F) transmitted between the local host and the remote host into a Large Flow (LF) and a Small Flow (SF); and adopting different strategies for the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels, so that the fine control of remote direct data access can be realized.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.

FIG. 1 illustrates a schematic diagram of an exemplary system architecture to which the remote direct data access method of embodiments of the present disclosure may be applied;

FIG. 2 is a flow chart of a remote direct data access method provided by an embodiment of the present disclosure;

FIG. 3 is a flow chart of a method for separating streams transmitted between the local host and the remote host into large streams and small streams provided by an embodiment of the present disclosure;

FIG. 4 is a flow chart of a method of completing a transfer between the local host and the remote host through the N queue pair channels using different policies for the large flow and the small flow provided by embodiments of the present disclosure;

Fig. 5 is a flowchart of a method for transmitting the large stream through M paths in the priority path and the normal path according to an embodiment of the present disclosure;

FIG. 6 is a flow chart of a method for completing a transfer between the local host and the remote host over the M paths provided by an embodiment of the present disclosure;

FIG. 7 illustrates a framework diagram of a remote direct data access method to which one embodiment of the present disclosure is applied;

FIG. 8 illustrates a framework diagram of a remote direct data access method to which one embodiment of the present disclosure is applied;

FIG. 9 is a block diagram of a remote direct data access device according to an embodiment of the present disclosure;

Fig. 10 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.

Detailed Description

The following description of the technical solutions in the embodiments of the present disclosure will be made clearly and completely with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are only some embodiments of the present disclosure, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without inventive effort, based on the embodiments in this disclosure are intended to be within the scope of this disclosure.

In an embodiment of the disclosure, N queue pair channels may be constructed between a local host and a remote host based on remote direct data access (RDMA, remote Direct Memory Access) technology, where N is an integer greater than or equal to 2, the N queue pair channels including a priority path and a normal path; dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and adopting different strategies for the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels, so that fine control of remote direct data access can be realized.

The following first describes some terms of the present disclosure:

RDMA (Remote Direct Memory Access, remote direct data Access), in order to solve the delay of server-side data processing in network transmission, RDMA transfers data directly into the storage area of a computer through the network, and moves data from a system to the remote system memory rapidly without any influence on the operating systems on both sides, so that the processing function of how many computers are needed is not needed. It eliminates the overhead of external memory copying and context switching, thus freeing up memory, bandwidth and CPU cycles for improved application system performance.

WQ (Work Queue), a Queue storing Work requests, is one of the most important concepts in RDMA technology.

WQEs (Work Queue Element, work queue elements), each operation (e.g., send, receive, or RDMA read/write) is described by a WQE in RDMA communication. Each WQE contains all the information needed to perform a particular operation, such as: the location, size, remote key (used to authenticate remote memory access), etc. of the data, WQE is the data structure describing a single operation in RDMA communication.

QP (Queue Pair), queue Pair, is the basic building block for RDMA communication. Each QP contains two work queues: a Send Work Queue (SWQ) and a receive work queue (Receive Work Queue, RWQ). In RDMA communication, each connection between two endpoints is represented by a QP. Thus, when a connection is established between two RDMA devices, they are actually establishing QP. One edge is a queue.

ECMP (Equal-Cost Multi-Path) is a strategy in network routing to distribute traffic among multiple paths with the same Cost, thereby providing load balancing and redundancy. ECMP allows routers to evenly distribute traffic among multiple paths of equal cost, providing load balancing and redundancy. When a router has multiple identical cost paths to a destination, it does not select only one path to send all packets, but uses ECMP policies to distribute packets among the paths. ECMP can improve network availability and fault tolerance. If one of the paths fails, traffic may automatically switch to the other available path.

The user layer, which refers to a hierarchy in a computer system, refers to the portion above the operating system kernel, consisting essentially of user processes and applications in user space. The user layer is a concept in operating system design for distinguishing the hierarchy corresponding to the kernel layer (core layer).

A flow (flow) generally refers to a set of network packets having the same source IP address, destination IP address, source port, and destination port. ECMP (Equal-Cost multi path) is a stream-based load balancing technique that uses multiple equivalent paths (with the same Cost) on routers to distribute traffic. Large flows (Big flows), which generally refer to a set of flows in a network that have the same characteristics or identities. This particular traffic may be referred to as "large flow" due to its large data volume, high transmission rate, or other particular characteristics. This term is more focused on the number and nature of streams than on the content of the streams.

Embodiments of the present disclosure provide solutions that relate to remote direct data access, RDMA, and the like.

Fig. 1 shows a schematic diagram of an exemplary system architecture 100 in which the remote direct data access method of embodiments of the present disclosure may be applied.

As shown in fig. 1, a system architecture 100 may include a local host farm 101 and a remote host farm 102.

It should be understood that the number of servers in the local host farm 101 and the remote host farm 102 in fig. 1 is merely illustrative. There may be any number of servers, as desired for implementation. The connection between the local host group 101 and the remote host group 102 may be a wired optical fiber connection or a wireless connection, which is not limited in this disclosure, as long as data transmission is possible.

The servers mentioned above may be cloud servers providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms, or may be independent physical servers, or may be server clusters or distributed systems formed by multiple physical servers.

Optionally, the data related to the embodiments of the present disclosure may be stored in a cloud platform, or may be stored based on a cloud storage technology and a blockchain technology, which is not limited herein.

In multipath RDMA transmission, ECMP is a common routing strategy for modern data centers and large-scale network environments in order to avoid overload and congestion of some paths while ensuring that other paths are not idle, improving network utilization. ECMP aims to distribute traffic evenly to all available paths, provides network redundancy by distributing RDMA traffic among multiple paths, and if one path fails, traffic can automatically switch to other available paths, thereby ensuring persistence of RDMA traffic. ECMP can encounter problems in handling elephant flows (flows that occupy a large amount of bandwidth for a long period of time) and mouse flows (short and small flows). Firstly, the hash mechanism in the ECMP has the problem of hash collision, so that the ECMP cannot well balance the traffic to different paths, thereby causing a part of the network to be very congested, and the other part of the network has little traffic, and reducing the overall utilization rate of the network. Second, since ECMP is flow-based load balancing, rather than traffic-based load balancing, ECMP treats an elephant flow as equal even if it occupies a much more bandwidth than a mouse flow. This may also cause some paths to be overloaded (caused by the elephant flow) while other paths are idle. This results in ECMP being better for the case of a comparable flow size, and not for the case of a larger flow size difference, e.g. a concurrent elephant flow and mouse flow. Finally, network traffic is dynamically changing, and elephant flows may begin or end at any time. However, the decision of ECMP is static and does not change once a flow is allocated to a path. In a data center network, the burst traffic is large, and the phenomenon that an elephant stream and a mouse stream coexist exists, so that it is important to allocate paths according to the transmission characteristics of large and small streams and schedule traffic in time according to the transmission states of different paths so as to improve the bandwidth utilization rate of the network.

Fig. 2 is a flowchart of a remote direct data access method according to an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure may be applied to the local host farm 101 and the remote host farm 102 in the embodiment of fig. 1. The subject of the inventive method may be any server of the local host farm 101 that requires multi-path RDMA transmissions.

As shown in fig. 2, the method provided by the embodiment of the present disclosure may include the following steps.

In step S210, N queue pair channels are constructed between the local host and the remote host, where N is an integer greater than or equal to 2, and the N queue pair channels include a priority path and a normal path.

In this step, the server builds N queue pair channels between the local host and the remote host, where N is an integer greater than or equal to 2, the N queue pair channels including a priority path and a normal path. The priority path has priority path sequence for common paths, and has better transmission rate, low packet loss rate and low time delay. The channel or path of the present application is, for example, a virtual path.

Where the transmit work queue of the queue is located, for example, at the local host and the receive work queue is located, for example, at the remote host.

In step S220, the stream transferred between the local host and the remote host is divided into a large stream and a small stream.

In this step, the server divides the stream transmitted between the local host and the remote host into a large stream and a small stream.

In step S230, the large flow and the small flow are transmitted between the local host and the remote host through the N queue pair channels by adopting different strategies.

In this step, the server takes the large flow and the small flow into different strategies to complete the transmission between the local host and the remote host through the N queue pair channels.

The remote direct data access method comprises the steps of constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and adopting different strategies for the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels, so that fine control of remote direct data access can be realized.

Fig. 3 is a flow chart of a method for separating streams transmitted between the local host and the remote host into large streams and small streams provided by an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure may be applied to the local host farm 101 and the remote host farm 102 in the embodiment of fig. 1. The subject of the inventive method may be any server of the local host farm 101 that requires multi-path RDMA transmissions.

As shown in fig. 3, the method provided by the embodiment of the present disclosure may include the following steps.

In step S310, a first number threshold of data packets is set.

In this step, the server sets a first number threshold for the data packets.

Wherein the initial value of the first number threshold may be specifically set according to the actual.

In step S320, a stream in which the number of data packets included in the stream is greater than the first number threshold is divided into the large streams.

In this step, the server divides the stream, in which the number of data packets included in the stream is greater than the first number threshold, into the large streams.

In step S330, the streams having the number of data packets included in the streams being equal to or less than the first number threshold are divided into the ministreams.

In this step, the server divides the stream, in which the number of packets included in the stream is equal to or less than the first number threshold, into the ministreams.

In one embodiment, when the ratio of the number of small flows to the number of large flows obtained by dividing according to the first number threshold is greater than a first ratio, the first number threshold is reduced so that the ratio of the number of small flows to the number of large flows is equal to the first ratio; the first number threshold is dynamically updated in subsequent transmissions. The initial value of the first proportion can be set according to an empirical value, for example, any proportion between 5% and 20%, and can also be set according to the service requirement of an application layer. FIG. 4 is a flow chart of a method for completing a transfer between the local host and the remote host through the N queue pair channels using different policies for the large flow and the small flow provided by embodiments of the present disclosure. The method provided by the embodiment of the present disclosure may be applied to the local host farm 101 and the remote host farm 102 in the embodiment of fig. 1. The subject of the inventive method may be any server of the local host farm 101 that requires multi-path RDMA transmissions.

As shown in fig. 4, the method provided by the embodiment of the present disclosure may include the following steps.

In step S410, each of the ministreams is transmitted over a priority path.

In this step, the server transmits each of the streamlets over a priority path.

Where by giving higher priority to delay sensitive streamlets, are passed through the priority path h-QP.

In step S420, the large stream is transmitted through M paths in the priority path and the normal path, where M is less than or equal to N.

In this step, the server transmits the large stream through M paths in the priority path h-QP and the normal path c-QP, where M is less than or equal to N.

In the subsequent transmission, the first quantity threshold is dynamically updated, and the adjustment method is as follows:

1) The first ratio value initial value is set to p0, whereby a first number threshold value initial value th0 is calculated, a combination (small stream) of streams having the number of data packets equal to or smaller than th0 included in the stream of the server is set to { SF ₁,…,SF_k } according to the threshold value, and a combination (large stream) of streams having the number of data packets equal to or larger than th0 included in the stream of the server is set to { LF1, …, LFj } according to the threshold value.

And updating the values of th0 and p0 by taking the time T as an observation period, respectively calculating average transmission rates r_h and r_c of h-QP and c-QP in the time T, and reducing the value of th0 until r_h is more than or equal to λr_c when r_h is less than λr_c (lambda is set according to the service condition, lambda is more than or equal to 1), wherein the latest first proportional value p1 can be obtained. The r_h, r_c formulas are (1) and (2):

wherein is the data amount { SF1, …, SFk } T1, and/ that the h-QP has completed transmitting in the T1 time, and c-QP has completed transmitting in the T1 time { LF1, …, LFj } T1, N is the number of queue pair channels, and M is M paths in the normal path.

In one embodiment, the remote direct data access method of the present application also requires the construction of a rate monitor for monitoring the data transmission rate of each path, and a sub-stream scheduler for transmission path scheduling of streams. During the transmission process, the rate monitor monitors the transmission rate of each sub-stream on the current path and judges whether the completion or the blocking situation occurs. The sub-flow scheduler dynamically allocates paths for the sub-flows based on completion and blocking conditions. The rate monitor will monitor the WQE rate for each QP (queue pair) and send it to the sub-stream scheduler.

Fig. 5 is a flowchart of a method for transmitting the large stream through M paths in the priority path and the normal path according to an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure may be applied to the local host farm 101 and the remote host farm 102 in the embodiment of fig. 1. The subject of the inventive method may be any server of the local host farm 101 that requires multi-path RDMA transmissions.

As shown in fig. 5, the method provided by the embodiment of the present disclosure may include the following steps.

In step S510, the large stream is divided into M sub-streams.

In this step, the server divides the large stream into M substreams.

In step S520, the M sub-streams complete transmission between the local host and the remote host through the M paths.

In this step, the server completes the transmission between the local host and the remote host through the M paths.

In one embodiment, when the current RDMA stream is determined to be a large stream, the RDMA stream is divided into M sub-streams of the same size, they are allocated to M common paths QP-i, and the WQE of each sub-stream is issued to QP-i, i being any natural number.

Fig. 6 is a flowchart of a method for completing a transmission between the local host and the remote host through the M paths provided by an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure may be applied to the local host farm 101 and the remote host farm 102 in the embodiment of fig. 1. The subject of the inventive method may be any server of the local host farm 101 that requires multi-path RDMA transmissions.

As shown in fig. 6, the method provided by the embodiment of the present disclosure may include the following steps.

In step S610, a transmission rate of each substream in the M paths is acquired.

In this step, the server acquires the transmission rate of each substream in the M paths.

Wherein the rate monitor, which may be a server in particular, will monitor the WQE rate of each path QP-i.

In step S620, a remaining transmission time of each sub-stream is obtained according to the transmission rate of each sub-stream and the remaining transmission data of each sub-stream.

In this step, the server obtains the remaining transmission time of each sub-stream based on the transmission rate of each sub-stream and the remaining transmission data of each sub-stream.

The sub-stream scheduler of the server calculates the residual time t-i (t=s/r) according to the residual data size S-i of QP-i and the WQE rate r-i; the initial value of r-i is the link bandwidth divided by M and updated at intervals T2.

In step S630, all or part of the data of the sub-stream with a large remaining time is allocated to the path with a completed transmission or with a small remaining transmission time for transmission.

In this step, the server allocates all or part of the data of the sub-stream having a large remaining time to the path having completed transmission or having a small remaining transmission time for transmission.

The sub-stream scheduler of the server defines a group of QPs with short residual time of the completed or current transmission task as { Fast-QP }, and the rest QP groups as { Slow-QP }, wherein the QP with the longest residual time is selected to take the transmission rate of r-Slow; the remaining data (WQEs) in { Slow-QP } is split into two parts according to r-Slow, greater than r-Slow and less than r-Slow, the latter part being reassigned to QP in { Fast-QP } to minimize the flow completion time.

In one embodiment, when the transmission rate of one sub-stream is zero, all of the data of the one sub-stream is allocated to a path for which transmission is completed or the remaining transmission time is small.

Fig. 7 is a schematic diagram showing a framework structure to which the remote direct data access method of one embodiment of the present disclosure is applied.

Referring to fig. 7, a rate monitor monitors the transmission rate of each sub-stream on the current path to determine whether a congestion condition is complete or has occurred. The sub-flow scheduler dynamically allocates paths for the sub-flows based on completion and blocking conditions. The rate monitor will monitor the WQE rate for each QP-i (queue pair) and send it to the sub-stream scheduler. Wherein the priority path h-QP is used to transport small streams and QP-i in the normal path c-QP is used to transport large streams.

Fig. 8 is a schematic diagram showing a framework structure to which the remote direct data access method of one embodiment of the present disclosure is applied.

Referring to fig. 8, a rate monitor monitors the transmission rate of each sub-stream on the current path to determine whether a congestion condition is complete or has occurred. The sub-flow scheduler dynamically allocates paths for the sub-flows based on completion and blocking conditions. The rate monitor will monitor the WQE rate for each QP (queue pair) and send it to the sub-stream scheduler. The method comprises the steps that a rate monitor obtains the transmission rate of each sub-stream in a path; the sub-stream scheduler obtains the residual transmission time of each sub-stream according to the transmission rate of each sub-stream and the residual transmission data of each sub-stream; the substream scheduler allocates all or part of the data { low-QP } of the substream with a large remaining time to the { Fast-QP } path with a completed transmission or a small remaining transmission time for transmission.

Fig. 9 is a structural data diagram of a remote direct data access device 900 according to an embodiment of the present disclosure.

Referring to fig. 9, the remote direct data access apparatus of the present application includes:

a building unit 910, configured to build N queue pair channels between a local host and a remote host, where N is an integer greater than or equal to 2, where the N queue pair channels include a priority path and a normal path;

A classification unit 920 for dividing a stream transmitted between the local host and the remote host into a large stream and a small stream;

a policy unit 930, configured to implement transmission between the local host and the remote host through the N queue pair channels by using different policies for the large flow and the small flow.

The remote direct data access device is used for constructing N queue pair channels between a local host and a remote host through a construction unit, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; a classification unit for dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and the strategy unit is used for adopting different strategies for the large flow and the small flow to complete transmission between the local host and the remote host through the N queue pair channels, so that the fine control of remote direct data access can be realized.

In one embodiment, the classifying unit 920 is configured to set a first number threshold of data packets; dividing a stream, in which the number of data packets included in the stream is greater than the first number threshold, into the large stream; and dividing the stream with the number of the data packets less than or equal to the first number threshold into the small streams.

In one embodiment, the policy unit 930 is configured to transmit each of the small flows through a priority path; and transmitting the large stream through M paths in the priority path and the common path, wherein M is less than or equal to N.

In one embodiment, a policy unit 930 is configured to divide the large stream into M sub-streams; and completing transmission between the local host and the remote host through the M paths by the M sub-streams.

In one embodiment, the policy unit 930 is configured to obtain a transmission rate of each sub-stream in the M paths; acquiring the residual transmission time of each sub-stream according to the transmission rate of each sub-stream and the residual transmission data of each sub-stream; all or part of the data of the sub-stream with more remaining time is distributed to the paths with completed transmission or less remaining transmission time for transmission.

In one embodiment, the policy unit 930 is configured to allocate all data of one sub-stream to a path with completed transmission or less remaining transmission time for transmission when the transmission rate of the one sub-stream is zero.

In one embodiment, the classifying unit 920 is configured to reduce the first number threshold when the ratio of the number of small flows to the number of large flows obtained by dividing according to the first number threshold is greater than a first ratio, so that the ratio of the number of small flows to the number of large flows is equal to the first ratio.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a computer device 1000 according to an embodiment of the present disclosure. As shown in fig. 10, a computer device in an embodiment of the present disclosure may include: one or more processors 1001, memory 1002, and an input-output interface 1003. The processor 1001, memory 1002, and input/output interface 1003 are connected by a bus 1004. The memory 1002 is used for storing a computer program including program instructions, and the input-output interface 1003 is used for receiving data and outputting data, such as for data interaction between a host and a computer device, or for data interaction between respective virtual machines in the host; the processor 1001 is configured to execute program instructions stored in the memory 1002.

The processor 1001 may perform the following operations, among others:

constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path; dividing a stream transmitted between the local host and the remote host into a large stream and a small stream; and adopting different strategies to the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels.

In some possible implementations, the processor 1001 may be a central processing unit (central processing unit, CPU), which may also be other general purpose processors, digital signal processors (DIGITAL SIGNAL processors, dsps), application specific integrated circuits (asics), off-the-shelf programmable gate arrays (fpgas) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 1002 may include read only memory and random access memory, and provides instructions and data to the processor 1001 and input output interface 1003. A portion of memory 1002 may also include non-volatile random access memory. For example, the memory 1002 may also store information of a device type.

In a specific implementation, the computer device may execute, through each functional module built in the computer device, an implementation manner provided by each step in the foregoing embodiment, and specifically may refer to an implementation manner provided by each step in the foregoing embodiment, which is not described herein again.

Embodiments of the present disclosure provide a computer device comprising: the processor, the input/output interface and the memory acquire the computer program in the memory through the processor, execute the steps of the method shown in the above embodiment, and perform the transmission operation.

The embodiments of the present disclosure further provide a computer readable storage medium, where the computer readable storage medium stores a computer program, where the computer program is adapted to be loaded by the processor and execute the method provided by each step in the foregoing embodiments, and specifically refer to an implementation manner provided by each step in the foregoing embodiments, which is not described herein in detail. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present disclosure, please refer to the description of the embodiments of the method according to the present disclosure. As an example, a computer program can be deployed to be executed on one computer device or on multiple computer devices at one site or distributed across multiple sites and interconnected by a communication network.

The computer readable storage medium may be an apparatus provided in any of the foregoing embodiments or an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device. The computer readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD), etc. that are provided on the computer device. Further, the computer-readable storage medium may also include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

The disclosed embodiments also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the methods provided in the various alternatives in the above embodiments.

The terms first, second and the like in the description and in the claims and drawings of the embodiments of the disclosure are used for distinguishing between different objects and not for describing a particular sequential order. Furthermore, the term "include" and any variations thereof is intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, article, or device that comprises a list of steps or elements is not limited to the list of steps or modules but may, in the alternative, include other steps or modules not listed or inherent to such process, method, apparatus, article, or device.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in this description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The methods and related devices provided by the embodiments of the present disclosure are described with reference to the method flowcharts and/or structure diagrams provided by the embodiments of the present disclosure, and each flowchart and/or block of the method flowcharts and/or structure diagrams may be implemented by computer program instructions, and combinations of flowcharts and/or block diagrams. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable transmission device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable transmission device, create means for implementing the functions specified in the flowchart flow or flows and/or block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable transmission apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or structural diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable transmission device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer implemented process such that the instructions which execute on the computer or other programmable device provide steps for implementing the functions specified in the flowchart flow or flows and/or structures.

The foregoing disclosure is merely illustrative of the presently preferred embodiments of the present disclosure, and it is not intended to limit the scope of the claims hereof, as defined by the appended claims.

Claims (10)

1. A remote direct data access method, comprising:

Constructing N queue pair channels between a local host and a remote host, wherein N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path;

Dividing a stream transmitted between the local host and the remote host into a large stream and a small stream;

And adopting different strategies to the large stream and the small stream to complete transmission between the local host and the remote host through the N queue pair channels.

2. The method of claim 1, wherein separating the stream transmitted between the local host and the remote host into a large stream and a small stream comprises:

setting a first quantity threshold of data packets;

dividing a stream, in which the number of data packets included in the stream is greater than the first number threshold, into the large stream;

And dividing the stream with the number of the data packets less than or equal to the first number threshold into the small streams.

3. The method of claim 1, wherein taking the large flow and the small flow into different policies to complete the transfer between the local host and the remote host through the N queue pair channels comprises:

Transmitting each of the streamlets through a priority path;

and transmitting the large stream through M paths in the priority path and the common path, wherein M is less than or equal to N.

4. The method of claim 3, wherein transmitting the large stream through M of the priority path and the normal path comprises:

Dividing the large stream into M substreams;

and completing transmission between the local host and the remote host through the M paths by the M sub-streams.

5. The method of claim 4, wherein completing the transmission of the M sub-streams between the local host and the remote host through the M paths comprises:

acquiring the transmission rate of each sub-stream in the M paths;

Acquiring the residual transmission time of each sub-stream according to the transmission rate of each sub-stream and the residual transmission data of each sub-stream;

all or part of the data of the sub-stream with more remaining time is distributed to the paths with completed transmission or less remaining transmission time for transmission.

6. The method as recited in claim 5, further comprising:

When the transmission rate of one sub-stream is zero, all the data of the one sub-stream is distributed to paths with completed transmission or less residual transmission time for transmission.

7. The method as recited in claim 2, further comprising:

Reducing the first number threshold when the ratio of the number of small flows to the number of large flows obtained by dividing according to the first number threshold is greater than a first ratio, so that the ratio of the number of small flows to the number of large flows is equal to the first ratio;

The first number threshold is dynamically updated in subsequent transmissions.

8. A remote direct data access device, comprising:

The system comprises a construction unit, a control unit and a control unit, wherein the construction unit is used for constructing N queue pair channels between a local host and a remote host, N is an integer greater than or equal to 2, and the N queue pair channels comprise a priority path and a common path;

A classification unit for dividing a stream transmitted between the local host and the remote host into a large stream and a small stream;

and the strategy unit is used for adopting different strategies to the large flow and the small flow to complete transmission between the local host and the remote host through the N queue pair channels.

9. A computer device, comprising a processor, a memory, and an input-output interface;

The processor is connected to the memory and the input/output interface, respectively, wherein the input/output interface is used for receiving data and outputting data, the memory is used for storing a computer program, and the processor is used for calling the computer program to enable the computer device to execute the method of any one of claims 1-7.

10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any of claims 1-7.