WO2015000302A1

WO2015000302A1 - Data transmission method, device, and system

Info

Publication number: WO2015000302A1
Application number: PCT/CN2014/072209
Authority: WO
Inventors: 林沐晖; 王俊捷; 卢广
Original assignee: 华为技术有限公司
Priority date: 2013-07-01
Filing date: 2014-02-19
Publication date: 2015-01-08
Also published as: CN103312720B; CN103312720A

Abstract

The present invention discloses a data transmission method, device, and system; kernel bypass technology is applied in an IPoPCIe data transmission scheme; a packet sent by a source node can directly reach a PCIe Switch by means of kernel bypass and enter a destination node, without passing through a conventional TCP/IP protocol stack; during transmission between two nodes, a data packet need not go through multi-step protocol processing and data duplication between layers, thus communications time delays are effectively reduced, increasing communications efficiency.

Description

Data transmission method, device and system

The present invention relates to the field of communications technologies, and in particular, to a data transmission method, device, and system. Background technique

PCIe (Peripheral Component Interconnect Express) is a data transmission bus used to connect CPUs and peripherals inside a node (such as a server host). The routing mechanism is: PCIe Switch (PCIe switch) is read by The address information is used for addressing, so as to achieve the purpose of transmitting the data packet, that is, when the CPU initializes the PCIe peripheral, the peripheral information is allocated to the peripheral, and the peripheral information is accessed through the address information, and the PCIe switch performs the data packet according to the address information. routing.

Since the routing mechanism of PCIe requires that the address information allocated to each peripheral device cannot be repeated within one node, if the PCIe is directly applied to the data transmission scenario between at least two nodes, it will be due to the relationship between the nodes. The address domains are independent of each other, and there is no problem that the address information between the nodes cannot be duplicated. In this regard, NTB (Non-Transparent Bridge) can be introduced in PCIe to solve the problem of direct interconnection between at least two nodes through PCIe.

The NTB technology converts the address information of the data packet sent by the source node to the destination node according to the preset address conversion rule, so as to ensure that the translated address information and the address information in the destination node are not duplicated, thereby realizing at least two nodes. Inter-PCIe data transmission.

Currently, the mainstream communication protocol between nodes is TCP/IP (Transmission Control Protocol/Internet Protocol), which is better compatible with TCP/IP on a network that uses PCIe for multi-host node interconnection. The industry has proposed the concept of IPoPCIe (TCP/IP protocol over PCIe, compatible with TCP/IP protocol on PCIe network), namely: Running TCP/IP protocol on PCIe network architecture, its schematic diagram is shown in Figure 1:

When PCIe communication is performed between at least two nodes, the PCIe driver inside each node simulates a network device called a Virtual Network Interface Card (VNIC). For external users, vNIC and common Ethernet devices have no difference, but also There are functions such as configuring ip address, mac address, mtu, etc., but the vNIC driver (PCIe vNIC driver) can be used as a link layer interface to receive packets sent by the upper layer application (ie, APP in Figure 1) through the TCP/IP stack, and other The packets transmitted by the node are sent to the upper application through the TCP/IP stack. Since the upper interface of the upper application is the TCP/IP stack, the upper layer application does not care about the content of the link layer. Therefore, the upper application in one node can transparently use IPoPCIe to transmit to the upper application in another node. data pack.

When using IPoPCIe for packet transmission between at least two nodes, taking the socket (socket) technology as an example, the normal transmission flow is:

The data packets sent by the upper-layer application in the source node pass through the TCP/IP layer to the link layer, and the PCIe vNIC driver transmits the data packet to the destination node through the PCIe Switch, and then enters the upper-layer application of the destination node through the inverse process with the source node. The data packet is transmitted by copy when it is transmitted between layers.

Since the transmission of data packets between two nodes goes through the above-mentioned multi-step protocol processing flow and data copy between layers, the problem of communication delay and data packets are transmitted between nodes when transmitting data packets between nodes through IPoPCIe. Inefficient problem.

Summary of the invention

The embodiments of the present invention provide a data transmission method, device, and system, which are used to solve the problem of communication delay and low transmission efficiency when transmitting data packets between nodes in the prior art through IPoPCIe.

In a first aspect, a data transmission method is provided, where the method includes:

The source node determines transmission information required when the data packet to be transmitted is transmitted in the component fast interconnect local bus PCIe, and the transmission information includes a transmission window address and a reception window address;

The source node transmits the to-be-transmitted data packet to the destination node through the PCIe switch according to the transmission information.

With reference to the first aspect, in a first possible implementation, the method further includes: determining, by the source node, a transmission mode of the data packet to be transmitted according to a preset configuration file. Transmission Control Protocol/Internet Protocol TCP/IP transmission mode or kernel bypass Kernel bypass transmission mode;

And determining, when the transmission mode of the data packet to be transmitted is a Kernel bypass transmission mode, determining the transmission information required when the data packet to be transmitted is transmitted in the PCIe.

In conjunction with the first aspect or the first possible implementation of the first aspect, in a second possible implementation,

The source node transmits the to-be-transmitted data packet to the destination node according to the transmission information, which specifically includes:

The source node stores the to-be-transmitted data packet into a buffer in its own kernel space; the source node converts the sending window address and the receiving window address into a physical address in a PCIe global space;

The source node transmits the to-be-transmitted data packet stored in the buffer from the physical address of the transmission window to the physical address of the receiving window, and is stored in a buffer in the kernel space of the destination node.

In conjunction with the first aspect or the first possible implementation of the first aspect, in a third possible implementation,

The source node converts a virtual address of an application that generates the data packet to be transmitted into a physical address and maps to a PCIe global space, and converts the sending window address and the receiving window address into a physical address in a PCIe global space. ;

The source node acquires the to-be-transmitted data packet from a physical address mapped by the application to the PCIe global space, and transmits the to-be-transmitted data packet from a physical address of the sending window to a physical address of the receiving window, and stores Direct memory access DMA transfers within the PCIe global space are implemented in the buffer of the destination node in kernel space.

A second aspect provides a node device, where the node device includes: a socket management module, configured to determine transmission information required when the data packet to be transmitted is transmitted in the PCIe, where the transmission information includes a sending window address and a receiving window address;

And a PCIe transmission module, configured to transmit the to-be-transmitted data packet to another node by using a PCIe switch according to the transmission information determined by the socket management module.

In combination with the second aspect, in the first possible implementation manner, the method further includes:

a mode selection module, configured to determine, according to a preset configuration file, whether a transmission mode of the to-be-transmitted data packet is a TCP/IP transmission mode or a Kernel bypass transmission mode, and determining that a transmission mode of the to-be-transmitted data packet is a Kernel bypass transmission In the mode, the socket management module is triggered.

In conjunction with the second aspect or the first possible implementation of the second aspect, in a second possible implementation,

The PCIe transmission module is specifically configured to store the to-be-transmitted data packet into a buffer in its own kernel space, and convert the sending window address and the receiving window address into a physical address in a PCIe global space. And transmitting the to-be-transmitted data packet stored in the buffer from a physical address of the sending window to a physical address of the receiving window, and storing in a buffer in the kernel space of the other node.

In conjunction with the second aspect or the first possible implementation of the second aspect, in a third possible implementation,

The PCIe transmission module is specifically configured to convert a virtual address of an application that generates the data packet to be transmitted into a physical address and map to a PCIe global space, and convert the sending window address and the receiving window address into a PCIe global space. a physical address within, and obtaining the data packet to be transmitted from a physical address mapped by the application to the PCIe global space, and transmitting the data packet to be transmitted from a physical address of the transmission window to a physical address of the receiving window Stored in the buffer of the other nodes in the kernel space to implement direct memory access DMA transfer in the PCIe global space.

In a third aspect, a data transmission system is provided, the system comprising:

a source node, configured to determine transmission information required for a data packet to be transmitted to be transmitted in the PCIe, And transmitting, according to the transmission information, the data packet to be transmitted to the destination node by using a PCIe switch, where the transmission information includes a sending window address and a receiving window address;

The destination node is configured to receive a data packet from the source node by using a PCIe switch. With reference to the third aspect, in a first possible implementation, the source node is further configured to determine, according to a preset configuration file, whether a transmission mode of the to-be-transmitted data packet is a TCP/IP transmission mode or a Kernel bypass transmission. And determining, when determining that the transmission mode of the data packet to be transmitted is a Kernel bypass transmission mode, determining the transmission information required when the data packet to be transmitted is transmitted in the PCIe.

In conjunction with the third aspect or the first possible implementation of the third aspect, in a second possible implementation,

The source node is specifically configured to store the to-be-transmitted data packet into a buffer in its own kernel space, and convert the sending window address and the receiving window address into a physical address in a PCIe global space. And transmitting the to-be-transmitted data packet stored in the buffer from the physical address of the sending window to the physical address of the receiving window, and storing in the buffer in the kernel space of the destination node.

In conjunction with the third aspect or the first possible implementation of the third aspect, in a third possible implementation,

The source node is specifically configured to convert a virtual address of an application that generates the data packet to be transmitted into a physical address and map to a PCIe global space, and convert the sending window address and the receiving window address into a PCIe global space. Physical address, and obtaining the data packet to be transmitted from a physical address mapped by the application to the PCIe global space, and transmitting the data packet to be transmitted from a physical address of the sending window to a physical address of the receiving window, Stored in the buffer of the destination node in the kernel space to implement direct memory access DMA transfer in the PCIe global space.

In the embodiment of the present invention, the kernel bypass technology is applied in the data transmission scheme of the IPoPCIe, and the data packet sent by the source node can reach the PCIe Switch and enter the destination node directly through the kernel bypass without going through the traditional TCP/IP protocol stack. Since the data packet is transmitted between two nodes without going through the multi-step protocol processing flow and data copy between the layers, the communication delay can be effectively reduced, and the communication is improved. effectiveness. DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the prior art or the embodiments will be briefly described below. Obviously, the drawings in the following description are only some implementations of the present invention. For example, other drawings may be obtained from those of ordinary skill in the art in light of the inventive work.

1 is a schematic diagram of an architecture of an IPoPCIe in the background art;

2 is a schematic diagram of steps of a data transmission method according to Embodiment 1 of the present invention;

3(a) and 3(b) are schematic diagrams showing the structure of a data transmission system according to Embodiment 2 of the present invention; FIG. 4 is a schematic diagram showing the working steps of the data transmission system according to Embodiment 2 of the present invention;

FIG. 5 is a schematic diagram of B-Copy transmission of a data packet according to Embodiment 2 of the present invention; FIG.

6 is a schematic diagram of Z-Copy transmission of a data packet in Embodiment 2 of the present invention;

FIG. 7 is a schematic structural diagram of a node device according to Embodiment 3 of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In order to reduce the transmission of data packets between at least two nodes through the multi-step protocol processing flow and data copy between the layers, the data transmission between the nodes through IPoPCIe will cause communication delay and the transmission of data packets between nodes. The problem of low efficiency, the embodiment of the present invention provides a data transmission scheme for applying the Kernel bypass technology to the IPoPCIe. In the solution of this embodiment, the data packet sent by the source node may not pass the traditional TCP. The /IP protocol stack, and directly enters the PCIe Switch through the Kernel bypass and enters the destination node. Since the data packet is transmitted between the two nodes, it does not need to go through the multi-step protocol processing flow and data copy between the layers, which can be effectively reduced. Less communication delay and improve communication efficiency.

The solution of the present invention will be described in detail below through specific embodiments.

Embodiment 1:

The embodiment of the present invention describes a data transmission method. As shown in FIG. 2, the method includes the following steps: Step 101: The source node determines, according to a preset configuration file, whether a transmission mode of a data packet to be transmitted is a TCP/IP transmission mode or a Kernel bypass. Transmission mode, if it is TCP/IP transmission mode, the data packet is transmitted according to the existing scheme; if it is the Kernel bypass transmission mode, the execution step is disordered.

This step 101 is a preferred step for achieving the object of the present invention. In the solution of the embodiment, the source node is not limited to directly adopting the Kernel bypass transmission mode for data packet transmission.

Step 102: The source node determines the transmission information required when the data packet to be transmitted is transmitted in the PCIe. The transmission information involved in this step 102 refers to: the address information of the source node transmitting the data to be transmitted and the address information of the destination node receiving the data to be transmitted, such as the transmission window address information and the reception window address information.

Step 103: The source node passes the to-be-transmitted data packet according to the transmission information, and adopts PCIe.

The Switch transmits to the destination node to implement data packet transmission.

Embodiment 2:

The method described in the first embodiment can be implemented by the data transmission system shown in FIG. 3( a ), and the data transmission system adds a mode selection module to each node on the basis of the IPoPCIe shown in FIG. 1 . 11 and the core bypass communication module 12. In the second embodiment, the data transmission system shown in FIG. 3( a ) is taken as an example to further describe the solution of the first embodiment.

Figure 3 (a) shows the working steps of the data transmission system as shown in Figure 4, with node A as the source node, node B as the destination node, and node A sending data packets to node B as an example. The data transmission process includes the following: Steps:

Step 201: The socket application in node A generates a data packet to be transmitted.

The socket application is an application in an upper layer application (ie, APP in FIG. 2) in node A. Program.

When the socket application generates a socket packet, it also creates corresponding socket state information for the socket packet, and the socket state information indicates the transmission request for transmitting the socket packet.

For example: Response time limit information, indicating the time limit for the response message to be returned after the socket data packet is transmitted to the destination node.

For another example, the packet size information indicates the size of the socket packet, and is used to determine the buffer size that needs to be requested for the socket packet when transmitting and receiving the socket packet.

The upper application in node A can contain multiple socket applications, and each socket application generates a socket packet with its corresponding socket status information.

Step 202: The mode selection module 11 in node A is called by the socket application. In this step 102, after the socket application in the node A generates the data packet to be transmitted, the socket interface in the C library can be called, and the mode selection module 11 is triggered.

Step 203: The mode selection module 11 in the node A determines, according to the preset configuration file, whether the transmission mode of the to-be-transmitted data packet is a TCP/IP transmission mode or a Kernel bypass transmission mode. If the TCP/IP transmission mode is performed, the step is performed. 204. If the mode is the Kernel bypass transmission mode, go to step 205.

In this step 203, the mode selection module 11 may preset a configuration file in which the selection condition of each transmission mode is recorded.

For example: Record the correspondence between the time limit of the data packet and the transmission mode in the configuration file. The time-sensitive packet configures the Kernel bypass transmission mode, and the time-sensitive packet configures the TCP/IP transmission mode. That is, the timeliness of the data packet is divided into multiple grades, and the timeliness is higher than the set grade (including the set grade). The data packet is configured with the Kernel bypass transmission mode, and the timeliness is lower than the set grade (excluding The data packet of the setting profile is configured with a TCP/IP transmission mode.

When the mode selection module 11 is called by the socket application module, according to the aging requirement of the data packet to be transmitted, an appropriate transmission mode is selected for the data packet to be transmitted, and TCP/IP transmission is realized. Mode switching in mode and Kernel bypass transmission mode.

Step 204: The node A transmits the to-be-transmitted data packet to the node B via the TCP/IP protocol stack, and ends the current data transmission process.

In this step 204, the socket data packet to be transmitted is transmitted from the TCP/IP NET CORE (network core) to the link layer PCIe vNIC driver, and finally the PCIe vNIC driver transmits the socket data packet to be transmitted to the node B through the PCIe Switch. Complete this data transfer process.

Step 205: The kernel bypass communication module 12 in the node A determines the transmission information required when the data packet to be transmitted is transmitted in the PCIe.

As shown in FIG. 3(b), the kernel bypass communication module 12 may include a socket management module 21, and the socket management module 21 has the function of performing this step 205.

Preferably, the socket management module 21 can also be used to maintain socket state information created by the socket application. Since the upper layer application in the node A can include multiple socket applications, the socket data packet generated by each socket application has its corresponding socket state information, and therefore, the socket management module 21 can pass a data structure. To maintain the corresponding socket state information of each socket application, the data structure can be called a socket structure. In order to improve the rate at which the socket management module 21 looks for socket status information in the socket structure, the sheath structure can be maintained in a list form.

The mode selection module 11 triggers the socket management module 21 when determining that the transmission mode of the data packet to be transmitted is the Kernel bypass transmission mode, and the socket management module 21 can determine the data to be transmitted after being triggered by the mode selection module 11. The socket structure of the packet and the transmission information required for the data packet to be transmitted to be transmitted in PCIe. The transmission information includes, but is not limited to, transmission window address information and reception window address information, where the transmission window address information and the reception window address information refer to logical addresses of the transmission window and the reception window.

Step 206: The kernel bypass communication module 12 in the node A transmits the to-be-transmitted data packet to the node B through the PCIe Switch according to the transmission information, and ends the current data transmission process.

The kernel bypass communication module 12 further includes: a private transmission interface 22 and a PCIe transmission module 23 , where:

The private transport interface 22 can be invoked after the socket management module 21 determines the transport information required for transmission in PCIe. The private transmission interface 22 is a private interface that implements a logical communication function, and the transmission of the data packet in the PCIe is realized by calling the underlying PCIe transmission module 23.

After the PCIe transmission module 23 is called by the private transmission interface 22, according to the data packet to be transmitted

The transmission information required for transmission in the PCIe transmits the to-be-transmitted data packet to the Node B through the PCIe Switch. Preferably, the PCIe transmission module 23 can also transmit the data packet according to the socket status information of the data packet to be transmitted.

Specifically, in this step 206, the following two ways are included to implement the transmission of the data packet:

Method 1: Implement B-Copy (Buffer Copy) transmission of the data packet in the kernel bypass.

As shown in FIG. 5, it is a schematic diagram of a B-Copy transmission process of a data packet, including the following contents: First, after the PCIe transmission module 23 is invoked by the private transmission interface 22, the data packet transmission function is called to start the transmission of the data packet. process.

In the second step, the PCIe transmission module 23 can read the socket status information of the data packet to be transmitted from the socket status information maintained by the socket management module 21; meanwhile, a buffer is also opened in the kernel space. The data packet to be transmitted is stored in the buffer.

If the socket status information of the data packet to be transmitted includes packet size information, the buffer may be opened according to the data packet size information, so that the buffer created has sufficient space to store the data to be transmitted. package.

The buffer opened in this step is actually the buffer of node A.

In the third step, the PCIe transmission module 23 converts the transmission window address and the reception window address into physical addresses in the PCIe global space.

Since the transmission window address information and the reception window address information determined in step 205 are logical address information, it is logically required to use the transmission window and the reception window for data packet transmission. The address is converted to a physical address in the PCIe global space. The specific conversion method is:

When the socket application creates the socket data packet, it also creates corresponding socket state information for the socket data packet, and the socket state information may include a mapping relationship between the logical address and the physical address, and the PCIe transmission The module 23 may obtain the mapping relationship from the socket state information of the data packet to be transmitted, and convert the transmission window address information and the receiving window address information acquired from the socket management module 21 into the PCIe global space. Physical address.

In the fourth step, the PCIe transmission module 23 transmits the to-be-transmitted data packet stored in the buffer of the node A from the physical address of the transmission window to the physical address of the receiving window, and stores the buffer in the kernel space of the node B. In the district.

In this step, the PCIe transmission module 23 sequentially copies the to-be-transmitted data packet stored in the buffer of the node A from the physical address of the transmission window to the physical address of the receiving window, and the data packet is stored to the node via the receiving window physical address. B is in the buffer in kernel space. After the transmission of the data packet to be transmitted is completed, the PCIe transmission module 23 can notify the node B through the interrupt message to complete the data transmission process. After receiving the interrupt message, the Node B determines that the data packet is cached in the buffer of the Node B, and can read the received data packet from the buffer of the Node B and copy it when needed. To the upper application of Node B.

By means of the B-Copy transmission process described in the first method, the overhead of the mapping can be reduced by storing and transmitting the data packets to be transmitted in the buffer, which is a relatively efficient transmission method for small data packets.

Method 2: Implement Z-Copy (Zero Copy) transmission of the data packet in the kernel bypass. The system diagram shown in Figure 3 (a) and Figure 3 (b) is the system diagram of Z-Copy transmission. .

As shown in FIG. 6, a schematic diagram of a Z-Copy transmission process of a data packet includes the following contents: First, after the PCIe transmission module 23 is invoked by the private transmission interface 22, the data packet transmission function is called to start the transmission of the data packet. process.

In the second step, the PCIe transmission module 23 can maintain a socket status letter from the socket management module 21. Reading the socket status information of the data packet to be transmitted; and acquiring the virtual address of the socket application that generates the data packet to be transmitted.

In this step, the virtual address of the socket application that generates the data packet to be transmitted may be obtained from the socket state information of the data packet to be transmitted.

In the third step, the PCIe transport module 23 converts the virtual address of the socket application into a physical address and maps to the PCIe global space, and converts the transmit window address and the receive window address into physical addresses within the PCIe global space.

In this step, the mapping relationship between the logical address and the physical address may be included in the socket state information, and the PCIe transmission module 23 may obtain the mapping relationship from the socket state information of the data packet to be transmitted, and then The virtual address of the socket application, the send window virtual address, and the receive window virtual address are all converted to physical addresses within the PCIe global space.

In a fourth step, the PCIe transmission module 23 obtains a data packet to be transmitted from a physical address mapped by the socket application to the PCIe global space, and transmits the data packet to be transmitted from a physical address of the sending window to a physical address of the receiving window. It is stored in the buffer of the node B in the kernel space, and realizes DMA (Direct Memory Access) transmission in the PCIe global space.

In this step, the PCIe transmission module 23 can implement DMA transmission in the PCIe global space by calling the DMA controller in the NTB.

After the DMA transfer is completed, the PCIe transmission module 23 can notify the node B by the interrupt message to complete the data transmission process. After receiving the interrupt message, the Node B determines that the data packet is cached in the buffer of the Node B, and can read the received data packet from the buffer of the Node B and copy it when needed. To the upper application of Node B.

Through the Z-Copy transmission process described in the second method, the data packet to be transmitted does not need to be copied, but is directly transmitted to the node B from the socket application that generates the data packet to be transmitted directly through the DMA transmission mode, thereby effectively reducing the data packet transmission. The delay of the process.

It should be noted that, in the solution of the second embodiment, the module division manner in the node A is an optional manner of the implementation scheme, and the second embodiment is not limited to other module division manners. The scheme of the embodiment; and the node A has the module division manner described in FIG. 3( a ) and FIG. 3 ( b ), and also has the data packet received by other nodes and copied to the local upper layer application. functional module.

Embodiment 3:

The third embodiment of the present invention further describes a node device. As shown in FIG. 7, the node device includes a processor 31 and a transmitter 32, where: the processor 31 is configured to determine when the data packet to be transmitted is transmitted in the PCIe. The required transmission information is used by the transmitter 32 to transmit the to-be-transmitted data packet to other nodes through the PCIe switch according to the transmission information determined by the processor 31.

The node device further includes a selector 33, configured to determine, according to a preset configuration file, whether a transmission mode of the data packet to be transmitted is a TCP/IP transmission mode or a Kernel bypass transmission mode, and determining the data packet to be transmitted. The processor 31 is triggered when the transmission mode is the Kernel bypass transmission mode.

The transmission information includes a transmission window address and a reception window address. method one:

The transmitter 32 is specifically configured to store the to-be-transmitted data packet into a buffer in its own kernel space, and convert the sending window address and the receiving window address into a physical address in a PCIe global space. And transmitting the to-be-transmitted data packet stored in the buffer from the physical address of the sending window to the physical address of the receiving window, and storing in the buffer in the kernel space of the other node.

Method 2:

The transmitter 32 is specifically configured to convert a virtual address of an application that generates the data packet to be transmitted into a physical address and map to a PCIe global space, and convert the sending window address and the receiving window address into a PCIe global space. a physical address, and obtaining the data packet to be transmitted from a physical address mapped by the application to the PCIe global space, and transmitting the data packet to be transmitted from a physical address of the sending window to a physical address of the receiving window, Stored in the buffer of the other nodes in the kernel space, realize direct memory access DMA transfer in the PCIe global space. Through the solution of the embodiment of the present invention, when the socket data packet is transmitted under the IPoPCIe-based architecture, the data packet is transmitted through the kernel bypass technology, and the protocol processing and data copy operation of the TCP/IP layer and the link layer are bypassed. The system resources allocated by the CPU for the protocol processing and the data copy operation can be translated to reduce the load of the CPU; meanwhile, since the communication delay is composed of the transmission packet transmission delay and the protocol processing delay, therefore, the bypass is performed. The protocol processing of the TCP/IP layer and the link layer can reduce the communication delay and improve the transmission efficiency. Therefore, the transmission throughput can be further improved by reducing the CPU overhead and reducing the communication delay.

Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system, or computer program product. Thus, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block of the flowchart and/or block diagrams, and combinations of processes and/or blocks in the flowcharts and/or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce a computer implemented The processing, such as instructions executed on a computer or other programmable device, provides the steps for implementing the functions specified in one or more blocks of the flowchart or in a block or blocks of the flowchart.

In a typical configuration, the computer device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. The memory may include non-persistent memory, random access memory (RAM), and/or non-volatile memory in a computer readable medium, such as read only memory (ROM) or flash memory. Memory is an example of a computer readable medium. Computer readable media, including both permanent and non-persistent, removable and non-removable media, can be stored by any method or technology. The information can be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory. (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage , magnetic tape cartridges, magnetic tape storage or other magnetic storage devices or any other non-transportable media that can be used to store information that can be accessed by computing devices. Computer-readable media, as defined herein, does not include non-persistent computer readable media, port-modulated data signals and carrier waves.

While the preferred embodiment of the present application has been described, those skilled in the art can make further changes and modifications to these embodiments once they are aware of the basic inventive concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all modifications and

It will be apparent to those skilled in the art that various modifications and changes can be made in the present application without departing from the spirit and scope of the application. Accordingly, it is intended that the present invention cover the modifications and variations of the invention, and the scope of the present invention.

Claims

Rights request

1. A data transmission method, characterized in that the method includes:

The source node determines the transmission information required when the data packet to be transmitted is transmitted in the Component Express Interconnect local bus PCIe. The transmission information includes a sending window address and a receiving window address;

The source node transmits the data packet to be transmitted to the destination node through the PCIe switch according to the transmission information.

2. The method of claim 1, wherein the method further includes: the source node determines, according to a preset configuration file, that the transmission mode of the data packet to be transmitted is Transmission Control Protocol/Internet Protocol TCP/IP transmission mode or Kernel bypass transmission mode;

When it is determined that the transmission mode of the data packet to be transmitted is the Kernel bypass transmission mode, the transmission information required when the data packet to be transmitted is transmitted in PCIe is determined.

3. The method according to claim 1 or 2, characterized in that,

The source node transmits the data packet to be transmitted to the destination node according to the transmission information, specifically including:

The source node stores the data packet to be transmitted in its own buffer in the kernel space; the source node converts the sending window address and receiving window address into physical addresses in the PCIe global space;

The source node transmits the data packet to be transmitted stored in the buffer from the physical address of the sending window to the physical address of the receiving window, and stores it in the buffer of the destination node in the kernel space.

4. The method according to claim 1 or 2, characterized in that,

The source node converts the virtual address of the application that generates the data packet to be transmitted into a physical address and maps it to the PCIe global space, and converts the sending window address and receiving window address. Is the physical address in the PCIe global space;

The source node obtains the data packet to be transmitted from the physical address mapped to the PCIe global space by the application program, transmits the data packet to be transmitted from the physical address of the sending window to the physical address of the receiving window, and stores it. In the buffer of the destination node in the kernel space, direct memory access DMA transfer in the PCIe global space is implemented.

5. A node device, characterized in that, the node device includes:

The socket management module is used to determine the transmission information required when the data packet to be transmitted is transmitted in PCIe. The transmission information includes the sending window address and the receiving window address;

The PCIe transmission module is configured to transmit the data packet to be transmitted to other nodes through the PCIe switch according to the transmission information determined by the socket management module.

6. The node device according to claim 5, further comprising:

A mode selection module, configured to determine whether the transmission mode of the data packet to be transmitted is the TCP/IP transmission mode or the Kernel bypass transmission mode according to the preset configuration file. After determining that the transmission mode of the data packet to be transmitted is the Kernel bypass transmission mode, trigger the socket management module.

7. The node device according to claim 5 or 6, characterized in that,

The PCIe transmission module is specifically used to store the data packet to be transmitted in a buffer in its own kernel space, and convert the sending window address and receiving window address into physical addresses in the PCIe global space. , and transfer the data packet to be transmitted stored in the buffer from the physical address of the sending window to the physical address of the receiving window, and store it in the buffer of the other node in the kernel space.

8. The node device according to claim 5 or 6, characterized in that,

The PCIe transmission module is specifically used to convert the virtual address of the application program that generates the data packet to be transmitted into a physical address and map it to the PCIe global space, and convert the sending window address and the receiving window address into the PCIe global space. the physical address within the PCIe global space, and obtain the data packet to be transmitted from the physical address mapped to the PCIe global space by the application program, and transmit the data packet to be transmitted from the physical address of the sending window to the physical address of the receiving window , stored in the other nodes In the buffer within the kernel space, direct memory access DMA transfers within the PCIe global space are implemented.

9. A data transmission system, characterized in that the system includes:

The source node is used to determine the transmission information required when the data packet to be transmitted is transmitted in PCIe, and transmit the data packet to be transmitted to the destination node through the PCIe switch according to the transmission information. The transmission information includes the transmission window address. and receiving window address;

The destination node is configured to receive data packets from the source node through the PCIe switch.

10. The system of claim 9, characterized in that,

The source node is also configured to determine whether the transmission mode of the data packet to be transmitted is the TCP/IP transmission mode or the Kernel bypass transmission mode according to the preset configuration file. After determining that the transmission mode of the data packet to be transmitted is the Kernel In the bypass transmission mode, the transmission information required when the data packet to be transmitted is transmitted in PCIe is determined.

11. The system according to claim 9 or 10, characterized in that,

The source node is specifically configured to store the data packet to be transmitted into a buffer in its own kernel space, and convert the sending window address and receiving window address into physical addresses in the PCIe global space, And the data packet to be transmitted stored in the buffer is transferred from the physical address of the sending window to the physical address of the receiving window, and is stored in the buffer of the destination node in the kernel space.

12. The system according to claim 9 or 10, characterized in that,

The source node is specifically used to convert the virtual address of the application program that generates the data packet to be transmitted into a physical address and map it to the PCIe global space, and convert the sending window address and receiving window address into the PCIe global space. and, obtain the data packet to be transmitted from the physical address mapped to the PCIe global space by the application program, and transmit the data packet to be transmitted from the physical address of the sending window to the physical address of the receiving window, Stored in the buffer of the destination node in the kernel space, direct memory access DMA transfer in the PCIe global space is implemented.