CN113422793A - Data transmission method and device, electronic equipment and computer storage medium - Google Patents

Data transmission method and device, electronic equipment and computer storage medium Download PDF

Info

Publication number
CN113422793A
CN113422793A CN202110164907.1A CN202110164907A CN113422793A CN 113422793 A CN113422793 A CN 113422793A CN 202110164907 A CN202110164907 A CN 202110164907A CN 113422793 A CN113422793 A CN 113422793A
Authority
CN
China
Prior art keywords
data
data packet
cache region
packet
receiving end
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110164907.1A
Other languages
Chinese (zh)
Inventor
高翼枭
李强
邱昊楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN202110164907.1A priority Critical patent/CN113422793A/en
Publication of CN113422793A publication Critical patent/CN113422793A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00—Digital computers in general; Data processing equipment in general
    • G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163—Interprocessor communication
    • G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17306—Intercommunication techniques
    • G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Communication Control (AREA)

Abstract

The embodiment of the application provides a data transmission method, a data transmission device, electronic equipment and a computer storage medium, wherein the data transmission method comprises the following steps: the data sending end determines a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, wherein the data packet occupies part of a storage space in the first cache region; the data sending end determines a target storage address of the data packet stored in a second cache region according to the storage address of the data packet in the first cache region and a pre-obtained storage address of the second cache region of the data receiving end, and accordingly executes RDMA (remote direct memory access) write operation to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address of the second cache region, wherein the storage address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end; and after the data sending end determines that the data packet is processed by the data receiving end, the storage space occupied by the data packet in the first cache region is released.

Description

Data transmission method and device, electronic equipment and computer storage medium
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to a data transmission method and device, electronic equipment and a computer storage medium.
Background
With the continuous development of cloud storage systems and the like, low latency becomes a hotspot requirement of the cloud storage system, and the traditional TCP protocol cannot meet the storage requirement of low latency, so that a Remote Direct Memory Access (RDMA) technology is widely applied. RDMA is a technique for performing data operations directly on a storage area of a device without intervention of an operating system and a processor when performing network data transfer. Because the intervention of an operating system and a processor is not needed, the transmission time can be greatly reduced, and the delay is reduced.
When data is transmitted by the RDMA technology, if the amount of the transmitted data is large, the data is generally directly sent to a buffer space of a data receiving end by way of RDMA Write operation (RDMA Write). When data transmission is performed through RDMA write operation, a storage space for storing a data packet is generally registered at a data receiving end, and then the data packet is directly sent to the storage space through RDMA write operation by a data sending end. However, each time an RDMA write operation is performed, a storage space for storing a data packet needs to be registered at a data receiving end, which results in low data transmission performance.
Disclosure of Invention
In view of the above, embodiments of the present application provide a data transmission scheme to at least partially solve the above problems.
According to a first aspect of embodiments of the present application, there is provided a data transmission method, including: the method comprises the steps that a data sending end determines a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, wherein the data packet occupies part of storage space in the first cache region; the data sending end determines a target storage address of the data packet stored in a second cache area according to a storage address of the data packet in a first cache area and a pre-obtained storage address of the second cache area of the data receiving end, wherein the storage address of the second cache area is obtained through one-time handshake interaction between the data sending end and the data receiving end; performing an RDMA write operation based on the storage address of the data packet in a first cache region and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored to the target storage address in a second cache region; and after the data sending end determines that the data packet is processed by the data receiving end, the data sending end releases the storage space occupied by the data packet in the first cache region.
According to a second aspect of embodiments of the present application, there is provided a data transmission method, including: writing a data packet sent by a data sending end through RDMA (remote direct memory Access) write operation into a target storage address in a second cache region by a data receiving end, wherein the RDMA write operation is executed based on the storage address of the data packet in a first cache region and the target storage address, the target storage address is determined by the data sending end according to the storage address of the data packet in the first cache region and a pre-obtained storage address of the second cache region of the data receiving end, and the storage address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end; and the data receiving end processes the data packet in the second cache region and sends a response message indicating the completion of data packet processing to the data sending end, so that the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end.
According to a third aspect of embodiments of the present application, there is provided a data transmission apparatus, including: the device comprises a first determining module, a first sending module and a second determining module, wherein the first determining module is used for determining a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, and the data packet occupies part of a storage space in the first cache region; an extracting module, configured to determine, according to a storage address of the data packet in a first cache region and a pre-obtained storage address of a second cache region of the data receiving end, a target storage address at which the data packet is stored in the second cache region, and perform an RDMA write operation based on the storage address of the data packet in the first cache region and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address in the second cache region, where the storage address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end; and the releasing module is used for releasing the storage space occupied by the data packet in the first cache region after the data packet is determined to be processed by the data receiving end.
According to a fourth aspect of embodiments of the present application, there is provided a data transmission apparatus, the apparatus including: a write module, configured to write a packet sent by a data sending end through an RDMA write operation to a target storage address in a second buffer, where the RDMA write operation is performed based on a storage address of the packet in a first buffer and the target storage address, the target storage address is determined by the data sending end according to the storage address of the packet in the first buffer and a pre-obtained storage address of a second buffer of the data receiving end, and the storage address of the second buffer is obtained through one-time handshake interaction between the data sending end and the data receiving end; and the first sending module is used for processing the data packet in the second cache region and sending a response message indicating the completion of data packet processing to the data sending end, so that the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end.
According to a fifth aspect of embodiments of the present application, there is provided an electronic apparatus, including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the data transmission method according to the first aspect or the second aspect.
According to a sixth aspect of embodiments of the present application, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the data transmission method according to the first or seventh aspect.
According to the data transmission scheme provided by the embodiment of the application, after one-time handshake interaction between the data sending end and the data receiving end, the data sending end can obtain the storage address of the second cache region; when data transmission is carried out, determining a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, wherein the data packet occupies part of a storage space in the first cache region; the data sending end can determine a target storage address of the data packet stored in a second cache region according to a storage address of the data packet in a first cache region and a pre-obtained storage address of the second cache region of the data receiving end, and accordingly perform an RDMA write operation to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address of the second cache region; the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end, so that a first cache region and a second cache region capable of storing a plurality of data packets can be obtained by registering through one-time handshake in advance, the data packet is directly stored in the first cache region before the data packet is sent each time, namely, the data packet can be sent through RDMA (remote direct memory access) writing operation based on the first cache region and the second cache region, the process that the data receiving end registers the storage space for storing the data packet in the data transmission process through RDMA writing operation is optimized, the handshake overhead of one-time RTT (round trip time) of the data sending end and the data receiving end is saved at least, and the transmission performance of the data transmission through RDMA writing operation is greatly improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present application, and other drawings can be obtained by those skilled in the art according to the drawings.
Fig. 1A is a schematic structural diagram of a system according to an embodiment of the present disclosure;
fig. 1B is a flowchart illustrating steps of a data transmission method according to a first embodiment of the present application;
FIG. 1C is a schematic view of a scenario in the embodiment shown in FIG. 1B;
fig. 2A is a flowchart illustrating steps of a data transmission method according to a second embodiment of the present application;
fig. 2B is a schematic view of a scenario in the second embodiment of the present application;
fig. 3 is a block diagram of a data transmission apparatus according to a third embodiment of the present application;
fig. 4 is a block diagram of a data transmission apparatus according to a fourth embodiment of the present application;
fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application shall fall within the scope of the protection of the embodiments in the present application.
The following further describes specific implementations of embodiments of the present application with reference to the drawings of the embodiments of the present application.
Example one
Fig. 1A is a system including an RDMA network card according to a first embodiment of the present application, and as shown in the figure, the system may include: a number of devices having data processing capabilities, device A, B, C, D is shown by way of example in fig. 1A, but is not limited thereto.
The plurality of devices may form any system, such as a distributed storage system, a cloud computing system, and the like, which is not limited in this embodiment.
The multiple devices are connected through a network, where the network may be the internet, an intranet, a local area network, a wide area network, a storage area network, and the like, as long as communication between the multiple devices can be achieved, and this embodiment does not limit this.
In this embodiment, each device may be provided with an RDMA network card (also referred to as RNIC), and a plurality of devices may perform remote direct memory access through the RDMA network card and the network. Illustratively, the RDMA network of the device a and the RDMA network card of the device B cooperate to enable the device a to directly access the memory of the device B through the network.
An RDMA network card is a network card that supports the RDMA protocol, which allows the network card to directly access buffers of an application, and is connected to the application using a work Queue (QP).
The work queues QP are established in pairs and include a Send Queue (SQ) and a Receive Queue (RQ), where SQ is established at the data sending end and used for storing an instruction for data transmission between a storage device of the data sending end and a storage device of the data receiving end, and RQ is established at the data receiving end and used for storing an instruction for storing received data to a preset storage location.
After the data transmitting end provides a Work Request (WR) to the Work Queue, a Work Queue Element (WQE) corresponding to the Work Request (WR) is added to the Work Queue.
The RDMA hardware may continually execute corresponding Work Requests (WRs) based on Work Queue Elements (WQEs) in a Work Queue (WQ) to enable the transfer of data.
Generally, if the amount of data sent by the data sender is small, an RDMA Send operation is performed to copy the data to the data receiver. Specifically, when the RDMA Send is executed, the RDMA network card of the data sending end copies data from the Buffer area of the data sending end to the Buffer of the RDMA network card, transmits the data to the Buffer of the RDMA network card of the data receiving end through the network, and then copies the data to the memory of the data receiving end by the RDMA network card of the data receiving end to complete the data transmission.
If the data sent by the data sending end is large and exceeds the load capacity of the RDMA network card Buffer, the RDMA Write operation is executed, and the data is directly sent to the cache of the data receiving end.
However, in general, when performing an RDMA Write operation, a memory space for storing a data packet needs to be registered at a data receiving end, which results in low data transmission performance.
To this end, the first embodiment of the present application provides another data transmission scheme to improve the transmission performance when performing an RDMA Write operation.
Referring to fig. 1B, a data transmission method provided in an embodiment of the present application includes:
s101, a data sending end determines a data packet to be sent to a data receiving end from a first buffer area registered by a first RDMA network card, wherein the data packet occupies part of a storage space in the first buffer area.
In this embodiment, the data sending end and the data receiving end may be any two devices in the above system. The data packet sent by the data sending end may also be a data packet corresponding to an application program of the data sending end, which is not limited in this embodiment.
The data sending end and the data receiving end in this embodiment may be any electronic device including an RDMA network card, for example, a server, a PC, a mobile terminal (such as a mobile phone), a computing node in a cloud computing system, a storage node in a distributed storage system, and the like.
The RDMA network card, i.e., a network card supporting the RDMA protocol, also called RNIC, can enable the data transmitting end and the data receiving end to perform data transmission based on the RDMA protocol through the RDMA network card.
In this embodiment, the first RDMA network card is registered with the first buffer.
The storage space of the first cache region and the subsequent second cache region can be larger, so that a plurality of data packets can be stored in the first cache region and the subsequent second cache region, and each data packet occupies part of the storage space of the first storage region or the second storage region; of course, in this embodiment, a plurality of packets can be stored in the first buffer, but in actual use, only one packet may be stored in the first buffer, which is not limited in this embodiment.
S102, the data sending end determines a target storage address of the data packet stored in a second cache region according to a storage address of the data packet in the first cache region and a pre-obtained storage address of the second cache region of the data receiving end.
And the storage address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end.
In this embodiment, the second RDMA network card at the data receiving end is registered with the second buffer area.
The address of the second buffer area is obtained through one-time handshake interaction between the data sending end and the data receiving end, the specific handshake scheme refers to the related technology, and is not repeated again, and the data sending end and the data receiving end can perform multiple data transmissions after the one-time handshake interaction.
In this embodiment, a data packet may be stored in a first cache region, and when the data packet needs to be sent, a target storage address where the data packet is stored in a second cache region is determined according to a storage address of the data packet in the first cache region and a pre-obtained storage address of the second cache region of the data receiving end.
For example, if the storage address of the packet in the first cache region is a +10kb to a +20kb, where a is the starting address of the first cache region, if the size of the first cache region is the same as that of the second cache region, it may be determined that the target storage address of the packet stored in the second cache region is B +10k to B +20k, where B is the starting address of the second cache region.
For example, if it is determined that the storage address of the data packet in the first buffer area is C to C +10kb, where C is the storage start address of the data packet in the first buffer area, and 10k is the size of the data packet; determining that a target storage address of the data packet stored in the second cache region is D-D +10k, where D is an end address of the data packet in the second cache region determined when the data packet was last sent, and if the data packet is currently sent to the data receiving end for the first time, D may default to the start address of the second cache region; in another embodiment of the present application, the end address of the data packet in the second buffer area determined when the data packet was last transmitted and the size of the data packet transmitted this time may be directly stored as the data packet to the target storage address of the second buffer area.
S103, performing RDMA write operation based on the storage address of the data packet in the first cache region and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored to the target storage address in the second cache region.
The data sending end submits a target storage address, a storage address of a data packet in a first cache region and RDMA write operation to a first RDMA network card, the first RDMA network card can extract the data packet from the storage address of a second cache region and send the data packet to a second cache region of a data receiving end, and the data packet can be directly stored to the target storage address in the second cache region through the second RDMA network card of the data receiving end.
S104, the data receiving end processes the data packet in the second cache region and sends a response message indicating the completion of data packet processing to the data sending end.
In this embodiment, the data receiving end may read the data packet from the second buffer area, and process the read data packet. If the data packet is successfully processed, a response message indicating the completion of the data packet processing can be sent to the data sending end through the second RDMA network card, so that the data sending end can know the completion of the data packet processing.
S105, after the data sending end determines that the data packet is processed by the data receiving end, the data sending end releases the storage space occupied by the data packet in the first cache region.
In this embodiment, after it is determined that processing of a certain data packet is completed, it may be determined that the data packet is not required to be sent again by the data sending end, and then the storage space occupied by the data packet in the first buffer area may be released, so that the released storage space may be allocated to a new data packet, and reuse of the first buffer area is achieved.
Referring to fig. 1C, a specific usage scenario of the embodiment of the present application is shown.
As shown in fig. 1C, the data sender registers a first buffer, and the first buffer may store a plurality of data packets, which are Packet1, Packet2, Packet3, and Packet 4; the first buffer may maintain an address area in which the packet is stored through a first start pointer Sbegin and a first end pointer Rend.
The Sender may submit the target storage address of Packet4 in the second buffer, the RDMA Write operation, and the storage address of Packet4 to the first RDMA network card, send Packet4 to the second buffer of the data receiving end Receiver through the first RDMA network card, where the last sent Packet is Packet3, and then the target storage address of Packet4 in the second buffer may be replaced by the end address of Packet3 in the second buffer, and the Packet size of Packet 4.
As shown in fig. 1C, a plurality of packets, namely Packet1, Packet2, and Packet3, are stored in the second buffer, and the newly written Packet4 may be stored after Packet 3.
Similar to the first buffer, the second buffer may also determine the address area of the data packet being processed through a second start pointer rbegin and a second end pointer rend.
An application program of the data receiving terminal Receiver may process the data Packet of the second buffer, for example, process Packet1, and after the processing is completed, update the second start pointer rbegin, and send information that the processing of the data Packet1 is completed to the sender through the second RDMA network card.
The Sender may release the storage space occupied by the Packet1 in the first buffer, and update the first start pointer Sbegin, and the released storage space may be used for writing other packets.
According to the scheme provided by the embodiment of the application, after one-time handshake interaction between the data sending end and the data receiving end, the data sending end can obtain the storage address of the second cache region; when data transmission is carried out, determining a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, wherein the data packet occupies part of a storage space in the first cache region; the data sending end can determine a target storage address of the data packet stored in a second cache region according to a storage address of the data packet in a first cache region and a pre-obtained storage address of the second cache region of the data receiving end, and accordingly perform an RDMA write operation to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address of the second cache region; the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end, so that a first cache region and a second cache region capable of storing a plurality of data packets can be obtained by registering through one-time handshake in advance, the data packet is directly stored in the first cache region before the data packet is sent each time, namely, the data packet can be sent through RDMA (remote direct memory access) writing operation based on the first cache region and the second cache region, the process that the data receiving end registers the storage space for storing the data packet in the data transmission process through RDMA writing operation is optimized, the handshake overhead of one-time RTT (round trip time) of the data sending end and the data receiving end is saved at least, and the transmission performance of the data transmission through RDMA writing operation is greatly improved.
Example two
Fig. 2A is a flowchart illustrating steps of a data transmission method according to a second embodiment of the present application. As shown in the figure, the data transmission method includes:
s201, the data sending end determines an end address of a storage space allocated to the data packet according to the length of the data packet of the storage space to be allocated and the end address of the allocated area in the first cache area, so that the data packet is stored in the first cache area.
Illustratively, referring to fig. 2B, the first buffer maintains an address area in which the packet is stored through a first start pointer Sbegin and a first end pointer Rend. Wherein the first end pointer Rend may be used to indicate the end address of the allocated area in the first buffer area.
And when the storage space allocated for the data packet from the first buffer area is used, the first end pointer Rend is used as a starting address, the length of the data packet of the storage space to be allocated is increased, and therefore the end address of the storage space allocated for the data packet is determined, and allocation is completed.
After the data packet is stored, the first end pointer Rend may be updated to the end address of the memory space allocated for the data packet, so that the memory space in the first buffer area may be continuously allocated for other data packets according to the first end pointer Rend.
The end address of the storage space allocated to the data packet is determined according to the length of the data packet of the storage space to be allocated and the end address of the allocated area in the first cache area, so that the addresses of a plurality of data packets in the first cache area are continuous, and the utilization rate of the first cache area is improved.
S202, the data sending end determines a data packet to be sent to the data receiving end from a first buffer area registered by the first RDMA network card.
For the specific determination process, reference may be made to related technologies, and details are not repeated again in this embodiment.
S203, the data sending end determines a target storage address of the data packet stored in a second cache region according to the storage address of the data packet in the first cache region and the pre-obtained storage address of the second cache region of the data receiving end, and accordingly performs RDMA write operation to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address of the second cache region.
In this embodiment, after determining the data packet to be sent to the data receiving end, the storage address, the target storage address, and the RDMA write operation of the data packet in the first buffer area may be submitted to the first RDMA network card, so that the data packet is extracted by the first RDMA network card of the data sending end and sent to the second buffer area of the data receiving end.
Optionally, in this embodiment, in order to facilitate the use of the first buffer area and the second buffer area, the second buffer area is a buffer area that is registered in advance by the data receiving end through a second RDMA network card, and the size of the second buffer area is the same as that of the first buffer area. Therefore, the storage addresses of the data packets in the first cache region and the storage addresses of the data packets in the second cache region can be mutually verified, and the accuracy of the data is further ensured.
In addition, when the size of the first buffer area is the same as that of the second buffer area, the storage address of the data packet in the first buffer area can be directly used as the target storage address of the data packet in the second buffer area.
Optionally, in this embodiment, in order to better determine whether to write the data packet into the second buffer, so as to facilitate application of the data transmission scheme provided in this embodiment, it may be identified that the data packet is stored into the second buffer of the data receiving end through the instant data of the RDMA write operation.
The instant data (imm data) of the RDMA write operation comprises 32 bits, and can be added when a first RDMA network card of a data sending end executes the RDMA write operation, and after a second RDMA network card of a data receiving end receives data corresponding to the write operation, the second RDMA network card can directly judge whether the data sent by the RDMA write operation is written into a second buffer area in a zero-copy mode according to the value of an identification bit in a second buffer area used for identifying and extracting a data packet and sending the data packet to the data receiving end in the instant data (imm data) of the RDMA write operation.
Illustratively, the identification may be by one or more bits in the immediate data of the RDMA write operation. When a bit in the instant data through the RDMA write operation is identified, when the value of the identification bit is 0, the data sent through the RDMA write operation is not written into the second buffer zone in a zero copy mode; otherwise, writing is carried out. With a multi-bit identification, for example with a two-bit identification, the two identifications are each used to indicate that data sent by an RDMA write operation is to be written to the second buffer with zero copy or to the other buffer.
Specifically, a scheme of extracting a data packet and sending the data packet to the second cache region of the data receiving end through an instant data (imm data) identifier of an RDMA write operation may be set by a person skilled in the art according to a requirement, which is not limited in this embodiment.
S204, the data receiving end processes the data packet in the second cache region, releases the storage space occupied by the data packet processed in the second cache region, and sends a response message indicating the completion of the data packet processing to the data sending end.
In this embodiment, after the data receiving end has processed a certain data packet, the storage space occupied by the processed data packet in the second buffer area may also be released.
For example, the second buffer may also determine an address region of the data packet being processed through a second start pointer rbegin and a second end pointer rend. After a certain data packet is processed, the second start pointer rbegin of the second buffer area may be updated to release the storage space occupied by the processed data packet in the second buffer area.
Optionally, in this embodiment, the processing, by the data receiving end, the data packet in the second buffer includes: verifying the integrity of the data packet according to the packet header of the data packet to be processed; and if the verification is passed, submitting the memory address mapped to the storage space occupied by the data packet to be processed in the second cache region to the application program of the data receiving end so as to process the data packet through the application program of the data receiving end. Therefore, the data packet in the second cache region can be directly processed through the application program, the data processing efficiency is improved, and the network card buffer does not need to be occupied.
For example, the second buffer may also determine an address region of the data packet being processed through a second start pointer rbegin and a second end pointer rend. When processing of a packet is to be started, a packet with the second end pointer rend as the start address may be read. In general, when the head of the packet is the header of the packet, the header of the packet may be read first when the packet with the second end pointer rend as the start address is read.
After reading the data packet, the second end pointer rend may be updated to the end position of the data packet according to the length of the data packet, so as to continue reading the next data packet according to the second end pointer rend.
Metadata, verification information and the like of the data body are stored in the packet head of the data packet, and whether the data packet is completely received or not is verified according to the packet head of the data packet.
Specifically, the packet header of the data packet includes a fragment number and a length of the data packet;
correspondingly, the verifying the integrity of the data packet according to the packet header of the data packet to be processed includes:
determining a storage space of the data packet according to a storage address of the packet header of the data packet in the second cache region and the length of the data packet included in the packet header of the data packet;
and verifying the integrity of the determined fragments in the storage space according to the fragment numbers included in the packet headers of the data packets.
Specifically, when the data packet includes a plurality of fragments, the number range of the fragment included in the header of the data packet.
If the verification is passed, the memory address mapped to the storage space occupied by the data packet in the second cache region may be submitted to the application program of the data receiving end, so as to process the data packet by the application program of the data receiving end. Therefore, the application program at the data receiving end can directly read and use the data of the data packet from the acquired memory address.
Optionally, in this embodiment, in order to reuse a storage space corresponding to a data packet of a submitted memory address in the second cache region, the memory address may be remapped for a storage space occupied by the data packet of the submitted memory address in the second cache region. When a new data packet is stored in the storage space occupied by the second cache region of the remapped memory, the data packet can be directly written into the reallocated memory.
S205, the data sending end receives a response message that the data receiving end responds to a sending message for sending a data packet at a certain time.
In this embodiment, the response message may include a storage address of the transmitted data packet in the second buffer and information of the number of times that the second buffer is released.
Specifically, after the cache release operation is performed, the second start pointer rbegin of the second cache region may be updated, and the updated second start pointer rbegin update ═ x may be included in the response message.
In addition, the data receiving end can also maintain release times Version ra, and the value of Version ra is added with 1 every time the cache release operation is executed. The updated value y of Version ra may be included in the response message as the information of the number of times the second buffer is released.
S206, comparing the number of times that the data sending end releases the buffer of the first buffer with the number of times indicated by the information of the number of times that the second buffer is released included in the response message.
If the number of times of the cache release performed by the data sending end on the first cache area is less than the number of times indicated by the number-of-times information that the second cache area is released and included in the response message, continue to execute step S207.
In this embodiment, the data receiving end may also maintain a release frequency Version sa, and each time the cache release operation on the first cache region is executed, the value of Version sa is incremented by 1.
Upon receiving the response message, Version ra and Version sa in the response message may be compared. If Version sa < Version ra, step S207 is performed.
And S207, the data sending end releases the corresponding storage space in the first cache region according to the storage address of the data packet in the second cache region, wherein the storage address is included in the response information.
And updating a first start pointer Sbox of the first cache region through a second start pointer rbbox included in the response information so as to release the corresponding storage space in the first cache region.
In addition, in the embodiment of the present application, in order to enhance the security of data, a loop back operation may be performed on the first cache region.
Specifically, since data transmission is performed by RDMA write operation, in order to implement zero copy in the transmission process, when allocating the storage space of the first buffer area for a data packet, a continuous storage address needs to be allocated for the data packet.
If the size of the unallocated area between the ending address of the last packet in the first cache area and the ending address of the first cache area is smaller than the size of the packet of which the storage space is to be allocated, allocating the storage space for the packet in the unallocated area between the starting address of the first cache area and the starting address of the first packet, so as to perform loop back operation on the first cache area.
For example, referring to fig. 2B, when allocating a storage space for a packet, if the size of an unallocated area located after the first ending pointer Rend in the first buffer area is smaller than the size of the packet to be allocated with the storage space, the storage space is allocated for the packet in the unallocated area from the starting address of the first buffer area to the starting pointer Sbegin, so as to perform a loopback operation on the first buffer area.
Optionally, in this embodiment, the information indicating the loop back operation is sent to the data receiving end by the first RDMA network card of the data sending end, so that the data receiving end performs the loop back operation on the second buffer area according to the information indicating the loop back operation. Therefore, the first cache region and the second cache region can synchronously carry out loop back operation, and the accuracy of data is further ensured.
For example, when the data receiving end performs loop back operation on the second cache region according to the information indicating loop back operation, if it is determined that no other data packet is stored in the subsequent storage space according to the information of the loop back operation when the last data packet in the second cache region is read, the data receiving end may jump to the start address of the second cache region to continue reading the data packet.
Specifically, in this embodiment, the information for indicating the loopback operation may include loopback number information of the data sending end and end address information of the allocated area in the first cache area. Correspondingly, the performing loop back operation on the second cache region according to the information indicating the loop back operation includes:
verifying whether the current loop times of the data receiving end are consistent with the times indicated by the loop time information;
if the data packets are the same, determining whether the storage address of the processed data packet in the second cache area is matched with the ending address indicated by the ending address information;
and if the first cache region is matched with the second cache region, performing loop back operation on the second cache region.
Therefore, the loop times of the first cache region and the second cache region can be ensured to be consistent through the times indicated by the loop time information; and after determining to read the correct storage address in the second cache region through the end address information of the allocated region in the first cache region, performing loop back operation to ensure the accuracy of the data.
For example, after performing the loopback operation, the data sending end may generate jump information, where the jump information indicates the ending address information m of the allocated area in the first buffer area. The data sender can also maintain a loop-back frequency Version sc, each pair of first cache regions execute a loop-back operation, and the value of the Version sc is added with 1. The information for indicating the loopback operation may include that the loopback number information of the data sender may be a value n of the updated Version sc.
If the current loop times of the data receiving end are larger than the times indicated by the loop time information, discarding the information for indicating the loop operation; or, if the current loop times of the data receiving end is less than the times indicated by the loop time information, queuing to retain information for indicating the loop operation. Therefore, the information used for indicating the loop back operation can be used, the disorder of the loop back operation is avoided, and the accuracy of the data is ensured.
In addition, during the data transmission, a transmission failure may also occur, and for this reason, in this embodiment, if the data sending end determines that the RDMA write operation fails, the data sending end copies the data packet corresponding to the failed RDMA write operation to the cache of the second RDMA network card at the data receiving end through an RDMA copy operation. Therefore, the data packet with the RDMA write operation failure can be sent to the data receiving end again, so that the data retransmission is realized, and the data integrity is ensured.
Specifically, in this embodiment, when performing an RDMA copy operation, the data sending end sends, to the data receiving end, information about the number of current loop back operations of the data sending end, so that the data receiving end verifies whether the number of loop back operations performed on the second buffer area is consistent with the number indicated by the number information about the loop back operations carried in the data packet;
if the data packet is consistent with the second RDMA network card, copying the data packet from the cache of the second RDMA network card to the second cache region;
or if the data packets are inconsistent, discarding the data packets.
For example, when the data packet a may be copied by an RDMA copy operation, the data packet a may be used as a data body of the RDMA copy operation, a new data packet header may be added thereto by the RDMA copy operation, and information on the number of current loop back operations of the data transmitting end may be added to the new data packet header, so as to generate a new data packet B. And copying the data packet B into a buffer of a second RDMA network card at the data receiving end through the RDMA copy operation.
And then judging whether the number of times of loop back operation carried out on the second buffer area by the data receiving end is verified by the data receiving end and is consistent with the number of times indicated by the number information of the loop back operation carried in the data packet or not through a second RDMA network card according to the number information of the current loop back operation carried in the packet head of the data packet B by the data transmitting end. If the data volume of the data packet A is consistent with the data volume of the data packet B, the data packet A which is the data volume of the data packet B is written into the second cache region. Otherwise packet a is discarded.
According to the scheme provided by the embodiment, the process that the data receiving end registers the storage space for storing the data packet in the data transmission process through RDMA write operation is optimized through the pre-registered first buffer area and the pre-registered second buffer area, so that the handshake overhead of one RTT (round trip time) of the data transmitting end and the data receiving end is saved at least once, and the transmission performance during data transmission through RDMA write operation is greatly improved; and the accuracy of data storage in the first cache region and the second cache region is ensured by managing the loop times of the first cache region and the second cache region.
EXAMPLE III
Fig. 3 is a block diagram of a data transmission apparatus according to a third embodiment of the present application, where as shown in the figure, the data transmission apparatus is configured at a data sending end, and the data transmission apparatus includes:
a first determining module 301, configured to determine, from a first buffer registered in a first RDMA network card, a data packet to be sent to a data receiving end, where the data packet occupies a part of a storage space in the first buffer;
an extracting module 302, configured to determine, according to a storage address of the data packet in a first buffer and a pre-obtained storage address of a second buffer of the data receiving end, a target storage address at which the data packet is stored in the second buffer, and perform an RDMA write operation based on the storage address of the data packet in the first buffer and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address in the second buffer, where the storage address of the second buffer is obtained through one-time handshake interaction between the data sending end and the data receiving end;
a releasing module 303, configured to release a storage space occupied by the data packet in the first cache region after it is determined that the data packet has been processed by the data receiving end.
Optionally, the second buffer area is a buffer area pre-registered by the data receiving end through a second RDMA network card, and the size of the second buffer area is the same as that of the first buffer area.
Optionally, the apparatus further comprises:
and the second determining module is used for determining the end address of the storage space allocated to the data packet according to the length of the data packet of the storage space to be allocated and the end address of the allocated area in the first cache region, so as to store the data packet into the first cache region.
Optionally, the apparatus further comprises:
and the allocation module is used for allocating a storage space for the data packet in the unallocated area from the start address of the first cache area to the start address of the first data packet to perform a loopback operation on the first cache area if the size of the unallocated area between the end address of the last data packet in the first cache area and the end address of the first cache area is smaller than the size of the data packet of which the storage space is to be allocated.
Optionally, the apparatus further comprises:
a second sending module, configured to send, to the data receiving end, information for indicating the loop back operation through the first RDMA network card of the data sending end, so that the data receiving end performs the loop back operation on the second buffer area according to the information for indicating the loop back operation.
Optionally, the apparatus further comprises:
the first copying module is configured to, if the data sending end determines that the RDMA write operation fails, copy, by the RDMA copy operation, a data packet corresponding to the failed RDMA write operation to a cache of the second RDMA network card of the data receiving end.
Optionally, the apparatus further comprises:
a third sending module, configured to send, to the data receiving end, information about the number of times of the current loop back operation of the data sending end, so that the data receiving end verifies whether the number of times of the loop back operation performed on the second cache area is consistent with the number of times indicated by the information about the number of times of the loop back operation carried in the data packet;
the second copying module is used for copying the data packet from the cache of the second RDMA network card to the second cache region when the data packet is consistent with the second RDMA network card;
or, a discarding module is included for discarding the data packet when there is an inconsistency.
Optionally, the apparatus further comprises:
a first receiving module, configured to receive a response message that the data receiving end responds to a sending message used for sending a data packet at a certain time, where the response message includes a storage address of the sent data packet in the second cache area and information of the number of times that the second cache area is released;
the releasing module is configured to, if the number of times that the data sending end releases the cache of the first cache area is less than the number of times indicated by the number-of-times information that the second cache area is released, the data sending end releases a corresponding storage space in the first cache area according to the storage address of the data packet in the second cache area, where the storage address is included in the response message.
Optionally, the apparatus further comprises:
and the fourth sending module is used for identifying and extracting a data packet through the instant data of the RDMA write operation and sending the data packet to the second cache region of the data receiving end.
Example four
Fig. 4 is a block diagram of a data transmission device according to a fourth embodiment of the present application, where the data transmission device is configured at a data receiving end, and as shown in the figure, the data transmission device includes:
a writing module 401, configured to write a packet sent by a data sending end through an RDMA write operation into a target storage address in a second buffer, where the RDMA write operation is performed based on a storage address of the packet in a first buffer and the target storage address, the target storage address is determined by the data sending end according to the storage address of the packet in the first buffer and a pre-obtained storage address of a second buffer of the data receiving end, and the storage address of the second buffer is obtained through one-time handshake interaction between the data sending end and the data receiving end, where the packet occupies a part of a storage space in the first buffer;
a first sending module 402, configured to process the data packet in the second cache region, and send a response message indicating that the data packet processing is completed to the data sending end, so that the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end.
Optionally, the apparatus further comprises: a second receiving module, configured to receive information sent by the data sending end and used to instruct the data sending end to perform a loop back operation on the first cache region;
and the loop back operation module is used for performing loop back operation on the second cache region according to the information for indicating the loop back operation.
Optionally, the information indicating the loopback operation includes loopback number information of the data sending end and end address information of an allocated area in the first cache region;
the loop back operation module comprises: the loop time verifying module is used for verifying whether the current loop time of the data receiving end is consistent with the time indicated by the loop time information;
if the data packets are consistent with the end address indicated by the end address information, determining whether the storage addresses of the processed data packets in the second cache region are matched with the end address indicated by the end address information through an address matching module; and if the first cache region is matched with the second cache region, performing loop back operation on the second cache region.
Optionally, if the current loop times of the data receiving end are greater than the times indicated by the loop time information, discarding information for indicating the loop operation;
or, if the current loop times of the data receiving end is less than the times indicated by the loop time information, queuing to retain information for indicating the loop operation.
Optionally, the apparatus further comprises: and the third receiving module is used for receiving a data packet sent by the data sending end through the RDMA copy operation, and storing the data packet into the cache of the second RDMA network card, wherein the data packet sent by the data sending end through the RDMA copy operation is a data packet corresponding to the failed RDMA write operation.
Optionally, the apparatus further comprises: a loop time obtaining module, configured to obtain information of a current loop operation time of the data sending end sent by the data sending end;
the loop time verifying module is further configured to verify whether the number of loop operations performed on the second cache area is consistent with the number indicated by the loop time information of the loop operations carried in the data packet;
the copying module is used for copying the data packet from the cache of the second RDMA network card to the second cache region if the data packet is consistent with the second RDMA network card; or if the data packets are inconsistent, discarding the data packets.
Optionally, the apparatus further comprises: and the second cache area releasing module is used for releasing the storage space occupied by the data packet which is managed and finished in the second cache area.
Optionally, the processing module comprises: the integrity verification module is used for verifying the integrity of the data packet according to the packet head of the data packet to be processed; and if the verification is passed, submitting the memory address mapped to the storage space occupied by the data packet to be processed in the second cache region to an application program of the data receiving end through a submitting module so as to process the data packet through the application program of the data receiving end.
Optionally, the apparatus further comprises: and the remapping module is used for remapping the memory address for the storage space occupied by the data packet submitted with the memory address in the second cache region.
Optionally, the fragment number included in the header of the data packet and the length of the data packet;
correspondingly, the integrity verification module comprises: the region determining module is used for determining the storage space of the data packet according to the storage address of the packet header of the data packet in the second cache region and the length of the data packet included in the packet header of the data packet; and the fragment verification module is used for verifying the integrity of the determined fragments in the storage space according to the fragment numbers included in the packet headers of the data packets.
The data transmission device in the embodiment of the application is used for implementing the corresponding data transmission method in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiment, which are not described herein again. In addition, the functional implementation of each module in the data transmission device of this embodiment can refer to the description of the corresponding part in the foregoing method embodiment, and is not repeated here.
Referring to fig. 5, a schematic structural diagram of an electronic device according to a fifth embodiment of the present application is shown, and the specific embodiment of the present application does not limit a specific implementation of the electronic device.
As shown in fig. 5, the electronic device may include: a processor (processor)502, a Communications Interface 504, a memory 506, and a communication bus 508.
Wherein:
the processor 502, communication interface 504, and memory 506 communicate with one another via a communication bus 508.
A communication interface 504 for communicating with other electronic devices or servers.
The processor 502 is configured to execute the program 510, and may specifically perform relevant steps in the above-described data transmission method embodiment.
In particular, program 510 may include program code that includes computer operating instructions.
The processor 502 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement embodiments of the present application. The intelligent device comprises one or more processors which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.
And a memory 506 for storing a program 510. The memory 506 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
For specific implementation of each step in the program 510, reference may be made to corresponding steps and corresponding descriptions in units in the foregoing data transmission method embodiments, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.
It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present application may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present application.
The above-described methods according to embodiments of the present application may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the methods described herein may be stored in such software processes on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the data transmission methods described herein. Further, when a general-purpose computer accesses code for implementing the data transmission method illustrated herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the data transmission method illustrated herein.
Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the embodiments of the present application.
The above embodiments are only used for illustrating the embodiments of the present application, and not for limiting the embodiments of the present application, and those skilled in the relevant art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application, so that all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the scope of patent protection of the embodiments of the present application should be defined by the claims.

Claims (23)

1. A method of data transmission, the method comprising:
the method comprises the steps that a data sending end determines a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, wherein the data packet occupies part of storage space in the first cache region;
the data sending end determines a target storage address of the data packet stored in a second cache area according to a storage address of the data packet in a first cache area and a pre-obtained storage address of the second cache area of the data receiving end, wherein the storage address of the second cache area is obtained through one-time handshake interaction between the data sending end and the data receiving end;
performing an RDMA write operation based on the storage address of the data packet in a first cache region and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored to the target storage address in a second cache region;
and after the data sending end determines that the data packet is processed by the data receiving end, the data sending end releases the storage space occupied by the data packet in the first cache region.
2. The method of claim 1, wherein the second buffer is a buffer pre-registered by the data-receiving end via a second RDMA network card, the second buffer being the same size as the first buffer.
3. The method of claim 1, wherein the method further comprises:
and the data sending end determines the end address of the storage space allocated to the data packet according to the length of the data packet of the storage space to be allocated and the end address of the allocated area in the first cache area, so as to store the data packet into the first cache area.
4. The method of claim 3, wherein the method further comprises:
if the size of an unallocated area between the ending address of the last data packet in the first cache area and the ending address of the first cache area is smaller than the size of a data packet of a storage space to be allocated, allocating a storage space for the data packet in the unallocated area between the starting address of the first cache area and the starting address of the first data packet so as to perform loop back operation on the first cache area.
5. The method of claim 4, wherein the method further comprises:
and sending information for indicating the loop back operation to the data receiving end through the first RDMA network card of the data sending end, so that the data receiving end performs the loop back operation on the second cache region according to the information for indicating the loop back operation.
6. The method of claim 4, wherein the method further comprises:
if the data sending end determines that the RDMA write operation fails, the data sending end copies a data packet corresponding to the failed RDMA write operation to a cache of the second RDMA network card of the data receiving end through RDMA copy operation.
7. The method of claim 6, wherein the method further comprises:
the data sending end sends the current loop back operation frequency information of the data sending end to the data receiving end, so that the data receiving end verifies whether the loop back operation frequency of the second cache area is consistent with the frequency indicated by the loop back operation frequency information carried in the data packet or not;
if the data packet is consistent with the second RDMA network card, copying the data packet from the cache of the second RDMA network card to the second cache region;
or if the data packets are inconsistent, discarding the data packets.
8. The method of claim 1, wherein the method further comprises:
the data sending end receives a response message of the data receiving end responding to a sending message used for sending a data packet at a certain time, wherein the response message comprises a storage address of the sent data packet in the second cache area and the information of the number of times that the second cache area is released;
the data sending end releases the storage space occupied by the data packet in the first cache region, and the method comprises the following steps:
if the number of times of cache release performed by the data sending end on the first cache area is less than the number of times indicated by the number-of-times information that the second cache area is released, which is included in the response message, the data sending end releases the corresponding storage space in the first cache area according to the storage address of the data packet in the second cache area, which is included in the response message.
9. The method of claim 1, wherein the method further comprises:
identifying that the data packet is stored in a second buffer of the data sink via immediate data of the RDMA write operation.
10. A method of data transmission, the method comprising:
writing, by a data receiving end, a data packet sent by a data sending end through an RDMA (remote direct memory access) write operation into a target storage address in a second cache region, wherein the RDMA write operation is executed based on the storage address of the data packet in a first cache region and the target storage address, the target storage address is determined by the data sending end according to the storage address of the data packet in the first cache region and a pre-obtained storage address of the second cache region of the data receiving end, and the address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end, wherein the data packet occupies part of a storage space in the first cache region;
and the data receiving end processes the data packet in the second cache region and sends a response message indicating the completion of data packet processing to the data sending end, so that the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end.
11. The method of claim 10, wherein the method further comprises:
receiving information which is sent by the data sending end and used for indicating the data sending end to carry out loop back operation on the first cache region;
and performing loop back operation on the second cache region according to the information for indicating the loop back operation.
12. The method according to claim 11, wherein the information indicating the loopback operation includes loopback number information of the data transmitting end and end address information of an allocated region in the first buffer;
the performing loop back operation on the second cache region according to the information indicating the loop back operation includes:
verifying whether the current loop times of the data receiving end are consistent with the times indicated by the loop time information;
if the data packets are consistent with the end address indicated by the end address information, determining whether the storage address of the processed data packets in the second cache region is matched with the end address indicated by the end address information;
and if the first cache region is matched with the second cache region, performing loop back operation on the second cache region.
13. The method of claim 12, wherein the method further comprises:
if the current loop times of the data receiving end are larger than the times indicated by the loop time information, discarding the information for indicating the loop operation;
or, if the current loop times of the data receiving end is less than the times indicated by the loop time information, queuing to retain information for indicating the loop operation.
14. The method of claim 10, wherein the method further comprises:
and receiving a data packet sent by the data sending end through RDMA copy operation, and storing the data packet into a cache of the second RDMA network card, wherein the data packet sent by the data sending end through the RDMA copy operation is a data packet corresponding to the failed RDMA write operation.
15. The method of claim 14, wherein the method further comprises:
acquiring the number information of loop back operation carried by the data packet when the data sending end sends the data packet;
verifying whether the times of loop back operation carried out on the second cache region are consistent with the times indicated by the information of the times of loop back operation carried in the data packet;
if the data packet is consistent with the second RDMA network card, copying the data packet from the cache of the second RDMA network card to the second cache region;
or if the data packets are inconsistent, discarding the data packets.
16. The method of claim 10, wherein the method further comprises:
and the data receiving end processes the data packet in the second cache region and releases the storage space occupied by the data packet processed in the second cache region.
17. The method of claim 10, wherein the data receiving end processes the data packet in the second buffer, comprising:
verifying the integrity of the data packet according to the packet header of the data packet to be processed;
and if the verification is passed, the memory address to which the storage space occupied by the data packet to be processed in the second cache region is mapped is submitted to the application program of the data receiving end, so that the data packet is processed by the application program of the data receiving end.
18. The method of claim 17, wherein the method further comprises: and remapping the memory address for the storage space occupied by the data packet submitted with the memory address in the second cache region.
19. The method of claim 17, wherein the fragment number included in the header of the data packet and the length of the data packet;
correspondingly, the verifying the integrity of the data packet according to the packet header of the data packet to be processed includes:
determining a storage space of the data packet according to a storage address of the packet header of the data packet in the second cache region and the length of the data packet included in the packet header of the data packet;
and verifying the integrity of the determined fragments in the storage space according to the fragment numbers included in the packet headers of the data packets.
20. A data transmission apparatus, the apparatus comprising:
the device comprises a first determining module, a first sending module and a second determining module, wherein the first determining module is used for determining a data packet to be sent to a data receiving end from a first cache region registered by a first RDMA network card, and the data packet occupies part of a storage space in the first cache region;
an extracting module, configured to determine, according to a storage address of the data packet in a first cache region and a pre-obtained storage address of a second cache region of the data receiving end, a target storage address at which the data packet is stored in the second cache region, and perform an RDMA write operation based on the storage address of the data packet in the first cache region and the target storage address to extract the data packet and send the data packet to the data receiving end, so that the data packet is stored in the target storage address in the second cache region, where the storage address of the second cache region is obtained through one-time handshake interaction between the data sending end and the data receiving end;
and the releasing module is used for releasing the storage space occupied by the data packet in the first cache region after the data packet is determined to be processed by the data receiving end.
21. A data transmission apparatus, the apparatus comprising:
a writing module, configured to write a packet sent by a data sending end through an RDMA write operation into a target storage address in a second buffer, where the RDMA write operation is performed based on a storage address of the packet in a first buffer and the target storage address, the target storage address is determined by the data sending end according to the storage address of the packet in the first buffer and a pre-obtained storage address of a second buffer of the data receiving end, and the storage address of the second buffer is obtained through one-time handshake interaction between the data sending end and the data receiving end, where the packet occupies a part of a storage space in the first buffer;
and the first sending module is used for processing the data packet in the second cache region and sending a response message indicating the completion of data packet processing to the data sending end, so that the data sending end releases the storage space occupied by the data packet in the first cache region after determining that the data packet is processed by the data receiving end.
22. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;
the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the operation corresponding to the data transmission method as claimed in any one of claims 1 to 9 or 10 to 19.
23. A computer storage medium, on which a computer program is stored which, when being executed by a processor, carries out the data transmission method according to any one of claims 1 to 9 or 10 to 19.
CN202110164907.1A 2021-02-05 2021-02-05 Data transmission method and device, electronic equipment and computer storage medium Pending CN113422793A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110164907.1A CN113422793A (en) 2021-02-05 2021-02-05 Data transmission method and device, electronic equipment and computer storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110164907.1A CN113422793A (en) 2021-02-05 2021-02-05 Data transmission method and device, electronic equipment and computer storage medium

Publications (1)

Publication Number Publication Date
CN113422793A true CN113422793A (en) 2021-09-21

Family

ID=77711805

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110164907.1A Pending CN113422793A (en) 2021-02-05 2021-02-05 Data transmission method and device, electronic equipment and computer storage medium

Country Status (1)

Country Link
CN (1) CN113422793A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114024874A (en) * 2021-10-29 2022-02-08 浪潮商用机器有限公司 RDMA (remote direct memory Access) -based data transmission method, device, equipment and storage medium
CN115442319A (en) * 2022-08-31 2022-12-06 北京天融信网络安全技术有限公司 Data transmission method, electronic device, and computer-readable storage medium
CN117573043A (en) * 2024-01-17 2024-02-20 济南浪潮数据技术有限公司 Transmission method, device, system, equipment and medium for distributed storage data

Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101409715A (en) * 2008-10-22 2009-04-15 中国科学院计算技术研究所 Method and system for communication using InfiniBand network
EP2611093A1 (en) * 2011-12-21 2013-07-03 Telefonaktiebolaget LM Ericsson (publ) Technique for handling memory resources of a memory buffer
CN103823721A (en) * 2014-02-26 2014-05-28 京信通信系统（广州）有限公司 Interprocess communication method and equipment
CN105302739A (en) * 2014-07-21 2016-02-03 深圳市中兴微电子技术有限公司 Memory management method and device
US20170155717A1 (en) * 2015-11-30 2017-06-01 Intel Corporation Direct memory access for endpoint devices
WO2017166298A1 (en) * 2016-04-01 2017-10-05 Nokia Technologies Oy Method, apparatus and computer program product for data distribution
CN107547508A (en) * 2017-06-29 2018-01-05 新华三信息安全技术有限公司 A kind of message sending, receiving method, device and the network equipment
CN108228476A (en) * 2017-12-22 2018-06-29 新华三技术有限公司 A kind of data capture method and device
CN110419034A (en) * 2017-04-14 2019-11-05 华为技术有限公司 A kind of data access method and device
CN110505112A (en) * 2019-07-09 2019-11-26 星融元数据技术(苏州)有限公司 A kind of network performance monitoring method, device and storage medium
CN110888827A (en) * 2018-09-10 2020-03-17 华为技术有限公司 Data transmission method, device, equipment and storage medium
US20200089649A1 (en) * 2018-09-13 2020-03-19 Microsoft Technology Licensing, Llc Transport Protocol and Interface for Efficient Data Transfer Over RDMA Fabric
CN110896406A (en) * 2018-09-13 2020-03-20 华为技术有限公司 Data storage method and device and server
CN111225042A (en) * 2019-12-27 2020-06-02 腾讯科技（深圳）有限公司 Data transmission method and device, computer equipment and storage medium
CN111400213A (en) * 2019-09-29 2020-07-10 杭州海康威视系统技术有限公司 Method, device and system for transmitting data
US20200274832A1 (en) * 2019-02-22 2020-08-27 Microsoft Technology Licensing, Llc Rdma transport with hardware integration and out of order placement
US20200322466A1 (en) * 2017-12-27 2020-10-08 Huawei Technologies Co., Ltd. Data Transmission Method and First Device

Patent Citations (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101409715A (en) * 2008-10-22 2009-04-15 中国科学院计算技术研究所 Method and system for communication using InfiniBand network
EP2611093A1 (en) * 2011-12-21 2013-07-03 Telefonaktiebolaget LM Ericsson (publ) Technique for handling memory resources of a memory buffer
CN103823721A (en) * 2014-02-26 2014-05-28 京信通信系统（广州）有限公司 Interprocess communication method and equipment
CN105302739A (en) * 2014-07-21 2016-02-03 深圳市中兴微电子技术有限公司 Memory management method and device
US20170155717A1 (en) * 2015-11-30 2017-06-01 Intel Corporation Direct memory access for endpoint devices
WO2017166298A1 (en) * 2016-04-01 2017-10-05 Nokia Technologies Oy Method, apparatus and computer program product for data distribution
CN110419034A (en) * 2017-04-14 2019-11-05 华为技术有限公司 A kind of data access method and device
CN107547508A (en) * 2017-06-29 2018-01-05 新华三信息安全技术有限公司 A kind of message sending, receiving method, device and the network equipment
CN108228476A (en) * 2017-12-22 2018-06-29 新华三技术有限公司 A kind of data capture method and device
US20200322466A1 (en) * 2017-12-27 2020-10-08 Huawei Technologies Co., Ltd. Data Transmission Method and First Device
CN110888827A (en) * 2018-09-10 2020-03-17 华为技术有限公司 Data transmission method, device, equipment and storage medium
US20200089649A1 (en) * 2018-09-13 2020-03-19 Microsoft Technology Licensing, Llc Transport Protocol and Interface for Efficient Data Transfer Over RDMA Fabric
CN110896406A (en) * 2018-09-13 2020-03-20 华为技术有限公司 Data storage method and device and server
US20200274832A1 (en) * 2019-02-22 2020-08-27 Microsoft Technology Licensing, Llc Rdma transport with hardware integration and out of order placement
CN110505112A (en) * 2019-07-09 2019-11-26 星融元数据技术(苏州)有限公司 A kind of network performance monitoring method, device and storage medium
CN111400213A (en) * 2019-09-29 2020-07-10 杭州海康威视系统技术有限公司 Method, device and system for transmitting data
CN111225042A (en) * 2019-12-27 2020-06-02 腾讯科技（深圳）有限公司 Data transmission method and device, computer equipment and storage medium

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
刘路;张磊;曹继军;戴艺;: "基于动态连接的RDMA可靠传输协议设计", 计算机工程与科学, no. 08, 15 August 2012 (2012-08-15) *
向敏;文成亮;王平;田力;: "基于IPv6的WIA-PA网络优化AODV路由算法", 仪器仪表学报, no. 12, 15 December 2015 (2015-12-15) *
韩文炳;陈小刚;李顺芬;李大刚;陈诗雁;段有康;宋志棠;: "基于新型非易失内存的远程零拷贝文件系统", 国防科技大学学报, no. 03, 28 June 2020 (2020-06-28) *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114024874A (en) * 2021-10-29 2022-02-08 浪潮商用机器有限公司 RDMA (remote direct memory Access) -based data transmission method, device, equipment and storage medium
CN115442319A (en) * 2022-08-31 2022-12-06 北京天融信网络安全技术有限公司 Data transmission method, electronic device, and computer-readable storage medium
CN115442319B (en) * 2022-08-31 2024-03-12 北京天融信网络安全技术有限公司 Data transmission method, electronic device, and computer-readable storage medium
CN117573043A (en) * 2024-01-17 2024-02-20 济南浪潮数据技术有限公司 Transmission method, device, system, equipment and medium for distributed storage data

Similar Documents

Publication Publication Date Title
US20200314181A1 (en) Communication with accelerator via RDMA-based network adapter
CN113422793A (en) Data transmission method and device, electronic equipment and computer storage medium
JP4504977B2 (en) Data processing for TCP connection using offload unit
CA2509404C (en) Using direct memory access for performing database operations between two or more machines
CN109690510B (en) Multicast apparatus and method for distributing data to multiple receivers in high performance computing networks and cloud-based networks
CN105630731A (en) Network card data processing method and device in multi-CPU (Central Processing Unit) environment
CN111447155A (en) Data transmission method, device, equipment and storage medium
CN104317716B (en) Data transmission method and distributed node equipment between distributed node
CN113014662A (en) Data processing method and storage system based on NVMe-oF protocol
WO2017032152A1 (en) Method for writing data into storage device and storage device
CN114442922A (en) Storage system, storage device and operation method thereof
CN109478171B (en) Improving throughput in openfabics environment
CN113076281B (en) Method, system, equipment and medium for communication of Ceph kernel client
US20160057068A1 (en) System and method for transmitting data embedded into control information
CN111404842B (en) Data transmission method, device and computer storage medium
CN116303173B (en) Method, device and system for reducing RDMA engine on-chip cache and chip
CN113238856A (en) RDMA (remote direct memory Access) -based memory management method and device
EP2760178A1 (en) Method, apparatus and system for obtaining object
CN113973091A (en) Message processing method, network equipment and related equipment
CN113422792A (en) Data transmission method and device, electronic equipment and computer storage medium
US20220263869A1 (en) Data validation for zero copy protocols
CN116032498A (en) Memory area registration method, device and equipment
CN113904976A (en) Multi-path data transmission method and device for lossy network based on RDMA
CN113326242A (en) Data processing method and device, electronic equipment and computer storage medium
US20170265103A1 (en) Communication device, communication method, and non-transitory computer readable medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40058772

Country of ref document: HK