WO2022151820A1 - Système de transmission de données, procédé de transmission de données, et dispositif de transmission - Google Patents

Système de transmission de données, procédé de transmission de données, et dispositif de transmission Download PDF

Info

Publication number
WO2022151820A1
WO2022151820A1 PCT/CN2021/129667 CN2021129667W WO2022151820A1 WO 2022151820 A1 WO2022151820 A1 WO 2022151820A1 CN 2021129667 W CN2021129667 W CN 2021129667W WO 2022151820 A1 WO2022151820 A1 WO 2022151820A1
Authority
WO
WIPO (PCT)
Prior art keywords
network device
data
pieces
message
host
Prior art date
Application number
PCT/CN2021/129667
Other languages
English (en)
Chinese (zh)
Inventor
卢胜文
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022151820A1 publication Critical patent/WO2022151820A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/06—Notations for structuring of protocol data, e.g. abstract syntax notation one [ASN.1]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/22—Parsing or analysis of headers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45583—Memory management, e.g. access or allocation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/45595—Network integration; Enabling network access in virtual machine instances

Definitions

  • the present application relates to the field of network communication technologies, and in particular, to a data transmission system, a data transmission method, and a network device.
  • the Remote Direct Memory Access (RDMA) protocol is a protocol that transfers data directly from one system to another in memory over a network without operating system intervention.
  • the RDMA protocol encapsulates the data to be transmitted into one or more RDMA packets, and sends the one or more RDMA packets from the sender to the receiver.
  • RDMA transfer is to transfer data from one system to the memory of another system through a send queue (SQ), wherein the data in the SQ only includes the data of one virtual machine (VM).
  • SQL send queue
  • the present application provides a data transmission system, a data transmission method and a network device for improving transmission efficiency.
  • a first aspect of the present application provides a data transmission system, the system includes: the data transmission system includes a first network device and a second network device, the first network device is set on the first host, and the second network device is set on the second network device On the host, there are N virtual machines running on the first host; the first network device is used to obtain N data, the N data comes from the N VMs, and the N data and the N data are converted according to the remote direct memory access RDMA protocol.
  • the write address of the data is encapsulated into a packet, and the packet is sent to the second network device, where N is an integer greater than 1; the second network device is used to receive the packet and decapsulate the packet to obtain N data and N data write addresses are stored on the second host according to the N data write addresses.
  • N data of N VMs running on the first host need to be transferred to the second host through RDMA, wherein the N VMs are VMs that can normally extract data, and the first network in the first host
  • the device may encapsulate the above-mentioned N pieces of data and the write addresses of the N pieces of data according to the RDMA protocol, and then send the encapsulated packet to the second network device on the second host.
  • the second network device may decapsulate the packet, extract the N data and the write address of the N data, and store the N data in the location indicated by the write address.
  • the first network device can directly send the data of the multiple virtual machines to the second network device to improve transmission efficiency.
  • the first network device is configured to acquire the identifiers and memory addresses of the N VMs; and acquire N pieces of data according to the identifiers and memory addresses of the N VMs.
  • the first network device can directly obtain N pieces of data according to the VM identifiers and VM memory addresses of the N pieces of data that it wants to RDMA to transmit to the second network device, and the write address received by the second network device includes VM. ID and VM memory address, the second network device can directly use the write address to store N pieces of data without going through a chip logical address (CLA), which can reduce the processing flow.
  • CLA chip logical address
  • an abnormal VM exists on the first host, and data cannot be obtained according to the identifier and memory address of the abnormal VM; the first network device is configured to obtain data according to the identifier and memory address of the abnormal VM and some of the N VMs.
  • the identifier and memory address of the VM are obtained, and M pieces of data are obtained, where M is a positive integer less than N; the M pieces of data are encapsulated into an exception message according to the RDMA protocol.
  • the first network device cannot obtain data according to the identification and memory address of the abnormal VM, that is, according to the identification and memory of the abnormal VM.
  • An exception message generated by encapsulating the address, the identifiers of some of the VMs in the N VMs, and the memory address cannot be sent to the second network device through a queue pair (queue pair, QP) message.
  • the first network device is configured to generate a packet sequence, where the packet sequence includes an abnormal packet and at least one packet.
  • the first network device when an abnormal VM exists in the N VMs, the first network device needs to generate a message sequence from the abnormal message and at least one of the above messages, so that the first network device can generate a message sequence according to the message of the message sequence.
  • the sequence number sends the abnormal message and the message in sequence, so as to improve the practicability of the solution.
  • the first network device is configured to modify the packet sequence, wherein modifying the packet sequence includes deleting abnormal packets, and adding padding packets to the packet sequence.
  • the present application can delete abnormal packets in the packet sequence, and then supplement the packet sequence with padding packets, which are invalid packets, so as to keep the data length unchanged and avoid retransmission due to inconsistent data lengths.
  • the second network device is configured to receive the modified packet sequence, determine the padding packet in the modified packet sequence, and delete the padding packet.
  • the second network device after receiving the above-mentioned modified packet sequence, can determine the invalid packet by checking, that is, determine the padding packet in the modified packet sequence, and then delete the modified packet.
  • the padding message is used to improve the reliability of message transmission.
  • a second aspect of the present application provides a data transmission method, the method includes: a first network device acquires N pieces of data, the first network device is set on a first host, and N virtual machines VM run on the first host, and N The data comes from N VMs, where N is an integer greater than 1; the first network device encapsulates the N data and the write address of the N data into one message according to the remote direct memory access RDMA protocol; the first network device Send the message to the second network device.
  • an abnormal VM runs on the first host, and the abnormal VM cannot obtain data according to the identifier and memory address of the abnormal VM.
  • the method further includes: the first network device according to the identifier and memory address of the abnormal VM and the The identifiers and memory addresses of some VMs in the N VMs are obtained, and M pieces of data are obtained, wherein M is a positive integer less than N; the first network device encapsulates the M pieces of data into an exception message according to the RDMA protocol.
  • the method further includes: the first network device generates a packet sequence, where the packet sequence includes an abnormal packet and at least one packet.
  • the method further includes: the first network device modifies the packet sequence, wherein modifying the packet sequence includes deleting abnormal packets, and adding padding packets to the packet sequence.
  • a third aspect of the present application provides a data transmission method, the method includes: a second network device receives a message from a first network device, the first network device is set on the first host, and the second network device is set on a second network device On the host, there are N virtual machines running on the first host, and the message is a message generated by encapsulating the N data and the write address of the N data according to the remote direct memory access RDMA protocol, and the N data comes from the N data.
  • VM where N is an integer greater than 1
  • the second network device decapsulates the packet to obtain N data and N data write addresses
  • the second network device decapsulates the N data in the first N pieces of data are stored on the second host.
  • the method further includes: the second network device receives a modified packet sequence, where the modified packet sequence includes a padding packet; and the second network device determines a packet in the modified packet sequence. Padding packets, and deleting padding packets.
  • a fourth aspect of the present application provides a network device, comprising: an acquisition unit configured to acquire N pieces of data, the network device is set on a first host, N virtual machines VM are running on the first host, and the N pieces of data come from N VMs, where N is an integer greater than 1; the encapsulation unit is used to encapsulate the N data and the write addresses of the N data into a message according to the remote direct memory access RDMA protocol; the sending unit is used to encapsulate the message The message is sent to the second network device.
  • the network device is configured to execute the method of the second aspect or any one of the implementation manners of the second aspect.
  • a fifth aspect of the present application provides a network device, including: a receiving unit configured to receive a message from a first network device, the first network device is set on the first host, the network device is set on the second host, and the first network device is set on the second host.
  • a receiving unit configured to receive a message from a first network device, the first network device is set on the first host, the network device is set on the second host, and the first network device is set on the second host.
  • N virtual machine VMs running on a host
  • the message is a message generated by encapsulating N data and N data write addresses according to the remote direct memory access RDMA protocol.
  • the N data comes from N VMs, where, N is an integer greater than 1; the decapsulation unit is used to decapsulate the message to obtain the N data and the write address of the N data; the storage unit is used to store the N data in the second according to the write address of the N data N pieces of data are stored on the host.
  • the network device is configured to execute the method of the third aspect or any one of the implementation manners of the third aspect.
  • a sixth aspect of the present application provides a network device, including: a processor, a memory, and a communication interface, where the processor is configured to execute instructions stored in the memory, so that the network device executes the second aspect or any one of the second aspects
  • the method provided by the optional manner, the communication interface is used for receiving or sending an indication.
  • the network device including: a processor, a memory, and a communication interface, where the processor is configured to execute instructions stored in the memory, so that the network device executes the second aspect or any one of the second aspects
  • the communication interface is used for receiving or sending an indication.
  • a seventh aspect of the present application provides a network device, including: a processor, a memory, and a communication interface, where the processor is configured to execute instructions stored in the memory, so that the network device executes the third aspect or any one of the third aspects
  • the method provided by the optional manner, the communication interface is used for receiving or sending an indication.
  • the network device including: a processor, a memory, and a communication interface, where the processor is configured to execute instructions stored in the memory, so that the network device executes the third aspect or any one of the third aspects
  • the communication interface is used for receiving or sending an indication.
  • An eighth aspect of the present application provides a computer-readable storage medium, where a program is stored in the computer-readable storage medium.
  • the computer executes the program, the computer executes the second aspect or any optional manner of the second aspect. method.
  • a ninth aspect of the present application provides a computer-readable storage medium, where a program is stored in the computer-readable storage medium, and when the computer executes the program, the computer executes the third aspect or any optional manner provided by the third aspect. method.
  • a tenth aspect of the present application provides a computer program product.
  • the computer program product When the computer program product is executed on a computer, the computer executes the method provided in the second aspect or any optional manner of the second aspect.
  • An eleventh aspect of the present application provides a computer program product.
  • the computer program product When the computer program product is executed on a computer, the computer executes the method provided in the third aspect or any optional manner of the third aspect.
  • FIG. 1 is a system frame diagram of a data transmission method provided by an embodiment of the present application
  • FIG. 2 is a schematic structural diagram of a data transmission system provided by an embodiment of the present application.
  • FIG. 3 is an embodiment of a data transmission method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a CLA address provided in an embodiment of the present application.
  • FIG. 5 is another embodiment of a data transmission method provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a combined strip provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a padding message provided by an embodiment of the present application.
  • FIG. 8 is another schematic diagram of filling a filling message provided by an embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of a network device provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a network device provided by an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a network device provided by an embodiment of the application.
  • FIG. 12 is a schematic structural diagram of a network device provided by an embodiment of the present application.
  • Embodiments of the present application provide a data transmission system, a data transmission method, and a network device, which are used to reduce processing flow and memory occupation.
  • the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Those of ordinary skill in the art know that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
  • the data transmission method of the embodiment of the present application is mainly applicable to the application scenario of RDMA transmission.
  • the system does not perform a data copy action, which reduces the amount of time spent in the kernel space and in the processing of network communication. The number of user space context switches. Without any kernel memory participation, RDMA requests are sent from the application running in user space to the local network card, and then sent to the remote network card through the network. Therefore, RDMA transmission does not require the participation of the operating system and will not increase the system load.
  • FIG. 1 is a system frame diagram of a data transmission method according to an embodiment of the present application.
  • the figure shows a transmission scenario of RDMA transmission: the first application fetches data from the memory to generate an RDMA message, and sends the RDMA message to the It is sent to the local network card through the cache, and then transmitted to the remote network card through the network.
  • the remote network card caches the received RDMA message, and the second application program takes out the data from the cache and writes it into the memory.
  • the process of reading the data in the memory of the second application by the first application is similar to the description of the above-mentioned writing process, and details are not repeated here.
  • the network card includes an RDMA-capable network interface card (RDMA network interface card, RNIC) or a host channel adapter (host channel adapter, HCA).
  • RDMA transfer is to transfer data from one system to the memory of another system through a send queue (SQ), wherein the data in the SQ only includes the data of a virtual machine (VM), when a service
  • SQ send queue
  • VM virtual machine
  • an embodiment of the present application provides a data transmission system, the structure of which can be seen in FIG. 2 , the data transmission system includes a first network device 21 and a second network device 22 , and the first network device 21 is arranged on the first network device 21 .
  • the second network device 22 is set on the second host 221, and N virtual machines 2111 run on the first host 211.
  • the first network device 21 is used to obtain N pieces of data, and the N pieces of data come from N pieces of VM 2111. According to the remote direct memory access RDMA protocol, the N pieces of data and the write addresses of the N pieces of data are encapsulated into a message, and the report The message is sent to the second network device 22, where N is an integer greater than 1.
  • the second network device 22 is configured to receive the message, decapsulate the message to obtain N data and N data write addresses, and store N data on the second host 221 according to the N data write addresses data.
  • N data of N VMs 2111 running on the first host 211 need to be transferred to the second host 221 through RDMA, wherein the N VMs 2111 are VMs that can normally extract data, and the first host 211
  • a network device 21 may encapsulate the above-mentioned N pieces of data and the write addresses of the N pieces of data according to the RDMA protocol, and then send the packet generated by encapsulation to the second network device 22 on the second host 221 .
  • the second network device 22 may decapsulate the packet, extract the N pieces of data and the write addresses of the N pieces of data, and store the N pieces of data in the positions indicated by the write addresses.
  • the first network device 21 can directly send data of multiple virtual machines to the second network device 22, which improves transmission efficiency.
  • the first network device 21 is used to acquire the identifiers and memory addresses of the N VMs 2111; and acquire N pieces of data according to the identifiers and memory addresses of the N VMs.
  • the first network device 21 can directly acquire N pieces of data according to the VM identifiers and VM memory addresses of the N pieces of data that it wants to RDMA to transmit to the second network device 22, and the write address received by the second network device 22 includes the VM identifier. and the VM memory address, the second network device 22 can directly use the write address to store N pieces of data without going through a chip logical address (chip logical address, CLA), which can reduce the processing flow.
  • chip logical address, CLA chip logical address
  • the first network device 21 is used for the identification and memory address of the abnormal VM and some VMs in the N VMs 2111.
  • the identifier and memory address are obtained, and M pieces of data are obtained, where M is a positive integer less than N; the M pieces of data are encapsulated into an exception message according to the RDMA protocol.
  • the first network device 21 cannot obtain data according to the identification and memory address of the abnormal VM, that is, according to the identification and memory address of the abnormal VM and
  • the abnormal packets generated by the identification and memory address encapsulation of some VMs in the N VMs cannot be sent to the second network device 22 through a queue pair (queue pair, QP) message.
  • the first network device 21 is configured to generate a packet sequence, where the packet sequence includes an abnormal packet and at least one packet.
  • the first network device 21 needs to generate a message sequence from the abnormal message and at least one of the foregoing messages, so that the first network device 21 can generate a message sequence according to the message sequence number of the message sequence.
  • the above-mentioned abnormal message and the above-mentioned message are sent in sequence, so as to improve the implementability of the solution.
  • the first network device 21 is configured to modify the packet sequence, where modifying the packet sequence includes deleting abnormal packets and adding padding packets to the packet sequence.
  • the present application can delete abnormal packets in the packet sequence, and then supplement the packet sequence with padding packets, which are invalid packets, so as to keep the data length unchanged and avoid retransmission due to inconsistent data lengths.
  • the second network device 22 is configured to receive the modified packet sequence, determine the padding packet in the modified packet sequence, and delete the padding packet.
  • the second network device 22 can determine the invalid packet by checking, that is, determine the padding packet in the modified packet sequence, and then delete the padding packet. message to improve the reliability of message transmission.
  • the first network device may directly send data of multiple VMs to the second network device, thereby improving transmission efficiency.
  • the VMs running on the first host where the first network device is set may all be VMs that can normally extract data, and may also include abnormal VMs that cannot extract data, which will be described separately below.
  • the VMs running on the first host are all VMs that can extract data normally.
  • the first network device acquires N pieces of data.
  • N (N is an integer greater than 1) pieces of data are data that the first network device needs to transmit and write to the second network device through RDMA, and the N pieces of data store the N data of the first host of the first network device.
  • the first network device may obtain the N pieces of data from the N VMs according to the identifiers and memory addresses of the N VMs where the N pieces of data are located.
  • the N VMs are all VMs that can normally extract data.
  • the first network device encapsulates the N pieces of data and the write addresses of the N pieces of data into one packet according to the RDMA protocol.
  • the above-mentioned encapsulated message may be an RDMA write message
  • the RDMA write message includes the following three types: RDMA write First, RDMA write Middle, and RDMA write Last
  • the RDMA extended transport header (RDMAextended transport header, RETH) is a field in the RDMA message format, which carries the destination address of the message data
  • the base transport header (BTH) is a field in the RDMA message format, including PSN. Therefore, the RDMA write First message includes the packet sequence number (PSN) and the write address, while the BTH field exists in the RDMA write Middle and RDMA write Last messages.
  • PSN packet sequence number
  • BTH packet sequence number
  • the first network device may complete the creation of the QP for RDMA in advance, which may be created either on a virtual machine manager (Hypervisor) of the host, or on another central processing unit (central processing unit, CPU) or On the device, at the same time, the RDMA hardware device (RNIC or HCA or other hardware device with RDMA capability) itself can directly access the memory space of the VM.
  • the RDMA transmission hardware is limited to the CLA
  • the first network device creates a CLA for RDMA hardware access in advance.
  • R_Key used for reading the memory of the remote device, L_key and R_Key can also use the same value
  • the corresponding CLA is unique.
  • the registered registry includes L_key, R_key, CLA start address, VM identifier, VM memory address and length, wherein there are multiple VM identifiers, and each VM identifier in the multiple VM identifiers corresponds to a VM memory address respectively and length.
  • the first network device may query the registry based on the CLA and length of the target data to obtain the VM identifier, VM memory address and length corresponding to the CLA and the length, and then extract the target data from the position corresponding to the VM identifier, the VM memory address and the length. to encapsulate.
  • the CLA address of the target data can be referred to as shown in FIG. 4 .
  • One CLA address includes addresses of data of multiple VMs.
  • VM1 includes data sg11 , sg12 and sg13
  • VM2 includes data sg21 , sg22 and sg23
  • VM3 includes data sg31, sg32 and sg33
  • the address of a CLA may include addresses adr11, adr21 and adr31, where address adr11 may indicate data sg11 in VM1, address adr21 may indicate data sg21 in VM2, and address adr31 may indicate VM3 Data in sg31.
  • the registry may only include L_key, R_key, VM identifier, VM memory address and length, and the first network device may directly use the VM identifier, VM memory address, and length of the target data to extract the target data from the corresponding VM. to encapsulate.
  • security verification needs to be performed through the registry for the VM identifier, the VM memory address and the length, to avoid when the indicated read-write position exceeds the memory area indicated by the VM identifier, the VM memory address and the length in the registry, The case of reading and writing VMs in other memory areas other than the registry.
  • the VM identifier in this embodiment may be a physical function (Physical Function, PF)/virtual function (Virtual Function, VF) based on a single-root I/O virtualization (SR-IOV) device.
  • Identification based on Scalable I/O virtualization (Scalable-IOV) devices can be assignable device interfaces (ADI), or other identification processes that can identify different VMs or address domains Address space identifier (Process Address Space identifier, PASID).
  • the storage may be the memory space of the application program in the second network device, and the target data may be data stored in the memory space of the application program in the first network device.
  • the maximum transmission unit (maximum transmission unit, MTU) is the maximum data packet size that can be transmitted in each RDMA transmission in the RDMA protocol, and the number of the above-mentioned first packets can be determined according to the data size of the target data and the MTU.
  • the first network device when the first network device transmits data to the second network device through RDMA transmission, the first network device may directly write data to the second network device RDMA, or the second network device may write data to the first network device.
  • the network device RDMA reads data.
  • the second network device reads data from the first network device RDMA, before step 301 , the second network device also needs to send a data read request to the first network device to trigger step 301 .
  • the first network device sends the packet to the second network device.
  • the first network device may directly send the packet to the second network device according to the PSN sequence of the packet.
  • the second network device receives the message.
  • the second network device decapsulates the packet to obtain N pieces of data and N pieces of data write addresses.
  • the second network device may directly decapsulate the packet, for example, disassemble the protocol packet, process the information in the packet header, and extract the N data in the payload and the write address of the N data.
  • the second network device stores N pieces of data on the second host according to the write addresses of the N pieces of data.
  • the second network device may store N pieces of data in the VM indicated by the write address.
  • the write address is the CLA address and length
  • the second network device may query the registration according to the CLA address and length.
  • the table is stored by matching the corresponding VM ID, VM memory address and length, or the write address is the VM ID, VM memory address and length, and the second network device directly stores N pieces of data according to the VM ID, VM memory address and length .
  • the first network device encapsulates N data of N VMs running on the first host into a packet, and sends the packet to the second network device.
  • the first network device can directly The data of each VM is sent to the second network device without performing multiple RDMA transmissions, which can improve transmission efficiency.
  • the VMs running on the first host include abnormal VMs that cannot extract data.
  • FIG. 5 another embodiment of the data transmission method provided by the embodiment of the present application includes:
  • the first network device acquires N pieces of data.
  • the first network device encapsulates the N pieces of data and the write addresses of the N pieces of data into one packet according to the RDMA protocol.
  • steps 501 and 502 reference may be made to the relevant descriptions of steps 301 and 302 in the data transmission method shown in FIG. 3, and details are not repeated here.
  • the first network device acquires M pieces of data according to the identifier and memory address of the abnormal VM and the identifiers and memory addresses of some VMs in the N VMs.
  • the abnormal VM is a VM from which data cannot be extracted normally.
  • the abnormal VM may be a VM failure, shutdown or restart, etc.
  • the data transmitted by the first network device through RDMA involves the data in the abnormal VM, and the RDMA command initiated by the first network device For example, if direct memory access (DMA) fails, data cannot be obtained according to the ID and memory address of the abnormal VM, and M data can only be obtained according to the ID and memory address of some VMs in N VMs.
  • DMA direct memory access
  • the first network device encapsulates the M pieces of data into an exception packet according to the RDMA protocol.
  • the first network device When the first network device encapsulates the data of the abnormal VM and the data of some VMs in the N VMs according to the RDMA protocol, since the data of the abnormal VM cannot be obtained, the encapsulated message is the abnormal message, and the first network device can directly return After the command is completed, configure the indication information for the abnormal message.
  • the indication information can indicate that the abnormal message is an error message.
  • the indication can be configured in the completion queue element (complete queue element, CQE) of the completion queue (complete queue, CQ). information.
  • sequence of steps 501 to 502 and steps 503 to 504 is not limited.
  • the first network device generates a packet sequence, where the packet sequence includes an abnormal packet and at least one packet.
  • the first network device RDMA When the first network device RDMA transmits data to the second network device, the data is sequentially transmitted according to the PSNs of the transmission packets.
  • the first device sorts the normally encapsulated packets and the abnormal packets to form a packet sequence, which is a normal packet. Configure PSN for encapsulated packets and abnormal packets.
  • the first network device modifies the packet sequence, where the modification of the packet sequence includes deleting an abnormal packet and adding a padding packet to the packet sequence.
  • the first network device may process the factors in the packet sequence that affect the related state received by the second network device.
  • the first network device sends the packet sequence to the second device through a QP message, and the first network device sends the packet sequence to the second device through a QP message.
  • a network device can modify the PSN in the message sequence, skip the error message and send the next message, so as to avoid the second network device finding that the PSN is discontinuous and thinking that the middle message is lost, so it tries to retransmit repeatedly. After several unsuccessful attempts, it is considered that the QP is faulty or the QP is disconnected. Exemplarily, as shown in the schematic diagram of the combined strips shown in FIG.
  • VM0 is represented by a thin solid line box.
  • the data D000, D001 and D020 of VM1, the data D110 and D111 of VM1, the data D210, D220 and D221 of VM2, the data D300, D301 and D310 of VM3, the data of VM4 and VM5 are represented by solid boxes. That is, the data strips from VM0 to VM5 can be combined to obtain MSG0 to MSG5 messages as shown in FIG. 6 , and the first network device can send MSG0 to the second network device according to the sequence in the send queue (SQ) in the figure. Message to MSG5.
  • SQ send queue
  • MSG2 and MSG3 cannot directly obtain data from VM2, which causes MSG2 and MSG3 to fail to send, and the connection will be broken after multiple retransmissions, resulting in other normal messages. If it cannot be sent, in this embodiment of the present application, the subsequent data of MSG4 and MSG5 can be moved forward to replace the data of MSG2 and MSG3, as shown in Figure 7, that is, the data of MSG4 is now numbered as MSG2', and the data of MSG5 is currently numbered. is MSG3'.
  • the padding message is supplemented as the data of the new MSG4 and MSG5 to obtain a new SQ' to ensure that the data length is correct, so that the RDMA message can be sent to the second network device normally.
  • the padding data in the padding packet may be other VM data or blank data.
  • the embodiment of the present application may also be a way of supplementing the message that does not need to reorder the PSN of the message, as shown in FIG. 8 ,
  • the first network device directly replaces the target VM data with data of other VMs or blank data to obtain a new SQ'.
  • the first network device sends the modified packet sequence to the second network device.
  • the first network device sequentially sends the message and the padding message to the second network device according to the PSN of the modified message sequence.
  • the second network device sequentially receives the message and the padding message in the modified message sequence according to the PSN sequence. Fill the message.
  • the second network device determines a padding packet in the modified packet sequence, and deletes the padding packet.
  • the second network device After the second network device receives the modified packet sequence, it can pass the data integrity field (DIF) check. For example, the data in the padding packet comes from other VMs that do not belong to the N VMs, then It can be verified as an invalid packet that does not conform to the VM type, or the padding packet includes blank data, which can be directly verified as an invalid packet.
  • the upper-layer data processing program will remove the invalid packet and save only the modified packet. the above message in the sequence.
  • the second network device decapsulates the packet sequence after deleting the padding packet, so as to obtain N data and N data write addresses.
  • the packet sequence after deleting the padding packet only includes the packet in step 502, and the packet can be decapsulated, such as disassembling the protocol packet, processing the information in the packet header, and taking out the N in the payload. write address of data and N data.
  • the second network device stores N pieces of data on the second host according to the write addresses of the N pieces of data.
  • the second network device may store N pieces of data in the VM indicated by the corresponding write address, for example, may query the registry to match the corresponding VM identifier and VM memory according to the CLA address and length.
  • the address and length are stored, or directly stored according to the VM ID, VM memory address and length.
  • the first network device encapsulates N data of N VMs running on the first host into a packet, and sends the packet to the second network device.
  • the first network device can directly When the data of each VM is sent to the second network device, it is not necessary to perform multiple RDMA transmissions, which can improve transmission efficiency.
  • the first network device modifies the message sequence including the abnormal message, deletes the abnormal message and adds a padding message, and sends the modified message sequence to the second network device to prevent the second network device from discovering the sequence. If the number is discontinuous, it is considered that the message in the middle is lost, which leads to repeated attempts to retransmit. After several unsuccessful attempts, it is considered that the QP is faulty or the QP chain is disconnected. In this way, the transmission efficiency can be improved.
  • FIG. 9 is a schematic diagram of an embodiment of a network device 90 in an embodiment of the present application.
  • an embodiment of the present application provides a network device, where the network device includes:
  • the obtaining unit 901 is used to obtain N pieces of data, the network device is set on the first host, and N virtual machines VMs are running on the first host, and the N pieces of data come from the N pieces of VM, where N is an integer greater than 1;
  • an encapsulation unit 902 configured to encapsulate the N data and the write addresses of the N data into a message according to the remote direct memory access RDMA protocol;
  • the sending unit 903 is configured to send the message to the second network device.
  • the obtaining unit 901 is specifically configured to: obtain identifiers and memory addresses of N VMs; and obtain N pieces of data according to the identifiers and memory addresses of N VMs.
  • the obtaining unit 901 is further configured to: obtain M pieces of data according to the identifier and memory address of the abnormal VM and the identifiers and memory addresses of some VMs in the N VMs, the abnormal VM runs on the first host, and the abnormal VM The identity and memory address of the VM cannot obtain data, where M is a positive integer less than N;
  • the encapsulation unit 902 is further configured to encapsulate the M pieces of data into an exception packet according to the RDMA protocol.
  • the network device 90 further includes a generating unit 904, and the generating unit 904 is specifically configured to: generate a packet sequence, where the packet sequence includes an abnormal packet and at least one packet.
  • the network device 90 further includes a modification unit 905, and the modification unit 905 is specifically configured to: modify the packet sequence, wherein modifying the packet sequence includes deleting abnormal packets and adding padding packets to the packet sequence.
  • the network device may perform the operations performed by the first network device in the foregoing embodiments shown in FIG. 3 and FIG. 5 , and details are not repeated here.
  • FIG. 10 is a schematic diagram of another embodiment of the network device 100 in the embodiment of the present application.
  • an embodiment of the present application provides a network device, where the network device includes:
  • the receiving unit 1001 is configured to receive a message from a first network device, the first network device is set on a first host, the network device is set on a second host, and N virtual machines VM are running on the first host, and the message A message generated by encapsulating N data and N data write addresses according to the remote direct memory access RDMA protocol, where N data comes from N VMs, where N is an integer greater than 1;
  • a decapsulation unit 1002 configured to decapsulate the message to obtain N data and N data write addresses;
  • the storage unit 1003 is configured to store N pieces of data on the second host according to the write addresses of the N pieces of data.
  • the receiving unit 1001 is further configured to: receive a modified message sequence, where the modified message sequence includes a padding message;
  • the network device 100 further includes a deletion unit 1004, and the deletion unit 1004 is specifically configured to: determine the padding message in the modified message sequence, and delete the padding message.
  • the network device may perform the operations performed by the second network device in the foregoing embodiments shown in FIG. 3 and FIG. 5 , and details are not described herein again.
  • the network device 110 includes: a processor 1101 , a communication interface 1102 , a storage system 1103 and a bus 1104 .
  • the processor 1101 , the communication interface 1102 , and the storage system 1103 are connected to each other through a bus 1104 .
  • the processor 1101 is configured to control and manage the actions of the network device 110.
  • the processor 1101 is configured to execute the steps performed by the first network device in the method embodiments of FIG. 3 and FIG. 5 .
  • the communication interface 1102 is used to support the communication of the network device 110 .
  • the storage system 1103 is used to store program codes and data of the network device 110 .
  • the processor 1101 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure.
  • the processor 1101 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.
  • the bus 1104 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like.
  • PCI Peripheral Component Interconnect
  • EISA Extended Industry Standard Architecture
  • the sending unit 903 in the network device 90 is equivalent to the communication interface 1102 in the network device 110 , and the acquiring unit 901 , the encapsulating unit 902 , the generating unit 904 and the modifying unit 905 in the network device 90 are equivalent to the processor 1101 in the network device 110 .
  • the network device 110 in this embodiment may correspond to the first network device in the foregoing method embodiments in FIG. 3 and FIG. 5 , and the communication interface 1102 in the network device 110 may implement the first network device in the foregoing method embodiments in FIG. 3 and FIG. 5 .
  • the functions possessed by the network device and/or the various steps implemented are not repeated here.
  • the network device 120 includes: a processor 1201 , a communication interface 1202 , a storage system 1203 and a bus 1204 .
  • the processor 1201 , the communication interface 1202 , and the storage system 1203 are connected to each other through a bus 1204 .
  • the processor 1201 is configured to control and manage the actions of the network device 120.
  • the processor 1201 is configured to execute the steps performed by the second network device in the method embodiments of FIG. 3 and FIG. 5 .
  • the communication interface 1202 is used to support the communication of the network device 120 .
  • the storage system 1203 is used to store program codes and data of the network device 120 .
  • the processor 1201 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure.
  • the processor 1201 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.
  • the bus 1204 may be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an Extended Industry Standard Architecture (Extended Industry Standard Architecture, EISA) bus or the like.
  • PCI peripheral component interconnect standard
  • EISA Extended Industry Standard Architecture
  • the receiving unit 1001 in the network device 100 is equivalent to the communication interface 1202 in the network device 120 , and the decapsulating unit 1002 , the storage unit 1003 and the deleting unit 1004 in the network device 100 may be equivalent to the processor 1201 .
  • the network device 120 in this embodiment may correspond to the second network device in the foregoing method embodiments in FIGS. 3 and 5 , and the processor 1201 and the communication interface 1202 in the network device 120 may implement the foregoing method embodiments in FIGS. 3 and 5 .
  • the functions and/or various steps performed by the second network device in are not repeated here.
  • a computer-readable storage medium is also provided, where computer-executable instructions are stored in the computer-readable storage medium.
  • the processor of the device executes the computer-executable instructions
  • the device executes the above-mentioned FIG. 3 and Steps of the data transmission method performed by the first network device in FIG. 5 .
  • a computer-readable storage medium is also provided, where computer-executable instructions are stored in the computer-readable storage medium.
  • the processor of the device executes the computer-executable instructions
  • the device executes the above-mentioned FIG. 3 and Steps of the data transmission method performed by the second network device in FIG. 5 .
  • a computer program product includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium; when a processor of a device executes the computer-executable instructions , the device executes the steps of the data transmission method executed by the first network device in the above-mentioned FIG. 3 and FIG. 5 .
  • a computer program product includes computer-executable instructions, and the computer-executable instructions are stored in a computer-readable storage medium; when a processor of a device executes the computer-executable instructions , the device executes the steps of the data transmission method executed by the second network device in the above-mentioned FIG. 3 and FIG. 5 .
  • the disclosed system, apparatus and method may be implemented in other manners.
  • the apparatus embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented.
  • the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

La présente demande divulgue un système de transmission de données, un procédé de transmission de données et un dispositif de réseau, lesquels sont utilisés pour améliorer l'efficacité de transmission. Ledit système est appliqué à une transmission de données d'accès direct à la mémoire à distance (RDMA). Ledit système comprend : N éléments de données de N machines virtuelles (VM) s'exécutant sur un premier hôte doivent être transmis à un second hôte au moyen du RDMA, les N VM étant des VM qui peuvent extraire des données normalement, et un premier dispositif de réseau dans le premier hôte peut encapsuler les N éléments de données et les adresses d'écriture des N éléments de données selon un protocole RDMA, puis envoyer un message généré au moyen d'une encapsulation à un second dispositif de réseau sur le second hôte. Le second dispositif de réseau peut désencapsuler le message, extraire les N éléments de données et les adresses d'écriture des N éléments de données à partir desdits éléments de données et stocker les N éléments de données à des emplacements indiqués par les adresses d'écriture.
PCT/CN2021/129667 2021-01-14 2021-11-10 Système de transmission de données, procédé de transmission de données, et dispositif de transmission WO2022151820A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110049814.4A CN114765631A (zh) 2021-01-14 2021-01-14 数据传输系统、数据传输方法以及网络设备
CN202110049814.4 2021-01-14

Publications (1)

Publication Number Publication Date
WO2022151820A1 true WO2022151820A1 (fr) 2022-07-21

Family

ID=82363926

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/129667 WO2022151820A1 (fr) 2021-01-14 2021-11-10 Système de transmission de données, procédé de transmission de données, et dispositif de transmission

Country Status (2)

Country Link
CN (1) CN114765631A (fr)
WO (1) WO2022151820A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116150078A (zh) * 2023-04-19 2023-05-23 湖南恩智绿源电子技术有限公司 板间数据通信传输方法、电子设备和计算机可读存储介质
CN117278504A (zh) * 2023-09-21 2023-12-22 中科驭数（北京）科技有限公司 报文数据转发方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117424849A (zh) * 2022-07-26 2024-01-19 中兴智能科技南京有限公司 数据传输方法、装置、计算机设备及可读介质
CN117707861A (zh) * 2022-09-07 2024-03-15 华为技术有限公司 数据访问方法、装置、网络接口卡、可读介质和电子设备

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102801729A (zh) * 2012-08-13 2012-11-28 福建星网锐捷网络有限公司 虚拟机报文转发方法、网络交换设备及通信系统
CN103248467A (zh) * 2013-05-14 2013-08-14 中国人民解放军国防科学技术大学 基于片内连接管理的rdma通信方法
CN106897106A (zh) * 2017-01-12 2017-06-27 北京三未信安科技发展有限公司 一种sr‑iov环境下多虚拟机并发dma的顺序调度方法及系统
WO2018000195A1 (fr) * 2016-06-28 2018-01-04 华为技术有限公司 Procédé de transmission de paquets, commutateur virtuel, et serveur
CN108228309A (zh) * 2016-12-21 2018-06-29 腾讯科技（深圳）有限公司 基于虚拟机的数据包发送和接收方法及装置
US20190079896A1 (en) * 2017-09-14 2019-03-14 Vmware, Inc. Virtualizing connection management for virtual remote direct memory access (rdma) devices
CN109983439A (zh) * 2016-12-28 2019-07-05 英特尔公司 虚拟化远程直接存储器访问
US20200110626A1 (en) * 2018-10-08 2020-04-09 Microsoft Technology Licensing, Llc Rdma with virtual address space
CN111193653A (zh) * 2019-12-31 2020-05-22 腾讯科技（深圳）有限公司 数据传输方法、装置、设备及存储介质

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102801729A (zh) * 2012-08-13 2012-11-28 福建星网锐捷网络有限公司 虚拟机报文转发方法、网络交换设备及通信系统
CN103248467A (zh) * 2013-05-14 2013-08-14 中国人民解放军国防科学技术大学 基于片内连接管理的rdma通信方法
WO2018000195A1 (fr) * 2016-06-28 2018-01-04 华为技术有限公司 Procédé de transmission de paquets, commutateur virtuel, et serveur
CN108228309A (zh) * 2016-12-21 2018-06-29 腾讯科技（深圳）有限公司 基于虚拟机的数据包发送和接收方法及装置
CN109983439A (zh) * 2016-12-28 2019-07-05 英特尔公司 虚拟化远程直接存储器访问
CN106897106A (zh) * 2017-01-12 2017-06-27 北京三未信安科技发展有限公司 一种sr‑iov环境下多虚拟机并发dma的顺序调度方法及系统
US20190079896A1 (en) * 2017-09-14 2019-03-14 Vmware, Inc. Virtualizing connection management for virtual remote direct memory access (rdma) devices
US20200110626A1 (en) * 2018-10-08 2020-04-09 Microsoft Technology Licensing, Llc Rdma with virtual address space
CN111193653A (zh) * 2019-12-31 2020-05-22 腾讯科技（深圳）有限公司 数据传输方法、装置、设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116150078A (zh) * 2023-04-19 2023-05-23 湖南恩智绿源电子技术有限公司 板间数据通信传输方法、电子设备和计算机可读存储介质
CN117278504A (zh) * 2023-09-21 2023-12-22 中科驭数（北京）科技有限公司 报文数据转发方法及装置

Also Published As

Publication number Publication date
CN114765631A (zh) 2022-07-19

Similar Documents

Publication Publication Date Title
WO2022151820A1 (fr) Système de transmission de données, procédé de transmission de données, et dispositif de transmission
CN111512603B (zh) 一种数据传输方法及第一设备
EP3042298B1 (fr) Porte pci express universelle
EP3660686B1 (fr) Procédé et dispositif de transmission d'une demande de traitement de données
US20200218688A1 (en) Data validation method and apparatus, and network interface card
CN101827072A (zh) 虚拟内存协议分段卸载
TW200846914A (en) Facilitating input/output processing of one or more guest processing systems
US11726666B2 (en) Network adapter with efficient storage-protocol emulation
US11231983B2 (en) Fault tolerance processing method, apparatus, and server
WO2019190859A1 (fr) Canal de message efficace et fiable entre un système hôte et un système d'accélération de circuit intégré
US12088688B2 (en) Packet processing method, network device, and related device
KR101559089B1 (ko) 장치의 컴포넌트들 간에 메모리 자원들을 공유하기 위한 통신 프로토콜
CN112866206A (zh) 一种单向数据传输方法及装置
US9769093B2 (en) Apparatus and method for performing InfiniBand communication between user programs in different apparatuses
CN113098780B (zh) 虚拟网络的报文处理方法、电子设备及存储介质
US9787805B2 (en) Communication control system and communication control method
US20190334826A1 (en) Connection management unit and connection management method
WO2023040330A1 (fr) Procédé, dispositif et système de traitement de données
CN101674219B (zh) 因特网安全协议智能卡隧道模式下的通信方法和设备
CN115982090B (zh) 软硬件协同实现远端内存访问的方法和装置
CN117544579A (zh) 数据传输方法、装置、设备及介质
CN115987609A (zh) 可信虚拟主机的识别方法、电子设备及存储介质
CN117544710A (zh) 数据通信方法、系统和车辆

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21919011

Country of ref document: EP

Kind code of ref document: A1