CN110602211A - Out-of-order RDMA method and device with asynchronous notification - Google Patents
Out-of-order RDMA method and device with asynchronous notification Download PDFInfo
- Publication number
- CN110602211A CN110602211A CN201910870364.8A CN201910870364A CN110602211A CN 110602211 A CN110602211 A CN 110602211A CN 201910870364 A CN201910870364 A CN 201910870364A CN 110602211 A CN110602211 A CN 110602211A
- Authority
- CN
- China
- Prior art keywords
- message
- packet
- response
- module
- rdma
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
- H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/22—Parsing or analysis of headers
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
An out-of-order RDMA method with asynchronous notification, comprising the steps of 1: the method comprises the steps that a source side obtains and records message packet information of RDMA messages, reads packet data from a source side main memory according to the message packet information, packages the packet data and corresponding message packet information into RDMA data packets and sends the RDMA data packets to a target side; step 2: after receiving a response packet returned by the target party, counting the response, and after finishing the response, sending an asynchronous notification message Send packet to the target party; and step 3: and writing a completion event after the target side writes the Send packet into the receiving queue and returns a response. The invention supports out-of-order transmission of the message packet, reduces the limitation on the network and the routing mode, and makes the construction of the network more flexible. And a reliable message transmission mechanism of source side counting is used, so that reliable transmission of messages is ensured, hardware design is simplified, and hardware resource overhead is saved. The source side hardware automatically initiates an asynchronous notification message to notify the target side of message completion, so that the rapid notification of a message completion event is realized, and the message delay is reduced.
Description
Technical Field
The invention relates to the field of out-of-order RDMA message communication, in particular to an out-of-order RDMA method and device with asynchronous notification.
Background
In high performance computing systems, RDMA messaging mechanisms are employed between computing nodes for remote host access, typically integrated in a network interface chip.
Currently, in the high performance computing field, the commercial network interface chip is basically monopolized by the company mellonox, and the RDMA mechanism thereof is implemented by strictly referring to the InfiniBand specification. RDMA message transmission in the InfiniBand specification must be strictly order-preserved, and the order-preserved transmission generally only can use deterministic routing and has poor dynamic adaptability to networks. And the message mechanism using out-of-order transmission can support adaptive routing, and has smaller constraint on the network.
In an out-of-order communication network, when an RDMA message completes data transmission and a source side needs to notify a target side of completing an event, if software initiates an asynchronous notification to the target side, the asynchronous notification message needs to be delivered after the RDMA message of the source side is completed, and therefore the time delay is large.
Disclosure of Invention
Aiming at the technical problem, the invention provides an out-of-order RDMA method with asynchronous notification, which comprises the following steps:
step 1: the method comprises the steps that a source side obtains and records message packet information of RDMA messages, reads packet data from a source side main memory according to the message packet information, packages the packet data and corresponding message packet information into RDMA data packets and sends the RDMA data packets to a target side;
step 2: after receiving the response packet returned by the target party, counting the response, and after finishing the response, sending an asynchronous notification message Send packet to the target party;
and step 3: and writing a completion event after the target side writes the Send packet into the receiving queue and returns a response.
The method supports message out-of-order transmission and message packet out-of-order transmission, reduces the limitation on the routing mode, enables the construction of the network to be more flexible, automatically initiates an asynchronous notification message by source side hardware to notify a target side of message completion, realizes the quick notification of a message completion event, and reduces the message delay. The method comprises the steps that when the length of an RDMA message is larger than the length of a maximum transmission packet, the RDMA message is split into a plurality of packet transmissions, each packet contains packet information, when different message packets of different RDMA messages or different message packets of the same RDMA message are transmitted, a source side obtains the message packet information in sequence, the packet data and the corresponding message packet information are read from a source side main memory according to the message packet information, the packet data and the corresponding message packet information are packaged into an RDMA data packet and are transmitted to a target side, the target side returns a response to the source side after receiving one RDMA data packet, the source side counts the responses, after the responses are collected completely (the response collection shows that one RDMA message transmission is completed), an asynchronous notification message Send packet is transmitted to the target side, the target side writes a receiving queue and returns one response, after the source side receives the response packet, the message execution is completed, and a write completion event is transmitted to.
Preferably, in step 1, the message packet information of the RDMA message is obtained through a suspension buffer unit, and includes a message ID, a remaining data amount, a next packet source data address, a next packet destination data address, an MTU, a remaining response number, an asynchronous notification reception queue number, a message descriptor, and a message status; in step 2, the target party returns a response packet which carries the message hanging buffer number and the message ID number.
Preferably, the specific method of response counting in step 2 is as follows: when the source side receives a response packet, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response quantity in the suspension buffer unit by one after the matching is successful; when the number of remaining replies is zero, it indicates that the message transmission is complete. The setting uses source side counting, ensures the reliability of message transmission, simplifies hardware design and saves hardware resource overhead.
Preferably, in step 1, the source side reads data from the source side main memory according to information such as the remaining data amount of the message, the source data address of the next packet, the destination data address of the next packet, and the MTU (maximum transmission packet length).
The invention also discloses an out-of-order RDMA device with asynchronous notification, which comprises a sending engine and a receiving engine, wherein the sending engine comprises a message hanging buffer module, a response processing module, a unpacking module, a packing module, an access interface module and a data buffer module; the receiving engine comprises a request packet processing module, a response queue module, a response packet processing module, a receiving queue management module and a receiving engine access interface module;
the message hanging buffer module is used for acquiring and registering a message descriptor of the RDMA message packet and recording message information of the RDMA message packet; the message information comprises information such as message ID, residual data volume, next packet source data address, next packet target data address, residual response number, asynchronous notification receiving queue number, message descriptor, message state, MTU and the like.
The unpacking module is electrically connected with the message hanging buffer module, the packaging module and the access interface module and is used for reading the message descriptor and unpacking the message descriptor to generate package control information, outputting the package information content to the packaging module, outputting a data reading request to the access interface module and updating the message state information and writing back to the message hanging buffer; when a signal that a message sent by a message hanging buffer enters a state of sending an asynchronous notification is acquired, the unpacking module generates asynchronous notification control information and message descriptor content and outputs the asynchronous notification control information and the message descriptor content to the packing module, wherein the message packet data content is a descriptor of an RDMA message;
the access interface module is electrically connected with the data buffer module and is used for processing all main memory accesses of the sending engine, initiating an access request to a main memory access after receiving a data fetching request of the unpacking module, and writing an access response into the data buffer; the memory access interface module is also electrically connected with the message suspension buffer module and is used for processing a completion event input by the message suspension buffer module;
the data buffer is used for storing the data content of the RDMA message packet;
the group package module is electrically connected with the unpacking module and the data buffer module and is used for being responsible for the group package of the message package to send to the Internet, receiving package control information from the unpacking module, receiving package data from the data buffer, packaging the package data into an RDMA message request package and sending the RDMA message request package to the Internet access; when the unpacking module obtains the message hanging buffer to send out the message and enters into the sending asynchronous notification state signal, the unpacking module obtains the asynchronous notification control information and the message descriptor, and then the asynchronous notification control information and the message descriptor are packaged into a request packet for sending the asynchronous notification message and sent to the internet access;
the request packet processing module is responsible for processing message request packets received on the network and is electrically connected with the response queue module, the memory access interface module and the receiving queue management module; when the message request packet is an RDMA message request packet, the request packet is analyzed, data in the packet is extracted, a write data request is generated and input to the memory access interface module, and meanwhile, an RDMA message response packet is generated according to the content of the message request packet and input to the response queue module; when the message request packet is a request packet for sending an asynchronous notification message, generating a response packet for sending the asynchronous notification message, sending the response packet to a response queue module, generating asynchronous notification information, and writing the asynchronous notification information into a receiving queue management module;
the response queue module is electrically connected with the request packet processing module, processes the response packet output by the request packet processing module and sends the response packet to the Internet;
the response packet processing module is responsible for processing the response packet received on the network, generating remote response control information and inputting the remote response control information into the response processing module;
the response processing module is electrically connected with the response packet processing module and the message hanging and buffering module, is responsible for processing remote response and processing according to the type of the response packet; when the response packet is an RDMA message response packet, controlling the message suspension buffer device to respond and count, and after finishing the response, controlling the message suspension buffer device to generate and send a message to enter a state signal for sending an asynchronous notification; when the response packet is a response packet for sending an asynchronous notification message, controlling a message hanging buffer module to generate a completion event and write the completion event into a main memory;
the receiving queue management module is electrically connected with the request packet processing module and is used for positioning to a specified position corresponding to a receiving queue space according to a receiving queue number when receiving asynchronous notification information input by the request packet processing module, generating a queue writing request and inputting the queue writing request to the receiving engine memory access interface module;
the receiving engine memory access interface module is responsible for processing all memory access requests of the receiving engine, including write data requests input by the request packet processing module and queue write requests input by the receiving queue management module, and all write main memory requests are sent to the main memory path.
In the out-of-order RDMA device, when different message packets of different RDMA messages or different message packets of the same RDMA message are sent, a sending engine acquires message packet information of each RDMA message in sequence, reads packet data from a main memory according to the message packet information, encapsulates the packet data and the corresponding message packet information into an RDMA message request packet and sends the RDMA message request packet to a receiving engine, the receiving engine returns a response to the sending engine when receiving one RDMA message request packet, a response processing module in the sending engine counts the responses, sends the message request packet with sending asynchronous notification messages to the receiving engine after the responses are received completely (the response is received completely to indicate that the transmission of one RDMA message is finished), a request packet processing module in the receiving engine generates the asynchronous notification information and writes the asynchronous notification information into a receiving queue management module and returns a response, and the response processing module in the sending engine receives the response packet, indicating that the message execution is complete, and writing a completion event to the specified main memory space. The device supports message out-of-order transmission and message packet out-of-order transmission, reduces the limitation on a routing mode, enables the construction of a network to be more flexible, automatically initiates an asynchronous notification message by source side hardware to notify a target side of message completion, realizes the quick notification of a message completion event, and reduces message delay.
Preferably, when the response packet is an RDMA message response packet, the response counting manner of the response processing module is as follows: and when a response packet is received, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response number in the suspension buffer unit by one after the matching is successful. The setting uses source side counting, ensures the reliability of message transmission, simplifies hardware design and saves hardware resource overhead.
Preferably, the receiving queue storage space of the receiving queue management module is located in the main memory space.
The invention has the beneficial effects that: the message packet supports out-of-order transmission, so that the limitation on a network and a routing mode can be reduced, and the construction of the network is more flexible. And a reliable message transmission mechanism of source side counting is used, so that the reliable transmission of the message is ensured, the hardware design is simplified, and the hardware resource overhead is saved. The hardware automatically sends the asynchronous notification message, the source side can complete the quick notification of the event to the target side under the disorder communication network, and the message delay is effectively reduced.
Drawings
FIG. 1 is a mechanical diagram of the out-of-order RDMA method with asynchronous notification of embodiment 1;
FIG. 2 is a schematic diagram showing a configuration of a transmission engine according to embodiment 2;
fig. 3 is a schematic structural diagram of a receiving engine in embodiment 2.
Detailed Description
The invention is described in further detail below:
example 1: as shown in fig. 1, an out-of-order RDMA method with asynchronous notification includes the following steps:
step 1: the method comprises the steps that a source side obtains and records message packet information of RDMA messages, reads packet data from a source side main memory according to the message packet information, packages the packet data and corresponding message packet information into RDMA data packets and sends the RDMA data packets to a target side; preferably, in this step, the message packet information of the RDMA message is obtained through the dangling buffer unit, which includes the message ID, the remaining data amount, the next packet source data address, the next packet destination data address, the MTU, the remaining response number, the asynchronous notification reception queue number, the message descriptor, and the message status;
step 2: after receiving the response packet returned by the target party, counting the response, and after finishing the response, sending an asynchronous notification message Send packet to the target party; in the step, a response packet returned by the target party carries a message hanging buffer number and a message ID number;
and step 3: and writing a completion event after the target side writes the Send packet into the receiving queue and returns a response.
The method supports message out-of-order transmission and message packet out-of-order transmission, reduces the limitation on the routing mode, enables the construction of the network to be more flexible, automatically initiates an asynchronous notification message by source side hardware to notify a target side of message completion, realizes the quick notification of a message completion event, and reduces the message delay. The method comprises the steps that when the length of an RDMA message is larger than the length of a maximum transmission packet, the RDMA message is split into a plurality of packet transmissions, each packet contains packet information, when different message packets of different RDMA messages or different message packets of the same RDMA message are transmitted, a source side obtains the message packet information in sequence, the packet data and the corresponding message packet information are read from a source side main memory according to the message packet information, the packet data and the corresponding message packet information are packaged into an RDMA data packet and are transmitted to a target side, the target side returns a response to the source side after receiving one RDMA data packet, the source side counts the responses, after the responses are collected completely (the response collection shows that one RDMA message transmission is completed), an asynchronous notification message Send packet is transmitted to the target side, the target side writes a receiving queue and returns one response, after the source side receives the response packet, the message execution is completed, and a write completion event is transmitted to.
Preferably, the specific method of response counting in step 2 is as follows: when the source side receives a response packet, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response quantity in the suspension buffer unit by one after the matching is successful; when the number of remaining replies is zero, it indicates that the message transmission is complete. The setting uses source side counting, ensures the reliability of message transmission, simplifies hardware design and saves hardware resource overhead.
Preferably, in step 1, the source side reads data from the source side main memory according to information such as the remaining data amount of the message, the source data address of the next packet, the destination data address of the next packet, and the MTU (maximum transmission packet length).
Example 2: 2-3, an out-of-order RDMA device with asynchronous notification comprises a sending engine and a receiving engine, wherein the sending engine comprises a message hanging buffer module, a response processing module, an unpacking module, a packing module, an access interface module and a data buffer module; the receiving engine comprises a request packet processing module, a response queue module, a response packet processing module, a receiving queue management module and a receiving engine access interface module;
the message hanging buffer module is used for acquiring and registering a message descriptor of the RDMA message packet and recording message information of the RDMA message packet; the message information comprises information such as message ID, residual data volume, next packet source data address, next packet target data address, residual response number, asynchronous notification receiving queue number, message descriptor, message state, MTU and the like.
The unpacking module is electrically connected with the message hanging buffer module, the packaging module and the access interface module and is used for reading the message descriptor and unpacking the message descriptor to generate package control information, outputting the package information content to the packaging module, outputting a data reading request to the access interface module and updating the message state information and writing back to the message hanging buffer; when a signal that a message sent by a message hanging buffer enters a state of sending an asynchronous notification is acquired, the unpacking module generates asynchronous notification control information and message descriptor content and outputs the asynchronous notification control information and the message descriptor content to the packing module, wherein the message packet data content is a descriptor of an RDMA message;
the access interface module is electrically connected with the data buffer module and is used for processing all main memory accesses of the sending engine, initiating an access request to a main memory access after receiving a data fetching request of the unpacking module, and writing an access response into the data buffer; the memory access interface module is also electrically connected with the message suspension buffer module and is used for processing a completion event input by the message suspension buffer module;
the data buffer is used for storing the data content of the RDMA message packet;
the group package module is electrically connected with the unpacking module and the data buffer module and is used for being responsible for the group package of the message package to send to the Internet, receiving package control information from the unpacking module, receiving package data from the data buffer, packaging the package data into an RDMA message request package and sending the RDMA message request package to the Internet access; when the unpacking module obtains the message hanging buffer to send out the message and enters into the sending asynchronous notification state signal, the unpacking module obtains the asynchronous notification control information and the message descriptor, and then the asynchronous notification control information and the message descriptor are packaged into a request packet for sending the asynchronous notification message and sent to the internet access;
the request packet processing module is responsible for processing message request packets received on the network and is electrically connected with the response queue module, the memory access interface module and the receiving queue management module; when the message request packet is an RDMA message request packet, the request packet is analyzed, data in the packet is extracted, a write data request is generated and input to the memory access interface module, and meanwhile, an RDMA message response packet is generated according to the content of the message request packet and input to the response queue module; when the message request packet is a request packet for sending an asynchronous notification message, generating a response packet for sending the asynchronous notification message, sending the response packet to a response queue module, generating asynchronous notification information, and writing the asynchronous notification information into a receiving queue management module;
the response queue module is electrically connected with the request packet processing module, processes the response packet output by the request packet processing module and sends the response packet to the Internet;
the response packet processing module is responsible for processing the response packet received on the network, generating remote response control information and inputting the remote response control information into the response processing module;
the response processing module is electrically connected with the response packet processing module and the message hanging and buffering module, is responsible for processing remote response and processing according to the type of the response packet; when the response packet is an RDMA message response packet, controlling the message suspension buffer device to respond and count, and after finishing the response, controlling the message suspension buffer device to generate and send a message to enter a state signal for sending an asynchronous notification; when the response packet is a response packet for sending an asynchronous notification message, controlling a message hanging buffer module to generate a completion event and write the completion event into a main memory;
the receiving queue management module is electrically connected with the request packet processing module and is used for positioning to a specified position corresponding to a receiving queue space according to a receiving queue number when receiving asynchronous notification information input by the request packet processing module, generating a queue writing request and inputting the queue writing request to the receiving engine memory access interface module;
the receiving engine memory access interface module is responsible for processing all memory access requests of the receiving engine, including write data requests input by the request packet processing module and queue write requests input by the receiving queue management module, and all write main memory requests are sent to the main memory path.
In the out-of-order RDMA device, when different message packets of different RDMA messages or different message packets of the same RDMA message are sent, a sending engine acquires message packet information of each RDMA message in sequence, reads packet data from a main memory according to the message packet information, encapsulates the packet data and the corresponding message packet information into an RDMA message request packet and sends the RDMA message request packet to a receiving engine, the receiving engine returns a response to the sending engine when receiving one RDMA message request packet, a response processing module in the sending engine counts the responses, sends the message request packet with sending asynchronous notification messages to the receiving engine after the responses are received completely (the response is received completely to indicate that the transmission of one RDMA message is finished), a request packet processing module in the receiving engine generates the asynchronous notification information and writes the asynchronous notification information into a receiving queue management module and returns a response, and the response processing module in the sending engine receives the response packet, indicating that the message execution is complete, and writing a completion event to the specified main memory space. The device supports message out-of-order transmission and message packet out-of-order transmission, reduces the limitation on a routing mode, enables the construction of a network to be more flexible, automatically initiates an asynchronous notification message by source side hardware to notify a target side of message completion, realizes the quick notification of a message completion event, and reduces message delay.
Preferably, when the response packet is an RDMA message response packet, the response counting manner of the response processing module is as follows: and when a response packet is received, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response number in the suspension buffer unit by one after the matching is successful. The setting uses source side counting, ensures the reliability of message transmission, simplifies hardware design and saves hardware resource overhead.
Preferably, the receiving queue storage space of the receiving queue management module is located in the main memory space.
The foregoing is only a preferred embodiment of the present invention and all equivalent changes or modifications in the structure, characteristics and principles described in the present patent application are included in the scope of the present patent application.
Claims (7)
1. An out-of-order RDMA method with asynchronous notification, comprising the steps of:
step 1: the method comprises the steps that a source side obtains and records message packet information of RDMA messages, reads packet data from a source side main memory according to the message packet information, packages the packet data and corresponding message packet information into RDMA data packets and sends the RDMA data packets to a target side;
step 2: after receiving the response packet returned by the target party, counting the response, and after finishing the response, sending an asynchronous notification message Send packet to the target party;
and step 3: and writing a completion event after the target side writes the Send packet into the receiving queue and returns a response.
2. The out-of-order RDMA method with asynchronous notification of claim 1, wherein in step 1, the message packet information of RDMA message is obtained through a pending buffer unit, which includes message ID, remaining data volume, next packet source data address, next packet destination data address, MTU, remaining number of replies, asynchronous notification receive queue number, message descriptor, message status; in step 2, the target party returns a response packet which carries the message hanging buffer number and the message ID number.
3. The out-of-order RDMA method with asynchronous notification of claim 2, wherein the specific method of response counting in step 2 is: when the source side receives a response packet, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response quantity in the suspension buffer unit by one after the matching is successful; when the number of remaining replies is zero, it indicates that the message transmission is complete.
4. The out-of-order RDMA method with asynchronous notification of claim 2, wherein in step 1, the source reads data from the source host according to the information of the remaining data volume of the message, the source address of the next packet, the destination address of the next packet, MTU (maximum transport packet length), etc.
5. An out-of-order RDMA device with asynchronous notification comprises a sending engine and a receiving engine, and is characterized in that the sending engine comprises a message hanging buffer module, a response processing module, an unpacking module, a packing module, an access interface module and a data buffer module; the receiving engine comprises a request packet processing module, a response queue module, a response packet processing module, a receiving queue management module and a receiving engine access interface module;
the message hanging buffer module is used for acquiring and registering a message descriptor of the RDMA message packet and recording message information of the RDMA message packet; the message information includes message ID, remaining data amount, next packet source data address, next packet destination data address, remaining response amount, asynchronous notification reception queue number, message descriptor, message status, MTU and the like,
the unpacking module is electrically connected with the message hanging buffer module, the packaging module and the access interface module and is used for reading the message descriptor and unpacking the message descriptor to generate package control information, outputting the package information content to the packaging module, outputting a data reading request to the access interface module and updating the message state information and writing back to the message hanging buffer; when a signal that a message sent by a message hanging buffer enters a state of sending an asynchronous notification is acquired, the unpacking module generates asynchronous notification control information and message descriptor content and outputs the asynchronous notification control information and the message descriptor content to the packing module, wherein the message packet data content is a descriptor of an RDMA message;
the access interface module is electrically connected with the data buffer module and is used for processing all main memory accesses of the sending engine, initiating an access request to a main memory access after receiving a data fetching request of the unpacking module, and writing an access response into the data buffer; the memory access interface module is also electrically connected with the message suspension buffer module and is used for processing a completion event input by the message suspension buffer module;
the data buffer is used for storing the data content of the RDMA message packet;
the group package module is electrically connected with the unpacking module and the data buffer module and is used for being responsible for the group package of the message package to send to the Internet, receiving package control information from the unpacking module, receiving package data from the data buffer, packaging the package data into an RDMA message request package and sending the RDMA message request package to the Internet access; when the unpacking module obtains the message hanging buffer to send out the message and enters into the sending asynchronous notification state signal, the unpacking module obtains the asynchronous notification control information and the message descriptor, and then the asynchronous notification control information and the message descriptor are packaged into a request packet for sending the asynchronous notification message and sent to the internet access;
the request packet processing module is responsible for processing message request packets received on the network and is electrically connected with the response queue module, the memory access interface module and the receiving queue management module; when the message request packet is an RDMA message request packet, the request packet is analyzed, data in the packet is extracted, a write data request is generated and input to the memory access interface module, and meanwhile, an RDMA message response packet is generated according to the content of the message request packet and input to the response queue module; when the message request packet is a request packet for sending an asynchronous notification message, generating a response packet for sending the asynchronous notification message, sending the response packet to a response queue module, generating asynchronous notification information, and writing the asynchronous notification information into a receiving queue management module;
the response queue module is electrically connected with the request packet processing module, processes the response packet output by the request packet processing module and sends the response packet to the Internet;
the response packet processing module is responsible for processing the response packet received on the network, generating remote response control information and inputting the remote response control information into the response processing module;
the response processing module is electrically connected with the response packet processing module and the message hanging and buffering module, is responsible for processing remote response and processing according to the type of the response packet; when the response packet is an RDMA message response packet, controlling the message suspension buffer device to respond and count, and after finishing the response, controlling the message suspension buffer device to generate and send a message to enter a state signal for sending an asynchronous notification; when the response packet is a response packet for sending an asynchronous notification message, controlling a message hanging buffer module to generate a completion event and write the completion event into a main memory;
the receiving queue management module is electrically connected with the request packet processing module and is used for positioning to a specified position corresponding to a receiving queue space according to a receiving queue number when receiving asynchronous notification information input by the request packet processing module, generating a queue writing request and inputting the queue writing request to the receiving engine memory access interface module;
the receiving engine memory access interface module is responsible for processing all memory access requests of the receiving engine, including write data requests input by the request packet processing module and queue write requests input by the receiving queue management module, and all write main memory requests are sent to the main memory path.
6. The out-of-order RDMA device with asynchronous notification of claim 5, wherein when the response packet is an RDMA message response packet, the response processing module has a response counting mode of: and when a response packet is received, matching the corresponding message information of the suspension buffer unit according to the message suspension buffer number and the message ID number in the response packet, and reducing the residual response number in the suspension buffer unit by one after the matching is successful.
7. The out-of-order RDMA device with asynchronous notification of claim 5, wherein the receive queue management module has receive queue storage space in main memory space.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910870364.8A CN110602211B (en) | 2019-09-16 | 2019-09-16 | Out-of-order RDMA method and device with asynchronous notification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910870364.8A CN110602211B (en) | 2019-09-16 | 2019-09-16 | Out-of-order RDMA method and device with asynchronous notification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110602211A true CN110602211A (en) | 2019-12-20 |
| CN110602211B CN110602211B (en) | 2022-06-14 |
Family
ID=68859782
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910870364.8A Active CN110602211B (en) | 2019-09-16 | 2019-09-16 | Out-of-order RDMA method and device with asynchronous notification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110602211B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022100312A1 (en) * | 2020-11-13 | 2022-05-19 | 苏州浪潮智能科技有限公司 | Method and apparatus for improving message processing efficiency for flash channel controller |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1997977A (en) * | 2003-12-11 | 2007-07-11 | 国际商业机器公司 | Reducing number of write operations relative to delivery of out-of-order RDMA send messages |
| US20070208820A1 (en) * | 2006-02-17 | 2007-09-06 | Neteffect, Inc. | Apparatus and method for out-of-order placement and in-order completion reporting of remote direct memory access operations |
| CN101409715A (en) * | 2008-10-22 | 2009-04-15 | 中国科学院计算技术研究所 | Method and system for communication using InfiniBand network |
| CN107925593A (en) * | 2015-09-23 | 2018-04-17 | 英特尔公司 | The technology of message synchronization based on polymerization |
| CN110113425A (en) * | 2019-05-16 | 2019-08-09 | 南京大学 | A kind of SiteServer LBS and equalization methods based on the unloading of RDMA network interface card correcting and eleting codes |
| CN110177118A (en) * | 2019-06-13 | 2019-08-27 | 上海海事大学 | A kind of RPC communication method based on RDMA |
-
2019
- 2019-09-16 CN CN201910870364.8A patent/CN110602211B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1997977A (en) * | 2003-12-11 | 2007-07-11 | 国际商业机器公司 | Reducing number of write operations relative to delivery of out-of-order RDMA send messages |
| US20070208820A1 (en) * | 2006-02-17 | 2007-09-06 | Neteffect, Inc. | Apparatus and method for out-of-order placement and in-order completion reporting of remote direct memory access operations |
| CN101409715A (en) * | 2008-10-22 | 2009-04-15 | 中国科学院计算技术研究所 | Method and system for communication using InfiniBand network |
| CN107925593A (en) * | 2015-09-23 | 2018-04-17 | 英特尔公司 | The technology of message synchronization based on polymerization |
| CN110113425A (en) * | 2019-05-16 | 2019-08-09 | 南京大学 | A kind of SiteServer LBS and equalization methods based on the unloading of RDMA network interface card correcting and eleting codes |
| CN110177118A (en) * | 2019-06-13 | 2019-08-27 | 上海海事大学 | A kind of RPC communication method based on RDMA |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022100312A1 (en) * | 2020-11-13 | 2022-05-19 | 苏州浪潮智能科技有限公司 | Method and apparatus for improving message processing efficiency for flash channel controller |
| US11977935B2 (en) | 2020-11-13 | 2024-05-07 | Inspur Suzhou Intelligent Technology Co., Ltd. | Method and apparatus for improving message processing efficiency of flash channel controller |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110602211B (en) | 2022-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11991072B2 (en) | System and method for facilitating efficient event notification management for a network interface controller (NIC) | |
| US7076569B1 (en) | Embedded channel adapter having transport layer configured for prioritizing selection of work descriptors based on respective virtual lane priorities | |
| US7603429B2 (en) | Network adapter with shared database for message context information | |
| US7263103B2 (en) | Receive queue descriptor pool | |
| US7860943B2 (en) | Enhanced network direct attached storage controller | |
| US11025564B2 (en) | RDMA transport with hardware integration and out of order placement | |
| US11068412B2 (en) | RDMA transport with hardware integration | |
| TW202016744A (en) | Host, nvme ssd and method for storage service | |
| US7457845B2 (en) | Method and system for TCP/IP using generic buffers for non-posting TCP applications | |
| CN111026324B (en) | Updating method and device of forwarding table entry | |
| US20050089033A1 (en) | Method, system, and program for constructing a packet | |
| CN110677220B (en) | RDMA (remote direct memory Access) message mechanism based on multi-track redundant response and implementation device thereof | |
| CN110602211B (en) | Out-of-order RDMA method and device with asynchronous notification | |
| US9544401B2 (en) | Device and method for data communication using a transmission ring buffer | |
| US20150199298A1 (en) | Storage and network interface memory share | |
| US7266614B1 (en) | Embedded channel adapter having link layer configured for concurrent retrieval of payload data during packet transmission | |
| WO2022188807A1 (en) | Data transmission system and related device | |
| CN116226027B (en) | Data transmission system, method, device, communication equipment and storage medium | |
| WO2022141250A1 (en) | Data transmission method and related apparatus | |
| CN115914144A (en) | Direct access of a data plane of a switch to a storage device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Gao Jiangang Inventor after: Ding Yajun Inventor after: Mou Huaxian Inventor after: Si Tian Hao Inventor after: Zhou Jianyi Inventor after: Feng Dongming Inventor after: Zhou Zhou Inventor before: Ding Yajun Inventor before: Mou Huaxian Inventor before: Si Tian Hao Inventor before: Zhou Jianyi Inventor before: Feng Dongming Inventor before: Zhou Zhou |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |