CN112559436B - Context access method and system of RDMA communication equipment - Google Patents
Context access method and system of RDMA communication equipment Download PDFInfo
- Publication number
- CN112559436B CN112559436B CN202011487278.8A CN202011487278A CN112559436B CN 112559436 B CN112559436 B CN 112559436B CN 202011487278 A CN202011487278 A CN 202011487278A CN 112559436 B CN112559436 B CN 112559436B
- Authority
- CN
- China
- Prior art keywords
- module
- context
- request
- information
- read
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004891 communication Methods 0.000 title claims abstract description 111
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000005540 biological transmission Effects 0.000 claims description 24
- 230000009191 jumping Effects 0.000 claims description 17
- 230000004044 response Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 description 12
- 239000000872 buffer Substances 0.000 description 8
- 238000012546 transfer Methods 0.000 description 7
- 238000007417 hierarchical cluster analysis Methods 0.000 description 4
- 230000003139 buffering effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
- G06F15/173—Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F15/17306—Intercommunication techniques
- G06F15/17331—Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Abstract
The invention provides a method and a system for accessing context by RDMA communication equipment, comprising a cache module, a resident storage module and an access management module. The cache module consists of a storage unit and a cache control unit and is used for providing an interface for non-key fields of the RDMA communication equipment for accessing the context information. The resident memory module is used for storing key field information of the context for the RDMA communication device to access. The access management module is used for performing read-write control on the context field information and ensuring the consistency of the access of the context information by a plurality of modules.
Description
Technical Field
The invention relates to the technical field of communication, in particular to a method and a system for storing context of communication equipment supporting RDMA.
Background
The remote direct memory access (Remote Direct Memory Access, RDMA) technology bypasses the remote central processing unit (Central Processing Unit, CPU) by directly accessing the remote memory, reduces the time delay in the data transmission process, and reduces the load of the server CPU. In general, RDMA enabled servers typically include a CPU, a memory unit, and an RDMA enabled host channel adapter (Host Channel Adapter, HCA). The HCA connects HCAs of other servers through cables, thereby enabling RDMA communication between servers.
The following analyzes the problems present in the RDMA communication devices during data transfer. FIG. 1 is a data transmission flow of an RDMA communication device. When the server is ready to send data, a doorbell message (1) is sent to the RDMA communication device. After the RDMA communication device receives the doorbell, the queue pair context (Queue Pair Context, QPC) information (2, 3) is obtained from the memory according to the doorbell information. Next, the RDMA communication device reads the descriptor information (4, 5) of the transmission data from the memory according to a part of the fields in the QPC, and then the RDMA communication device acquires the data to be transmitted (6, 7) from the memory using the QPC and the descriptor information and transmits it outwards. When the RDMA communication device completes the data transfer process, it will fetch completion queue context (Completion Queue Context, CQC) and event queue context (Event Queue Context, EQC) information (8, 9) from memory, and finally write completion event information (10) to the completion queue and event queue.
FIG. 2 is a data reception flow for an RDMA communication device. When a remote data packet is received, the RDMA communication device first parses the packet header and obtains QPC information (1, 2) for processing the packet from the memory based on the header information. Next, the RDMA communication device reads descriptors (3, 4) of the received data from the memory according to a partial field in the QPC; writing the received data packet into the memory according to the QPC and the descriptor (5); finally, the CQC, EQC are read from memory (6, 7), and completion event information is written to the completion queue and event queue (8).
As can be seen from the above flow, the RDMA communication device must obtain information of a portion of the fields in the corresponding context (including but not limited to QPC, CQC, EQC) during processing of the data, which increases the processing delay per packet. In the existing design method, a context buffer module is added in the RDMA communication equipment, and the context buffer module buffers the context information read for the first time to the RDMA communication equipment, so that the context is not required to be read from a memory when the context is reused for data transmission or reception next time, and the context reading time delay is reduced.
However, the above method still has the following problems: 1. context information is cached in the RDMA communication equipment, and as the number of queue pairs increases, on-chip storage resources of the RDMA communication equipment cannot save all the contexts, so that the processing delay of part of queue pairs is higher; 2. not all information in the context has a large impact on data transfer latency, and opening up a large cache memory context in an RDMA communication device can result in waste of on-chip memory resources.
Disclosure of Invention
In view of the foregoing, an object of the present invention is to provide a method and system for accessing a context of an RDMA communication device (e.g., HCA) stored by fields, i.e., the context is divided into a field stored on-chip in the RDMA communication device and a field partially cached on-chip according to the criticality of each field affecting the time delay. The part resident on the chip for storage can effectively reduce the reading time delay of the context in the data transmission process; while the on-chip portion may reduce frequent reading of memory by RDMA communication devices.
In order to overcome the defects in the prior art, the invention provides a context access method of RDMA communication equipment, which comprises the following steps:
step 1, creating a context item, storing a delay key field of the context item to a resident storage module of a context management system in RDMA communication equipment, and adding the context item to a cache module of the context management system in the RDMA communication equipment;
step 2, the RDMA communication equipment sends a delay key field access request to the context management system according to the received doorbell information, the context management system returns the delay key field, reads non-delay key information of the context item from the memory and stores the non-delay key information into the cache unit, and meanwhile the RDMA communication equipment acquires descriptor information from the memory by utilizing the delay key field; judging whether the data transmission request in the descriptor information is a read request, if so, executing the step 3, otherwise, executing the step 4, wherein the data transmission request is a write request;
step 3, the RDMA communication device interacts context information with the context management system, and simultaneously, sends out a reading request and obtains a response by utilizing the read descriptor information so as to realize data reading;
and 4, the RDMA communication equipment acquires data to be transmitted from the memory according to the descriptor information, and orderly transmits the data to be transmitted outwards through interacting the context information with the context management system so as to realize data transmission.
The context access method of the RDMA communication device, wherein the step 1 comprises the following steps: and writing the content of the item except the time delay key field in the context item into the memory.
The context access method of the RDMA communication device, wherein the step 4 includes:
step 41, judging whether the write request is the first write to the context entry, if yes, setting the context entry corresponding to the write request to be valid, executing step 42, otherwise, directly jumping to step 42;
step 42, judging whether the write request needs to write the field in the resident memory module, if yes, writing the information in the write request into the resident memory module, executing step 43, otherwise, directly executing step 43;
and 43, judging whether the write request needs to be written into a field in the cache module, if so, writing information in the write request into the cache module, otherwise, giving up executing the write request.
The context access method of the RDMA communication device, wherein the step 3 comprises the following steps:
step 31, judging whether the context entry accessed by the read request is valid, if so, executing step 32, otherwise, discarding the read request;
step 32, judging whether the read request needs to read the field in the resident memory module, if yes, reading the field in the resident memory module, and reading the corresponding entry of the read request from the server memory to the cache module, executing step 33, otherwise, directly executing step 33;
step 33, judging whether the read request needs to read the field in the cache module, if yes, reading the field in the cache module, and executing step 34, otherwise, directly executing step 34;
step 34, returning the information read in step 32 and step 33 to the RDMA communication device.
The context access method of the RDMA communication device, before executing the step 41, determines whether the write request is a request for canceling a context entry, if yes, sets a corresponding entry in the resident memory module and the cache module as invalid, cancels the invalid entry, and otherwise, executes the step 41.
The invention also provides a context access system of the RDMA communication device, which comprises:
the module 1 is used for creating a context item, storing a delay key field of the context item into a resident storage module of a context management system in the RDMA communication equipment, and adding the context item into a cache module of the context management system in the RDMA communication equipment;
a module 2, configured to enable the RDMA communication device to send a latency key field access request to the context management system according to the received doorbell information, where the context management system returns a latency key field, reads non-latency key information of the context entry from the memory, and stores the non-latency key information in the cache unit, and meanwhile, the RDMA communication device obtains descriptor information from the memory using the latency key field; judging whether the data transmission request in the descriptor information is a read request, if so, calling the module 3, otherwise, calling the module 4, wherein the data transmission request is a write request;
a module 3, configured to enable the RDMA communication device to interact with the context management system by using the context information, and send a read request and obtain a response to implement data reading by using the read descriptor information;
and a module 4, configured to enable the RDMA communication device to obtain data to be sent from the memory according to the descriptor information, and sequentially send the data to be sent outwards by interacting with the context management system to implement data sending.
The context access system of the RDMA communication device, wherein the module 1 comprises: and writing the content of the item except the time delay key field in the context item into the memory.
The context access system of the RDMA communication device, wherein the module 4 comprises:
the module 41 judges whether the write request is the first writing of the context entry, if yes, the context entry corresponding to the write request is set to be valid, the module 42 is called, and if not, the module 42 is directly skipped;
the module 42 judges whether the write request needs to write the field in the resident memory module, if yes, the information in the write request is written into the resident memory module, the module 43 is called, otherwise, the module 43 is directly called;
and a module 43 for judging whether the write request needs to write the field in the cache module, if yes, writing the information in the write request into the cache module, otherwise, giving up executing the write request.
The context access system of the RDMA communication device, wherein the module 3 comprises:
a module 31 for judging whether the context entry accessed by the read request is valid, if so, calling a module 32, otherwise, giving up executing the read request;
the module 32, judge whether the read request needs to read the field in the resident memory module, if yes, read the field in the resident memory module, and read the corresponding entry of the read request from the server memory to the cache module, call the module 33, otherwise call the module 33 directly;
the module 33 judges whether the read request needs to read the field in the cache module, if yes, reads the field in the cache module, and calls the module 34, otherwise directly calls the module 34;
block 34, returns the information read by blocks 32 and 33 to the RDMA communication device.
The context access system of the RDMA communication device determines, before executing the module 41, whether the write request is a request for canceling a context entry, if so, sets a corresponding entry in the resident memory module and the cache module to be invalid, cancels the invalid entry, and otherwise, executes the module 41.
The advantages of the invention are as follows:
1. after the context entry is created, the time delay key field information in the context entry information is stored on the resident memory module, and no cache miss exists for the first time of reading the context entry.
2. And the field information of the key time delay in the context is all stored in the resident memory module in the equipment, so that the time delay in the data transmission process is reduced, and the data processing rate is improved.
3. And when the resident module is read, the non-time-delay critical context segment information is read into the device (in the cache module), so that the reading time delay under the condition of cache miss is hidden, the time delay under the condition of the context cache miss is reduced, and the data processing rate is improved.
4. The non-delay critical field information in the context adopts a strategy of partial buffering on the device, so that frequent reading of the context is avoided, and the contention of the system bus bandwidth is reduced.
Drawings
FIG. 1 is a flow of a conventional RDMA communication device in data transmission;
FIG. 2 is a flow of a conventional RDMA communication device in data reception;
FIG. 3 is a context access system architecture in an RDMA enabled communication device of the present invention;
FIG. 4 is a context access flow in the RDMA communication device data transfer process of the present invention;
FIG. 5 is a context access flow in an RDMA communication device data reception process of the present invention;
FIG. 6 is an RDMA enabled communication device context read-write system access flow of the present invention;
FIG. 7 is an embodiment of a context access method of an RDMA communication device of the present invention;
FIG. 8 is an embodiment of a context access system architecture of the present invention;
FIG. 9 is an embodiment of a creation and cancellation process of a context entry in a context access system of the present invention;
FIG. 10 is an embodiment of an access flow of a context entry in a context access system of the present invention.
Detailed Description
In order to make the above features and effects of the present invention more clearly understood, the following specific examples are given with reference to the accompanying drawings.
The overall system structure is shown in fig. 3, and comprises: the system comprises a caching module, a resident storage module and an access management module. The system can effectively reduce the time delay in the data transmission process of the RDMA communication equipment.
The buffer memory module is characterized by being independent of a storage module outside a host memory and is a buffer memory attached to the RDMA communication device. The cache module comprises a storage unit and a cache control unit, wherein the storage unit is used for caching the possibly used context information; the cache control unit is used for controlling the replacement strategy of the cache module. The content in the cache module is written in when the context is created, and partial items in the cache are written in the memory in a cache replacement mode when the cache space is full; when the context is read, if the context item to be read does not exist in the storage unit, the cache module reads the corresponding item from the memory to the storage unit.
A resident memory module that functions to hold fields in the context that are resident on the device. These fields are written at the time of context creation and are always kept in the on-chip memory space of the device. When the context is logged off, it is invalidated.
And the access management module is used for intensively controlling write operation or read operation of different modules in the RDMA communication device on the context resource. If the access type is a write operation, the access management module writes the context information into the resident memory module or the cache module according to the field to be written. If the access type is a read operation, the access management module reads the context information from the resident memory module or the cache module according to the field to be read out, and performs merging output.
The invention also relates to a context read-write method of the RDMA communication device, which introduces a queue pair context access flow of the RDMA communication device in the data transmission process:
step 410. If the rdma communication device receives doorbell information, then it jumps to step 420; otherwise still execute step 410;
step 420, reading the context information (located in the resident memory module) of the queue with critical delay, wherein the context information is completely stored on the device, and jumping to step 430;
step 430, according to the QPC information read in step 420, the descriptor information related to data transmission is read in the memory, and meanwhile, the corresponding non-time-delay critical context information (including but not limited to QPC, CQC, EQC, etc.) is read and stored in the buffer module for subsequent descriptor processing. At this point, the information of the designated QPC will be all cached on the device, and the process goes to step 440;
step 440, reading the data to be transmitted from the memory according to the descriptor information, and jumping to step 450;
step 450. The QPC information is read from the context access system for processing and transmitting data.
The invention also relates to a context read-write method of the RDMA communication device, which introduces a queue pair context access flow of the RDMA communication device in the data receiving process:
step 510.Rdma communication device receives the remote data packet, and jumps to step 520; otherwise, continuing to execute step 510;
step 520, reading delay critical QPC information (located in a resident memory module), the information being completely stored on the device, and jumping to step 530;
step 530, reading descriptor information in the memory according to the QPC information read in step 520, and simultaneously reading non-time-delay critical context information (including but not limited to QPC, CQC, EQC, etc.), where the specified context information is all cached on the device, and jumping to step 540;
step 540, writing the remote data into the memory according to the descriptor information and the context information obtained in step 530.
The invention relates to a context read-write system of RDMA communication equipment, which comprises the following internal context access processing flow:
step 610, if the context management system receives an access request of the RDMA communication device, jumping to step 620; otherwise, still execute step 610;
step 620, judging whether the incoming access request is a write request, if so, jumping to step 630; otherwise, a read request is received and the process jumps to step 660;
step 630, judging whether the write request is the first write to the context entry, if so, setting the entry as valid, and jumping to step 640; otherwise, directly jump to step 640;
step 640, judging whether the write request needs to write the field in the resident memory module, if so, writing the corresponding information in the request into the resident memory module, and jumping to step 650; otherwise, directly jump to step 650;
step 650, judging whether the write request needs to write the field in the cache module, if so, writing the corresponding information in the request into the cache module, and jumping to step 610; otherwise, the execution of the write request is abandoned, and the process goes directly to step 610;
step 660, the request is a read request, whether the context entry accessed by the read request is valid is judged, if so, the step 670 is skipped; otherwise, the execution of the read request is aborted to step 610;
step 670, judging whether the read request needs to read the field in the resident memory module, if so, reading the corresponding field in the resident memory module, reading the content of the item to be read from the memory to the cache module, and jumping to step 680; otherwise, directly jump to step 680;
step 680, judging whether the read request needs to read the field in the cache module, if so, reading the field in the cache module, and jumping to step 690; otherwise, directly jump to step 690;
step 690. Combine the read context information, return a context read response to the RDMA communication device, and jump to step 610.
One embodiment of a context access system on an RDMA communication device is shown in FIG. 7. The RDMA communication device comprises: RDMA engine, control plane management system, context access system, transport interface, system bus controller.
An RDMA engine for RDMA related protocol processing, the module using context information in data sending and data receiving processes; the control plane management system is used for managing the works of creation, allocation, cancellation and the like of various resources on the RDMA communication equipment; a context access system for managing access to context resources on the RDMA communication device; a transport interface including a link layer and a physical layer of the RDMA communication device, the interface including but not limited to being an ethernet or InfiniBand link; a system bus controller for controlling data transfer between the CPU and the RDMA communication device, in this embodiment using an external device interconnect express channel (Peripheral Component Interconnect Express, PCIe) as the system bus for the RDMA communication device and the CPU.
The flow of an RDMA device accessing context resources during data transfer is described below using the RDMA communication device of fig. 7 as an example. First, the control plane system in the RDMA communication device writes the context entry information sent from the CPU to the context access system (1); next, the RDMA engine receives a data transmission request, where the request may be a data transmission request sent by the CPU, or may be a data packet coming from a remote end; then, the RDMA engine sends a queue pair address reading request (2) to the context access system, and uses the memory address to access the descriptor information of the queue pair in the memory; finally, the RDMA engine reads and writes other context fields (including but not limited to queue pair status field, destination queue pair number, send packet sequence number, etc.) from the context access system to complete the data transfer process.
FIG. 8 is an embodiment of a context access system architecture. This case contains three modules: the access management module, the cache module and the resident storage module. The access management module is used for controlling the RDMA engine and the control plane management system to read-write access of the context; the buffer module comprises a storage unit and a buffer control unit, and is used for storing information such as a queue pair state field, a destination end queue pair number, a data packet serial number and the like in the embodiment; and the resident memory module is used for storing the head address of the sending queue memory and the head address of the receiving queue memory in the context and the valid flag bit of the context entry in the embodiment.
The invention relates to a creation and cancellation flow of a context item in a context access system, which comprises the following steps:
step 910, the context access system determines whether a write request of the control plane management system is received, if so, the step 920 is skipped; otherwise still execute step 910;
step 920. If the request is a create context entry request, step 930 is skipped; otherwise, the request is a log-off context entry request, jumping to step 950;
step 930, the valid position of the corresponding entry in the resident memory module is 1, which indicates that the entry is a valid entry, and simultaneously, the queue address information is written, and step 940 is skipped;
step 940, writing the information of the item into the cache module, if the cache is full, replacing one item and writing the item;
step 950, the effective position of the corresponding entry in the resident memory module is 0, which indicates that the cache entry is invalid at present, and step 960 is skipped;
if the entry is in the cache, the valid bit of the cache entry is set to 0, indicating that the cache location may write a new entry.
The invention relates to a read-write flow of a context item in a context access system, which comprises the following steps:
step 1010, the context access system judges whether a context read-write request from an RDMA engine is received, if so, the step 1020 is skipped; otherwise, still execute step 1010;
step 1020, reading the valid bit of the request entry from the resident memory module, if the valid bit is 1, jumping to step 1030; otherwise, jumping to step 1010;
step 1030, in this embodiment, if the request is a request for reading a queue memory address, including a send queue address, a receive queue address, etc., then step 1040 is skipped; otherwise, jump to step 1060;
step 1040. Returning the queue address information of the request from the resident memory cell, and jumping to step 1050;
step 1050. The cache module writes the read data from the memory into the cache module. If the cache module is full, replacing one context entry in the cache module into the memory module by adopting a replacement strategy;
the RDMA engine's context access request will access the context information in the cache module 1060.
The following is a system example corresponding to the above method example, and this embodiment mode may be implemented in cooperation with the above embodiment mode. The related technical details mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the related technical details mentioned in the present embodiment can also be applied to the above-described embodiments.
The invention also provides a context access system of the RDMA communication device, which comprises:
the module 1 is used for creating a context item, storing a delay key field of the context item into a resident storage module of a context management system in the RDMA communication equipment, and adding the context item into a cache module of the context management system in the RDMA communication equipment;
a module 2, configured to enable the RDMA communication device to send a latency key field access request to the context management system according to the received doorbell information, where the context management system returns a latency key field, reads non-latency key information of the context entry from the memory, and stores the non-latency key information in the cache unit, and meanwhile, the RDMA communication device obtains descriptor information from the memory using the latency key field; judging whether the data transmission request in the descriptor information is a read request, if so, calling the module 3, otherwise, calling the module 4, wherein the data transmission request is a write request;
a module 3, configured to enable the RDMA communication device to interact with the context management system by using the context information, and send a read request and obtain a response to implement data reading by using the read descriptor information;
and a module 4, configured to enable the RDMA communication device to obtain data to be sent from the memory according to the descriptor information, and sequentially send the data to be sent outwards by interacting with the context management system to implement data sending.
The context access system of the RDMA communication device, wherein the module 1 comprises: and writing the content of the item except the time delay key field in the context item into the memory.
The context access system of the RDMA communication device, wherein the module 4 comprises:
the module 41 judges whether the write request is the first writing of the context entry, if yes, the context entry corresponding to the write request is set to be valid, the module 42 is called, and if not, the module 42 is directly skipped;
the module 42 judges whether the write request needs to write the field in the resident memory module, if yes, the information in the write request is written into the resident memory module, the module 43 is called, otherwise, the module 43 is directly called;
and a module 43 for judging whether the write request needs to write the field in the cache module, if yes, writing the information in the write request into the cache module, otherwise, giving up executing the write request.
The context access system of the RDMA communication device, wherein the module 3 comprises:
a module 31 for judging whether the context entry accessed by the read request is valid, if so, calling a module 32, otherwise, giving up executing the read request;
the module 32, judge whether the read request needs to read the field in the resident memory module, if yes, read the field in the resident memory module, and read the corresponding entry of the read request from the server memory to the cache module, call the module 33, otherwise call the module 33 directly;
the module 33 judges whether the read request needs to read the field in the cache module, if yes, reads the field in the cache module, and calls the module 34, otherwise directly calls the module 34;
block 34, returns the information read by blocks 32 and 33 to the RDMA communication device.
The context access system of the RDMA communication device determines, before executing the module 41, whether the write request is a request for canceling a context entry, if so, sets a corresponding entry in the resident memory module and the cache module to be invalid, cancels the invalid entry, and otherwise, executes the module 41.
Claims (6)
1. A method of context access for an RDMA communication device, comprising:
step 1, creating a context item, storing a delay key field of the context item to a resident storage module of a context management system in RDMA communication equipment, and adding the context item to a cache module of the context management system in the RDMA communication equipment;
step 2, the RDMA communication equipment sends a delay key field access request to the context management system according to the received doorbell information, the context management system returns the delay key field, reads non-delay key information of the context item from the memory and stores the non-delay key information into the cache unit, and meanwhile the RDMA communication equipment acquires descriptor information from the memory by utilizing the delay key field; judging whether the data transmission request in the descriptor information is a read request, if so, executing the step 3, otherwise, executing the step 4, wherein the data transmission request is a write request;
step 3, the RDMA communication device interacts context information with the context management system, and simultaneously, sends out a reading request and obtains a response by utilizing the read descriptor information so as to realize data reading;
step 4, the RDMA communication equipment acquires data to be transmitted from the memory according to the descriptor information, and orderly transmits the data to be transmitted outwards through interacting the context information with the context management system so as to realize data transmission;
wherein the step 3 comprises:
step 31, judging whether the context entry accessed by the read request is valid, if so, executing step 32, otherwise, discarding the read request;
step 32, judging whether the read request needs to read the field in the resident memory module, if yes, reading the field in the resident memory module, and reading the corresponding entry of the read request from the server memory to the cache module, executing step 33, otherwise, directly executing step 33;
step 33, judging whether the read request needs to read the field in the cache module, if yes, reading the field in the cache module, and executing step 34, otherwise, directly executing step 34;
step 34, returning the information read in the steps 32 and 33 to the RDMA communication device;
the step 4 comprises the following steps:
step 41, judging whether the write request is the first write to the context entry, if yes, setting the context entry corresponding to the write request to be valid, executing step 42, otherwise, directly jumping to step 42;
step 42, judging whether the write request needs to write the field in the resident memory module, if yes, writing the information in the write request into the resident memory module, executing step 43, otherwise, directly executing step 43;
and 43, judging whether the write request needs to be written into a field in the cache module, if so, writing information in the write request into the cache module, otherwise, giving up executing the write request.
2. The method of context access in an RDMA communication device of claim 1, wherein step 1 comprises: and writing the content of the item except the time delay key field in the context item into the memory.
3. The method for accessing context of an RDMA communication device according to claim 1, wherein before executing the step 41, it is determined whether the write request is a request to cancel a context entry, if so, the corresponding entry in the resident memory module and cache module is set to be invalid, and if not, the invalid entry is canceled, otherwise, the step 41 is executed.
4. A context access system for an RDMA communication device, comprising:
the module 1 is used for creating a context item, storing a delay key field of the context item into a resident storage module of a context management system in the RDMA communication equipment, and adding the context item into a cache module of the context management system in the RDMA communication equipment;
a module 2, configured to enable the RDMA communication device to send a latency key field access request to the context management system according to the received doorbell information, where the context management system returns a latency key field, reads non-latency key information of the context entry from the memory, and stores the non-latency key information in the cache unit, and meanwhile, the RDMA communication device obtains descriptor information from the memory using the latency key field; judging whether the data transmission request in the descriptor information is a read request, if so, calling the module 3, otherwise, calling the module 4, wherein the data transmission request is a write request;
a module 3, configured to enable the RDMA communication device to interact with the context management system by using the context information, and send a read request and obtain a response to implement data reading by using the read descriptor information;
the module 4 is configured to enable the RDMA communication device to obtain data to be sent from the memory according to the descriptor information, and sequentially send the data to be sent outwards by interacting with the context management system to implement data sending;
wherein the module 3 comprises:
a module 31 for judging whether the context entry accessed by the read request is valid, if so, calling a module 32, otherwise, giving up executing the read request;
the module 32, judge whether the read request needs to read the field in the resident memory module, if yes, read the field in the resident memory module, and read the corresponding entry of the read request from the server memory to the cache module, call the module 33, otherwise call the module 33 directly;
the module 33 judges whether the read request needs to read the field in the cache module, if yes, reads the field in the cache module, and calls the module 34, otherwise directly calls the module 34;
a module 34 for returning the information read by the modules 32 and 33 to the RDMA communication device;
the module 4 comprises:
the module 41 judges whether the write request is the first writing of the context entry, if yes, the context entry corresponding to the write request is set to be valid, the module 42 is called, and if not, the module 42 is directly skipped;
the module 42 judges whether the write request needs to write the field in the resident memory module, if yes, the information in the write request is written into the resident memory module, the module 43 is called, otherwise, the module 43 is directly called;
and a module 43 for judging whether the write request needs to write the field in the cache module, if yes, writing the information in the write request into the cache module, otherwise, giving up executing the write request.
5. The RDMA communication device context access system of claim 4, wherein the module 1 comprises: and writing the content of the item except the time delay key field in the context item into the memory.
6. The RDMA communication device context access system as recited in claim 4, wherein prior to executing the module 41, determining whether the write request is a request to cancel a context entry, if so, setting a corresponding entry in a resident memory module and cache module to invalid, canceling the invalid entry, otherwise executing the module 41.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011487278.8A CN112559436B (en) | 2020-12-16 | 2020-12-16 | Context access method and system of RDMA communication equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011487278.8A CN112559436B (en) | 2020-12-16 | 2020-12-16 | Context access method and system of RDMA communication equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112559436A CN112559436A (en) | 2021-03-26 |
| CN112559436B true CN112559436B (en) | 2023-11-03 |
Family
ID=75064084
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011487278.8A Active CN112559436B (en) | 2020-12-16 | 2020-12-16 | Context access method and system of RDMA communication equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112559436B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113572582B (en) * | 2021-07-15 | 2022-11-22 | 中国科学院计算技术研究所 | Data transmission and retransmission control method and system, storage medium and electronic device |
| CN115934625B (en) * | 2023-03-15 | 2023-05-16 | 珠海星云智联科技有限公司 | Doorbell knocking method, equipment and medium for remote direct memory access |
| CN116303173B (en) * | 2023-05-19 | 2023-08-08 | 深圳云豹智能有限公司 | Method, device and system for reducing RDMA engine on-chip cache and chip |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6742075B1 (en) * | 2001-12-03 | 2004-05-25 | Advanced Micro Devices, Inc. | Arrangement for instigating work in a channel adapter based on received address information and stored context information |
| CN101377788A (en) * | 2008-09-28 | 2009-03-04 | 中国科学院计算技术研究所 | Method and system of caching management in cluster file system |
| CN103902486A (en) * | 2014-04-08 | 2014-07-02 | 华为技术有限公司 | System, device and method for implementation of remote direct memory access |
| CN110191194A (en) * | 2019-06-13 | 2019-08-30 | 华中科技大学 | A kind of Distributed File System Data transmission method and system based on RDMA network |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7543037B2 (en) * | 2003-12-02 | 2009-06-02 | International Business Machines Corporation | RDMA completion and retransmit system and method |
-
2020
- 2020-12-16 CN CN202011487278.8A patent/CN112559436B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6742075B1 (en) * | 2001-12-03 | 2004-05-25 | Advanced Micro Devices, Inc. | Arrangement for instigating work in a channel adapter based on received address information and stored context information |
| CN101377788A (en) * | 2008-09-28 | 2009-03-04 | 中国科学院计算技术研究所 | Method and system of caching management in cluster file system |
| CN103902486A (en) * | 2014-04-08 | 2014-07-02 | 华为技术有限公司 | System, device and method for implementation of remote direct memory access |
| CN110191194A (en) * | 2019-06-13 | 2019-08-30 | 华中科技大学 | A kind of Distributed File System Data transmission method and system based on RDMA network |
Non-Patent Citations (1)
| Title |
|---|
| 一种加速广域文件系统读写访问的缓存策略;马留英;蔡杰明;刘浏;刘振军;;计算机研究与发展(第S1期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112559436A (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112559436B (en) | Context access method and system of RDMA communication equipment | |
| US11899596B2 (en) | System and method for facilitating dynamic command management in a network interface controller (NIC) | |
| US8019902B2 (en) | Network adapter with shared database for message context information | |
| US6327625B1 (en) | FIFO-based network interface supporting out-of-order processing | |
| US7870306B2 (en) | Shared memory message switch and cache | |
| US5606665A (en) | Buffer descriptor prefetch in network and I/O design | |
| US6757768B1 (en) | Apparatus and technique for maintaining order among requests issued over an external bus of an intermediate network node | |
| US8542693B2 (en) | Managing free packet descriptors in packet-based communications | |
| US20110225168A1 (en) | Hash processing in a network communications processor architecture | |
| US8943507B2 (en) | Packet assembly module for multi-core, multi-thread network processors | |
| CN113225307B (en) | Optimization method, system and terminal for pre-reading descriptors in uninstalling engine network card | |
| US7647436B1 (en) | Method and apparatus to interface an offload engine network interface with a host machine | |
| KR20110099796A (en) | Message communication techniques | |
| US7788437B2 (en) | Computer system with network interface retransmit | |
| CN115934625B (en) | Doorbell knocking method, equipment and medium for remote direct memory access | |
| US9336162B1 (en) | System and method for pre-fetching data based on a FIFO queue of packet messages reaching a first capacity threshold | |
| EP1891503B1 (en) | Concurrent read response acknowledge enhanced direct memory access unit | |
| EP2383658B1 (en) | Queue depth management for communication between host and peripheral device | |
| CN113157465B (en) | Message sending method and device based on pointer linked list | |
| CN116471242A (en) | RDMA-based transmitting end, RDMA-based receiving end, data transmission system and data transmission method | |
| CN113992608B (en) | Network transceiver packet path optimization method, device and storage medium | |
| CN115955441A (en) | Management scheduling method and device based on TSN queue | |
| CN115495406A (en) | Message transmission method, device, equipment and storage medium based on PCIe | |
| US7480739B1 (en) | Segregated caching of linked lists for USB | |
| JP3873589B2 (en) | Processor system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |