CN117573603A - Data processing method and computer equipment for remote direct memory access - Google Patents

Data processing method and computer equipment for remote direct memory access Download PDF

Info

Publication number
CN117573603A
CN117573603A CN202410063040.4A CN202410063040A CN117573603A CN 117573603 A CN117573603 A CN 117573603A CN 202410063040 A CN202410063040 A CN 202410063040A CN 117573603 A CN117573603 A CN 117573603A
Authority
CN
China
Prior art keywords
queue
elements
queue pair
work queue
pair
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202410063040.4A
Other languages
Chinese (zh)
Other versions
CN117573603B (en
Inventor
陈雅民
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhuhai Xingyun Zhilian Technology Co Ltd
Original Assignee
Zhuhai Xingyun Zhilian Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhuhai Xingyun Zhilian Technology Co Ltd filed Critical Zhuhai Xingyun Zhilian Technology Co Ltd
Priority to CN202410063040.4A priority Critical patent/CN117573603B/en
Publication of CN117573603A publication Critical patent/CN117573603A/en
Application granted granted Critical
Publication of CN117573603B publication Critical patent/CN117573603B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F15/00Digital computers in general; Data processing equipment in general
    • G06F15/16Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
    • G06F15/163Interprocessor communication
    • G06F15/173Interprocessor communication using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
    • G06F15/17306Intercommunication techniques
    • G06F15/17331Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to the technical field of computers and provides a data processing method for remote direct memory access and computer equipment. The data processing method comprises the following steps: creating a first queue pair context associated with the first queue pair; and reading the first queue pair context, and reading a first work queue element in the first queue pair, wherein when the first queue pair is created, the length of a dispersion aggregation element respectively included by each of the plurality of work queue elements, the number of dispersion aggregation elements respectively included by each of the plurality of work queue elements, and the length of each of the plurality of work queue elements can be adjusted. Therefore, the resource utilization rate is improved, and the requirements of application environments can be flexibly met.

Description

Data processing method and computer equipment for remote direct memory access
Technical Field
The present disclosure relates to the field of computer technologies, and in particular, to a data processing method and a computer device for remote direct memory access.
Background
In a computer network, remote direct memory access is a direct access from the memory of one computer to the memory of another computer, without involving any one party's operating system. Remote direct memory access directly copies data from a physical line to the memory of an application program through hardware, such as a network adapter, or directly copies data from the memory of an application program to the physical line, the process does not need to copy data between the memory of the application program and a data buffer of an operating system, does not need to participate in a central processing unit, does not involve context switching, and data transmission and other system operations can be performed in parallel. In the prior art, a fixed-length programming strategy is generally adopted at the data processing level of remote direct memory access, and after the length is determined, the lengths of all work queue elements are the same, page-crossing processing is not supported, and variable-length processing is not supported, so that the waste of storage space is caused, and the resource utilization rate is low.
For this reason, the present application provides a data processing method and a computer device for remote direct memory access, which are used to solve the technical problems in the prior art.
Disclosure of Invention
In a first aspect, the present application provides a data processing method for remote direct memory access. The data processing method comprises the following steps: creating a first queue pair context associated with a first queue pair, wherein the first queue pair comprises a plurality of work queue elements, each of the plurality of work queue elements is a decentralized aggregation table comprising at least one decentralized aggregation element, the first queue pair context comprises first queue page address information and first page change information, the first queue page address information is used for indicating the position of a physical page corresponding to each of the plurality of work queue elements, and the first page change information is used for indicating page crossing information of the decentralized aggregation element respectively included by each of the plurality of work queue elements among the physical pages corresponding to the work queue elements; and reading a first work queue element in the first queue pair by reading the first queue pair context, wherein the first work queue element is any work queue element in the plurality of work queue elements, wherein when the first queue pair is created, the length of a scatter aggregate element included in each of the plurality of work queue elements, the number of scatter aggregate elements included in each of the plurality of work queue elements, and the length of each of the plurality of work queue elements are all adjustable.
By the first aspect of the application, on the basis of reading the first work queue element in the first queue pair by reading the first queue pair context, the queue pair with the work queue element with fixed length and the scattered aggregation element with fixed number and length can be avoided, so that the first queue pair can have the work queue element capable of being dynamically adjusted, the scattered aggregation element capable of being dynamically adjusted and the scattered aggregation element capable of being dynamically adjusted in number and length; therefore, the application environment of the remote direct memory access with complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment of the remote direct memory access can be better adapted to the requirements.
In a possible implementation manner of the first aspect of the present application, the scatter aggregate elements included in the first work queue element are distributed on at least two physical pages in the physical pages corresponding to the first work queue element, and the page crossing information of the scatter aggregate elements included in the first work queue element between the at least two physical pages is determined by reading the first page changing information of the first queue pair context.
In a possible implementation manner of the first aspect of the present application, at least two of the decentralized aggregation elements included in the first work queue element have different lengths.
In a possible implementation manner of the first aspect of the present application, the first queue pair is adapted to a physical page size by adjusting a length of a scatter aggregate element included in each of the plurality of work queue elements, a number of scatter aggregate elements included in each of the plurality of work queue elements, and a length of each of the plurality of work queue elements.
In a possible implementation manner of the first aspect of the present application, when the first queue pair is created, physical pages allocated to the first queue pair are in the same physical memory, and by reading the first queue pair context, a page crossing process associated with the first queue pair is implemented.
In a possible implementation manner of the first aspect of the present application, the first queue pair belongs to a plurality of queue pairs, the first queue pair context belongs to a plurality of queue pair contexts corresponding to the plurality of queue pairs one to one, and the plurality of queue pair contexts are created based on a first scheduling capability, wherein a maximum value of respective lengths of work queue elements included in each of the plurality of queue pair contexts is determined based on a lower limit of the first scheduling capability.
In a possible implementation manner of the first aspect of the present application, the data processing method further includes: a first read operation is performed on the plurality of queue pairs based on the first scheduling capability, wherein a lower limit of an operating range of the first read operation is determined based on the lower limit of the first scheduling capability and is not less than a maximum value of a respective length of work queue elements included in each of the plurality of queue pair contexts.
In a possible implementation manner of the first aspect of the present application, the operation range of the first reading operation may be adjusted upward by multiplying by 2 or adjusted downward by dividing by 2.
In a possible implementation manner of the first aspect of the present application, on the basis of performing the first read operation on the plurality of queue pairs based on the first scheduling capability, performing a second read operation on the plurality of queue pairs based on the first scheduling capability, wherein a queue pair context associated with a queue pair of the plurality of queue pairs that is read by the second read operation is used to adjust the first scheduling capability.
In a possible implementation manner of the first aspect of the present application, the data processing method further includes: creating a second queue pair context associated with a second queue pair, wherein the length of the scatter gather elements each included by a work queue element of the second queue pair, the number of scatter gather elements each included by a work queue element of the second queue pair, and the length of each work queue element of the second queue pair are all fixed; and executing remote direct memory access in a dynamic adjustable mode by using the first queue pair and the first queue pair context, or executing remote direct memory access in a fixed-length mode by using the second queue pair and the second queue pair context.
In a possible implementation manner of the first aspect of the present application, when the application scenario of the remote direct memory access is short video, voice communication, the remote direct memory access in the dynamically adjustable mode is performed, and when the application scenario of the remote direct memory access is industrial monitoring, traffic monitoring, logistics automation, the remote direct memory access in the fixed-length mode is performed.
In a possible implementation manner of the first aspect of the present application, the scatter gather element included in each of the plurality of work queue elements is a data segment including an address field, a length field, and a key value field.
In a possible implementation manner of the first aspect of the present application, the first queue pair includes a transmit queue and a receive queue, and the plurality of work queue elements belong to the transmit queue or the receive queue.
In a second aspect, embodiments of the present application further provide a computer device, where the computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements a method according to any implementation manner of any one of the foregoing aspects when the computer program is executed.
In a third aspect, embodiments of the present application also provide a computer-readable storage medium storing computer instructions that, when run on a computer device, cause the computer device to perform a method according to any one of the implementations of any one of the above aspects.
In a fourth aspect, embodiments of the present application also provide a computer program product comprising instructions stored on a computer-readable storage medium, which when run on a computer device, cause the computer device to perform a method according to any one of the implementations of any one of the above aspects.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a data processing method for remote direct memory access according to an embodiment of the present application;
FIG. 2 is a schematic diagram of a first queue pair context provided in an embodiment of the present application;
fig. 3 is a schematic structural diagram of a computing device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
It should be understood that in the description of this application, "at least one" means one or more than one, and "a plurality" means two or more than two. In addition, the words "first," "second," and the like, unless otherwise indicated, are used solely for the purposes of description and are not to be construed as indicating or implying a relative importance or order.
Fig. 1 is a flow chart of a data processing method for remote direct memory access according to an embodiment of the present application. As shown in fig. 1, the data processing method includes the following steps.
Step S110: creating a first queue pair context associated with a first queue pair, wherein the first queue pair comprises a plurality of work queue elements, each of the plurality of work queue elements is a decentralized aggregation table comprising at least one decentralized aggregation element, the first queue pair context comprises first queue page address information and first page change information, the first queue page address information is used for indicating a position of a physical page corresponding to each of the plurality of work queue elements, and the first page change information is used for indicating page crossing information of the decentralized aggregation element respectively included by each of the plurality of work queue elements among the physical pages corresponding to the work queue elements.
Step S120: and reading a first work queue element in the first queue pair by reading the first queue pair context, wherein the first work queue element is any work queue element in the plurality of work queue elements, wherein when the first queue pair is created, the length of a scatter aggregate element included in each of the plurality of work queue elements, the number of scatter aggregate elements included in each of the plurality of work queue elements, and the length of each of the plurality of work queue elements are all adjustable.
The data processing method for remote direct memory access shown in fig. 1 can be applied to any application scenario in which a remote direct memory access technology is adopted. Here, remote direct memory access refers to direct access from the memory of one computer to the memory of another computer, where data is directly copied from a physical line to the memory of an application program through hardware, such as a network adapter, or from the memory of an application program to the physical line, and in this process, without going through an operating system, a central processor, and a system kernel, and without copying data between the application program memory and a data buffer of the operating system, delay of data transmission can be greatly reduced and computational resources can be saved. At the data processing level of remote direct memory access, a data organization form of a Scatter/Gather List (SGL) is adopted, and the Scatter/Gather List is an array including at least one Scatter/Gather Element (SGE). Each scatter gather element is a data segment. The remote direct memory access technology utilizes a dispersion aggregation table and dispersion aggregation elements to perform dispersion and aggregation operations, and can disperse discontinuous cache spaces on a plurality of target hosts and aggregate the plurality of discontinuous cache spaces to continuous cache spaces of the target hosts. Remote direct memory access is performed between two hosts supporting a remote direct memory access communication protocol, bypassing the host kernel and the operating system, and generally adopting a Queue Pair (QP) data format. The sending queue and the receiving queue both adopt data structures of queue pairs, the sending queue is used for sending remote direct memory access messages, and the receiving queue is used for receiving remote direct memory access messages. One queue pair may contain multiple work queue elements (Work Queue Element, WQE). The work queue element is a task that the software issues to the hardware and can be used to instruct the hardware where to read how long the data is to be sent to which destination. Queue pair contexts (Queue Pair Context, QPC), sometimes referred to as queue management contexts, are used to store queue pair related attributes, such as queue pair contexts containing pre-applied physical memory space and order of storage, etc. The hardware can determine the position of the corresponding queue pair, the sequence number of the queue pair, the size of the queue pair and the like by reading the context of the queue pair, and further can read a certain work queue element in the queue pair to execute the corresponding task. Taking a communication process based on a remote direct memory access technology as an example, the initiating direction queue pair adds a work queue element, that is, issues a request to the hardware so that the hardware performs an action, thereby receiving data or sending data. One work queue element may be a scatter gather table containing one or more scatter gather elements, e.g., one work queue element may support 6 scatter gather elements or 32 scatter gather elements. The actual physical memory used to store the queue pairs and work queue elements therein is typically divided into storage spaces in physical pages, each of which is fixed in size, e.g., 4 Kilobytes (KB) in size. If the size of the scatter gather elements included by each work queue element is fixed and the length of each work queue element is also fixed, it can be inferred that the number of scatter gather elements included by each work queue element is the same, a queue pair with a fixed length of work queue elements and a fixed number and length of scatter gather elements may result in a waste of storage space, difficulty in fully utilizing the storage resources of the physical pages, and may present challenges in processing across the physical pages. In view of the application environment of the complex and variable remote direct memory access caused by various communication standards, network protocols, interface standards, user requirements, and the like, it is necessary to provide a queue pair with dynamically adjustable work queue elements and dynamically adjustable number and length of distributed aggregation elements, so as to flexibly adapt to the application environment of the complex and variable remote direct memory access, which is described in further detail below in conjunction with fig. 1.
With continued reference to FIG. 1, at step S110, a first queue pair context associated with a first queue pair is created, and then, at step S120, a first work queue element in the first queue pair is read by reading the first queue pair context. Here, the first queue pair includes a plurality of work queue elements, each of the plurality of work queue elements is a scatter gather table including at least one scatter gather element, the first queue pair context includes first queue page address information and first page change information, the first queue page address information is used for indicating a position of a physical page corresponding to each of the plurality of work queue elements, and the first page change information is used for indicating page crossing information of the scatter gather element included in each of the plurality of work queue elements between the physical pages corresponding to the work queue elements. Thus, by reading the context of the first queue pair, the first queue page address information and the first page changing information can be utilized, so that on the basis of determining, for example, the position of the first queue pair, the sequence number of the first queue pair, the size of the first queue pair, and the like, the position of a physical page corresponding to a work queue element to be read, for example, the first work queue element, and the page crossing information of the work queue element to be read, for example, the scattered aggregation element included in the first work queue element, between the physical pages corresponding to the work queue element, can be determined. This means that a physical page crossing process can be implemented and that the respective page crossing information can be updated by updating the first page changing information, thereby adapting to the various possible lengths and numbers of discrete aggregation elements. Therefore, on the basis of reading the first work queue element in the first queue pair by reading the first queue pair context, the queue pair with the work queue element with the fixed length and the scattered aggregation element with the fixed number and the fixed length can be avoided, so that the first queue pair can have the work queue element capable of being dynamically adjusted and the scattered aggregation element with the dynamically adjustable number and the fixed length. When the first queue pair is created, a length of a scatter gather element included by each of the plurality of work queue elements, a number of scatter gather elements included by each of the plurality of work queue elements, and a length of each of the plurality of work queue elements may be adjusted. Therefore, the application environment of the complicated and changeable remote direct memory access can be combined, for example, various communication standards, network protocols, interface standards and user requirements are considered, when the first queue pair context is created, the information of the related physical pages is recorded through the first queue page address information, and the related page crossing information is recorded through the first page changing information, so that the configuration of the physical page size can be flexibly adapted, the storage space is fully utilized, the resource utilization rate is improved, and the application environment requirement of the remote direct memory access can be better adapted.
In summary, the data processing method for remote direct memory access shown in fig. 1 can avoid using a queue pair with a work queue element of a fixed length and a distributed aggregation element of a fixed number and length on the basis of reading the first work queue element in the first queue pair by reading the first queue pair context, so that the first queue pair can have a dynamically adjustable work queue element, a dynamically adjustable distributed aggregation element of a number and length; therefore, the application environment of the remote direct memory access with complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment of the remote direct memory access can be better adapted to the requirements.
In one possible implementation manner, the scatter aggregate elements included in the first work queue element are distributed on at least two physical pages in the physical pages corresponding to the first work queue element, and page crossing information of the scatter aggregate elements included in the first work queue element between the at least two physical pages is determined by reading the first page changing information of the first queue pair context. Therefore, the physical page crossing processing is realized, the complex and changeable remote direct memory access application environment can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the requirements of the remote direct memory access application environment can be better met.
In one possible implementation, at least two of the scatter gather elements included in the first work queue element are different in length. Therefore, the first queue pair can be provided with the work queue elements which can be dynamically adjusted, the distributed aggregation elements which can be dynamically adjusted in number and length, and the application environment of the remote direct memory access which is complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment requirement of the remote direct memory access can be better adapted.
In one possible implementation, the first queue pair is adapted to the physical page size by adjusting a length of a scatter gather element that is included in each of the plurality of work queue elements, a number of scatter gather elements that are included in each of the plurality of work queue elements, and a length of each of the plurality of work queue elements. Therefore, the configuration of the physical page size can be flexibly adapted, the storage space is fully utilized, the resource utilization rate is improved, and the demand of the application environment of remote direct memory access can be better adapted.
In one possible implementation, when the first queue pair is created, the physical pages allocated to the first queue pair are in the same physical memory, and the page crossing process associated with the first queue pair is implemented by reading the first queue pair context. Therefore, the loss of the cross-physical memory is avoided, the processing efficiency is improved, the cross-physical page processing is realized, the complex and changeable application environment of remote direct memory access can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the requirements of the application environment of remote direct memory access can be better met.
In one possible implementation, the first queue pair belongs to a plurality of queue pairs, the first queue pair context belongs to a plurality of queue pair contexts corresponding to the plurality of queue pairs one to one, and the plurality of queue pair contexts are created based on a first scheduling capability, wherein a maximum value of respective lengths of work queue elements included in each of the plurality of queue pair contexts is determined based on a lower limit of the first scheduling capability. As mentioned above, on the basis of reading the first work queue element in the first queue pair by reading the first queue pair context, a queue pair having a fixed length of work queue elements and a fixed number and length of scatter gather elements may be avoided, so that the first queue pair may have a dynamically adjustable work queue element, a dynamically adjustable number and length of scatter gather elements. Here, considering the application scenario of the complex and changeable remote direct memory access, in order to optimize the delay and the resource occupation situation, the consideration of the first scheduling capability is also added into the design of the data processing method. Specifically, the plurality of queue pair contexts are created based on a first scheduling capability, and a maximum value of a respective length of work queue elements included in each of the plurality of queue pair contexts is determined based on a lower limit of the first scheduling capability. As such, the respective lengths of the work queue elements included in each of the plurality of queue pair contexts are adjustable, i.e., variable, and the range of variation of the work queue elements is limited by the lower limit of the first scheduling capability, i.e., the maximum value of the respective lengths of the work queue elements, i.e., the maximum possible length, is limited by the current first scheduling capability. This means that the work queue elements comprised by each of the plurality of queue pair contexts may be constrained in a global scope by dynamically adjusting the first scheduling capability. Further, the dynamically adjustable first scheduling capability can be combined with the above-mentioned first queue pair context supporting reading across physical pages, that is, the first queue pair with dynamically adjustable work queue elements and dynamically adjustable number and length of scattered aggregation elements is combined, so that the application requirements under various load scenes can be better adapted. For example, the length of the work queue element may be set to be variable (limited by the maximum length) and the size of the scatter aggregate element may be also variable, and the number of scatter aggregate elements per work queue element may be also variable. More global settings may also be provided, for example setting the minimum size of a scatter gather element to 16 bytes, with one work queue element having at least 1 scatter gather element and at most 30 scatter gather elements.
In some embodiments, the data processing method further comprises: a first read operation is performed on the plurality of queue pairs based on the first scheduling capability, wherein a lower limit of an operating range of the first read operation is determined based on the lower limit of the first scheduling capability and is not less than a maximum value of a respective length of work queue elements included in each of the plurality of queue pair contexts. Thus, utilizing the first scheduling capability and setting the lower limit of the operating range of the first read operation is based on the lower limit of the first scheduling capability, which means that executing the first read operation will read at least one work queue element. Therefore, the work queue elements included in each queue pair context in the plurality of queue pair contexts are constrained in the global scope, and the application requirements under various load scenes can be better adapted.
In some embodiments, the operating range of the first read operation may be adjusted upward by a factor of 2 or downward by a factor of 2. Thus, the delay of the cross-physical page processing and the optimization of the delay and the resource occupation can be reduced by upward adjustment and downward adjustment. For example, information contained in the read queue pair context relating to processing across physical pages may be utilized in conjunction with the expected change law of work queue elements to predict the change law of work queue elements currently to be scheduled and transmitted. For example, when the distribution of the work queue elements of the previous reading and the next reading in the aspect of cross-physical page processing has certain regularity, corresponding upward adjustment or downward adjustment can be made according to a statistical method, so that particularly drastic abrupt change does not occur under certain regularity. Wherein the upward adjustment by a multiplication of 2 may be, for example, from an operating range of 16 bytes to an operating range of 32 bytes to a range of 64 bytes or even higher. The downward adjustment by dividing by 2 may be, for example, from an operating range of 16 bytes to an operating range of 8 bytes, and typically the operating range may be set to a minimum of 8 bytes.
In some embodiments, a second read operation is performed on the plurality of queue pairs based on the first scheduling capability based on performing the first read operation on the plurality of queue pairs based on the first scheduling capability, wherein a queue pair context of the plurality of queue pair contexts associated with a queue pair of the plurality of queue pairs that was read by the second read operation is used to adjust the first scheduling capability. Thus, the first scheduling capability may be adjusted by the second read operation. The second read operation may be a pre-read operation. In this way, the first reading operation can be performed according to the dynamically configurable first scheduling capability, for example, a plurality of work queue elements of one batch are read, then the work queue elements of the batch can be scheduled and sent, and in addition, a plurality of work queue elements of the next batch are also read in advance through the second reading operation, so that feedback can be made and the first scheduling capability can be adjusted based on the corresponding queue pair context information and the related information of the inter-physical page processing therein, and the subsequent scheduling and reading can be performed by using the adjusted first scheduling capability, which is beneficial to improving the processing efficiency.
In one possible implementation manner, the data processing method further includes: creating a second queue pair context associated with a second queue pair, wherein the length of the scatter gather elements each included by a work queue element of the second queue pair, the number of scatter gather elements each included by a work queue element of the second queue pair, and the length of each work queue element of the second queue pair are all fixed; and executing remote direct memory access in a dynamic adjustable mode by using the first queue pair and the first queue pair context, or executing remote direct memory access in a fixed-length mode by using the second queue pair and the second queue pair context. As mentioned above, the first queue pair may have dynamically adjustable work queue elements, dynamically adjustable numbers and lengths of decentralized aggregated elements based on reading the first work queue elements in the first queue pair by reading the first queue pair context. Here, considering the application scenario of the complex and changeable remote direct memory access, in order to optimize the delay and resource occupation situation, the second queue pair with the work queue element with a fixed length and the scattered aggregation elements with a fixed number and length and the corresponding second queue pair context may be combined with the first queue pair and the corresponding first queue pair context, so as to better adapt to the requirement of the application scenario. In some application scenarios, the length of the packet is small, the packet data is not different, for example, video data streams are generally sent at fixed intervals in the application scenarios such as closed-circuit television monitoring, traffic monitoring and the like; thus, a second queue pair with a fixed length of work queue elements and a fixed number and length of scatter gather elements and corresponding second queue pair contexts may be adapted to the requirements of such an application scenario. In other application scenes, the length of the packet is changed greatly, and the change of the data flow is difficult to predict, for example, a large number of sessions with different durations can be generated in the application scenes such as short video, voice communication and the like; thus, a first queue pair with dynamically adjustable work queue elements, dynamically adjustable numbers and lengths of decentralized aggregation elements, and corresponding first queue pair contexts, may be adapted to the needs of such application scenarios. In this way, remote direct memory access in a dynamic adjustable mode can be performed by using the first queue pair and the first queue pair context, or remote direct memory access in a fixed-length mode can be performed by using the second queue pair and the second queue pair context, so that the complex and changeable requirements of an application scene can be better adapted.
In some embodiments, the remote direct memory access in the dynamically adjustable mode is performed when the application scenario of the remote direct memory access is short video, voice communication, and the remote direct memory access in the fixed length mode is performed when the application scenario of the remote direct memory access is industrial monitoring, traffic monitoring, logistics automation. Thus, when the application scene of the remote direct memory access is short video and voice communication, the remote direct memory access in the dynamic adjustable mode is suitable to be executed. When the application scene of the remote direct memory access is industrial monitoring, traffic monitoring and logistics automation, the flow front-back change is stable and the flow burst and other conditions occur at a lower probability under the similar application scene, so the remote direct memory access method is suitable for executing the remote direct memory access under the fixed-length mode.
In one possible implementation, the scatter gather element included in each of the plurality of work queue elements is a data segment that includes an address field, a length field, and a key value field. Therefore, the application environment of the remote direct memory access with complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment of the remote direct memory access can be better adapted to the requirements.
In one possible implementation, the first queue pair includes a transmit queue and a receive queue, and the plurality of work queue elements belong to the transmit queue or the receive queue. Therefore, the application environment of the remote direct memory access with complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment of the remote direct memory access can be better adapted to the requirements.
Fig. 2 is a schematic diagram of a first queue pair context provided in an embodiment of the present application. As shown in fig. 2, the first queue pair context 210 includes first queue page address information 212 and first page change information 214. Wherein the first queue page address information 212 indicates the location of physical page A220 and the location of physical page B222, physical page A220 and physical page B222 corresponding to a plurality of work queue elements, work queue element A230, work queue element B232 and work queue element C234, respectively. Work queue element A230 includes a scatter gather element A240, among other things. Work queue element B232 includes a scatter aggregate element B242, a scatter aggregate element C244, and a scatter aggregate element D246. Work queue element C234 includes a scatter gather element E248 and a scatter gather element F250. It can be seen that work queue element A230 and the scatter gather element A240 it includes are distributed across physical page A220. The scatter gather elements B242 and C244 included in the work queue element B232 are distributed on the physical page a 220, while the scatter gather elements D246 included in the work queue element B232 are distributed on the physical page B222. Work queue element C234 and the scatter gather elements E248 and F250 included therein are distributed across physical page B222. As schematically illustrated in FIG. 2, work queue element A230, work queue element B232, and work queue element C234 each include a non-uniform number of scatter gather elements, and the plurality of scatter gather elements included in work queue element B232 (i.e., scatter gather element B242, scatter gather element C244, and scatter gather element D246) are distributed over two different physical pages, namely physical page A220 and physical page B222. Here, the first page-changing information 214 indicates page-crossing information of the scatter aggregate element included in each of the plurality of work queue elements between the physical pages corresponding to the work queue element, and thus, the first page-changing information 214 indicates page-crossing information of the plurality of scatter aggregate elements included in the work queue element B232 between the physical page a 220 and the physical page B222. Thus, with the first queue pair context 210 shown in fig. 2, referring to the data processing method for remote direct memory access shown in fig. 1, the application environment of the remote direct memory access with complex and variable characteristics can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space can be fully utilized, the resource utilization rate can be improved, and the requirement of the application environment of the remote direct memory access can be better adapted.
In some embodiments, the lengths of the scatter gather elements included in each of work queue element A230, work queue element B232, and work queue element C234 may also be non-uniform, e.g., the length of scatter gather element A240 included in work queue element A230 may be different from the length of scatter gather element E248 included in work queue element C234. In some embodiments, the lengths of work queue element A230, work queue element B232, and work queue element C234 may also be inconsistent. Therefore, the application environment of the remote direct memory access with complex and changeable can be combined, for example, the configuration of the physical page size can be flexibly adapted in consideration of various communication standards, network protocols, interface standards and user requirements, the storage space is fully utilized, the resource utilization rate is improved, and the application environment of the remote direct memory access can be better adapted to the requirements.
Fig. 3 is a schematic structural diagram of a computing device provided in an embodiment of the present application, where the computing device 300 includes: one or more processors 310, a communication interface 320, and a memory 330. The processor 310, the communication interface 320 and the memory 330 are interconnected by a bus 340. Optionally, the computing device 300 may further include an input/output interface 350, where the input/output interface 350 is connected to an input/output device for receiving parameters set by a user, etc. The computing device 300 can be used to implement some or all of the functionality of the device embodiments or system embodiments described above in the embodiments of the present application; the processor 310 can also be used to implement some or all of the operational steps of the method embodiments described above in the embodiments of the present application. For example, specific implementations of the computing device 300 performing various operations may refer to specific details in the above-described embodiments, such as the processor 310 being configured to perform some or all of the steps of the above-described method embodiments or some or all of the operations of the above-described method embodiments. For another example, in the embodiment of the present application, the computing device 300 may be used to implement some or all of the functions of one or more components in the apparatus embodiments described above, and the communication interface 320 may be used in particular for communication functions and the like necessary for implementing the functions of these apparatuses, components, and the processor 310 may be used in particular for processing functions and the like necessary for implementing the functions of these apparatuses, components.
It should be appreciated that the computing device 300 of fig. 3 may include one or more processors 310, and that the plurality of processors 310 may cooperatively provide processing power in a parallelized connection, a serialized connection, a serial-parallel connection, or any connection, or the plurality of processors 310 may constitute a processor sequence or processor array, or the plurality of processors 310 may be separated into primary and secondary processors, or the plurality of processors 310 may have different architectures such as employing heterogeneous computing architectures. In addition, the computing device 300 shown in FIG. 3, the associated structural and functional descriptions are exemplary and not limiting. In some example embodiments, computing device 300 may include more or fewer components than shown in fig. 3, or combine certain components, or split certain components, or have a different arrangement of components.
Processor 310 may take many specific forms, for example, processor 310 may include one or more combinations of a central processing unit (central processing unit, CPU), a graphics processor (graphic processing unit, GPU), a neural network processor (neural-network processing unit, NPU), a tensor processor (tensor processing unit, TPU), or a data processor (data processing unit, DPU), and embodiments of the present application are not limited in detail. Processor 310 may also be a single-core processor or a multi-core processor. The processor 310 may be formed by a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (programmable logic device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), general-purpose array logic (generic array logic, GAL), or any combination thereof. The processor 310 may also be implemented solely with logic devices incorporating processing logic, such as an FPGA or digital signal processor (digital signal processor, DSP) or the like. The communication interface 320 may be a wired interface, which may be an ethernet interface, a local area network (local interconnect network, LIN), etc., or a wireless interface, which may be a cellular network interface, or use a wireless local area network interface, etc., for communicating with other modules or devices.
The memory 330 may be a nonvolatile memory such as a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an electrically Erasable EPROM (EEPROM), or a flash memory. Memory 330 may also be volatile memory, which may be random access memory (random access memory, RAM) used as external cache. By way of example, and not limitation, many forms of RAM are available, such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous DRAM (SLDRAM), and direct memory bus RAM (DR RAM). Memory 330 may also be used to store program code and data such that processor 310 invokes the program code stored in memory 330 to perform some or all of the operational steps of the method embodiments described above, or to perform corresponding functions in the apparatus embodiments described above. Moreover, computing device 300 may contain more or fewer components than shown in FIG. 3, or may have a different configuration of components.
Bus 340 may be a peripheral component interconnect express (peripheral component interconnect express, PCIe) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, a unified bus (Ubus or UB), a computer quick link (compute express link, CXL), a cache coherent interconnect protocol (cache coherent interconnect for accelerators, CCIX), or the like. The bus 340 may be divided into an address bus, a data bus, a control bus, and the like. The bus 340 may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus. But is shown with only one bold line in fig. 3 for clarity of illustration, but does not represent only one bus or one type of bus.
The method and the device provided in the embodiments of the present application are based on the same inventive concept, and because the principles of solving the problems by the method and the device are similar, the embodiments, implementations, examples or implementation of the method and the device may refer to each other, and the repetition is not repeated. Embodiments of the present application also provide a system that includes a plurality of computing devices, each of which may be structured as described above. The functions or operations that may be implemented by the system may refer to specific implementation steps in the above method embodiments and/or specific functions described in the above apparatus embodiments, which are not described herein.
Embodiments of the present application also provide a computer-readable storage medium having stored therein computer instructions which, when executed on a computer device (e.g., one or more processors), may implement the method steps in the above-described method embodiments. The specific implementation of the processor of the computer readable storage medium in executing the above method steps may refer to specific operations described in the above method embodiments and/or specific functions described in the above apparatus embodiments, which are not described herein again.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. The present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, the above-described embodiments may be implemented in whole or in part in the form of a computer program product. The present application may take the form of a computer program product embodied on one or more computer-usable storage media having computer-usable program code embodied therein. The computer program product includes one or more computer instructions. When loaded or executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). Computer readable storage media can be any available media that can be accessed by a computer or data storage devices, such as servers, data centers, etc. that contain one or more collections of available media. Usable media may be magnetic media (e.g., floppy disks, hard disks, tape), optical media, or semiconductor media. The semiconductor medium may be a solid state disk, or may be a random access memory, flash memory, read only memory, erasable programmable read only memory, electrically erasable programmable read only memory, register, or any other form of suitable storage medium.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. Each flow and/or block of the flowchart and/or block diagrams, and combinations of flows and/or blocks in the flowchart and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments. It will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. The steps in the method of the embodiment of the application can be sequentially adjusted, combined or deleted according to actual needs; the modules in the system of the embodiment of the application can be divided, combined or deleted according to actual needs. Such modifications and variations of the embodiments of the present application are intended to be included herein, if they fall within the scope of the claims and their equivalents.

Claims (14)

1. A data processing method for remote direct memory access, the data processing method comprising:
creating a first queue pair context associated with a first queue pair, wherein the first queue pair comprises a plurality of work queue elements, each of the plurality of work queue elements is a decentralized aggregation table comprising at least one decentralized aggregation element, the first queue pair context comprises first queue page address information and first page change information, the first queue page address information is used for indicating the position of a physical page corresponding to each of the plurality of work queue elements, and the first page change information is used for indicating page crossing information of the decentralized aggregation element respectively included by each of the plurality of work queue elements among the physical pages corresponding to the work queue elements;
Reading a first work queue element in the first queue pair by reading the first queue pair context, wherein the first work queue element is any work queue element in the plurality of work queue elements,
wherein when the first queue pair is created, a length of a scatter gather element included by each of the plurality of work queue elements, a number of scatter gather elements included by each of the plurality of work queue elements, and a length of each of the plurality of work queue elements are adjustable,
the scattered aggregation elements included in the first work queue element are distributed on at least two physical pages in the physical pages corresponding to the first work queue element, and page crossing information of the scattered aggregation elements included in the first work queue element between the at least two physical pages is determined by reading the first page changing information of the first queue pair context.
2. The data processing method according to claim 1, wherein at least two of the scatter gather elements included in the first work queue element are different in length.
3. The data processing method of claim 1, wherein the first queue pair is adapted to a physical page size by adjusting a length of a scatter gather element included in each of the plurality of work queue elements, a number of scatter gather elements included in each of the plurality of work queue elements, and a length of each of the plurality of work queue elements.
4. The data processing method according to claim 1, wherein when the first queue pair is created, physical pages allocated to the first queue pair are in the same physical memory, and wherein the page crossing process associated with the first queue pair is realized by reading the first queue pair context.
5. The data processing method of claim 1, wherein the first queue pair belongs to a plurality of queue pairs, the first queue pair context belongs to a plurality of queue pair contexts that are in one-to-one correspondence with the plurality of queue pairs, the plurality of queue pair contexts are created based on a first scheduling capability, wherein a maximum value of a respective length of work queue elements included in each of the plurality of queue pair contexts is determined based on a lower limit of the first scheduling capability.
6. The data processing method according to claim 5, characterized in that the data processing method further comprises: a first read operation is performed on the plurality of queue pairs based on the first scheduling capability, wherein a lower limit of an operating range of the first read operation is determined based on the lower limit of the first scheduling capability and is not less than a maximum value of a respective length of work queue elements included in each of the plurality of queue pair contexts.
7. The data processing method of claim 6, wherein the operating range of the first read operation is adjusted upward by multiplying by 2 or downward by dividing by 2.
8. The data processing method of claim 6, wherein a second read operation is performed on the plurality of queue pairs based on the first scheduling capability based on performing the first read operation on the plurality of queue pairs based on the first scheduling capability, wherein a queue pair context of the plurality of queue pair contexts associated with a queue pair of the plurality of queue pairs that was read by the second read operation is used to adjust the first scheduling capability.
9. The data processing method according to claim 1, characterized in that the data processing method further comprises:
creating a second queue pair context associated with a second queue pair, wherein the length of the scatter gather elements each included by a work queue element of the second queue pair, the number of scatter gather elements each included by a work queue element of the second queue pair, and the length of each work queue element of the second queue pair are all fixed;
and executing remote direct memory access in a dynamic adjustable mode by using the first queue pair and the first queue pair context, or executing remote direct memory access in a fixed-length mode by using the second queue pair and the second queue pair context.
10. The data processing method of claim 9, wherein the remote direct memory access in the dynamically adjustable mode is performed when the application scenario of the remote direct memory access is short video, voice communication, and the remote direct memory access in the fixed length mode is performed when the application scenario of the remote direct memory access is industrial monitoring, traffic monitoring, logistics automation.
11. The data processing method of claim 1, wherein the scatter gather element included in each of the plurality of work queue elements is a data segment comprising an address field, a length field, and a key value field.
12. The data processing method of claim 1, wherein the first queue pair comprises a transmit queue and a receive queue, the plurality of work queue elements belonging to either the transmit queue or the receive queue.
13. A computer device, characterized in that it comprises a memory, a processor and a computer program stored on the memory and executable on the processor, which processor implements the method according to any of claims 1 to 12 when executing the computer program.
14. A computer readable storage medium storing computer instructions which, when run on a computer device, cause the computer device to perform the method of any one of claims 1 to 12.
CN202410063040.4A 2024-01-17 2024-01-17 Data processing method and computer equipment for remote direct memory access Active CN117573603B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410063040.4A CN117573603B (en) 2024-01-17 2024-01-17 Data processing method and computer equipment for remote direct memory access

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410063040.4A CN117573603B (en) 2024-01-17 2024-01-17 Data processing method and computer equipment for remote direct memory access

Publications (2)

Publication Number Publication Date
CN117573603A true CN117573603A (en) 2024-02-20
CN117573603B CN117573603B (en) 2024-04-19

Family

ID=89892274

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410063040.4A Active CN117573603B (en) 2024-01-17 2024-01-17 Data processing method and computer equipment for remote direct memory access

Country Status (1)

Country Link
CN (1) CN117573603B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060230119A1 (en) * 2005-04-08 2006-10-12 Neteffect, Inc. Apparatus and method for packet transmission over a high speed network supporting remote direct memory access operations
CN115858160A (en) * 2022-12-07 2023-03-28 江苏为是科技有限公司 Remote direct memory access virtualization resource allocation method and device and storage medium
CN115934625A (en) * 2023-03-15 2023-04-07 珠海星云智联科技有限公司 Doorbell knocking method, device and medium for remote direct memory access
CN115964319A (en) * 2023-01-30 2023-04-14 珠海星云智联科技有限公司 Data processing method for remote direct memory access and related product
US20230231914A1 (en) * 2020-09-04 2023-07-20 Huawei Technologies Co., Ltd. Devices and methods for remote direct memory access

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060230119A1 (en) * 2005-04-08 2006-10-12 Neteffect, Inc. Apparatus and method for packet transmission over a high speed network supporting remote direct memory access operations
US20230231914A1 (en) * 2020-09-04 2023-07-20 Huawei Technologies Co., Ltd. Devices and methods for remote direct memory access
CN115858160A (en) * 2022-12-07 2023-03-28 江苏为是科技有限公司 Remote direct memory access virtualization resource allocation method and device and storage medium
CN115964319A (en) * 2023-01-30 2023-04-14 珠海星云智联科技有限公司 Data processing method for remote direct memory access and related product
CN115934625A (en) * 2023-03-15 2023-04-07 珠海星云智联科技有限公司 Doorbell knocking method, device and medium for remote direct memory access

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
肖侬;赵英杰;刘芳;陈志广;: "基于顺序检测的双队列缓存替换算法", 中国科学:信息科学, no. 04, 15 April 2011 (2011-04-15), pages 47 - 57 *

Also Published As

Publication number Publication date
CN117573603B (en) 2024-04-19

Similar Documents

Publication Publication Date Title
JP6224244B2 (en) Power balancing to increase working density and improve energy efficiency
CN110602156A (en) Load balancing scheduling method and device
WO2020019743A1 (en) Traffic control method and device
CN110391873B (en) Method, apparatus and computer program product for determining a data transfer mode
CN112445857A (en) Resource quota management method and device based on database
CN115858184B (en) RDMA memory management method, device, equipment and medium
CN109194721A (en) A kind of asynchronous RDMA communication dynamic memory management method and system
CN112600761A (en) Resource allocation method, device and storage medium
CN116049085A (en) Data processing system and method
CN113268329B (en) Request scheduling method, device and storage medium
CN113992600A (en) Multi-rate mode data transmission control method and device based on cache queue
CN117573603B (en) Data processing method and computer equipment for remote direct memory access
CN112770358B (en) Multi-rate mode data transmission control method and device based on service data
CN113590666B (en) Data caching method, system, equipment and computer medium in AI cluster
CN110928693B (en) Computing equipment and resource allocation method
CN115878309A (en) Resource allocation method, device, processing core, equipment and computer readable medium
CN116340246B (en) Data pre-reading method and medium for direct memory access read operation
CN114531389A (en) Routing table optimization method, controller and router
CN111611068A (en) Data writing method in distributed system, server and client
CN117573602A (en) Method and computer device for remote direct memory access message transmission
JP6646340B2 (en) Dispersing apparatus and dispersing method
WO2023093200A1 (en) Method and apparatus for asynchronously processing tasks, and storage medium and electronic apparatus
CN114979232B (en) Construction method of mobile web augmented reality decentralizing collaborative rendering computing network
WO2023105671A1 (en) Computer and program
CN112965796B (en) Task scheduling system, method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant