WO2017124917A1 - 数据处理方法和装置 - Google Patents

数据处理方法和装置 Download PDF

Info

Publication number
WO2017124917A1
WO2017124917A1 PCT/CN2017/000065 CN2017000065W WO2017124917A1 WO 2017124917 A1 WO2017124917 A1 WO 2017124917A1 CN 2017000065 W CN2017000065 W CN 2017000065W WO 2017124917 A1 WO2017124917 A1 WO 2017124917A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
queue
page address
current
processed
Prior art date
Application number
PCT/CN2017/000065
Other languages
English (en)
French (fr)
Inventor
刘卯银
秦长鹏
戴庆军
牛克强
张翼
舒坦
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2017124917A1 publication Critical patent/WO2017124917A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/0802—Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
    • G06F12/0806—Multiuser, multiprocessor or multiprocessing cache systems
    • G06F12/0815—Cache consistency protocols
    • G06F12/0831—Cache consistency protocols using a bus scheme, e.g. with bus monitoring or watching means
    • G06F12/0835—Cache consistency protocols using a bus scheme, e.g. with bus monitoring or watching means for main memory peripheral accesses (e.g. I/O or DMA)
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/0024—Peripheral component interconnect [PCI]

Definitions

  • Embodiments of the present invention relate to the field of communications, and in particular, to a data processing method and apparatus.
  • PCIe PCI-Express
  • NTB Non-transparent bridge
  • the PCIe protocol can support high-speed interconnection between nodes, and the interconnection nodes exchange the address domain through NTB.
  • the DMA engine on the node can access the memory on the peer node by accessing the virtual address mapped by the NTB.
  • DMA Direct Memory Acess
  • PCIe's Remote Direct Data Access (RDMA) operation is performed only on the transmitting node
  • RDMA Remote Direct Data Access
  • the peer when PCIe interacts with the peer, for example, accessing the peer through the DMA engine, or When the peer sends data, it needs to obtain the physical address on the peer node exactly, so that the RDMA data is RDMA to the memory on the peer node during the RDMA operation of the source. That is to say, when the source end obtains the physical address of the RDMA operation on the peer node from the peer end, it also needs to perform multiple interactions between the nodes to apply for the memory on the peer node.
  • RDMA Remote Direct Data Access
  • the DEMO software provided by the PCIe switch manufacturer generally uses PCIe to simulate an Ethernet interface, but the analog Ethernet interface often requires multiple copies of data during data transmission, and cannot fully utilize the high bandwidth provided by the PCIe protocol. .
  • using the related technology only through multiple interactions When the RDMA operation is performed in the manner of obtaining the physical address of the peer, the problem of low data processing efficiency is caused.
  • the embodiment of the invention provides a data processing method and device, which can solve at least the data processing efficiency caused by performing RDMA operations in a manner that RDMA can only perform multiple times of interaction through the underlying protocol stack and multiple times copying data in the related art. The problem.
  • a data processing method includes: acquiring an amount of data of a current data to be processed, where the current to-be-processed data includes an input/output I/ currently performing a remote direct data access RDMA operation. If the data volume of the current data to be processed is greater than the first predetermined threshold, the RDMA operation of the current data to be processed is directly performed on the page address corresponding to the current data to be processed; if the current data to be processed is If the amount of data is less than or equal to the first predetermined threshold, the RDMA operation of the current data to be processed is performed on the page address in the receiving device through a pre-established data transmission channel.
  • the method further includes: directly acquiring the page address from the local page address buffer pool, where The page address cache pool is used to cache one or more page addresses.
  • the method includes: detecting a number of page addresses cached in the page address buffer pool; and if the number of the page addresses is less than or equal to a second predetermined threshold, The data transfer channel gets a new page address.
  • the obtaining, by the foregoing data transmission channel, the new page address comprises: sending an acquisition request for requesting to obtain the page address as the data to be transmitted to the receiving device by using the data transmission channel; and acquiring the foregoing sending by the receiving device Page address; add the above page address to the above page address cache pool.
  • the foregoing performing, by using the pre-established data transmission channel, the foregoing RDMA operation of the current to-be-processed data on the foregoing page address in the receiving device includes: Processing the data as the data to be transmitted is directly sent to the receiving device through the data transmission channel, and stored in the data transmission channel in the channel buffer of the receiving device, so that the receiving device uses the current waiting in the channel buffer The processing data performs the above RDMA operation on the above page address.
  • the performing, by the receiving device, performing the RDMA operation on the page address by using the current to-be-processed data in the channel buffer comprises: the receiving device copying the current to-be-processed data in the channel buffer to the page The address corresponds to the page memory.
  • the method before acquiring the data volume of the current data to be processed, the method further includes: establishing the foregoing data transmission channel between the sending device and the receiving device, where the data transmission channel is connected to the control queue in the sending device, and the receiving A circular queue in the device, wherein the control queue includes control information for controlling a change in data state in the circular queue.
  • the establishing the data transmission channel between the sending device and the receiving device includes: setting, in the control queue and the ring queue, a pointer for indicating a state change of a data in the queue, where the control queue is The position indicated by the pointer in the circular queue changes synchronously; wherein the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate the foregoing The receiving device has not confirmed the received data; the data between the queue receiving pointer and the queue tail pointer is used to indicate that the receiving device has confirmed receipt and has not processed the data.
  • the synchronization change between the control queue and the location indicated by the pointer in the ring queue includes: when the sending device performs a sending operation, the queue head pointer of the control queue moves forward by N data bytes. And the position indicated by the queue head pointer is synchronized to the circular queue of the receiving device, wherein the N is a natural number greater than or equal to 1; when the receiving device performs a receiving operation, the queue tail pointer of the circular queue Moving M data bytes forward, and synchronizing the position indicated by the queue tail pointer to the above control queue of the above sending device, where M ⁇ N, M is a natural number greater than or equal to 1; wherein, in the above sending When the queue tail pointer in the above control queue of the device moves to the queue head pointer, the position of the queue receiving pointer is updated.
  • a data processing method including: when the data amount of the current to-be-processed data acquired by the transmitting device is greater than a first predetermined threshold, receiving the sending device directly and currently waiting Performing a remote direct data access RDMA operation on the current data to be processed on the page address corresponding to the data, wherein the current to-be-processed data includes input/output I/O data currently performing the RDMA operation; and the current pending data
  • the RDMA operation of performing the current to-be-processed data on the page address in the receiving device through the pre-established data transmission channel is received.
  • the method before receiving the RDMA operation that is performed by the sending device directly on the page address corresponding to the current data to be processed, the method further includes: receiving, by the sending device, requesting to acquire the page address.
  • the acquisition request; the above page address is sent in response to the above acquisition request.
  • the foregoing performing, by using the pre-established data transmission channel, the foregoing RDMA operation of the current to-be-processed data on the page address corresponding to the current data to be processed includes: receiving the current to-be-processed data; and saving the to-be-processed data in the The data transmission channel is in the channel buffer of the receiving device; and the current pending data in the channel buffer is copied to the page corresponding to the page address.
  • the method further includes: pre-establishing the foregoing data transmission channel between the sending device and the receiving device, wherein the data transmission channel is connected to the control queue in the sending device and the ring queue in the receiving device, wherein the foregoing control The queue includes control information for controlling changes in data state in the circular queue described above.
  • the establishing the data transmission channel between the sending device and the receiving device includes: setting, in the control queue and the ring queue, a pointer for indicating a state change of a data in the queue, where the control queue is The position indicated by the pointer in the circular queue changes synchronously; wherein the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate the foregoing The receiving device has not confirmed the received data; the above queue receiving pointer and the above queue tail pointer The data in between is used to indicate that the above-mentioned receiving device has confirmed receipt and has not processed the data.
  • the synchronization change between the control queue and the location indicated by the pointer in the ring queue includes: when the sending device performs a sending operation, the queue head pointer of the control queue moves forward by N data bytes. And the position indicated by the queue head pointer is synchronized to the circular queue of the receiving device, wherein the N is a natural number greater than or equal to 1; when the receiving device performs a receiving operation, the queue tail pointer of the circular queue Moving M data bytes forward, and synchronizing the position indicated by the queue tail pointer to the above control queue of the above sending device, where M ⁇ N, M is a natural number greater than or equal to 1; wherein, in the above sending When the queue tail pointer in the above control queue of the device moves to the queue head pointer, the position of the queue receiving pointer is updated.
  • a data processing apparatus including: a first acquiring unit, configured to acquire a data amount of current data to be processed, where the current to-be-processed data includes currently executing remote direct data storage Taking the input/output I/O data of the RDMA operation; the first processing unit is configured to perform the current current on the page address corresponding to the current data to be processed directly when the data amount of the current to-be-processed data is greater than the first predetermined threshold The RDMA operation of the data to be processed; the second processing unit is configured to execute on the page address in the receiving device by using the pre-established data transmission channel when the data amount of the current data to be processed is less than or equal to the first predetermined threshold The above RDMA operation of the current data to be processed.
  • the foregoing apparatus further includes: a second obtaining unit, configured to: from the local page address buffer pool, before performing the foregoing RDMA operation of the current to-be-processed data directly on the page address corresponding to the current to-be-processed data The above page address is directly obtained, wherein the page address buffer pool is used to cache one or more page addresses.
  • a second obtaining unit configured to: from the local page address buffer pool, before performing the foregoing RDMA operation of the current to-be-processed data directly on the page address corresponding to the current to-be-processed data The above page address is directly obtained, wherein the page address buffer pool is used to cache one or more page addresses.
  • the device further includes: a detecting unit, configured to detect the number of page addresses cached in the page address buffer pool before directly acquiring the page address from the local page address buffer pool; and the third obtaining unit is configured to When the number of page addresses is less than or equal to a second predetermined threshold, a new page address is obtained through the data transmission channel.
  • a detecting unit configured to detect the number of page addresses cached in the page address buffer pool before directly acquiring the page address from the local page address buffer pool
  • the third obtaining unit is configured to When the number of page addresses is less than or equal to a second predetermined threshold, a new page address is obtained through the data transmission channel.
  • the foregoing third obtaining unit includes: a sending module, configured to send, as the to-be-transmitted data, the acquiring request for acquiring the page address to the receiving device by using the data transmission channel; and the acquiring module is configured to obtain the receiving The above page address sent by the device; adding a module, and setting to add the above page address to the above page address buffer pool.
  • the foregoing second processing unit includes: a processing module, configured to send the current to-be-processed data as the data to be transmitted to the receiving device directly through the data transmission channel, and save the data transmission channel in the receiving device
  • the channel buffer is configured to enable the receiving device to perform the RDMA operation on the page address by using the current to-be-processed data in the channel buffer.
  • the method further includes: an establishing unit, configured to establish the foregoing data transmission channel between the sending device and the receiving device before acquiring the data amount of the current data to be processed, where the data transmission channel is connected to the transmitting device Controlling the queue and the circular queue in the receiving device, wherein the control queue includes control information for controlling a change in data state in the circular queue.
  • an establishing unit configured to establish the foregoing data transmission channel between the sending device and the receiving device before acquiring the data amount of the current data to be processed, where the data transmission channel is connected to the transmitting device Controlling the queue and the circular queue in the receiving device, wherein the control queue includes control information for controlling a change in data state in the circular queue.
  • the establishing unit includes: a setting module, configured to respectively set, in the control queue and the ring queue, a pointer for indicating a state change of a data in the queue, wherein the control queue and the pointer in the circular queue are The indicated position synchronization changes; wherein the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein the data between the queue tail pointer and the queue head pointer is used to indicate that the receiving device has not confirmed the received data. The data between the queue receiving pointer and the queue tail pointer is used to indicate that the receiving device has confirmed receipt and has not processed the data.
  • a data processing apparatus including: a first processing unit, configured to receive the sending when a data amount of current to-be-processed data acquired by a transmitting device is greater than a first predetermined threshold The device directly performs the remote direct data access RDMA operation of the current to-be-processed data on the page address corresponding to the current data to be processed, wherein the current to-be-processed data includes input/output I/O data currently performing the RDMA operation; a second processing unit, configured to receive, when the data amount of the current to-be-processed data is less than or equal to the first predetermined threshold, receiving the foregoing current pending data on the page address in the receiving device by using a pre-established data transmission channel RDMA operation.
  • the foregoing apparatus further includes: a receiving unit, configured to receive, before receiving the RDMA operation that the sending device directly performs the current to-be-processed data on a page address corresponding to the current to-be-processed data, receive the sending by the sending device And an sending unit, configured to send the foregoing page address in response to the obtaining request.
  • a receiving unit configured to receive, before receiving the RDMA operation that the sending device directly performs the current to-be-processed data on a page address corresponding to the current to-be-processed data, receive the sending by the sending device
  • an sending unit configured to send the foregoing page address in response to the obtaining request.
  • the foregoing second processing unit includes: a receiving module, configured to receive the current to-be-processed data; and a saving module, configured to save the to-be-processed data in the data transmission channel in a channel buffer of the receiving device; copy And a module, configured to copy the foregoing current to-be-processed data in the channel buffer to a page corresponding to the page address.
  • a computer storage medium is further provided, and the computer storage medium may store an execution instruction for executing the data processing method in the foregoing embodiment.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of executing the current data to be processed directly on the page address is adopted, and It is not necessary to obtain the corresponding page address through interaction each time, thereby reducing the purpose of data interaction; the data amount of the currently pending data is relatively small In the hour, the RDMA operation on the current data to be processed is directly completed by the memory copy in the receiving device through the data transmission channel, thereby realizing the hardware acceleration effect on the data processing process.
  • the data processing efficiency caused by performing RDMA operations in a manner that can only obtain the physical address of the opposite end through multiple interactions in the related art is low, thereby improving data processing. The effect of efficiency.
  • FIG. 1 is a flow chart of an alternative data processing method in accordance with an embodiment of the present invention.
  • FIG. 2 is a schematic diagram of an alternative data processing method according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of another alternative data processing method according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of an application of an optional data processing method according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of an application of another optional data processing method according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of an application of another optional data processing method according to an embodiment of the present invention.
  • FIG. 7 is a flow chart of another alternative data processing method in accordance with an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of an alternative data processing apparatus in accordance with an embodiment of the present invention.
  • FIG. 9 is a schematic diagram of another alternative data processing apparatus in accordance with an embodiment of the present invention.
  • FIG. 1 is a flowchart of a data processing method according to an embodiment of the present invention. As shown in FIG. 1, the process includes the following steps:
  • Step S102 Acquire a data amount of current data to be processed, where the current data to be processed includes input and output I/O data currently performing a remote direct data access RDMA operation;
  • Step S104 if the data amount of the current to-be-processed data is greater than the first predetermined threshold, the RDMA operation of the current to-be-processed data is directly performed on the page address corresponding to the current to-be-processed data;
  • Step S106 If the data amount of the current data to be processed is less than or equal to the first predetermined threshold, the RDMA operation of the current data to be processed is performed on the page address in the receiving device through the pre-established data transmission channel.
  • the foregoing data processing method may be, but is not limited to, being applied to a node communication process of a PCIe (PCI-Express) link, and after the sending device acquires the data amount of the current data to be processed,
  • the data volume is selected according to different data processing modes: when the data volume is less than or equal to the first predetermined threshold, the data to be processed may be sent to the receiving device through the pre-established data transmission channel, so that the receiving device is Performing a memory copy to complete a remote direct data access RDMA operation on the current pending data on the channel cache corresponding to the page address; when the data amount is greater than the first predetermined threshold, the data may be acquired at the destination node After the upper page address, the RDMA operation of the current data to be processed is directly executed on the page address.
  • PCIe PCI-Express
  • the current to-be-processed data includes input/output I/O data that currently performs an RDMA operation.
  • the interaction duration of sending a request once and receiving the request response is fixed, if the amount of data transmitted at one time is small, the utilization rate of the data transmission channel is obviously lowered. Therefore, in the embodiment, when the amount of data (that is, the overhead) is small, the current data to be processed can be directly transmitted to the receiving device through the data transmission channel, so that the receiving device completes the data processing through one memory copy, thereby realizing Hardware acceleration of the data processing process without pre-fetching the page address for performing RDMA operations, and avoiding multiple copying of data. That is to say, when the amount of data is small, the processing duration of the data will be determined according to the copying duration of the data. Further, when the amount of data (i.e., overhead) is large, the time required to perform copying is long, and thus, the RDMA operation of executing the current data to be processed directly on the page address is employed.
  • the amount of data i.e., overhead
  • the page address is obtained, where the manner of obtaining the page address may include, but is not limited to, at least one of the following: Sending a request for obtaining a page address to the receiving device through the data transmission channel, and directly obtaining the cached page address from the local page address buffer pool.
  • the page address in the page address buffer pool is obtained by: detecting a number of page addresses cached in the page address buffer pool; and if the number of page addresses is less than or equal to a second predetermined threshold, The data transfer channel gets a new page address.
  • the page address buffer pool will add and update the page address in the pool in time according to the current cache amount. Therefore, when the amount of data is large, the problem of processing delay caused by requesting the receiving device to obtain the page address every time is avoided. Further, when the new page address is added to the page address buffer pool, the normal RDMA operation on the current data to be processed is not affected, that is, the two can be simultaneously performed asynchronously, thereby further improving the efficiency of data processing. .
  • the foregoing data transmission channel may be, but is not limited to, a cross-node communication channel established between nodes (such as between the sending device and the receiving device) based on a message directly accessing the peer memory.
  • the foregoing data transmission channel established by the foregoing cross-node may be, but not limited to, being established based on data queues on both sides.
  • the receiving device sets an annular queue
  • the transmitting device sets a control queue for controlling the circular queue, wherein the control queue includes control information for controlling the state change of the data in the circular queue.
  • the synchronous update sends and receives the location indicated by the data pointer in the data queues on both sides to achieve accurate control of the transmission status of the current data to be processed.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of directly executing the current data to be processed on the page address is adopted. Without having to obtain the corresponding page address by interaction each time, the purpose of reducing the data interaction is achieved; when the data volume of the current data to be processed is small, the current pending processing is completed through the memory copy directly in the receiving device through the data transmission channel.
  • the RDMA operation of the data thereby achieving the effect of hardware acceleration on the data processing process.
  • the method before performing the RDMA operation of the current data to be processed directly on the page address corresponding to the current data to be processed, the method further includes:
  • S1 Obtain a page address directly from a local page address cache pool, where the page address cache pool is used to cache one or more page addresses.
  • the foregoing page address buffer pool may be, but is not limited to, setting a plurality of different page address buffer pools according to different services (also referred to as applications). As shown in FIG. 2, it can be divided into a page address buffer pool 202-1 to a page address buffer pool 202-N.
  • the page address in the page address buffer pool may be, but is not limited to, added to the local page address buffer pool after being acquired by the peer (ie, the receiving device).
  • the manner in which the page address buffer pool obtains the page address may include, but is not limited to, at least one of the following: when the node 1 detects that the buffer amount is lower than the predetermined threshold, the node 1 requests the supplementary page address through the data transmission channel; the node 2 transmits the data through the data.
  • the channel actively adds a new page address to node 1 according to a predetermined period.
  • the sending device takes the node 1 as an example
  • the receiving device takes the node 2 as an example.
  • step S206-S208 when the node 1 sends data to the node 2,
  • the page address buffer pool is directly requested to obtain the page address without sending a request for obtaining the request to the node 2, the time for the node 1 to obtain the page address is reduced, thereby reducing the processing delay.
  • the node 1 may set a plurality of page address buffer pools, such as a page address buffer pool 202-1 to a page address buffer pool 202-N, for the peer node 2.
  • the acquisition request may be sent to the correspondent node 2 through the data transmission channel, so that the node 2 supplements the node 1 with a new page address.
  • the process of acquiring the page address by the node 1 ie, steps S202-S204
  • the process of acquiring the page address by the page address buffer pool ie, steps S206-S208
  • the sending device can directly obtain the page address from the local and perform the RDMA operation directly on the page address, thereby reducing the processing delay of the data to be processed.
  • the method includes:
  • the foregoing second predetermined threshold may be, but is not limited to, set to different values according to different application scenarios.
  • the node 1 ie, the sending device
  • can set multiple page address buffer pools for the node 2 and can set a second predetermined threshold, that is, whether the total number of page addresses in all the page address buffer pools meets a second predetermined threshold.
  • the second predetermined threshold of different values may be set for each page address buffer pool, that is, whether the number of page addresses in each page address buffer pool meets a corresponding second predetermined threshold value, which is not used in this embodiment. limited.
  • acquiring a new page address through the data transmission channel includes: S22: Send the request for obtaining the page address as the data to be transmitted to the receiving device through the data transmission channel; S24, obtain the page address sent by the receiving device; and S26, add the page address to the page address buffer pool.
  • steps S202-S204 may be performed to obtain a page address for supplementing with the peer node.
  • steps S202-S204 may be performed to obtain a page address for supplementing with the peer node.
  • the page address in the page address buffer pool is replenished in time by detecting the number of page addresses in the page address buffer pool, thereby ensuring that the current pending data performs RDMA operations normally. It is also possible to cache new page addresses in time to further improve the efficiency of data processing.
  • the RDMA operation of executing the current data to be processed on the page address in the receiving device through the pre-established data transmission channel includes:
  • the current to-be-processed data is directly sent to the receiving device as the data to be transmitted through the data transmission channel, and is stored in the channel buffer of the receiving device in the data transmission channel, so that the receiving device utilizes the current data to be processed in the channel buffer.
  • the current page address may not be acquired, and the current to-be-processed data is directly sent to the receiving device as the data to be transmitted through the data transmission channel, and The device is stored in the channel buffer of the receiving device, and the application of the receiving device directly reads the current data to be processed from the channel buffer and copies the data to the corresponding page memory for RDMA operation.
  • the receiving device performs the RDMA operation on the page address by using the current to-be-processed data in the channel buffer, where: the S12, the receiving device copies the current to-be-processed data in the channel buffer to the page address. Corresponding page memory.
  • the channel buffer of the data transmission channel is used to complete the RDMA operation on the current data to be processed directly in the receiving device, thereby achieving the effect of improving data processing efficiency.
  • the method before acquiring the data amount of the current data to be processed, the method further includes:
  • a data transmission channel is established between the sending device and the receiving device, wherein the data transmission channel is connected to the control queue in the sending device and the ring queue in the receiving device, wherein the control queue includes a data queue for controlling the change of the data state in the ring queue. Control information.
  • establishing a data transmission channel between the sending device and the receiving device includes:
  • a pointer for indicating a change of a data state in the queue is respectively set in the control queue and the ring queue, wherein the control queue changes synchronously with the position indicated by the pointer in the ring queue;
  • the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate that the receiving device has not confirmed the received data; the queue receiving pointer and the queue tail pointer The data between the two is used to indicate that the receiving device has confirmed receipt and has not processed the data.
  • the queue head pointer HEAD, the queue tail pointer TAIL, and the queue receiving pointer RECV_TAIL are respectively set in the control queue and the ring queue, and the transmission of the data to be transmitted is realized by the change of the position indicated by the pointer. control.
  • the new message of the sending device is performed by adding to the header, and the receiving device starts receiving from the tail.
  • the transmitting device determines that when HEAD is equal to TAIL, the queue is empty, and when HEAD+1 is equal to RECV_TAIL, the queue is full.
  • a data transmission channel is established based on the sequence by establishing a corresponding sequence on both sides of the sending and receiving, so as to implement flexible transmission control of the data to be processed.
  • control queue changes synchronously with the location indicated by the pointer in the circular queue includes:
  • the queue head pointer of the control queue will move forward by N data bytes, and the position indicated by the queue head pointer will be synchronized to the circular queue of the receiving device, where N is greater than a natural number equal to 1;
  • the queue tail pointer of the circular queue will move forward M data bytes, and synchronize the position indicated by the queue tail pointer to the control queue of the sending device.
  • the position of the queue receiving pointer is updated.
  • the cross-node data transmission channel established between the node 1 and the node 2 shown in FIG. 3 is based on the control of the data queue.
  • the specific interaction process is as follows:
  • the node 2 when the node 2 receives the interrupt signal of the node 1 or polls the pointer HEAD change, the message processing is performed, and the message is handed over to the application module in the node 2 for processing.
  • the node 2 increases the pointer TAIL of the circular queue, for example, the pointer TAIL moves forward 2 data bytes, then synchronizes the position indicated by the pointer TAIL to the node 1;
  • node 1 when the node 1 receives the interrupt signal of the node 2 or polls the pointer TAIL, it indicates that there is data in the queue to be acknowledged, and the node 1 processes the message in the queue to be acknowledged, and notifies the application module in the node 1.
  • node 1 When the message processing is completed, node 1 will update the position of the pointer RECV_TAIL, that is, release the current position of the pointer RECV_TAIL.
  • the position indicated by the pointer in the queues on both sides is synchronized, thereby accurately controlling the transmission state of the transmitted data, and achieving the purpose of accurately completing the data transmission as the data transmission channel.
  • the foregoing data processing method may be applied to the system shown in FIG. 4, where the communication module 402 located in the sending device is respectively connected to the corresponding service module 1 to the service module N.
  • the communication module 404 located at the receiving device is also respectively connected to the corresponding service module 1 to the service module N, and the transmitting device and the receiving device implement the RDMA operation through the network 406.
  • node 1 and node 2 are directly connected through a PCIe link.
  • the address domain isolation of PCIe is performed between nodes through a non-transparent bridge of NTB (Non-Transport).
  • NTB Non-Transport
  • Node 1 and Node 2 access using a pre-defined fixed address access to obtain the status of the cross-node message queue of the peer.
  • Each business module registers a callback function with the communication module.
  • the registered callback function contains:
  • the communication module invokes the page memory application function of the service module to apply for the memory of the service module, and fills the page address buffer pool on the peer node.
  • the communication module calls the page release function of the service module to release the memory allocated to the destination node.
  • the message is sent to the peer end, and after the peer end responds, the communication module will call the message sending result notification function registered by the service module to notify the service module that the message has been delivered.
  • the business module registers a successful callback function. After the communication module completes the information interaction between the nodes, the interaction process of one message is as follows:
  • the service module 1 invokes the interface of the communication module to send a message to the service module 2 on the opposite node;
  • the communication module receives the message of the service module 1, and sends the message to the message memory on the peer node according to the queue information of the peer end saved by the local end;
  • the communication module on node 1 updates the HEAD pointer of the queue on node 2;
  • the node 2 detects the change of the HEAD pointer, copies the message in the queue, and delivers the message to the service module 2 according to the destination module of the message;
  • node 1 and node 7 are connected by a PCIe switch.
  • PCIe switches are typically configured with a redundant, dual-star architecture. Two links per node are connected to two PCIe switches. In the embodiment, a single link is taken as an example for specific description.
  • the address domain isolation of the PCIe is performed between the nodes through the NTB (Non-Transport) non-transparent bridge, and the PCIe switch allocates different address domains for each node.
  • the RDMA operation is performed using the DMA engine on the PCIe switch, which can directly access the memory on other nodes through the PCIe switching network.
  • the nodes use an out-of-band (Ethernet) link to interact with other nodes in the cluster to obtain the address information of the producer consumer queues across nodes on each node, and the status of the nodes.
  • Ethernet out-of-band
  • Each business module registers a callback function with the communication module.
  • the registered callback function contains:
  • the communication module invokes the page memory application function of the service module to apply for the memory of the business module, and fills the page memory buffer on the opposite node, the page pool in FIG.
  • the communication module calls the page release function of the service module to release the memory allocated to the destination node.
  • the message is sent to the peer end, and after the peer end responds, the communication module will call the message sending result notification function registered by the service module to notify the service module that the message has been delivered.
  • the service module registers a successful callback function. After the communication module completes the information interaction between the nodes, the message interaction process with the page is as follows:
  • the service module 1 on the node 1 invokes the interface of the communication module to send a message with page data to the service module 2 on the node 7;
  • the communication module receives the message of the service module 1, and applies the page address to the page address buffer pool of the service module 2 on the locally saved node 7;
  • the communication module on node 1 updates the HEAD pointer of the queue on node 7;
  • the node 7 detects the change of the HEAD pointer, copies the message in the queue, and delivers the message to the service module 2 according to the destination module of the message;
  • node 1 when node 1 detects the change of the TAIL pointer, it processes the sent pending queue on node 1, until the queue is empty, and updates the RECV_TAIL pointer.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases, the former is A better implementation.
  • the technical solution of the present invention in essence or the contribution to the related art can be embodied in the form of a software product stored in a storage medium (such as ROM/RAM, disk, CD-ROM).
  • the instructions include a number of instructions for causing a terminal device (which may be a cell phone, computer, server, or network device, etc.) to perform the methods described in various embodiments of the present invention.
  • FIG. 7 is a flowchart of a data processing method according to an embodiment of the present invention. As shown in FIG. 7, the process includes the following steps:
  • the foregoing data processing method may be, but is not limited to, being applied to a node communication process of a PCIe (PCI-Express) link, and after the sending device acquires the data amount of the current data to be processed,
  • the data volume is selected according to different data processing modes: when the data volume is less than or equal to the first predetermined threshold, the data to be processed may be sent to the receiving device through the pre-established data transmission channel, so that the receiving device is Performing a memory copy to complete a remote direct data access RDMA operation on the current pending data on the channel cache corresponding to the page address; when the data amount is greater than the first predetermined threshold, the data may be acquired at the destination node After the upper page address, the RDMA operation of the current data to be processed is directly executed on the page address.
  • PCIe PCI-Express
  • the current to-be-processed data includes input/output I/O data that currently performs an RDMA operation.
  • the interaction duration of sending a request once and receiving the request response is fixed, if the amount of data transmitted at one time is small, the utilization rate of the data transmission channel will obviously be lowered, and thus, in this embodiment, when the data is When the quantity (that is, the overhead) is small, the current data to be processed can be directly transmitted to the receiving device through the above data transmission channel, so that the receiving device passes once.
  • the memory copy completes the data processing, thereby realizing the hardware acceleration of the data processing process, without pre-acquiring the page address for performing the RDMA operation, and avoiding the multiple copying process of the data. That is to say, when the amount of data is small, the processing duration of the data will be determined according to the copying duration of the data. Further, when the amount of data (i.e., overhead) is large, the time required to perform copying is long, and thus, the RDMA operation of executing the current data to be processed directly on the page address is employed.
  • the foregoing data transmission channel may be, but is not limited to, a cross-node communication channel established between nodes (such as between the sending device and the receiving device) based on a message directly accessing the peer memory.
  • the foregoing data transmission channel established by the foregoing cross-node may be, but not limited to, being established based on data queues on both sides.
  • the receiving device sets an annular queue
  • the transmitting device sets a control queue for controlling the circular queue, wherein the control queue includes control information for controlling the state change of the data in the circular queue. That is to say, the transmission control of the current data to be processed is realized by directly controlling the data queues on both sides of the transmission and reception.
  • the synchronous update sends and receives the location indicated by the data pointer in the data queues on both sides to achieve accurate control of the transmission status of the current data to be processed.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of directly executing the current data to be processed on the page address is adopted. Without having to obtain the corresponding page address by interaction each time, the purpose of reducing the data interaction is achieved; when the data volume of the current data to be processed is small, the current pending processing is completed through the memory copy directly in the receiving device through the data transmission channel.
  • the RDMA operation of the data thereby achieving the effect of hardware acceleration on the data processing process.
  • the method before the receiving, by the sending device, the RDMA operation of the current data to be processed directly on the page address corresponding to the current data to be processed, the method further includes:
  • the page address in response to the acquisition request sent by the sending device, the page address is sent to the sending device, so as to ensure that the current page to be processed performs the RDMA operation normally, and the new page address can be cached to the sending device in time. Achieve the effect of improving data processing efficiency.
  • the RDMA operation of executing the current data to be processed on the page address corresponding to the current data to be processed through the pre-established data transmission channel includes:
  • the data to be processed is saved in the data transmission channel in the channel buffer of the receiving device;
  • the current page address may not be acquired, and the current to-be-processed data is directly sent to the receiving device as the data to be transmitted through the data transmission channel, and The device is stored in the channel buffer of the receiving device, and the application of the receiving device directly reads the current data to be processed from the channel buffer and copies the data to the corresponding page memory for RDMA operation.
  • the channel buffer of the data transmission channel is used to complete the RDMA operation on the current data to be processed directly in the receiving device, thereby achieving the effect of improving data processing efficiency.
  • a data transmission channel is established between the sending device and the receiving device in advance, wherein the data transmission channel is connected to the control queue in the sending device and the ring queue in the receiving device, wherein the control queue includes a data state change for controlling the ring queue. Control information.
  • establishing a data transmission channel between the sending device and the receiving device includes:
  • a pointer for indicating a state change of a data in the queue is respectively set in the control queue and the ring queue, wherein the control queue changes synchronously with the position indicated by the pointer in the ring queue;
  • the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate that the receiving device has not confirmed the received data; between the queue receiving pointer and the queue tail pointer The data is used to indicate that the receiving device has acknowledged receipt and has not processed the data.
  • the queue head pointer HEAD, the queue tail pointer TAIL, and the queue receiving pointer RECV_TAIL are respectively set in the control queue and the ring queue, and the transmission of the data to be transmitted is realized by the change of the position indicated by the pointer. control.
  • the new message of the sending device is performed by adding to the header, and the receiving device starts receiving from the tail.
  • the transmitting device determines that when HEAD is equal to TAIL, the queue is empty, and when HEAD+1 is equal to RECV_TAIL, the queue is full.
  • a data transmission channel is established based on the sequence by establishing a corresponding sequence on both sides of the sending and receiving, so as to implement flexible transmission control of the data to be processed.
  • control queue changes synchronously with the location indicated by the pointer in the circular queue includes:
  • the queue head pointer of the control queue will move forward by N data bytes, and the position indicated by the queue head pointer will be synchronized to the circular queue of the receiving device, where N is greater than a natural number equal to 1;
  • the queue tail pointer of the circular queue will move forward M data bytes, and synchronize the position indicated by the queue tail pointer to the control queue of the sending device, where M ⁇ N, M is a natural number greater than or equal to 1;
  • the position of the queue receiving pointer is updated.
  • the cross-node data transmission channel established between the node 1 and the node 2 shown in FIG. 3 is based on the control of the data queue.
  • the specific interaction process is as follows:
  • the node 2 when the node 2 receives the interrupt signal of the node 1 or polls the pointer HEAD change, the message processing is performed, and the message is handed over to the application module in the node 2 for processing.
  • the node 2 increases the pointer TAIL of the circular queue, for example, the pointer TAIL moves forward 2 data bytes, then synchronizes the position indicated by the pointer TAIL to the node 1;
  • node 1 when the node 1 receives the interrupt signal of the node 2 or polls the pointer TAIL, it indicates that there is data in the queue to be acknowledged, and the node 1 processes the message in the queue to be acknowledged, and notifies the application module in the node 1.
  • node 1 When the message processing is completed, node 1 will update the position of the pointer RECV_TAIL, that is, release the current position of the pointer RECV_TAIL.
  • the position indicated by the pointer in the queues on both sides is synchronized, thereby accurately controlling the transmission state of the transmitted data, and achieving the purpose of accurately completing the data transmission as the data transmission channel.
  • FIG. 8 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. As shown in FIG. 8, the apparatus includes:
  • the first obtaining unit 802 is configured to acquire the data amount of the current data to be processed, where the current data to be processed includes input and output I/O data currently performing a remote direct data access RDMA operation;
  • the first processing unit 804 is configured to perform an RDMA operation of the current data to be processed directly on the page address corresponding to the current data to be processed when the data amount of the current data to be processed is greater than the first predetermined threshold;
  • the second processing unit 806 is configured to set, on the page address in the receiving device, the pre-established data transmission channel when the data amount of the current data to be processed is less than or equal to the first predetermined threshold Execute the RDMA operation of the current data to be processed.
  • the foregoing data processing method may be, but is not limited to, being applied to a node communication process of a PCIe (PCI-Express) link, and after the sending device acquires the data amount of the current data to be processed,
  • the data volume is selected according to different data processing modes: when the data volume is less than or equal to the first predetermined threshold, the data to be processed may be sent to the receiving device through the pre-established data transmission channel, so that the receiving device is Performing a memory copy to complete a remote direct data access RDMA operation on the current pending data on the channel cache corresponding to the page address; when the data amount is greater than the first predetermined threshold, the data may be acquired at the destination node After the upper page address, the RDMA operation of the current data to be processed is directly executed on the page address.
  • PCIe PCI-Express
  • the current to-be-processed data includes input/output I/O data that currently performs an RDMA operation.
  • the interaction duration of sending a request once and receiving the request response is fixed, if the amount of data transmitted at one time is small, the utilization rate of the data transmission channel will obviously be lowered, and thus, in this embodiment, when the data is When the quantity (that is, the overhead) is small, the current data to be processed can be directly transmitted to the receiving device through the above data transmission channel, so that the receiving device completes the data processing through one memory copy, thereby realizing hardware acceleration of the data processing process without The page address for performing the RDMA operation is acquired in advance, and the multiple copying process for the data is also avoided. That is to say, when the amount of data is small, the processing duration of the data will be determined according to the copying duration of the data. Further, when the amount of data (i.e., overhead) is large, the time required to perform copying is long, and thus, the RDMA operation of executing the current data to be processed directly on the page address is employed.
  • the amount of data i.e., overhead
  • the page address is obtained, where the manner of obtaining the page address may include, but is not limited to, at least one of the following: Sending a request for obtaining a page address to the receiving device through the data transmission channel, and directly obtaining the cached page address from the local page address buffer pool.
  • the page address in the page address buffer pool is obtained by: detecting a number of page addresses cached in the page address buffer pool; and if the number of page addresses is less than or equal to a second predetermined threshold, The data transfer channel gets a new page address.
  • the page address buffer pool will add and update the page address in the pool in time according to the current cache amount. Therefore, when the amount of data is large, the problem of processing delay caused by requesting the receiving device to obtain the page address every time is avoided. Further, when the new page address is added to the page address buffer pool, the normal RDMA operation on the current data to be processed is not affected, that is, the two can be simultaneously performed asynchronously, thereby further improving the efficiency of data processing. .
  • the foregoing data transmission channel may be, but is not limited to, a cross-node communication channel established between nodes (such as between the sending device and the receiving device) based on a message directly accessing the peer memory.
  • the foregoing data transmission channel established by the foregoing cross-node may be, but not limited to, being established based on data queues on both sides.
  • the receiving device sets an annular queue
  • the transmitting device sets a control queue for controlling the circular queue, wherein the control queue includes control information for controlling the state change of the data in the circular queue. That is to say, the transmission control of the current data to be processed is realized by directly controlling the data queues on both sides of the transmission and reception.
  • the synchronous update sends and receives the location indicated by the data pointer in the data queues on both sides to achieve accurate control of the transmission status of the current data to be processed.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of directly executing the current data to be processed on the page address is adopted. Without having to obtain the corresponding page address by interaction each time, the purpose of reducing the data interaction is achieved; when the data volume of the current data to be processed is small, the current pending processing is completed through the memory copy directly in the receiving device through the data transmission channel.
  • the RDMA operation of the data thereby achieving the effect of hardware acceleration on the data processing process.
  • the second obtaining unit is configured to directly obtain the page address from the local page address buffer pool before performing the RDMA operation of the current data to be processed directly on the page address corresponding to the current data to be processed, wherein the page address cache The pool is used to cache one or more page addresses.
  • the foregoing page address buffer pool may be, but is not limited to, setting a plurality of different page address buffer pools according to different services (also referred to as applications). As shown in FIG. 2, it can be divided into a page address buffer pool 202-1 to a page address buffer pool 202-N.
  • the page address in the page address buffer pool may be, but is not limited to, added to the local page address buffer pool after being acquired by the peer (ie, the receiving device).
  • the manner in which the page address buffer pool obtains the page address may include, but is not limited to, at least one of the following: when the node 1 detects that the buffer amount is lower than the predetermined threshold, the node 1 requests the supplementary page address through the data transmission channel; the node 2 transmits the data through the data.
  • the channel actively adds a new page address to node 1 according to a predetermined period.
  • the sending device takes the node 1 as an example
  • the receiving device takes the node 2 as an example.
  • the steps S206-S208 when the node 1 sends data to the node 2, it can directly address the page address.
  • the cache pool application obtains the page address without sending a request for obtaining the request to the node 2, thereby reducing the time for the node 1 to obtain the page address each time, thereby reducing the processing delay.
  • the node 1 may set a plurality of page address buffer pools, such as a page address buffer pool 202-1 to a page address buffer pool 202-N, for the peer node 2.
  • the acquisition request may be sent to the correspondent node 2 through the data transmission channel, so that the node 2 supplements the node 1 with a new page address.
  • the process of acquiring the page address by the node 1 ie, steps S202-S204
  • the process of acquiring the page address by the page address buffer pool ie, steps S206-S208
  • the sending device can directly obtain the page address from the local and perform the RDMA operation directly on the page address, thereby reducing the processing delay of the data to be processed.
  • the detecting unit is configured to detect the number of page addresses cached in the page address buffer pool before directly obtaining the page address from the local page address buffer pool;
  • the third obtaining unit is configured to acquire a new page address through the data transmission channel when the number of page addresses is less than or equal to a second predetermined threshold.
  • the foregoing second predetermined threshold may be, but is not limited to, set to different values according to different application scenarios.
  • the node 1 ie, the sending device
  • can set multiple page address buffer pools for the node 2 and can set a second predetermined threshold, that is, whether the total number of page addresses in all the page address buffer pools meets a second predetermined threshold.
  • the second predetermined threshold of different values may be set for each page address buffer pool, that is, whether the number of page addresses in each page address buffer pool meets a corresponding second predetermined threshold value, which is not used in this embodiment. limited.
  • the third obtaining unit includes: 1) a sending module, configured to send an acquisition request for requesting the acquisition of the page address as the data to be transmitted to the receiving device through the data transmission channel; 2) acquiring the module , set to get the page address sent by the receiving device; 3) add a module, set to add the page address to the page address buffer pool.
  • steps S202-S204 may be performed to obtain a page address for supplementing with the peer node.
  • steps S202-S204 may be performed to obtain a page address for supplementing with the peer node.
  • the page address in the page address buffer pool is replenished in time by detecting the number of page addresses in the page address buffer pool, thereby ensuring that the current pending data performs RDMA operations normally. It is also possible to cache new page addresses in time to further improve the efficiency of data processing.
  • the second processing unit includes:
  • the processing module is configured to send the current to-be-processed data as the data to be transmitted directly to the receiving device through the data transmission channel, and save the data transmission channel in the channel buffer of the receiving device, so that the receiving device uses the channel buffer.
  • the current pending data performs an RDMA operation on the page address.
  • the current page address may not be acquired, and the current to-be-processed data is directly sent to the receiving device as the data to be transmitted through the data transmission channel, and The device is stored in the channel buffer of the receiving device, and the application of the receiving device directly reads the current data to be processed from the channel buffer and copies the data to the corresponding page memory for RDMA operation.
  • the channel buffer of the data transmission channel is used to complete the RDMA operation on the current data to be processed directly in the receiving device, thereby achieving the effect of improving data processing efficiency.
  • control queue includes control information for controlling changes in data state in the circular queue.
  • the establishing unit includes:
  • (1) setting a module configured to respectively set a pointer for indicating a state change of the data in the queue in the control queue and the ring queue, wherein the control queue changes synchronously with the position indicated by the pointer in the circular queue;
  • the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate that the receiving device has not confirmed the received data; between the queue receiving pointer and the queue tail pointer The data is used to indicate that the receiving device has acknowledged receipt and has not processed the data.
  • a queue head pointer HEAD, a queue tail pointer TAIL, and a queue receiving pointer RECV_TAIL are respectively set in the control queue and the ring queue, and the foregoing refers to The change of the position indicated by the needle realizes the transmission control of the data to be transmitted.
  • the new message of the sending device is performed by adding to the header, and the receiving device starts receiving from the tail.
  • the transmitting device determines that when HEAD is equal to TAIL, the queue is empty, and when HEAD+1 is equal to RECV_TAIL, the queue is full.
  • a data transmission channel is established based on the sequence by establishing a corresponding sequence on both sides of the sending and receiving, so as to implement flexible transmission control of the data to be processed.
  • the setting module controls the synchronization change of the position indicated by the pointer in the control queue and the ring queue by:
  • the queue head pointer of the control queue will move N data bytes forward, and the position indicated by the queue head pointer will be synchronized to the circular queue of the receiving device, where N is greater than a natural number equal to 1;
  • the queue tail pointer of the circular queue will move forward M data bytes, and synchronize the position indicated by the queue tail pointer to the control queue of the sending device, where M ⁇ N, M is a natural number greater than or equal to 1;
  • the position of the queue receiving pointer is updated.
  • the cross-node data transmission channel established between the node 1 and the node 2 shown in FIG. 3 is based on the control of the data queue.
  • the specific interaction process is as follows:
  • the node 2 when the node 2 receives the interrupt signal of the node 1 or polls the pointer HEAD change, the message processing is performed, and the message is handed over to the application module in the node 2 for processing.
  • the node 2 increases the pointer TAIL of the circular queue, for example, the pointer TAIL moves forward 2 data bytes, then synchronizes the position indicated by the pointer TAIL to the node 1;
  • node 1 when the node 1 receives the interrupt signal of the node 2 or polls the pointer TAIL, it indicates that there is data in the queue to be acknowledged, and the node 1 processes the message in the queue to be acknowledged, and notifies the application module in the node 1.
  • node 1 When the message processing is completed, node 1 will update the position of the pointer RECV_TAIL, that is, release the current position of the pointer RECV_TAIL.
  • the position indicated by the pointer in the queues on both sides is synchronized, thereby accurately controlling the transmission state of the transmitted data, and achieving the purpose of accurately completing the data transmission as the data transmission channel.
  • FIG. 9 is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. As shown in FIG. 9, the apparatus includes:
  • the first processing unit 902 is configured to: when the data amount of the current to-be-processed data acquired by the sending device is greater than the first predetermined threshold, the receiving and transmitting device directly executes the current to-be-processed data on the page address corresponding to the current to-be-processed data.
  • Remote direct data access RDMA operation wherein the current data to be processed includes input and output I/O data that currently performs an RDMA operation;
  • the second processing unit 904 is configured to receive an RDMA operation of executing current data to be processed on a page address in the receiving device by using a pre-established data transmission channel when the data amount of the current data to be processed is less than or equal to the first predetermined threshold.
  • the foregoing data processing method may be, but is not limited to, being applied to a node communication process of a PCIe (PCI-Express) link, and after the sending device acquires the data amount of the current data to be processed,
  • the data volume is selected according to different data processing modes: when the data volume is less than or equal to the first predetermined threshold, the data to be processed may be sent to the receiving device through the pre-established data transmission channel, so that the receiving device is Performing a memory copy to complete a remote direct data access RDMA operation on the current pending data on the channel cache corresponding to the page address; when the data amount is greater than the first predetermined threshold, the data may be acquired at the destination node After the page address is uploaded, the current pending address is directly executed on the page address.
  • PCIe PCI-Express
  • the current to-be-processed data includes input/output I/O data that currently performs an RDMA operation.
  • the interaction duration of sending a request once and receiving the request response is fixed, if the amount of data transmitted at one time is small, the utilization rate of the data transmission channel will obviously be lowered, and thus, in this embodiment, when the data is When the quantity (that is, the overhead) is small, the current data to be processed can be directly transmitted to the receiving device through the above data transmission channel, so that the receiving device completes the data processing through one memory copy, thereby realizing hardware acceleration of the data processing process without The page address for performing the RDMA operation is acquired in advance, and the multiple copying process for the data is also avoided. That is to say, when the amount of data is small, the processing duration of the data will be determined according to the copying duration of the data. Further, when the amount of data (i.e., overhead) is large, the time required to perform copying is long, and thus, the RDMA operation of executing the current data to be processed directly on the page address is employed.
  • the amount of data i.e., overhead
  • the foregoing data transmission channel may be, but is not limited to, a cross-node communication channel established between nodes (such as between the sending device and the receiving device) based on a message directly accessing the peer memory.
  • the foregoing data transmission channel established by the foregoing cross-node may be, but not limited to, being established based on data queues on both sides.
  • the receiving device sets an annular queue
  • the transmitting device sets a control queue for controlling the circular queue, wherein the control queue includes control information for controlling the state change of the data in the circular queue. That is to say, the transmission control of the current data to be processed is realized by directly controlling the data queues on both sides of the transmission and reception.
  • the synchronous update sends and receives the location indicated by the data pointer in the data queues on both sides to achieve accurate control of the transmission status of the current data to be processed.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of directly executing the current data to be processed on the page address is adopted. Without having to obtain the corresponding page address through interaction each time, the purpose of reducing data interaction is achieved; when the amount of data of the current data to be processed is small, the data transmission channel is directly completed by the memory copy in the receiving device.
  • the RDMA operation of the data to be processed before thereby achieving the effect of hardware acceleration on the data processing process.
  • a receiving unit configured to receive an acquisition request sent by the sending device for requesting to acquire a page address, before the receiving sending device directly performs an RDMA operation of the current data to be processed on the page address corresponding to the current data to be processed;
  • the sending unit is set to send the page address in response to the acquisition request.
  • the page address in response to the acquisition request sent by the sending device, the page address is sent to the sending device, so as to ensure that the current page to be processed performs the RDMA operation normally, and the new page address can be cached to the sending device in time. Achieve the effect of improving data processing efficiency.
  • the second processing unit includes:
  • a receiving module configured to receive current pending data
  • the copy module is set to copy the current pending data in the channel buffer to the page memory corresponding to the page address.
  • the current page address may not be acquired, and the current to-be-processed data is directly sent to the receiving device as the data to be transmitted through the data transmission channel, and The device is stored in the channel buffer of the receiving device, and the application of the receiving device directly reads the current data to be processed from the channel buffer and copies the data to the corresponding page memory for RDMA operation.
  • the channel buffer of the data transmission channel is utilized to complete the RDMA operation on the current data to be processed directly in the receiving device. Thereby achieving the effect of improving data processing efficiency.
  • the establishing unit includes:
  • the pointer includes: a queue head pointer, a queue tail pointer, and a queue receiving pointer, wherein data between the queue tail pointer and the queue head pointer is used to indicate that the receiving device has not confirmed the received data; between the queue receiving pointer and the queue tail pointer The data is used to indicate that the receiving device has acknowledged receipt and has not processed the data.
  • the queue head pointer HEAD, the queue tail pointer TAIL, and the queue receiving pointer RECV_TAIL are respectively set in the control queue and the ring queue, and the transmission of the data to be transmitted is realized by the change of the position indicated by the pointer. control.
  • the new message of the sending device is performed by adding to the header, and the receiving device starts receiving from the tail.
  • the transmitting device determines that when HEAD is equal to TAIL, the queue is empty, and when HEAD+1 is equal to RECV_TAIL, the queue is full.
  • a data transmission channel is established based on the sequence by establishing a corresponding sequence on both sides of the sending and receiving, so as to implement flexible transmission control of the data to be processed.
  • the setting module realizes the synchronization change of the position indicated by the pointer in the control queue and the circular queue by:
  • the queue head pointer of the control queue will move forward N data bytes, and the position indicated by the queue head pointer is synchronized to the circular queue of the receiving device, where N is a natural number greater than or equal to 1;
  • the queue tail pointer of the circular queue will move forward M data bytes, and synchronize the position indicated by the queue tail pointer to the control queue of the sending device, where M ⁇ N, M is a natural number greater than or equal to 1;
  • the position of the queue receiving pointer is updated.
  • the cross-node data transmission channel established between the node 1 and the node 2 shown in FIG. 3 is based on the control of the data queue.
  • the specific interaction process is as follows:
  • the node 2 when the node 2 receives the interrupt signal of the node 1 or polls the pointer HEAD change, the message processing is performed, and the message is handed over to the application module in the node 2 for processing.
  • the node 2 increases the pointer TAIL of the circular queue, for example, the pointer TAIL moves forward 2 data bytes, then synchronizes the position indicated by the pointer TAIL to the node 1;
  • node 1 when the node 1 receives the interrupt signal of the node 2 or polls the pointer TAIL, it indicates that there is data in the queue to be acknowledged, and the node 1 processes the message in the queue to be acknowledged, and notifies the application module in the node 1.
  • node 1 When the message processing is completed, node 1 will update the position of the pointer RECV_TAIL, that is, release the current position of the pointer RECV_TAIL.
  • the position indicated by the pointer in the queues on both sides is synchronized, thereby accurately controlling the transmission state of the transmitted data, and achieving the purpose of accurately completing the data transmission as the data transmission channel.
  • Embodiments of the present invention also provide a storage medium.
  • the foregoing storage medium may be configured to store program code for performing the following steps:
  • the RDMA operation of the current data to be processed is performed on the page address in the receiving device through the pre-established data transmission channel.
  • the foregoing storage medium may include, but not limited to, a USB flash drive, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, and a magnetic memory.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • a mobile hard disk e.g., a hard disk
  • magnetic memory e.g., a hard disk
  • modules or steps of the present invention described above can be implemented by a general-purpose computing device that can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they may be implemented by program code executable by the computing device such that they may be stored in the storage device by the computing device and, in some cases, may be different from the order herein.
  • the steps shown or described are performed, or they are separately fabricated into individual integrated circuit modules, or a plurality of modules or steps thereof are fabricated as a single integrated circuit module.
  • the invention is not limited to any specific combination of hardware and software.
  • a reasonable data processing mode is selected according to the data amount of the current data to be processed: when the data amount of the current data to be processed is large, the RDMA operation of executing the current data to be processed directly on the page address is adopted. It is not necessary to obtain the corresponding page address by interaction each time, thereby reducing the purpose of data interaction; when the data volume of the current data to be processed is small, the data to be processed is directly completed by the memory copy in the receiving device through the data transmission channel.
  • RDMA operations to achieve hardware acceleration for the data processing process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Systems (AREA)
  • Communication Control (AREA)

Abstract

一种数据处理方法和装置。该方法包括：获取当前待处理数据的数据量，其中，当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据（S102）；若当前待处理数据的数据量大于第一预定阈值，则直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作（S104）；若当前待处理数据的数据量小于等于第一预定阈值，则通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作（S106）。解决了相关技术中RDMA只能通过底层协议栈多次交互，多次拷贝来传输数据的方式执行RDMA操作所导致的数据处理效率较低的问题，进而达到了提高数据处理效率的效果。

Description

数据处理方法和装置 技术领域
本发明实施例涉及通信领域，具体而言，涉及一种数据处理方法和装置。
背景技术
随着用户数据的不断膨胀，信息技术的兴起。各种通信设备间的带宽越来越大。PCIe(PCI-Express)链路作为一种节点内主流的高速传输协议被广泛的应用。在不断提高带宽的同时，PCIe协议逐步开始从节点内中央处理器(CPU,Central Processing Unit)和外部设备之间的互联协议中走出来，利用PCIe的非透明桥(NTB，Non Transparent bridge)技术，PCIe协议可以支持节点间的高速互联，互联的节点间通过NTB进行地址域的隔离。经过NTB的地址映射后，本节点上的DMA引擎通过访问NTB映射过来的虚拟地址，就可以实现对对端的节点上的内存的访问。虽然通过NTB和存储器直接访问(DMA，Direct Memory Acess)技术，PCIe协议物理上实现了直接访问对端节点上的内存，但是这离高效的节点间的数据交互还有一些距离。
具体来说，由于PCIe的远程直接数据存取RDMA(RDMA，Remote Direct Memory Acess)操作只在发送节点上进行，因而PCIe在与对端进行交互时，例如，通过DMA引擎访问对端，或向对端发送数据时，就需要确切的获取对端节点上的物理地址，从而实现在源端的一次RDMA操作过程中，就将数据RDMA到对端节点上的内存。也就是说，在源端向对端获取对端节点上执行RDMA操作的物理地址时，还需要在节点间进行多次交互，才能申请到对端节点上的内存。此外，PCIe交换器厂商提供的DEMO软件一般都采用PCIe去模拟一个以太网接口，但是模拟的以太网接口在数据传输时，往往需要多次的数据拷贝，不能充分利用到PCIe协议提供的高带宽。也就是说，采用相关技术中所提供的只能通过多次交互 获取对端物理地址的方式执行RDMA操作时，将导致数据处理效率较低的问题。
发明内容
本发明实施例提供了一种数据处理方法和装置，以至少解决相关技术中RDMA只能通过底层协议栈多次交互，多次拷贝来传输数据的方式执行RDMA操作所导致的数据处理效率较低的问题。
根据本发明实施例的一个方面，提供了一种数据处理方法，包括：获取当前待处理数据的数据量，其中，上述当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；若上述当前待处理数据的数据量大于第一预定阈值，则直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作；若上述当前待处理数据的数据量小于等于上述第一预定阈值，则通过预先建立的数据传输通道在接收设备中的上述页面地址上执行上述当前待处理数据的上述RDMA操作。
可选地，在直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作之前，还包括：从本地的页面地址缓存池中直接获取上述页面地址，其中，上述页面地址缓存池用于缓存一个或多个页面地址。
可选地，在从本地的页面地址缓存池中直接获取上述页面地址之前，包括：检测上述页面地址缓存池中缓存的页面地址数量；若上述页面地址数量小于等于第二预定阈值，则通过上述数据传输通道获取新的页面地址。
可选地，上述通过上述数据传输通道获取新的页面地址包括：将用于请求获取上述页面地址的获取请求作为待传输数据通过上述数据传输通道发送给上述接收设备；获取上述接收设备发送的上述页面地址；添加上述页面地址到上述页面地址缓存池中。
可选地，上述通过预先建立的数据传输通道在接收设备中的上述页面地址上执行上述当前待处理数据的上述RDMA操作包括：将上述当前待 处理数据作为待传输数据通过上述数据传输通道直接发送给上述接收设备，并保存在上述数据传输通道在上述接收设备的通道缓存器中，以使上述接收设备利用上述通道缓存器中的上述当前待处理数据在上述页面地址上执行上述RDMA操作。
可选地，上述接收设备利用上述通道缓存器中的上述当前待处理数据在上述页面地址上执行上述RDMA操作包括：上述接收设备将上述通道缓存器中的上述当前待处理数据拷贝到与上述页面地址对应页面内存中。
可选地，在获取当前待处理数据的数据量之前，还包括：在发送设备与上述接收设备之间建立上述数据传输通道，其中，上述数据传输通道连接上述发送设备中的控制队列与上述接收设备中的环形队列，其中，上述控制队列包括用于控制上述环形队列中数据状态变化的控制信息。
可选地，上述在发送设备与上述接收设备之间建立上述数据传输通道包括：在上述控制队列及上述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，上述控制队列与上述环形队列中的上述指针所指示的位置同步变化；其中，上述指针包括：队列头指针、队列尾指针及队列接收指针，其中，上述队列尾指针与上述队列头指针之间的数据用于表示上述接收设备尚未确认接收的数据；上述队列接收指针与上述队列尾指针之间的数据用于表示上述接收设备已确认接收，且尚未处理的数据。
可选地，上述控制队列与上述环形队列中的上述指针所指示的位置同步变化包括：在上述发送设备执行发送操作时，上述控制队列的上述队列头指针将向前移动N个数据字节，并将上述队列头指针所指示的位置将同步到上述接收设备的上述环形队列中，其中，上述N为大于等于1的自然数；在上述接收设备执行接收操作时，上述环形队列的上述队列尾指针将向前移动M个数据字节，并将上述队列尾指针所指示的位置同步到上述发送设备的上述控制队列中，其中，M≤N，M为大于等于1的自然数；其中，在上述发送设备的上述控制队列中上述队列尾指针移动到上述队列头指针时，更新上述队列接收指针的位置。
根据本发明实施例的另一方面，提供了一种数据处理方法，包括：在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收上述发送设备直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的远程直接数据存取RDMA操作，其中，上述当前待处理数据包括当前执行上述RDMA操作的输入输出I/O数据；在上述当前待处理数据的数据量小于等于上述第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的上述页面地址上执行上述当前待处理数据的上述RDMA操作。
可选地，在接收上述发送设备直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作之前，还包括：接收上述发送设备发送的用于请求获取上述页面地址的获取请求；响应上述获取请求发送上述页面地址。
可选地，上述通过预先建立的数据传输通道在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作包括：接收上述当前待处理数据；将上述待处理数据保存在上述数据传输通道在上述接收设备的通道缓存器中；将上述通道缓存器中的上述当前待处理数据一次拷贝到与上述页面地址对应页面内存中。
可选地，还包括：预先在发送设备与上述接收设备之间建立上述数据传输通道，其中，上述数据传输通道连接上述发送设备中的控制队列与上述接收设备中的环形队列，其中，上述控制队列包括用于控制上述环形队列中数据状态变化的控制信息。
可选地，上述在发送设备与上述接收设备之间建立上述数据传输通道包括：在上述控制队列及上述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，上述控制队列与上述环形队列中的上述指针所指示的位置同步变化；其中，上述指针包括：队列头指针、队列尾指针及队列接收指针，其中，上述队列尾指针与上述队列头指针之间的数据用于表示上述接收设备尚未确认接收的数据；上述队列接收指针与上述队列尾指针 之间的数据用于表示上述接收设备已确认接收，且尚未处理的数据。
可选地，上述控制队列与上述环形队列中的上述指针所指示的位置同步变化包括：在上述发送设备执行发送操作时，上述控制队列的上述队列头指针将向前移动N个数据字节，并将上述队列头指针所指示的位置将同步到上述接收设备的上述环形队列中，其中，上述N为大于等于1的自然数；在上述接收设备执行接收操作时，上述环形队列的上述队列尾指针将向前移动M个数据字节，并将上述队列尾指针所指示的位置同步到上述发送设备的上述控制队列中，其中，M≤N，M为大于等于1的自然数；其中，在上述发送设备的上述控制队列中上述队列尾指针移动到上述队列头指针时，更新上述队列接收指针的位置。
根据本发明实施例的又一方面，提供了一种数据处理装置，包括：第一获取单元，设置为获取当前待处理数据的数据量，其中，上述当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；第一处理单元，设置为在上述当前待处理数据的数据量大于第一预定阈值时，直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作；第二处理单元，设置为在上述当前待处理数据的数据量小于等于上述第一预定阈值时，通过预先建立的数据传输通道在接收设备中的上述页面地址上执行上述当前待处理数据的上述RDMA操作。
可选地，上述装置还包括：第二获取单元，设置为在直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作之前，从本地的页面地址缓存池中直接获取上述页面地址，其中，上述页面地址缓存池用于缓存一个或多个页面地址。
可选地，上述装置还包括：检测单元，设置为在从本地的页面地址缓存池中直接获取上述页面地址之前，检测上述页面地址缓存池中缓存的页面地址数量；第三获取单元，设置为在上述页面地址数量小于等于第二预定阈值时，通过上述数据传输通道获取新的页面地址。
可选地，上述第三获取单元包括：发送模块，设置为将用于请求获取上述页面地址的获取请求作为待传输数据通过上述数据传输通道发送给上述接收设备；获取模块，设置为获取上述接收设备发送的上述页面地址；添加模块，设置为添加上述页面地址到上述页面地址缓存池中。
可选地，上述第二处理单元包括：处理模块，设置为将上述当前待处理数据作为待传输数据通过上述数据传输通道直接发送给上述接收设备，并保存在上述数据传输通道在上述接收设备的通道缓存器中，以使上述接收设备利用上述通道缓存器中的上述当前待处理数据在上述页面地址上执行上述RDMA操作。
可选地，还包括：建立单元，设置为在获取当前待处理数据的数据量之前，在发送设备与上述接收设备之间建立上述数据传输通道，其中，上述数据传输通道连接上述发送设备中的控制队列与上述接收设备中的环形队列，其中，上述控制队列包括用于控制上述环形队列中数据状态变化的控制信息。
可选地，上述建立单元包括：设置模块，设置为在上述控制队列及上述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，上述控制队列与上述环形队列中的上述指针所指示的位置同步变化；其中，上述指针包括：队列头指针、队列尾指针及队列接收指针，其中，上述队列尾指针与上述队列头指针之间的数据用于表示上述接收设备尚未确认接收的数据；上述队列接收指针与上述队列尾指针之间的数据用于表示上述接收设备已确认接收，且尚未处理的数据。
可选地，上述设置模块通过以下方式控制上述控制队列与上述环形队列中的上述指针所指示的位置同步变化包括：在上述发送设备执行发送操作时，上述控制队列的上述队列头指针将向前移动N个数据字节，并将上述队列头指针所指示的位置将同步到上述接收设备的上述环形队列中，其中，上述N为大于等于1的自然数；在上述接收设备执行接收操作时，上述环形队列的上述队列尾指针将向前移动M个数据字节，并将上述队列 尾指针所指示的位置同步到上述发送设备的上述控制队列中，其中，M≤N，M为大于等于1的自然数；其中，在上述发送设备的上述控制队列中上述队列尾指针移动到上述队列头指针时，更新上述队列接收指针的位置。
根据本发明实施例的又一方面，提供了一种数据处理装置，包括：第一处理单元，设置为在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收上述发送设备直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的远程直接数据存取RDMA操作，其中，上述当前待处理数据包括当前执行上述RDMA操作的输入输出I/O数据；第二处理单元，设置为在上述当前待处理数据的数据量小于等于上述第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的上述页面地址上执行上述当前待处理数据的上述RDMA操作。
可选地，上述装置还包括：接收单元，设置为在接收上述发送设备直接在与上述当前待处理数据对应的页面地址上执行上述当前待处理数据的上述RDMA操作之前，接收上述发送设备发送的用于请求获取上述页面地址的获取请求；发送单元，设置为响应上述获取请求发送上述页面地址。
可选地，上述第二处理单元包括：接收模块，设置为接收上述当前待处理数据；保存模块，设置为将上述待处理数据保存在上述数据传输通道在上述接收设备的通道缓存器中；拷贝模块，设置为将上述通道缓存器中的上述当前待处理数据拷贝到与上述页面地址对应页面内存中。
在本发明实施例中，还提供了一种计算机存储介质，该计算机存储介质可以存储有执行指令，该执行指令用于执行上述实施例中的数据处理方法。
通过本发明实施例，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较 小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。
附图说明
此处所说明的附图用来提供对本发明的进一步理解，构成本申请的一部分，本发明的示意性实施例及其说明用于解释本发明，并不构成对本发明的不当限定。在附图中：
图1是根据本发明实施例的一种可选的数据处理方法的流程图；
图2是根据本发明实施例的一种可选的数据处理方法的示意图；
图3是根据本发明实施例的另一种可选的数据处理方法的示意图；
图4是根据本发明实施例的一种可选的数据处理方法的应用示意图；
图5是根据本发明实施例的另一种可选的数据处理方法的应用示意图；
图6是根据本发明实施例的又一种可选的数据处理方法的应用示意图；
图7是根据本发明实施例的另一种可选的数据处理方法的流程图；
图8是根据本发明实施例的一种可选的数据处理装置的示意图；以及
图9是根据本发明实施例的另一种可选的数据处理装置的示意图。
具体实施方式
下文中将参考附图并结合实施例来详细说明本发明。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
需要说明的是，本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序 或先后次序。
实施例1
在本实施例中提供了一种数据处理方法，图1是根据本发明实施例的数据处理方法的流程图，如图1所示，该流程包括如下步骤：
步骤S102，获取当前待处理数据的数据量，其中，当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；
步骤S104，若当前待处理数据的数据量大于第一预定阈值，则直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作；
步骤S106，若当前待处理数据的数据量小于等于第一预定阈值，则通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据处理方法可以但不限于应用于PCIe(PCI-Express)链路的节点通信过程中，发送设备在获取到当前待处理数据的数据量后，将根据上述数据量选择不同的数据处理方式：在数据量小于等于第一预定阈值时，由于数据开销较小，则可以将当前待处理数据通过预先建立的数据传输通道发送给接收设备，以使接收设备进行内存拷贝，以完成在页面地址对应的通道缓存上对当前待处理数据的远程直接数据存取RDMA操作；在数据量大于第一预定阈值时，由于数据开销较大，则可以在获取目的节点上的页面地址后，直接在该页面地址上执行当前待处理数据的RDMA操作。其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据。通过根据当前待处理数据的数据量选择合理的数据处理方式，从而实现提高数据处理效率的效果，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低。
需要说明的是，由于一次发送请求并接收到请求响应的交互时长固定，如果一次传输的数据量较小时，显然将使得数据传输通道的利用率变低， 因而，在本实施例中，当数据量(即开销)较小时，就可以直接通过上述数据传输通道将当前待处理数据传输给接收设备，以使接收设备通过一次内存拷贝完成数据处理，从而实现对数据处理过程的硬件加速，而无需预先获取执行RDMA操作的页面地址，也避免了对数据的多次拷贝过程。也就是说，当数据量较小时，数据的处理时长将根据数据的拷贝时长决定。进一步，当数据量(即开销)较大时，执行拷贝所需的时间很长，因而，则采用直接在页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，在直接在页面地址上执行当前待处理数据的RDMA操作之前，还需获取页面地址，其中，上述页面地址的获取方式可以包括但不限于以下至少之一：通过数据传输通道向接收设备发送用于获取页面地址的获取请求、从本地的页面地址缓存池中直接获取已缓存的页面地址。
可选地，在本实施例中，上述页面地址缓存池中的页面地址可以通过以下方式获取：检测页面地址缓存池中缓存的页面地址数量；若页面地址数量小于等于第二预定阈值，则通过数据传输通道获取新的页面地址。
需要说明的是，在本实施例中，上述页面地址缓存池将根据当前的缓存量，对池中的页面地址进行及时添加更新。从而实现在数据量较大时，避免每次都向接收设备请求获取页面地址所造成的处理延时的问题。进一步，在获取新的页面地址添加到上述页面地址缓存池时，并不影响对当前待处理数据正常的RDMA操作，也就是说，二者可以异步同时进行，从而进一步实现提高数据处理效率的效果。
可选地，在本实施例中，上述数据传输通道可以但不限于是基于直接访问对端内存的消息，在节点之间(如发送设备和接收设备之间)建立的跨节点通信通道。其中，上述跨节点建立的上述数据传输通道可以但不限于是基于两侧的数据队列建立。具体而言，在接收设备设置一个环形队列，在发送设备设置一个用于控制环形队列控制队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。也就是说，通过直接控制发 送和接收两侧的数据队列，实现对当前待处理数据的传输控制。例如，同步更新发送和接收两侧的数据队列中的数据指针所指示的位置，以达到准确控制当前待处理数据的传输状态。
通过本申请提供的实施例，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。
作为一种可选的方案，在直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作之前，还包括：
S1，从本地的页面地址缓存池中直接获取页面地址，其中，页面地址缓存池用于缓存一个或多个页面地址。
可选地，在本实施例中，上述页面地址缓存池可以但不限于根据不同的业务(也称之为应用)设置多个不同的页面地址缓存池。如图2所示，可以划分为页面地址缓存池202-1至页面地址缓存池202-N。
可选地，在本实施例中，上述页面地址缓存池中的页面地址可以但不限于在由对端(即接收设备)获取后，添加更新到本地页面地址缓存池中。其中，页面地址缓存池获取页面地址的方式可以包括但不限于以下至少之一：节点1检测到缓存量低于预定阈值时，通过数据传输通道向节点2请求补充页面地址；节点2通过数据传输通道主动按照预定周期为节点1补充新的页面地址。
具体结合以下示例进行说明，如图2所示，发送设备以节点1为例，接收设备以节点2为例，如步骤S206-S208，当节点1向节点2发送数据 时，可以直接向页面地址缓存池申请获取页面地址而无需向节点2发送申请获取的请求，从而减少节点1每次获取页面地址的时间，进而达到减少处理延时的效果。
进一步，如步骤S202-S204，上述节点1为对端节点2可以设置多个页面地址缓存池，如页面地址缓存池202-1至页面地址缓存池202-N。当检测到一个缓存池中的缓存量低于预定阈值时，则可以通过数据传输通道向对端节点2发送获取请求，以实现由节点2为节点1补充新的页面地址。
需要说明的是，在本示例中，上述节点1获取页面地址的过程(即步骤S202-S204)与页面地址缓存池获取页面地址的过程(即步骤S206-S208)并不限于如图所示的顺序，上述两个过程可以但不限异步同时进行，本实施例中对此不作任何限定。
通过本申请提供的实施例，通过在本地设置页面地址缓存池，以使发送设备可以直接从本地获取页面地址，并直接在页面地址上进行RDMA操作，从而达到减少对待处理数据的处理延时。
作为一种可选的方案，在从本地的页面地址缓存池中直接获取所述页面地址之前，包括：
S1，检测页面地址缓存池中缓存的页面地址数量；
S2，若页面地址数量小于等于第二预定阈值，则通过数据传输通道获取新的页面地址。
可选地，在本实施例中，上述第二预定阈值可以但不限于根据不同的应用场景设置为不同取值。其中，节点1(即发送设备)可以为节点2设置多个页面地址缓存池，可以设置一个第二预定阈值，即检测所有页面地址缓存池中页面地址的数量的总量是否满足第二预定阈值；也可以为每个页面地址缓存池设置不同取值的第二预定阈值，即分别检测各个页面地址缓存池中页面地址的数量是否满足对应的第二预定阈值，本实施例中对此不作任何限定。
可选地，在本实施例中，通过数据传输通道获取新的页面地址包括： S22，将用于请求获取页面地址的获取请求作为待传输数据通过数据传输通道发送给接收设备；S24，获取接收设备发送的页面地址；S26，添加页面地址到页面地址缓存池中。
具体如图2所示，当检测到页面地址缓存池中的页面地址数量较小时，可以执行步骤S202-S204，向对端节点获取页面地址进行补充。具体过程可以参见上述示例，本示例在此不再赘述。
通过本申请提供的实施例，通过实时检测页面地址缓存池中的页面地址的数量，实现对页面地址缓存池中的页面地址的及时补充，从而保证在当前待处理数据正常执行RDMA操作的同时，还可以及时缓存新的页面地址，进一步实现提高数据处理效率的效果。
作为一种可选的方案，通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作包括：
S1，将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在数据传输通道在接收设备的通道缓存器中，以使接收设备利用通道缓存器中的当前待处理数据在页面地址上执行RDMA操作。
可选地，在本实施例中，在执行当前待处理数据的RDMA操作时，还可以不获取确切的页面地址，将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在接收设备的通道缓存器中，接收设备的应用会直接从上述通道缓存器中将当前待处理数据读取出来并以此拷贝到对应的页面内存中，进行RDMA操作。
可选地，在本实施例中，接收设备利用通道缓存器中的当前待处理数据在页面地址上执行RDMA操作包括：S12，接收设备将通道缓存器中的当前待处理数据拷贝到与页面地址对应页面内存中。
通过本申请提供的实施例，在数据量较小时，利用数据传输通道的通道缓存，实现直接在接收设备中完成对当前待处理数据的RDMA操作，从而达到提高数据处理效率的效果。
作为一种可选的方案，在获取当前待处理数据的数据量之前，还包括：
S1，在发送设备与接收设备之间建立数据传输通道，其中，数据传输通道连接发送设备中的控制队列与接收设备中的环形队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。
可选地，在本实施例中，在发送设备与接收设备之间建立数据传输通道包括：
S12，在控制队列及环形队列中分别设置用于指示队列中数据状态变化的指针，其中，控制队列与环形队列中的指针所指示的位置同步变化；
其中，上述指针包括：队列头指针、队列尾指针及队列接收指针，其中，队列尾指针与队列头指针之间的数据用于表示接收设备尚未确认接收的数据；队列接收指针与队列尾指针之间的数据用于表示接收设备已确认接收，且尚未处理的数据。
可选地，在本实施例中，在控制队列及环形队列中分别设置队列头指针HEAD、队列尾指针TAIL及队列接收指针RECV_TAIL，通过上述指针所指示的位置的变化，实现对待传输数据的传输控制。
需要说明的是，发送设备的新消息采用加入到头部的方式进行，接收设备从尾部开始接收。发送设备判定当HEAD等于TAIL时，则队列为空，当HEAD+1等于RECV_TAIL时，则队列为满。
通过本申请提供的实施例，通过在发送和接收两侧分别建立对应的数列，实现基于数列建立数据传输通道，以实现对待处理数据的灵活传输控制。
作为一种可选的方案，控制队列与环形队列中的指针所指示的位置同步变化包括：
S1，在发送设备执行发送操作时，控制队列的队列头指针将向前移动N个数据字节，并将队列头指针所指示的位置将同步到接收设备的环形队列中，其中，N为大于等于1的自然数；
S2，在接收设备执行接收操作时，环形队列的队列尾指针将向前移动M个数据字节，并将队列尾指针所指示的位置同步到发送设备的控制队列 中，其中，M≤N，M为大于等于1的自然数；
其中，在发送设备的控制队列中队列尾指针移动到队列头指针时，更新队列接收指针的位置。
具体结合以下示例进行说明，如图3所示，图3所示的节点1和节点2之间建立的跨节点数据传输通道是基于对数据队列的控制，具体的交互流程如下：
S1，节点1向节点2发送消息时，将消息数据写入控制队列后，将控制队列的指针HEAD增加，例如，指针HEAD向前移动5个字节，并将指针HEAD所指示的位置同步给节点2；
S2，节点2收到节点1的中断信号或者轮询到指针HEAD变化时，进行消息处理，将消息交给节点2中的应用模块处理。消息处理后，节点2将环形队列的指针TAIL增加，例如，指针TAIL向前移动2个数据字节，则将指针TAIL所指示的位置同步给节点1；
S3，节点1收到节点2的中断信号或者轮询到指针TAIL变化时，说明已发送待确认的队列里有数据，节点1处理已发送待确认队列里的消息，通知节点1中的应用模块消息处理完成，则节点1将更新指针RECV_TAIL的位置，即释放指针RECV_TAIL的当前位置。
需要说明的是，图3所示的实线表示在本侧执行处理后得到的位置，虚线表示在对侧执行处理后同步得到的位置。
通过本申请提供的实施例，同步两侧队列中指针所指示的位置，从而实现准确控制所传输的数据的传输状态，达到作为数据传输通道准确完成数据传输的目的。
可选地，在本实施例中，上述数据处理方法可以但不限于应用于如图4所示的系统中，系统中位于发送设备的通信模块402分别与对应的业务模块1至业务模块N相连，位于接收设备的通信模块404也分别与对应的业务模块1至业务模块N相连，发送设备和接收设备通过网络406实现RDMA操作。
作为一种可选的实施方式，如图5所示的双节点系统,节点1和节点2通过PCIe链路直连。节点间通过NTB(Non-Transport)非透明桥进行PCIe的地址域隔离。使用节点上的CPU带的DMA引擎，这个DMA引擎可以通过PCIe交换网络直接访问其它节点上的内存。
在系统上电时，节点1和节点2使用事先约定好的固定地址的访问进行访问，获取对端的跨节点消息队列的状态。各个业务模块会向通信模块注册回调函数，注册的回调函数包含：
1)业务模块对应的消息处理函数将执行以下步骤：
S1，当通信模块收到目的地是该业务模块(通过模块号或者端口号来区分)时，调用业务模块注册的消息处理函数处理消息接收。
2)业务模块的页面内存申请、释放函数将执行以下步骤：
S1，通信模块会调用业务模块的页面内存申请函数申请业务模块的内存，并填充到对端节点上的页面地址缓存池中。当对端节点故障或者离线时，通信模块会调用业务模块的页面释放函数释放分配到目的节点上的内存。
3)消息发送结果通知函数将执行以下步骤：
S1，消息发送给对端，对端回了响应后，通信模块会调用业务模块注册的消息发送结果通知函数来通知业务模块，消息已经送达。
业务模块注册成功回调函数，通信模块完成节点间的信息交互后，一次消息的交互过程如下：
S1，业务模块1调用通信模块的接口向对端节点上的业务模块2发送消息；
S2，通信模块接收到业务模块1的消息，根据本端保存的对端的队列信息，将消息发送到对端节点上的消息内存中；
S3，累计发送1个或者多个消息后，待发送队列为空时，节点1上的通信模块更新节点2上队列的HEAD指针；
S4，节点2检测到HEAD指针变化，将队列中的消息拷贝出来，根据消息的目的模块，交给业务模块2进行处理；
S5，节点2上的接收队列处理为空的时候，节点2更新节点1上的TAIL指针；
S6，节点1检测到TAIL指针变化时，处理节点1上的已发送待确认队列，直到队列为空，更新RECV_TAIL指针。
作为另一种可选的实施方式，如图6所示的多节点系统,节点1和节点7通过PCIe交换机进行连接。为了保持冗余，防止单点故障，PCIe交换机通常采用冗余的双星架构进行配置。每个节点出两条链路分别连接到两台PCIe交换机。本实施例中以单链路的情况为例进行具体描述。节点间通过NTB(Non-Transport)非透明桥进行PCIe的地址域隔离，PCIe交换机会为每个节点分配不同的地址域。使用PCIe交换器上带的DMA引擎进行RDMA操作，这个DMA引擎可以通过PCIe交换网络直接访问其它节点上的内存。
节点间使用带外(以太网)链路和集群内的其它节点进行信息交互，获取各个节点上跨节点的生产者消费者队列的地址信息，以及节点的状态。各个业务模块会向通信模块注册回调函数，注册的回调函数包含：
1)业务模块对应的消息处理函数将执行以下步骤：
S1，当通信模块收到目的地是该业务模块(通过模块号或者端口号来区分)时，调用业务模块注册的消息处理函数处理消息接收。
2)业务模块的页面内存申请、释放函数执行以下步骤：
S1，通信模块会调用业务模块的页面内存申请函数申请业务模块的内存，并填充到对端节点上的页面内存缓冲区，图3中的页面池。当对端节点故障或者离线时，通信模块会调用业务模块的页面释放函数释放分配到目的节点上的内存。
3)消息发送结果通知函数执行以下步骤：
S1，消息发送给对端，对端回了响应后，通信模块会调用业务模块注册的消息发送结果通知函数来通知业务模块，消息已经送达。
业务模块注册成功回调函数，通信模块完成节点间的信息交互后，一次带页面的消息交互过程如下：
S1，节点1上的业务模块1调用通信模块的接口向节点7上的业务模块2发送带页面数据的消息；
S2，通信模块接收到业务模块1的消息，向本地保存的节点7上的业务模块2的页面地址缓存池申请页面地址；
S3，将页面数据写到对端的页面内存，并将页面地址，消息发送到对端节点上的消息内存中；
S4，累计发送1个或者多个消息后，待发送队列为空时，节点1上的通信模块更新节点7上队列的HEAD指针；
S5，节点7检测到HEAD指针变化，将队列中的消息拷贝出来，根据消息的目的模块，交给业务模块2进行处理；
S6，节点7上的接收队列处理为空的时候，节点7更新节点1上的TAIL指针；
S7，节点1检测到TAIL指针变化时，处理节点1上的已发送待确认队列，直到队列为空，更新RECV_TAIL指针。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本发明的技术方案本质上或者说对相关技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本发明各个实施例所述的方法。
实施例2
在本实施例中提供了一种数据处理方法，图7是根据本发明实施例的数据处理方法的流程图，如图7所示，该流程包括如下步骤：
S702，在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收发送设备直接在与当前待处理数据对应的页面地址上执行当前待处理数据的远程直接数据存取RDMA操作，其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据；
S704，在当前待处理数据的数据量小于等于第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据处理方法可以但不限于应用于PCIe(PCI-Express)链路的节点通信过程中，发送设备在获取到当前待处理数据的数据量后，将根据上述数据量选择不同的数据处理方式：在数据量小于等于第一预定阈值时，由于数据开销较小，则可以将当前待处理数据通过预先建立的数据传输通道发送给接收设备，以使接收设备进行内存拷贝，以完成在页面地址对应的通道缓存上对当前待处理数据的远程直接数据存取RDMA操作；在数据量大于第一预定阈值时，由于数据开销较大，则可以在获取目的节点上的页面地址后，直接在该页面地址上执行当前待处理数据的RDMA操作。其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据。通过根据当前待处理数据的数据量选择合理的数据处理方式，从而实现提高数据处理效率的效果，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低。
需要说明的是，由于一次发送请求并接收到请求响应的交互时长固定，如果一次传输的数据量较小时，显然将使得数据传输通道的利用率变低，因而，在本实施例中，当数据量(即开销)较小时，就可以直接通过上述数据传输通道将当前待处理数据传输给接收设备，以使接收设备通过一次 内存拷贝完成数据处理，从而实现对数据处理过程的硬件加速，而无需预先获取执行RDMA操作的页面地址，也避免了对数据的多次拷贝过程。也就是说，当数据量较小时，数据的处理时长将根据数据的拷贝时长决定。进一步，当数据量(即开销)较大时，执行拷贝所需的时间很长，因而，则采用直接在页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据传输通道可以但不限于是基于直接访问对端内存的消息，在节点之间(如发送设备和接收设备之间)建立的跨节点通信通道。其中，上述跨节点建立的上述数据传输通道可以但不限于是基于两侧的数据队列建立。具体而言，在接收设备设置一个环形队列，在发送设备设置一个用于控制环形队列控制队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。也就是说，通过直接控制发送和接收两侧的数据队列，实现对当前待处理数据的传输控制。例如，同步更新发送和接收两侧的数据队列中的数据指针所指示的位置，以达到准确控制当前待处理数据的传输状态。
通过本申请提供的实施例，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。
作为一种可选的方案，在接收发送设备直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作之前，还包括：
S1，接收发送设备发送的用于请求获取页面地址的获取请求；
S2，响应获取请求发送页面地址。
通过本申请提供的实施例，响应发送设备发送的获取请求，向发送设备发送页面地址，从而保证在当前待处理数据正常执行RDMA操作的同时，还可以及时向发送设备缓存新的页面地址，进一步实现提高数据处理效率的效果。
作为一种可选的方案，通过预先建立的数据传输通道在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作包括：
S1，接收当前待处理数据；
S2，将待处理数据保存在数据传输通道在接收设备的通道缓存器中；
S3，将通道缓存器中的当前待处理数据一次拷贝到与页面地址对应页面内存中。
可选地，在本实施例中，在执行当前待处理数据的RDMA操作时，还可以不获取确切的页面地址，将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在接收设备的通道缓存器中，接收设备的应用会直接从上述通道缓存器中将当前待处理数据读取出来并以此拷贝到对应的页面内存中，进行RDMA操作。
通过本申请提供的实施例，在数据量较小时，利用数据传输通道的通道缓存，实现直接在接收设备中完成对当前待处理数据的RDMA操作，从而达到提高数据处理效率的效果。
作为一种可选的方案，还包括：
S1，预先在发送设备与接收设备之间建立数据传输通道，其中，数据传输通道连接发送设备中的控制队列与接收设备中的环形队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。
可选地，在本实施例中，在发送设备与接收设备之间建立数据传输通道包括：
S1，在控制队列及环形队列中分别设置用于指示队列中数据状态变化的指针，其中，控制队列与环形队列中的指针所指示的位置同步变化；
其中，指针包括：队列头指针、队列尾指针及队列接收指针，其中，队列尾指针与队列头指针之间的数据用于表示接收设备尚未确认接收的数据；队列接收指针与队列尾指针之间的数据用于表示接收设备已确认接收，且尚未处理的数据。
可选地，在本实施例中，在控制队列及环形队列中分别设置队列头指针HEAD、队列尾指针TAIL及队列接收指针RECV_TAIL，通过上述指针所指示的位置的变化，实现对待传输数据的传输控制。
需要说明的是，发送设备的新消息采用加入到头部的方式进行，接收设备从尾部开始接收。发送设备判定当HEAD等于TAIL时，则队列为空，当HEAD+1等于RECV_TAIL时，则队列为满。
通过本申请提供的实施例，通过在发送和接收两侧分别建立对应的数列，实现基于数列建立数据传输通道，以实现对待处理数据的灵活传输控制。
作为一种可选的方案，控制队列与环形队列中的指针所指示的位置同步变化包括：
S1，在发送设备执行发送操作时，控制队列的队列头指针将向前移动N个数据字节，并将队列头指针所指示的位置将同步到接收设备的环形队列中，其中，N为大于等于1的自然数；
S2，在接收设备执行接收操作时，环形队列的队列尾指针将向前移动M个数据字节，并将队列尾指针所指示的位置同步到发送设备的控制队列中，其中，M≤N，M为大于等于1的自然数；
其中，在发送设备的控制队列中队列尾指针移动到队列头指针时，更新队列接收指针的位置。
具体结合以下示例进行说明，如图3所示，图3所示的节点1和节点2之间建立的跨节点数据传输通道是基于对数据队列的控制，具体的交互流程如下：
S1，节点1向节点2发送消息时，将消息数据写入控制队列后，将控 制队列的指针HEAD增加，例如，指针HEAD向前移动5个字节，并将指针HEAD所指示的位置同步给节点2；
S2，节点2收到节点1的中断信号或者轮询到指针HEAD变化时，进行消息处理，将消息交给节点2中的应用模块处理。消息处理后，节点2将环形队列的指针TAIL增加，例如，指针TAIL向前移动2个数据字节，则将指针TAIL所指示的位置同步给节点1；
S3，节点1收到节点2的中断信号或者轮询到指针TAIL变化时，说明已发送待确认的队列里有数据，节点1处理已发送待确认队列里的消息，通知节点1中的应用模块消息处理完成，则节点1将更新指针RECV_TAIL的位置，即释放指针RECV_TAIL的当前位置。
需要说明的是，图3所示的实线表示在本侧执行处理后得到的位置，虚线表示在对侧执行处理后同步得到的位置。
通过本申请提供的实施例，同步两侧队列中指针所指示的位置，从而实现准确控制所传输的数据的传输状态，达到作为数据传输通道准确完成数据传输的目的。
实施例3
在本实施例中提供了一种数据处理装置，图8是根据本发明实施例的数据处理装置的示意图，如图8所示，该装置包括：
1)第一获取单元802，设置为获取当前待处理数据的数据量，其中，当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；
2)第一处理单元804，设置为在当前待处理数据的数据量大于第一预定阈值时，直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作；
3)第二处理单元806，设置为在当前待处理数据的数据量小于等于第一预定阈值时，通过预先建立的数据传输通道在接收设备中的页面地址上 执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据处理方法可以但不限于应用于PCIe(PCI-Express)链路的节点通信过程中，发送设备在获取到当前待处理数据的数据量后，将根据上述数据量选择不同的数据处理方式：在数据量小于等于第一预定阈值时，由于数据开销较小，则可以将当前待处理数据通过预先建立的数据传输通道发送给接收设备，以使接收设备进行内存拷贝，以完成在页面地址对应的通道缓存上对当前待处理数据的远程直接数据存取RDMA操作；在数据量大于第一预定阈值时，由于数据开销较大，则可以在获取目的节点上的页面地址后，直接在该页面地址上执行当前待处理数据的RDMA操作。其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据。通过根据当前待处理数据的数据量选择合理的数据处理方式，从而实现提高数据处理效率的效果，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低。
需要说明的是，由于一次发送请求并接收到请求响应的交互时长固定，如果一次传输的数据量较小时，显然将使得数据传输通道的利用率变低，因而，在本实施例中，当数据量(即开销)较小时，就可以直接通过上述数据传输通道将当前待处理数据传输给接收设备，以使接收设备通过一次内存拷贝完成数据处理，从而实现对数据处理过程的硬件加速，而无需预先获取执行RDMA操作的页面地址，也避免了对数据的多次拷贝过程。也就是说，当数据量较小时，数据的处理时长将根据数据的拷贝时长决定。进一步，当数据量(即开销)较大时，执行拷贝所需的时间很长，因而，则采用直接在页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，在直接在页面地址上执行当前待处理数据的RDMA操作之前，还需获取页面地址，其中，上述页面地址的获取方式可以包括但不限于以下至少之一：通过数据传输通道向接收设备发送用于获取页面地址的获取请求、从本地的页面地址缓存池中直接获取已缓存的页面地址。
可选地，在本实施例中，上述页面地址缓存池中的页面地址可以通过以下方式获取：检测页面地址缓存池中缓存的页面地址数量；若页面地址数量小于等于第二预定阈值，则通过数据传输通道获取新的页面地址。
需要说明的是，在本实施例中，上述页面地址缓存池将根据当前的缓存量，对池中的页面地址进行及时添加更新。从而实现在数据量较大时，避免每次都向接收设备请求获取页面地址所造成的处理延时的问题。进一步，在获取新的页面地址添加到上述页面地址缓存池时，并不影响对当前待处理数据正常的RDMA操作，也就是说，二者可以异步同时进行，从而进一步实现提高数据处理效率的效果。
可选地，在本实施例中，上述数据传输通道可以但不限于是基于直接访问对端内存的消息，在节点之间(如发送设备和接收设备之间)建立的跨节点通信通道。其中，上述跨节点建立的上述数据传输通道可以但不限于是基于两侧的数据队列建立。具体而言，在接收设备设置一个环形队列，在发送设备设置一个用于控制环形队列控制队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。也就是说，通过直接控制发送和接收两侧的数据队列，实现对当前待处理数据的传输控制。例如，同步更新发送和接收两侧的数据队列中的数据指针所指示的位置，以达到准确控制当前待处理数据的传输状态。
通过本申请提供的实施例，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。
作为一种可选的方案，还包括：
1)第二获取单元，设置为在直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作之前，从本地的页面地址缓存池中直接获取页面地址，其中，页面地址缓存池用于缓存一个或多个页面地址。
可选地，在本实施例中，上述页面地址缓存池可以但不限于根据不同的业务(也称之为应用)设置多个不同的页面地址缓存池。如图2所示，可以划分为页面地址缓存池202-1至页面地址缓存池202-N。
可选地，在本实施例中，上述页面地址缓存池中的页面地址可以但不限于在由对端(即接收设备)获取后，添加更新到本地页面地址缓存池中。其中，页面地址缓存池获取页面地址的方式可以包括但不限于以下至少之一：节点1检测到缓存量低于预定阈值时，通过数据传输通道向节点2请求补充页面地址；节点2通过数据传输通道主动按照预定周期为节点1补充新的页面地址。
具体结合以下示例进行说明，如图2所示，发送设备以节点1为例，接收设备以节点2为例，如步骤S206-S208，当节点1向节点2发送数据时，可以直接向页面地址缓存池申请获取页面地址而无需向节点2发送申请获取的请求，从而减少节点1每次获取页面地址的时间，进而达到减少处理延时的效果。
进一步，如步骤S202-S204，上述节点1为对端节点2可以设置多个页面地址缓存池，如页面地址缓存池202-1至页面地址缓存池202-N。当检测到一个缓存池中的缓存量低于预定阈值时，则可以通过数据传输通道向对端节点2发送获取请求，以实现由节点2为节点1补充新的页面地址。
需要说明的是，在本示例中，上述节点1获取页面地址的过程(即步骤S202-S204)与页面地址缓存池获取页面地址的过程(即步骤S206-S208)并不限于如图所示的顺序，上述两个过程可以但不限异步同时进行，本实施例中对此不作任何限定。
通过本申请提供的实施例，通过在本地设置页面地址缓存池，以使发送设备可以直接从本地获取页面地址，并直接在页面地址上进行RDMA操作，从而达到减少对待处理数据的处理延时。
作为一种可选的方案，还包括：
1)检测单元，设置为在从本地的页面地址缓存池中直接获取页面地址之前，检测页面地址缓存池中缓存的页面地址数量；
2)第三获取单元，设置为在页面地址数量小于等于第二预定阈值时，通过数据传输通道获取新的页面地址。
可选地，在本实施例中，上述第二预定阈值可以但不限于根据不同的应用场景设置为不同取值。其中，节点1(即发送设备)可以为节点2设置多个页面地址缓存池，可以设置一个第二预定阈值，即检测所有页面地址缓存池中页面地址的数量的总量是否满足第二预定阈值；也可以为每个页面地址缓存池设置不同取值的第二预定阈值，即分别检测各个页面地址缓存池中页面地址的数量是否满足对应的第二预定阈值，本实施例中对此不作任何限定。
可选地，在本实施例中，第三获取单元包括：1)发送模块，设置为将用于请求获取页面地址的获取请求作为待传输数据通过数据传输通道发送给接收设备；2)获取模块，设置为获取接收设备发送的页面地址；3)添加模块，设置为添加页面地址到页面地址缓存池中。
具体如图2所示，当检测到页面地址缓存池中的页面地址数量较小时，可以执行步骤S202-S204，向对端节点获取页面地址进行补充。具体过程可以参见上述示例，本示例在此不再赘述。
通过本申请提供的实施例，通过实时检测页面地址缓存池中的页面地址的数量，实现对页面地址缓存池中的页面地址的及时补充，从而保证在当前待处理数据正常执行RDMA操作的同时，还可以及时缓存新的页面地址，进一步实现提高数据处理效率的效果。
作为一种可选的方案，第二处理单元包括：
1)处理模块，设置为将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在数据传输通道在接收设备的通道缓存器中，以使接收设备利用通道缓存器中的当前待处理数据在页面地址上执行RDMA操作。
可选地，在本实施例中，在执行当前待处理数据的RDMA操作时，还可以不获取确切的页面地址，将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在接收设备的通道缓存器中，接收设备的应用会直接从上述通道缓存器中将当前待处理数据读取出来并以此拷贝到对应的页面内存中，进行RDMA操作。
通过本申请提供的实施例，在数据量较小时，利用数据传输通道的通道缓存，实现直接在接收设备中完成对当前待处理数据的RDMA操作，从而达到提高数据处理效率的效果。
作为一种可选的方案，还包括：
1)建立单元，设置为在获取当前待处理数据的数据量之前，在发送设备与接收设备之间建立数据传输通道，其中，数据传输通道连接发送设备中的控制队列与接收设备中的环形队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。
可选地，在本实施例中，建立单元包括：
(1)设置模块，设置为在控制队列及环形队列中分别设置用于指示队列中数据状态变化的指针，其中，控制队列与环形队列中的指针所指示的位置同步变化；
其中，指针包括：队列头指针、队列尾指针及队列接收指针，其中，队列尾指针与队列头指针之间的数据用于表示接收设备尚未确认接收的数据；队列接收指针与队列尾指针之间的数据用于表示接收设备已确认接收，且尚未处理的数据。
可选地，在本实施例中，在控制队列及环形队列中分别设置队列头指针HEAD、队列尾指针TAIL及队列接收指针RECV_TAIL，通过上述指 针所指示的位置的变化，实现对待传输数据的传输控制。
需要说明的是，发送设备的新消息采用加入到头部的方式进行，接收设备从尾部开始接收。发送设备判定当HEAD等于TAIL时，则队列为空，当HEAD+1等于RECV_TAIL时，则队列为满。
通过本申请提供的实施例，通过在发送和接收两侧分别建立对应的数列，实现基于数列建立数据传输通道，以实现对待处理数据的灵活传输控制。
作为一种可选的方案，设置模块通过以下方式控制控制队列与环形队列中的指针所指示的位置同步变化包括：
1)在发送设备执行发送操作时，控制队列的队列头指针将向前移动N个数据字节，并将队列头指针所指示的位置将同步到接收设备的环形队列中，其中，N为大于等于1的自然数；
2)在接收设备执行接收操作时，环形队列的队列尾指针将向前移动M个数据字节，并将队列尾指针所指示的位置同步到发送设备的控制队列中，其中，M≤N，M为大于等于1的自然数；
其中，在发送设备的控制队列中队列尾指针移动到队列头指针时，更新队列接收指针的位置。
具体结合以下示例进行说明，如图3所示，图3所示的节点1和节点2之间建立的跨节点数据传输通道是基于对数据队列的控制，具体的交互流程如下：
S1，节点1向节点2发送消息时，将消息数据写入控制队列后，将控制队列的指针HEAD增加，例如，指针HEAD向前移动5个字节，并将指针HEAD所指示的位置同步给节点2；
S2，节点2收到节点1的中断信号或者轮询到指针HEAD变化时，进行消息处理，将消息交给节点2中的应用模块处理。消息处理后，节点2将环形队列的指针TAIL增加，例如，指针TAIL向前移动2个数据字节，则将指针TAIL所指示的位置同步给节点1；
S3，节点1收到节点2的中断信号或者轮询到指针TAIL变化时，说明已发送待确认的队列里有数据，节点1处理已发送待确认队列里的消息，通知节点1中的应用模块消息处理完成，则节点1将更新指针RECV_TAIL的位置，即释放指针RECV_TAIL的当前位置。
需要说明的是，图3所示的实线表示在本侧执行处理后得到的位置，虚线表示在对侧执行处理后同步得到的位置。
通过本申请提供的实施例，同步两侧队列中指针所指示的位置，从而实现准确控制所传输的数据的传输状态，达到作为数据传输通道准确完成数据传输的目的。
实施例4
在本实施例中提供了一种数据处理装置，图9是根据本发明实施例的数据处理装置的示意图，如图9所示，该装置包括：
1)第一处理单元902，设置为在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收发送设备直接在与当前待处理数据对应的页面地址上执行当前待处理数据的远程直接数据存取RDMA操作，其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据；
2)第二处理单元904，设置为在当前待处理数据的数据量小于等于第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据处理方法可以但不限于应用于PCIe(PCI-Express)链路的节点通信过程中，发送设备在获取到当前待处理数据的数据量后，将根据上述数据量选择不同的数据处理方式：在数据量小于等于第一预定阈值时，由于数据开销较小，则可以将当前待处理数据通过预先建立的数据传输通道发送给接收设备，以使接收设备进行内存拷贝，以完成在页面地址对应的通道缓存上对当前待处理数据的远程直接数据存取RDMA操作；在数据量大于第一预定阈值时，由于数据开销较大，则可以在获取目的节点上的页面地址后，直接在该页面地址上执行当前待 处理数据的RDMA操作。其中，当前待处理数据包括当前执行RDMA操作的输入输出I/O数据。通过根据当前待处理数据的数据量选择合理的数据处理方式，从而实现提高数据处理效率的效果，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低。
需要说明的是，由于一次发送请求并接收到请求响应的交互时长固定，如果一次传输的数据量较小时，显然将使得数据传输通道的利用率变低，因而，在本实施例中，当数据量(即开销)较小时，就可以直接通过上述数据传输通道将当前待处理数据传输给接收设备，以使接收设备通过一次内存拷贝完成数据处理，从而实现对数据处理过程的硬件加速，而无需预先获取执行RDMA操作的页面地址，也避免了对数据的多次拷贝过程。也就是说，当数据量较小时，数据的处理时长将根据数据的拷贝时长决定。进一步，当数据量(即开销)较大时，执行拷贝所需的时间很长，因而，则采用直接在页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述数据传输通道可以但不限于是基于直接访问对端内存的消息，在节点之间(如发送设备和接收设备之间)建立的跨节点通信通道。其中，上述跨节点建立的上述数据传输通道可以但不限于是基于两侧的数据队列建立。具体而言，在接收设备设置一个环形队列，在发送设备设置一个用于控制环形队列控制队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。也就是说，通过直接控制发送和接收两侧的数据队列，实现对当前待处理数据的传输控制。例如，同步更新发送和接收两侧的数据队列中的数据指针所指示的位置，以达到准确控制当前待处理数据的传输状态。
通过本申请提供的实施例，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当 前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。
作为一种可选的方案，还包括：
1)接收单元，设置为在接收发送设备直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作之前，接收发送设备发送的用于请求获取页面地址的获取请求；
2)发送单元，设置为响应获取请求发送页面地址。
通过本申请提供的实施例，响应发送设备发送的获取请求，向发送设备发送页面地址，从而保证在当前待处理数据正常执行RDMA操作的同时，还可以及时向发送设备缓存新的页面地址，进一步实现提高数据处理效率的效果。
作为一种可选的方案，第二处理单元包括：
1)接收模块，设置为接收当前待处理数据；
2)保存模块，设置为将待处理数据保存在数据传输通道在接收设备的通道缓存器中；
3)拷贝模块，设置为将通道缓存器中的当前待处理数据拷贝到与页面地址对应页面内存中。
可选地，在本实施例中，在执行当前待处理数据的RDMA操作时，还可以不获取确切的页面地址，将当前待处理数据作为待传输数据通过数据传输通道直接发送给接收设备，并保存在接收设备的通道缓存器中，接收设备的应用会直接从上述通道缓存器中将当前待处理数据读取出来并以此拷贝到对应的页面内存中，进行RDMA操作。
通过本申请提供的实施例，在数据量较小时，利用数据传输通道的通道缓存，实现直接在接收设备中完成对当前待处理数据的RDMA操作， 从而达到提高数据处理效率的效果。
作为一种可选的方案，还包括：
1)建立单元，设置为预先在发送设备与接收设备之间建立数据传输通道，其中，数据传输通道连接发送设备中的控制队列与接收设备中的环形队列，其中，控制队列包括用于控制环形队列中数据状态变化的控制信息。
可选地，在本实施例中，建立单元包括：
1)设置模块，设置为在控制队列及环形队列中分别设置用于指示队列中数据状态变化的指针，其中，控制队列与环形队列中的指针所指示的位置同步变化；
其中，指针包括：队列头指针、队列尾指针及队列接收指针，其中，队列尾指针与队列头指针之间的数据用于表示接收设备尚未确认接收的数据；队列接收指针与队列尾指针之间的数据用于表示接收设备已确认接收，且尚未处理的数据。
可选地，在本实施例中，在控制队列及环形队列中分别设置队列头指针HEAD、队列尾指针TAIL及队列接收指针RECV_TAIL，通过上述指针所指示的位置的变化，实现对待传输数据的传输控制。
需要说明的是，发送设备的新消息采用加入到头部的方式进行，接收设备从尾部开始接收。发送设备判定当HEAD等于TAIL时，则队列为空，当HEAD+1等于RECV_TAIL时，则队列为满。
通过本申请提供的实施例，通过在发送和接收两侧分别建立对应的数列，实现基于数列建立数据传输通道，以实现对待处理数据的灵活传输控制。
作为一种可选的方案，设置模块通过以下方式实现控制队列与环形队列中的指针所指示的位置同步变化包括：
1)在发送设备执行发送操作时，控制队列的队列头指针将向前移动 N个数据字节，并将队列头指针所指示的位置将同步到接收设备的环形队列中，其中，N为大于等于1的自然数；
2)在接收设备执行接收操作时，环形队列的队列尾指针将向前移动M个数据字节，并将队列尾指针所指示的位置同步到发送设备的控制队列中，其中，M≤N，M为大于等于1的自然数；
其中，在发送设备的控制队列中队列尾指针移动到队列头指针时，更新队列接收指针的位置。
具体结合以下示例进行说明，如图3所示，图3所示的节点1和节点2之间建立的跨节点数据传输通道是基于对数据队列的控制，具体的交互流程如下：
S1，节点1向节点2发送消息时，将消息数据写入控制队列后，将控制队列的指针HEAD增加，例如，指针HEAD向前移动5个字节，并将指针HEAD所指示的位置同步给节点2；
S2，节点2收到节点1的中断信号或者轮询到指针HEAD变化时，进行消息处理，将消息交给节点2中的应用模块处理。消息处理后，节点2将环形队列的指针TAIL增加，例如，指针TAIL向前移动2个数据字节，则将指针TAIL所指示的位置同步给节点1；
S3，节点1收到节点2的中断信号或者轮询到指针TAIL变化时，说明已发送待确认的队列里有数据，节点1处理已发送待确认队列里的消息，通知节点1中的应用模块消息处理完成，则节点1将更新指针RECV_TAIL的位置，即释放指针RECV_TAIL的当前位置。
需要说明的是，图3所示的实线表示在本侧执行处理后得到的位置，虚线表示在对侧执行处理后同步得到的位置。
通过本申请提供的实施例，同步两侧队列中指针所指示的位置，从而实现准确控制所传输的数据的传输状态，达到作为数据传输通道准确完成数据传输的目的。
实施例5
本发明的实施例还提供了一种存储介质。可选地，在本实施例中，上述存储介质可以被设置为存储用于执行以下步骤的程序代码：
S1，获取当前待处理数据的数据量，其中，当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；
S2，若当前待处理数据的数据量大于第一预定阈值，则直接在与当前待处理数据对应的页面地址上执行当前待处理数据的RDMA操作；
S3，若当前待处理数据的数据量小于等于第一预定阈值，则通过预先建立的数据传输通道在接收设备中的页面地址上执行当前待处理数据的RDMA操作。
可选地，在本实施例中，上述存储介质可以包括但不限于：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
可选地，本实施例中的具体示例可以参考上述实施例及可选实施方式中所描述的示例，本实施例在此不再赘述。
显然，本领域的技术人员应该明白，上述的本发明的各模块或各步骤可以用通用的计算装置来实现，它们可以集中在单个的计算装置上，或者分布在多个计算装置所组成的网络上，可选地，它们可以用计算装置可执行的程序代码来实现，从而，可以将它们存储在存储装置中由计算装置来执行，并且在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤，或者将它们分别制作成各个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。这样，本发明不限制于任何特定的硬件和软件结合。
以上所述仅为本发明的优选实施例而已，并不用于限制本发明，对于本领域的技术人员来说，本发明可以有各种更改和变化。凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明 的保护范围之内。
工业实用性
在本发明实施例中，通过根据当前待处理数据的数据量选择合理的数据处理方式：在当前待处理数据的数据量较大时，采用直接在页面地址上执行当前待处理数据的RDMA操作，而无需每次都通过交互获取对应的页面地址，从而达到减少数据交互的目的；在当前待处理数据的数据量较小时，直接通过数据传输通道在接收设备中通过内存拷贝完成对当前待处理数据的RDMA操作，从而实现对数据处理过程的硬件加速的效果。通过根据不同的数据开销，选择合理的数据处理方式，以克服相关技术中只能通过多次交互获取对端物理地址的方式执行RDMA操作所所导致的数据处理效率较低，从而实现提高数据处理效率的效果。

Claims (26)

  1. 一种数据处理方法，包括：
    获取当前待处理数据的数据量，其中，所述当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；
    若所述当前待处理数据的数据量大于第一预定阈值，则直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作；
    若所述当前待处理数据的数据量小于等于所述第一预定阈值，则通过预先建立的数据传输通道在接收设备中的所述页面地址上执行所述当前待处理数据的所述RDMA操作。
  2. 根据权利要求1所述的方法，其中，在直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作之前，还包括：
    从本地的页面地址缓存池中直接获取所述页面地址，其中，所述页面地址缓存池用于缓存一个或多个页面地址。
  3. 根据权利要求2所述的方法，其中，在从本地的页面地址缓存池中直接获取所述页面地址之前，包括：
    检测所述页面地址缓存池中缓存的页面地址数量；
    若所述页面地址数量小于等于第二预定阈值，则通过所述数据传输通道获取新的页面地址。
  4. 根据权利要求3所述的方法，其中，所述通过所述数据传输通道获取新的页面地址包括：
    将用于请求获取所述页面地址的获取请求作为待传输数据通过所述数据传输通道发送给所述接收设备；
    获取所述接收设备发送的所述页面地址；
    添加所述页面地址到所述页面地址缓存池中。
  5. 根据权利要求1所述的方法，其中，所述通过预先建立的数 据传输通道在接收设备中的所述页面地址上执行所述当前待处理数据的所述RDMA操作包括：
    将所述当前待处理数据作为待传输数据通过所述数据传输通道直接发送给所述接收设备，并保存在所述数据传输通道在所述接收设备的通道缓存器中，以使所述接收设备利用所述通道缓存器中的所述当前待处理数据在所述页面地址上执行所述RDMA操作。
  6. 根据权利要求5所述的方法，其中，所述接收设备利用所述通道缓存器中的所述当前待处理数据在所述页面地址上执行所述RDMA操作包括：
    所述接收设备将所述通道缓存器中的所述当前待处理数据拷贝到与所述页面地址对应页面内存中。
  7. 根据权利要求1所述的方法，其中，在获取当前待处理数据的数据量之前，还包括：
    在发送设备与所述接收设备之间建立所述数据传输通道，其中，所述数据传输通道连接所述发送设备中的控制队列与所述接收设备中的环形队列，其中，所述控制队列包括用于控制所述环形队列中数据状态变化的控制信息。
  8. 根据权利要求7所述的方法，其中，所述在发送设备与所述接收设备之间建立所述数据传输通道包括：
    在所述控制队列及所述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，所述控制队列与所述环形队列中的所述指针所指示的位置同步变化；
    其中，所述指针包括：队列头指针、队列尾指针及队列接收指针，其中，所述队列尾指针与所述队列头指针之间的数据用于表示所述接收设备尚未确认接收的数据；所述队列接收指针与所述队列尾指针之间的数据用于表示所述接收设备已确认接收，且尚未处理的数据。
  9. 根据权利要求8所述的方法，其中，所述控制队列与所述环 形队列中的所述指针所指示的位置同步变化包括：
    在所述发送设备执行发送操作时，所述控制队列的所述队列头指针将向前移动N个数据字节，并将所述队列头指针所指示的位置将同步到所述接收设备的所述环形队列中，其中，所述N为大于等于1的自然数；
    在所述接收设备执行接收操作时，所述环形队列的所述队列尾指针将向前移动M个数据字节，并将所述队列尾指针所指示的位置同步到所述发送设备的所述控制队列中，其中，M≤N，M为大于等于1的自然数；
    其中，在所述发送设备的所述控制队列中所述队列尾指针移动到所述队列头指针时，更新所述队列接收指针的位置。
  10. 一种数据处理方法，包括：
    在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收所述发送设备直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的远程直接数据存取RDMA操作，其中，所述当前待处理数据包括当前执行所述RDMA操作的输入输出I/O数据；
    在所述当前待处理数据的数据量小于等于所述第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的所述页面地址上执行所述当前待处理数据的所述RDMA操作。
  11. 根据权利要求10所述的方法，其中，在接收所述发送设备直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作之前，还包括：
    接收所述发送设备发送的用于请求获取所述页面地址的获取请求；
    响应所述获取请求发送所述页面地址。
  12. 根据权利要求10所述的方法，其中，所述通过预先建立的数据传输通道在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作包括：
    接收所述当前待处理数据；
    将所述待处理数据保存在所述数据传输通道在所述接收设备的通道缓存器中；
    将所述通道缓存器中的所述当前待处理数据一次拷贝到与所述页面地址对应页面内存中。
  13. 根据权利要求10所述的方法，其中，还包括：
    预先在发送设备与所述接收设备之间建立所述数据传输通道，其中，所述数据传输通道连接所述发送设备中的控制队列与所述接收设备中的环形队列，其中，所述控制队列包括用于控制所述环形队列中数据状态变化的控制信息。
  14. 根据权利要求13所述的方法，其中，所述在发送设备与所述接收设备之间建立所述数据传输通道包括：
    在所述控制队列及所述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，所述控制队列与所述环形队列中的所述指针所指示的位置同步变化；
    其中，所述指针包括：队列头指针、队列尾指针及队列接收指针，其中，所述队列尾指针与所述队列头指针之间的数据用于表示所述接收设备尚未确认接收的数据；所述队列接收指针与所述队列尾指针之间的数据用于表示所述接收设备已确认接收，且尚未处理的数据。
  15. 根据权利要求14所述的方法，其中，所述控制队列与所述环形队列中的所述指针所指示的位置同步变化包括：
    在所述发送设备执行发送操作时，所述控制队列的所述队列头指针将向前移动N个数据字节，并将所述队列头指针所指示的位置将同 步到所述接收设备的所述环形队列中，其中，所述N为大于等于1的自然数；
    在所述接收设备执行接收操作时，所述环形队列的所述队列尾指针将向前移动M个数据字节，并将所述队列尾指针所指示的位置同步到所述发送设备的所述控制队列中，其中，M≤N，M为大于等于1的自然数；
    其中，在所述发送设备的所述控制队列中所述队列尾指针移动到所述队列头指针时，更新所述队列接收指针的位置。
  16. 一种数据处理装置，包括：
    第一获取单元，设置为获取当前待处理数据的数据量，其中，所述当前待处理数据包括当前执行远程直接数据存取RDMA操作的输入输出I/O数据；
    第一处理单元，设置为在所述当前待处理数据的数据量大于第一预定阈值时，直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作；
    第二处理单元，设置为在所述当前待处理数据的数据量小于等于所述第一预定阈值时，通过预先建立的数据传输通道在接收设备中的所述页面地址上执行所述当前待处理数据的所述RDMA操作。
  17. 根据权利要求16所述的装置，其中，还包括：
    第二获取单元，设置为在直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作之前，从本地的页面地址缓存池中直接获取所述页面地址，其中，所述页面地址缓存池用于缓存一个或多个页面地址。
  18. 根据权利要求17所述的装置，其中，还包括：
    检测单元，设置为在从本地的页面地址缓存池中直接获取所述页面地址之前，检测所述页面地址缓存池中缓存的页面地址数量；
    第三获取单元，设置为在所述页面地址数量小于等于第二预定阈值时，通过所述数据传输通道获取新的页面地址。
  19. 根据权利要求18所述的装置，其中，所述第三获取单元包括：
    发送模块，设置为将用于请求获取所述页面地址的获取请求作为待传输数据通过所述数据传输通道发送给所述接收设备；
    获取模块，设置为获取所述接收设备发送的所述页面地址；
    添加模块，设置为添加所述页面地址到所述页面地址缓存池中。
  20. 根据权利要求16所述的装置，其中，所述第二处理单元包括：
    处理模块，设置为将所述当前待处理数据作为待传输数据通过所述数据传输通道直接发送给所述接收设备，并保存在所述数据传输通道在所述接收设备的通道缓存器中，以使所述接收设备利用所述通道缓存器中的所述当前待处理数据在所述页面地址上执行所述RDMA操作。
  21. 根据权利要求16所述的装置，其中，还包括：
    建立单元，设置为在获取当前待处理数据的数据量之前，在发送设备与所述接收设备之间建立所述数据传输通道，其中，所述数据传输通道连接所述发送设备中的控制队列与所述接收设备中的环形队列，其中，所述控制队列包括用于控制所述环形队列中数据状态变化的控制信息。
  22. 根据权利要求21所述的装置，其中，所述建立单元包括：
    设置模块，设置为在所述控制队列及所述环形队列中分别设置用于指示队列中数据状态变化的指针，其中，所述控制队列与所述环形队列中的所述指针所指示的位置同步变化；
    其中，所述指针包括：队列头指针、队列尾指针及队列接收指针， 其中，所述队列尾指针与所述队列头指针之间的数据用于表示所述接收设备尚未确认接收的数据；所述队列接收指针与所述队列尾指针之间的数据用于表示所述接收设备已确认接收，且尚未处理的数据。
  23. 根据权利要求22所述的装置，其中，所述设置模块通过以下方式控制所述控制队列与所述环形队列中的所述指针所指示的位置同步变化包括：
    在所述发送设备执行发送操作时，所述控制队列的所述队列头指针将向前移动N个数据字节，并将所述队列头指针所指示的位置将同步到所述接收设备的所述环形队列中，其中，所述N为大于等于1的自然数；
    在所述接收设备执行接收操作时，所述环形队列的所述队列尾指针将向前移动M个数据字节，并将所述队列尾指针所指示的位置同步到所述发送设备的所述控制队列中，其中，M≤N，M为大于等于1的自然数；
    其中，在所述发送设备的所述控制队列中所述队列尾指针移动到所述队列头指针时，更新所述队列接收指针的位置。
  24. 一种数据处理装置，包括：
    第一处理单元，设置为在发送设备获取到的当前待处理数据的数据量大于第一预定阈值时，接收所述发送设备直接在与所述当前待处理数据对应的页面地址上执行所述当前待处理数据的远程直接数据存取RDMA操作，其中，所述当前待处理数据包括当前执行所述RDMA操作的输入输出I/O数据；
    第二处理单元，设置为在所述当前待处理数据的数据量小于等于所述第一预定阈值时，接收通过预先建立的数据传输通道在接收设备中的所述页面地址上执行所述当前待处理数据的所述RDMA操作。
  25. 根据权利要求24所述的装置，其中，还包括：
    接收单元，设置为在接收所述发送设备直接在与所述当前待处理 数据对应的页面地址上执行所述当前待处理数据的所述RDMA操作之前，接收所述发送设备发送的用于请求获取所述页面地址的获取请求；
    发送单元，设置为响应所述获取请求发送所述页面地址。
  26. 根据权利要求24所述的装置，其中，所述第二处理单元包括：
    接收模块，设置为接收所述当前待处理数据；
    保存模块，设置为将所述待处理数据保存在所述数据传输通道在所述接收设备的通道缓存器中；
    拷贝模块，设置为将所述通道缓存器中的所述当前待处理数据拷贝到与所述页面地址对应页面内存中。
PCT/CN2017/000065 2016-01-18 2017-01-03 数据处理方法和装置 WO2017124917A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610031757.6 2016-01-18
CN201610031757.6A CN106980582B (zh) 2016-01-18 2016-01-18 数据处理方法和装置

Publications (1)

Publication Number Publication Date
WO2017124917A1 true WO2017124917A1 (zh) 2017-07-27

Family

ID=59340607

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/000065 WO2017124917A1 (zh) 2016-01-18 2017-01-03 数据处理方法和装置

Country Status (2)

Country Link
CN (1) CN106980582B (zh)
WO (1) WO2017124917A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113778914A (zh) * 2020-06-09 2021-12-10 华为技术有限公司 用于执行数据处理的装置、方法、和计算设备

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106980582B (zh) * 2016-01-18 2022-05-13 中兴通讯股份有限公司 数据处理方法和装置
CN107947799B (zh) * 2017-11-28 2021-06-29 郑州云海信息技术有限公司 一种数据压缩方法及装置
CN110865960B (zh) * 2018-08-28 2022-07-22 上海忆芯实业有限公司 在网络上模拟PCIe总线
CN109800202B (zh) * 2019-01-17 2022-02-18 郑州云海信息技术有限公司 一种基于pcie的数据传输系统、方法及装置
CN110046114B (zh) * 2019-03-06 2020-08-14 上海熠知电子科技有限公司 基于pcie协议的dma控制器及dma数据传输方法
CN110490308B (zh) * 2019-09-16 2022-02-18 普联技术有限公司 加速库的设计方法、终端设备及存储介质
CN111225042B (zh) * 2019-12-27 2021-09-17 腾讯科技（深圳）有限公司 数据传输的方法、装置、计算机设备以及存储介质
CN112162875A (zh) * 2020-10-12 2021-01-01 上交所技术有限责任公司 一种交易系统内部高可靠消息传输方法
CN112887196B (zh) * 2021-01-20 2023-03-24 远景智能国际私人投资有限公司 消息发送方法、系统、装置、设备及可读存储介质
CN115174484A (zh) * 2022-06-16 2022-10-11 阿里巴巴（中国）有限公司 基于rdma的数据传输方法、装置、设备及存储介质
CN115955437B (zh) * 2023-03-14 2023-05-30 苏州浪潮智能科技有限公司 一种数据传输方法、装置、设备及介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101599991A (zh) * 2009-07-07 2009-12-09 浪潮电子信息产业股份有限公司 一种基于高速通信协议的提升网络文件系统性能的方法
CN103051793A (zh) * 2012-12-18 2013-04-17 广东欧珀移动通信有限公司 一种移动终端控制数据传输模式的方法及装置
CN103440202A (zh) * 2013-08-07 2013-12-11 华为技术有限公司 一种基于rdma的通信方法、系统及通信设备
CN103634932A (zh) * 2012-08-27 2014-03-12 联想(北京)有限公司 一种数据传输方法及电子设备
CN104202391A (zh) * 2014-08-28 2014-12-10 浪潮(北京)电子信息产业有限公司 共享系统地址空间的非紧耦合系统间的rdma通信方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8874680B1 (en) * 2011-11-03 2014-10-28 Netapp, Inc. Interconnect delivery process
CN106980582B (zh) * 2016-01-18 2022-05-13 中兴通讯股份有限公司 数据处理方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101599991A (zh) * 2009-07-07 2009-12-09 浪潮电子信息产业股份有限公司 一种基于高速通信协议的提升网络文件系统性能的方法
CN103634932A (zh) * 2012-08-27 2014-03-12 联想(北京)有限公司 一种数据传输方法及电子设备
CN103051793A (zh) * 2012-12-18 2013-04-17 广东欧珀移动通信有限公司 一种移动终端控制数据传输模式的方法及装置
CN103440202A (zh) * 2013-08-07 2013-12-11 华为技术有限公司 一种基于rdma的通信方法、系统及通信设备
CN104202391A (zh) * 2014-08-28 2014-12-10 浪潮(北京)电子信息产业有限公司 共享系统地址空间的非紧耦合系统间的rdma通信方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113778914A (zh) * 2020-06-09 2021-12-10 华为技术有限公司 用于执行数据处理的装置、方法、和计算设备

Also Published As

Publication number Publication date
CN106980582A (zh) 2017-07-25
CN106980582B (zh) 2022-05-13

Similar Documents

Publication Publication Date Title
WO2017124917A1 (zh) 数据处理方法和装置
US11290544B2 (en) Data transmission methods applied to a proxy server or a backend server, and data transmission system
US20210117224A1 (en) Virtual machine migration while maintaining live network links
US10592464B2 (en) Methods for enabling direct memory access (DMA) capable devices for remote DMA (RDMA) usage and devices thereof
JP5607067B2 (ja) 複数のアプリケーションによる制御をサポートする無線アクセスポイント
US20150169418A1 (en) Priming Failover of Stateful Offload Adapters
JP6536677B2 (ja) Ｃｐｕおよびマルチｃｐｕシステム管理方法
EP2744166B1 (en) Method for migrating a user link, method for migrating a user service, apparatuses and system
US20220369410A1 (en) Efficient Context Handling for RRC-Inactive in 5G
WO2014177085A1 (zh) 分布式多副本数据存储方法及装置
TW200920027A (en) Intelligent failover in a load-balanced networking environment
CN104782081B (zh) 用于转移套接字状态的系统以及用于迁移tcp连接的方法
WO2014079308A1 (zh) 一种时隙资源占用处理方法及装置
US7159010B2 (en) Network abstraction of input/output devices
US11589273B2 (en) Context placement in the mobile communications network
JP2008167359A (ja) Ｉｐ電話システムにおける所分割方法，ファイル更新方法及びｉｐ電話システム
WO2019062725A1 (zh) 一种上行数据传输方法及装置
WO2023193671A1 (zh) 数据传输方法以及系统
US20140143441A1 (en) Chip multi processor and router for chip multi processor
CN113204517B (zh) 一种电力专用以太网控制器的核间共享方法
US11469955B2 (en) Stacked computer network devices having multiple master nodes
JP2010176511A (ja) ホットスタンバイシステム及びホットスタンバイ方法
JP5591737B2 (ja) サービス提供システムおよびその動作方法
KR102224327B1 (ko) Rdma 기반의 노드 간 데이터 전송 방법 및 노드 장치
WO2023124494A1 (zh) 网络切换处理方法、装置、计算机可读介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17740948

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17740948

Country of ref document: EP

Kind code of ref document: A1