CN113596085A

CN113596085A - Data processing method, system and device

Info

Publication number: CN113596085A
Application number: CN202110707903.3A
Authority: CN
Inventors: 方璟; 章颖强; 王波
Original assignee: Alibaba Cloud Computing Ltd
Current assignee: Alibaba Cloud Computing Ltd
Priority date: 2021-06-24
Filing date: 2021-06-24
Publication date: 2021-11-02

Abstract

The embodiment of the specification provides a data processing method, a system and a device, wherein the data processing method comprises the following steps: the method comprises the steps of obtaining data to be stored, inquiring network connection in an idle state in a network connection pool, determining a data buffer area corresponding to a storage address of the network connection in a central node, generating a first communication message based on the data to be stored, calling a remote direct memory access module to write the first communication message into the data buffer area through the network connection, receiving a second communication message returned by the central node, analyzing the first communication message by the central node, and processing and generating the data to be stored obtained through analysis.

Description

Data processing method, system and device

Technical Field

The embodiment of the specification relates to the technical field of computers, in particular to a data processing method. One or more embodiments of the present specification also relate to a data processing system, a data processing apparatus, a computing device, and a computer-readable storage medium.

Background

With the rapid development of information technology, a large number of server nodes are generally involved in project execution in network systems in many fields (e.g., finance, communication, production, transportation, etc.). In order to ensure normal project execution, data consistency and accuracy among various server nodes of the distributed database are generally required.

Under the circumstance, in the conventional TCP (Transmission Control Protocol)/IP technology, during the data processing process, an operating system and other software layers of the application server need to occupy a large amount of server resources and memory bus bandwidth, that is, during the data Transmission process using the TCP, a large amount of CPU resources of the server node of the data receiving party need to be occupied, which causes heavy burden to the CPU and the memory of the server node of the data receiving party, especially mismatch between network bandwidth, processor speed and memory bandwidth, thereby causing network communication delay.

Disclosure of Invention

In view of this, the embodiments of the present specification provide a data processing method. One or more embodiments of the present specification also relate to a data processing system, a data processing apparatus, a computing device, and a computer-readable storage medium to address technical deficiencies in the prior art.

According to a first aspect of embodiments of the present specification, there is provided a data processing method applied to a compute node, including:

acquiring data to be stored, inquiring network connection in an idle state in a network connection pool, and determining a data buffer area corresponding to a storage address of the network connection in a central node;

generating a first communication message based on the data to be stored;

calling a remote direct memory access module to write the first communication message into the data buffer area through the network connection;

and receiving a second communication message returned by the central node, wherein the central node analyzes the first communication message in the second communication message, and processes and generates the data to be stored obtained by analysis.

Optionally, after querying the network connection in the idle state in the network connection pool, the method further includes:

determining a first data buffer corresponding to a first storage address connected with the network, and determining a second data buffer corresponding to a second storage address connected with the network in a central node;

correspondingly, the invoking the remote direct memory access module writes the first communication packet into the data buffer through the network connection, including:

and writing the first communication message into the first data buffer area, and calling a remote direct memory access module to write the first communication message in the first data buffer area into the second data buffer area through the network connection.

Optionally, before querying the network connection in the idle state in the network connection pool, the method further includes:

initiating a network connection request to a central node, wherein the network connection request comprises a first storage address of at least one first data buffer area;

receiving a network connection establishment result returned by the central node, wherein the network connection establishment result comprises a second storage address and access authority information of at least one second data buffer area in the central node;

putting the network connection establishment result into the network connection pool, and storing the corresponding relation of the storage addresses of the data buffer areas in the network connection establishment result; the correspondence is used for sending a first communication packet to the central node based on the network connection.

Optionally, after generating the first communication packet based on the data to be stored, the method further includes:

determining a data transmission mode between the computing nodes and the central node according to the number of the computing nodes in the distributed network; or the like, or, alternatively,

and determining a data transmission mode between the computing nodes and the central node according to the number of network connections between the computing nodes and the central node in the distributed network.

Optionally, the determining a data transmission mode between the compute node and the central node according to the number of the compute nodes in the distributed network includes:

judging whether the number of the computing nodes in the distributed network is greater than or equal to a first preset number threshold value or not;

if not, determining that the data transmission mode between the computing node and the central node is a unilateral transmission mode, and executing the step of inquiring the network connection in an idle state in the network connection pool;

and if so, determining that the data transmission mode between the computing node and the central node is a bilateral transmission mode.

Optionally, the generating a first communication packet based on the data to be stored includes:

acquiring access authority information of the first data buffer area, and taking the access authority information and the first storage address as message header information of a first communication message;

packaging the data to be stored, and taking a packaging result as a message body of the first communication message; and the number of the first and second groups,

and setting the notification mark of the first communication message to be 1 so as to generate the first communication message.

Optionally, the receiving the second communication packet returned by the central node includes:

receiving a second communication message written in the data buffer area by the central node by calling the remote direct memory access module;

and under the condition that the notification mark of the second communication message is detected to be changed into 1, analyzing the message body of the second communication message, and storing the analysis result.

Optionally, the data processing method further includes:

and calling a remote direct memory access module to read a data processing result generated by processing the data to be stored from the data buffer area according to the storage address corresponding to the data buffer area in the central node.

According to a second aspect of embodiments herein, there is provided a data processing system comprising:

a central node and at least one compute node;

the at least one computing node is configured to acquire data to be stored, query network connection in an idle state in a network connection pool, determine a data buffer area corresponding to a storage address of the network connection in the central node, generate a first communication message based on the data to be stored, and call a remote direct memory access module to write the first communication message into the data buffer area through the network connection;

the central node is configured to analyze the first communication message, process the data to be stored obtained through analysis, generate a second communication message based on a processing result and return the second communication message;

the at least one computing node configured to receive the second communication message.

Optionally, the central node is further configured to:

calling a main thread to establish network connection with the at least one computing node;

sending the network connection establishment result to a working thread, and calling the working thread to detect at least one network connection in the network connection establishment result;

and when detecting that the notification mark of the first communication message is 1 in the data buffer corresponding to the storage address of the at least one network connection, analyzing the first communication message.

Optionally, the central node is further configured to:

analyzing the first communication message, and processing the data to be stored obtained by analysis;

generating a second communication message based on the processing result;

and setting a notification flag of a second communication message to be 1, and calling a remote direct memory access module to write the second communication message into a data buffer area of the at least one computing node.

Optionally, the central node is further configured to:

receiving a data access request submitted by the at least one computing node for shared data;

and under the condition that the shared data is determined not to be accessed according to the global lock state of the shared data, distributing a global lock aiming at the shared data to the at least one computing node so as to allow the at least one computing node to access the shared data.

According to a third aspect of embodiments herein, there is provided a data processing apparatus comprising:

the system comprises an acquisition module, a storage module and a data processing module, wherein the acquisition module is configured to acquire data to be stored, inquire network connection in an idle state in a network connection pool, and determine a data buffer area corresponding to a storage address of the network connection in a central node;

the generating module is configured to generate a first communication message based on the data to be stored;

the writing module is configured to call a remote direct memory access module to write the first communication message into the data buffer area through the network connection;

and the receiving module is configured to receive a second communication message returned by the central node, wherein the second communication message is generated by analyzing the first communication message by the central node and processing the data to be stored obtained by analyzing.

According to a fourth aspect of embodiments herein, there is provided a computing device comprising:

a memory and a processor;

the memory is to store computer-executable instructions, and the processor is to execute the computer-executable instructions to:

generating a first communication message based on the data to be stored;

According to a fifth aspect of embodiments herein, there is provided a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the data processing method.

In an embodiment of the present description, data to be stored is acquired, a network connection in an idle state is queried in a network connection pool, a data buffer corresponding to a storage address of the network connection in a central node is determined, a first communication packet is generated based on the data to be stored, a remote direct memory access module is invoked to write the first communication packet into the data buffer through the network connection, and a second communication packet returned by the central node is received, where the second communication packet is generated by processing the data to be stored, which is obtained through analysis, by the central node.

In the embodiment of the present description, the computing node transmits data to be stored to the data buffer of the central node by using a remote direct memory access module (RDMA), and since RDMA can directly and quickly transmit the data to be stored from the data buffer of the computing node to the data buffer of the central node, and does not need the participation of a central node CPU in the whole data transmission process, it is beneficial to reduce the consumption of the central node CPU in the data transmission process, thereby being beneficial to improving the processing performance of the system and reducing the network communication delay.

Drawings

FIG. 1 is a process flow diagram of a data processing method provided in one embodiment of the present description;

FIG. 2 is a schematic diagram of a first data processing procedure provided in one embodiment of the present description;

fig. 3(a) is a schematic diagram of a first communication packet provided in an embodiment of the present specification;

fig. 3(b) is a schematic diagram of a second communication packet provided in an embodiment of the present specification;

FIG. 4 is a diagram illustrating a second data processing procedure provided in one embodiment of the present description;

FIG. 5 is a schematic diagram of a data processing system provided by one embodiment of the present description;

FIG. 6 is an interaction diagram of a data processing method provided in an embodiment of the present specification;

FIG. 7 is a schematic diagram of a data processing apparatus provided in one embodiment of the present description;

fig. 8 is a block diagram of a computing device according to an embodiment of the present disclosure.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present description. This description may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, as those skilled in the art will be able to make and use the present disclosure without departing from the spirit and scope of the present disclosure.

The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It will be understood that, although the terms first, second, etc. may be used herein in one or more embodiments to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, a first can also be referred to as a second and, similarly, a second can also be referred to as a first without departing from the scope of one or more embodiments of the present description. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.

First, the noun terms to which one or more embodiments of the present specification relate are explained.

RDMA: in the field of data centers, Remote Direct Memory Access (RDMA) is a technology for accessing data in a memory by bypassing a kernel of a remote host operating system, and since the RDMA does not pass through the operating system, a large amount of CPU resources are saved, the system throughput is also improved, the network communication delay of the system is reduced, and the RDMA is particularly suitable for being widely applied to large-scale parallel computer clusters.

In the present specification, a data processing method is provided, and the present specification relates to a data processing system, a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

When the current application program adopts TCP/Socket to transmit data, a CPU of a server needs to copy the data from a user mode to a kernel protocol stack for processing, then copies the data to a network card driver, finally, network card hardware acquires the data from the network card driver in a DMA mode, and then the network card processes the data and sends the data to a network. After the data reaches the network card hardware of the destination node, a series of reverse copy operations are carried out to reach the memory of the target application. It can be seen that, in the process of data transmission through TCP, a large number of memory copies are made by the CPU of the server, which may cause a heavy burden on the CPU and the memory of the server for receiving data, especially a mismatch between a network bandwidth, a processor speed, and a memory bandwidth, thereby causing a network communication delay.

In addition, in a part of the existing distributed networks, only one data read-write node and a plurality of data read-only nodes may exist, data writing can be performed only on one data read-write node, and the writing performance is affected under the condition that data to be written is more, so for the condition, an improvement mode is that a plurality of read-write nodes are arranged in the distributed network, but in the condition, in order to ensure that data written by different read-write nodes can be synchronized in a distributed database, a central node is arranged in the distributed network in the embodiment of the present specification, so that data written by a plurality of read-write nodes is transmitted to the central node, and then the central node synchronizes the data to other read-write nodes or read-only nodes.

Specifically, an embodiment of the present disclosure provides a specific processing manner for performing data transmission from a compute node (read/write node) to a central node, that is, a network connection is required to be established between each compute node of a distributed network and the central node, and after the network connection is established, the compute nodes are first placed in a network connection pool, and during a process that data transmission is required, idle network connections are queried in the network connection pool, and then data to be stored is transmitted to a data cache center of the central node through the network connection by using a remote direct memory access technology.

Therefore, the embodiments of the present specification combine the RDMA technology in the data transmission process between the server nodes of the distributed network to implement an RDMA-based data processing method. The method is favorable for reducing the interference to the central node CPU in the data transmission process between the computing node and the central node, thereby reducing the consumption of the central node CPU in the data transmission process, improving the processing performance of the system and being favorable for reducing the network communication delay.

Fig. 1 shows a process flow diagram of a data processing method provided according to an embodiment of the present specification, including steps 102 to 108.

Step 102, obtaining data to be stored, inquiring network connection in an idle state in a network connection pool, and determining a data buffer area corresponding to a storage address of the network connection in a central node.

Specifically, the data to be stored, that is, the data to be transmitted from the computing node to the central node; the network connection pool comprises a plurality of network connections, and the network connections are established between data buffers of the computing nodes and the central node.

The data processing method provided in the embodiments of the present specification is applied to a computing node of a distributed network, where the distributed network includes a central node and a plurality of computing nodes, and the computing nodes are one of the plurality of computing nodes, and before data transmission, network connections between the central node and the plurality of computing nodes (network connections between storage addresses of data buffers in the central node and storage addresses of data buffers in the computing nodes) may be respectively established, and the established network connections are placed in a network connection pool.

When data to be synchronized (data to be stored) exists in the computing node, the idle network connection can be inquired from the network connection pool, a data buffer area corresponding to the storage address of the idle network connection in the central node is determined, and then the data to be stored is transmitted to the data buffer area of the central node through the network connection by using Remote Direct Memory Access (RDMA).

In specific implementation, before querying the network connection in the idle state in the network connection pool, the process of establishing the network connection between the computing node and the central node may be specifically implemented in the following manner:

Specifically, in the process of establishing the network connection between the computing node and the central node, a plurality of persistent buffers (persistent buffers) with fixed sizes may be registered in the central node and the computing node in advance as data buffers for transferring data. The data buffers of the central node and the computing nodes are in one-to-one correspondence, and when the network connection is established between the data buffers of the central node and the computing nodes, the central node and the computing nodes can respectively transmit the storage addresses and the access authority information of the newly registered data buffers to each other through the RDMA data buffers of the central node and the computing nodes for storing private data, so that after the network connection between the computing nodes and the central node is established between the computing nodes and the central node, the computing nodes and the central node can both write data in the data buffers of each other remotely through a remote memory direct access technology.

In practical application, after registering data buffer areas in a computing node and a central node, the computing node can initiate a network connection request to the central node, then the central node responds to the network connection request, establishes network connection between the data buffer areas in the central node and data buffer areas in the computing node, and returns a network connection establishment result to the computing node; after receiving the network connection establishment result, the computing node is placed in a network connection pool, and the corresponding relation of the storage addresses of the data buffer areas contained in the network connection pool is stored, so that data transmission is carried out between the computing node and the central node through the established network connection under the condition of demand.

Fig. 2 is a schematic diagram of a first data processing process provided in an embodiment of this specification, where fig. 2 illustrates a network connection pool of a compute node, where the network connection pool includes network connections between data buffer areas of the compute node and data buffer areas between a central node. When the data to be stored exists in the computing node, the idle network connection can be inquired from the network connection pool, and the idle network connection is utilized for data transmission.

In the embodiment of the present specification, a computing node manages RDMA network connection through a network connection pool, establishes a network connection between the computing node and a central node in advance and stores the network connection in the network connection pool, acquires the network connection from the network connection pool when communication (data transmission) with the central node is required, and puts the network connection back into the network connection pool after the communication is finished, thereby avoiding the overhead of frequently creating/consuming the network connection.

And 104, generating a first communication message based on the data to be stored.

In specific implementation, the first communication packet is generated based on the data to be stored, and the method can be specifically implemented in the following manner:

Specifically, during the process of transmitting data from the computing node to the central node by using the remote direct memory access technology, data transmission can be performed by using a single-side operation. In the unilateral operation process, the central node needs to determine which data buffer area of the computing node the data to be stored is stored in, that is, the storage address of the data buffer area where the data to be stored is located and the access authority information of the data buffer area need to be determined, so that when the computing node transmits a first communication message to the central node, the first communication message needs to contain the information, and the information can be specifically encapsulated into the first communication message and serves as the message header information of the first communication message; in addition, when the data to be stored is transmitted, the data to be stored also needs to be transmitted in a form of a message, so that the data to be stored can be encapsulated, and the encapsulation result is used as a message body of the first communication message.

A schematic diagram of a first communication packet provided in the embodiment of the present specification is shown in fig. 3(a), where the first communication packet is a request packet, and information included in a packet Header (Req Header) of the request packet is a storage address of a data buffer where data to be stored in a compute node is located, and access right information of the data buffer; the information contained in the message Body (Req Body) is the data to be stored; in addition, the first communication message further includes a notification Flag (Notify Flag), and when the computing node writes the first communication message to the data buffer of the central node remotely, the notification Flag is set to 1, and the first communication message is transmitted to the central node.

In addition, after generating the first communication packet based on the data to be stored, the method further includes:

Further, the data transmission mode between the computing node and the central node is determined according to the number of the computing nodes in the distributed network, which can be specifically realized by the following modes:

Specifically, a traditional RDMA network library (for example, XRDMA) generally uses a double-side transmission interface (send/receive two-side API) to implement Remote Procedure Call (RPC), because double-side transmission requires participation of both communication parties, and receive needs to be executed before send, the other party can send data, the process is similar to traditional communication, a central node manages and issues RDMA descriptors, which generates additional overhead, which further reduces communication performance, and thus, the transmission rate of double-side transmission is slower than that of single-side transmission.

In one embodiment provided in this specification, a Remote write (Remote write) may be performed between a compute node and a central node by using a single-side operation, and this write process is performed by the compute node, and the central node does not need to participate in a CPU. The computing node directly transmits the data to be stored from the local data buffer to the continuous data buffer of the central node.

In practical application, the low-delay/high-throughput data processing mode can be realized by using unilateral operation for remote writing, and the method is suitable for scenes with fewer computing nodes and higher system throughput requirements in a distributed network; for a scene with more computing nodes in a distributed network or more network connections between the computing nodes and a central node, a high expansion mode can be realized by using bilateral operation.

Therefore, a data transmission mode between the computing nodes and the central node can be determined according to the number of the computing nodes in the distributed network, and under the condition that the number of the computing nodes is smaller than a first preset number threshold, the data transmission is performed by using a single-side transmission mode; determining to use a bilateral transmission mode under the condition that the number of the computing nodes is greater than or equal to a first preset number threshold; or, the data transmission mode between the computing node and the central node can be determined according to the number of network connections between the computing node and the central node in the distributed network, and under the condition that the number of network connections between the computing node and the central node is large, the data transmission is performed by using the bilateral transmission mode.

The embodiment of the specification realizes data transmission and processing by using a single-side transmission mode (one-side) and a double-side transmission mode (two-side) in a mixed mode, and simultaneously meets a low-delay/high-throughput mode and a high-expansion mode of a large number of computing nodes. Specifically, the embodiments of the present disclosure may support two modes, namely, a single-side transmission mode and a double-side transmission mode, and when there are fewer computing nodes, all RPC communications may use the single-side transmission mode, so that the inter-node communication delay is as small as possible, and throughput is ensured. With the increase of the number of the computing nodes or the increase of the number of the network connections from the computing nodes to the central node, the polling efficiency of the working threads in the nodes is lowered, the overall throughput of the distributed network may be lowered, at this time, a part of the network connections which do not need to frequently interact with the central node may be changed into a bilateral transmission communication mode, and the network connections which need to frequently communicate with the central node still adopt a unilateral transmission mode, so that the requirements of high throughput and high expansion are considered.

And 106, calling a remote direct memory access module to write the first communication message into the data buffer area through the network connection.

Specifically, after a first communication packet is generated based on data to be stored, a remote direct access memory module (RDMA) may be called to write the first communication packet into a data buffer corresponding to a storage address connected to the network in a central node.

In specific implementation, after network connection in an idle state is inquired in a network connection pool, a first data buffer corresponding to a first storage address of the network connection can be determined, and a second data buffer corresponding to a second storage address of the network connection in a central node can be determined;

correspondingly, the process of transmitting the first communication packet of the data to be stored by using the network connection specifically includes:

Specifically, in the embodiments of the present specification, an RDMA technology may be used to directly transmit a first communication packet of data to be stored from a data buffer of a computing node to a data buffer of another central node through a network connection, specifically, the first communication packet is first copied to a network card (a first data buffer) of the computing node in a DMA manner, and then the first communication packet in the network card is transmitted to a network card (a remote network card, i.e., a second data buffer) of the central node through the network connection, and then the first communication packet directly reaches a memory (a remote memory) of the central node, without requiring an operating system to copy data among the data buffers for multiple times and without requiring a CPU to participate, so that the CPU is saved for improving the overall system performance, thereby reducing the requirements for bandwidth and processor overhead, and significantly reducing the time delay.

Therefore, after the idle network connection is inquired in the network connection pool, a first data buffer corresponding to the first storage address of the idle network connection in the computing node and a second data buffer corresponding to the second storage address of the idle network connection in the central node can be determined, so that the first communication message is written into the first data buffer, and a remote direct memory access module is called to write the first communication message in the first data buffer of the computing node into the second data buffer of the central node through the idle network connection.

And 108, receiving a second communication message returned by the central node, wherein the second communication message is generated by analyzing the first communication message by the central node and processing the data to be stored obtained by analyzing.

Specifically, a schematic diagram of a second data processing process provided in this specification is shown in fig. 4, where a computing node transmits a first communication packet to a central node through a data buffer of the computing node through a network connection, and a main thread (main thread) in the central node is used for processing RDMA network connection events, including events such as connection establishment and disconnection. Each newly-built RDMA network connection is distributed to a worker thread (worker thread) in a round robin (round robin) mode, each worker thread is responsible for processing a certain number of RDMA network connections, and communication messages are sensed and processed in a data buffer area in a central node in a circular polling (polling) mode.

Therefore, the working thread of the central node traverses all the RDMA network connections managed by the working thread, and when detecting that the notification flag of the first communication message in a certain data buffer is set to 1, the network connection receives a message, and because the notification flag is the last byte of the first communication message and RDMA remote write is order-preserving, under the condition that the notification flag is set to 1, the data in the first communication message can be guaranteed to be completely written into the data buffer of the central node.

Based on this, the working thread starts to analyze and process the data contained in the message body of the first communication message, after the processing is completed, the processing result is written into the data buffer area of the central node, and then the processing result is remotely written into the data buffer area of the computing node by using the remote direct memory access module.

Fig. 3(b) shows a schematic diagram of the second communication packet provided in this embodiment, where a last byte of the second communication packet (response packet) is a completion Flag, and after the processing of the data to be stored is completed, the central node sets the completion Flag (Complete Flag) to 1, which indicates that the message has been processed. The second communication message is then returned to the computing node.

In specific implementation, a second communication message returned by the central node is received, specifically, the receiving central node calls a second communication message written in a data buffer by a remote direct memory access module, analyzes a message body of the second communication message when detecting that a notification flag of the second communication message becomes 1, and stores an analysis result.

Specifically, after the computing node transmits the first communication packet to the central node, the computing node continuously checks the last byte (i.e., Complete Flag) of the second communication packet in the local data buffer, and once the last byte is set to 1, it indicates that the central node has processed the message and sent the response packet.

In addition, after the computing node transmits the data to be stored to the central node, other computing nodes can also read the data from the central node to realize the consistency of the data among different computing nodes, and specifically, a data processing result generated by processing the data to be stored can be read from the data buffer area by calling a remote direct memory access module according to a storage address corresponding to the data buffer area in the central node.

In practical application, after receiving a data access request submitted by a computing node for shared data (generated by processing data to be stored), a central node needs to determine whether the shared data is being accessed according to a global lock state of the shared data, and if not, a global lock for the shared data can be allocated to the computing node to allow the computing node to access the shared data.

Based on this, in an embodiment of the present specification, data to be stored is acquired, a network connection in an idle state is queried in a network connection pool, a data buffer corresponding to a storage address of the network connection in a central node is determined, a first communication packet is generated based on the data to be stored, a remote direct memory access module is invoked to write the first communication packet into the data buffer through the network connection, and a second communication packet returned by the central node is received, where the second communication packet is generated by analyzing the first communication packet by the central node and processing the data to be stored obtained through analysis.

FIG. 5 illustrates a schematic diagram of a data processing system provided in accordance with one embodiment of the present description, including:

a central node 502 and at least one compute node 504;

the at least one computing node 504 is configured to acquire data to be stored, query a network connection in an idle state in a network connection pool, determine a data buffer corresponding to a storage address of the network connection in the central node, generate a first communication packet based on the data to be stored, and call a remote direct memory access module to write the first communication packet into the data buffer through the network connection;

the central node 502 is configured to analyze the first communication packet, process the data to be stored obtained by analysis, generate a second communication packet based on a processing result, and return the second communication packet;

the at least one computing node 504 configured to receive the second communication message.

Specifically, the data processing system provided in the embodiment of the present disclosure includes a central node 502 and a plurality of computing nodes 504, and before data transmission, network connections between the central node 502 and the plurality of computing nodes 504 may be respectively established, and the established network connections are placed in a network connection pool.

When there is data to be synchronized (data to be stored) in the compute node 504, the idle network connection may be queried from the network connection pool, and a data buffer corresponding to the storage address of the idle network connection in the central node 502 is determined, and then a first communication packet generated based on the data to be stored is transmitted to the data buffer of the central node 502 through the network connection by using a Remote Direct Memory Access (RDMA) technique.

After receiving the first communication packet, the central node 502 may analyze the first communication packet, process the data to be stored obtained by the analysis, generate a second communication packet based on the processing result, and return the second communication packet to the computing node 504.

In practical applications, a plurality of computing nodes for data reading and writing are arranged in a database cluster of a server-less distributed database, and the computing nodes can communicate with a centralized central node (Cache Center) through an RDMA network framework (the computing nodes and the central node form the data processing system in the embodiments of the present specification).

As shown in fig. 5, the RDMA network framework can directly transmit data from the memory of the computing node to the memory of the central node through the network based on the RDMA network framework, specifically, the data in the computing node is firstly copied to the network card (data buffer) of the computing node in a DMA manner, then transmitted to the network card of the central node through the network, and then directly reaches the memory of the central node without the involvement of the CPU of the central node and the data copied between the data buffers by the operating system for many times, so that the CPU can be saved for improving the overall system performance, the requirements for bandwidth and processor overhead are reduced, and the time delay is significantly reduced.

In addition, the establishment of network connection and data transmission involved in the data transmission process can be realized by calling related interfaces in the RDMA network framework.

For example, RDMA native interface may be invoked for registration of data buffers and creation of message queues, etc.; a central node network connection manager and a computing node network connection manager can be called to establish network connection between the central node and the computing node; a central node data flow manager and a computing node data flow manager can be called to poll and detect whether a local data buffer area has a message to be processed (so that data flow and control flow are separated); the remote memory direct access module can be called to transmit communication messages and the like.

In addition, the central node is responsible for caching global page, lock, and transaction information. Wherein, a Global Buffer (GBP) in the central node is responsible for synchronization and invalidation of the Global page, and Global consistency is ensured; a Global Lock Manager (GLM) realizes the management of Global locks and ensures the correctness of concurrent control among nodes; a Global Transaction Manager (GTM) maintains Global Transaction information; similarly, the local buffer in the compute node is responsible for synchronization and invalidation of the local page; the local lock manager realizes the management of the local lock; the local transaction manager maintains local transaction information.

Optionally, the central node is further configured to:

generating a second communication message based on the processing result;

Optionally, the central node is further configured to:

The above is a schematic scheme of a data processing system of the present embodiment. It should be noted that the technical solution of the data processing system and the technical solution of the data processing method belong to the same concept, and details that are not described in detail in the technical solution of the data processing system can be referred to the description of the technical solution of the data processing method.

The data processing method is further described below with reference to fig. 6. Fig. 6 shows an interaction diagram of a data processing method provided in an embodiment of the present specification, and specific steps include step 602 to step 620.

Step 602, the computing node sends a network connection request to the central node.

Step 604, the central node establishes a network connection with the computing node.

Specifically, the central node establishes a network connection between its data buffer and the data buffer of the compute node.

Step 606, the central node returns the network connection establishment result to the compute node.

And after receiving the network connection establishment result, the computing node puts the network connection contained in the network connection establishment result into a network connection pool.

Step 608, the computing node obtains the data to be stored.

Step 610, the computing node generates a first communication packet based on the data to be stored.

Step 612, the computing node remotely writes the first communication packet into a data buffer of the central node.

After the computing node generates the first communication message, the computing node can search for an idle network connection in the network connection pool, and transfer the remote direct memory access module to transmit the first communication message to the central node through the idle network connection.

And 614, the central node analyzes the first communication message, processes the analysis result and generates a corresponding processing result.

At step 616, the central node generates a second communication message based on the processing result.

At step 618, the central node remotely writes the second communication message to the data buffer of the compute node.

Specifically, the central node returns the second communication message to the computing node by calling the remote direct memory access module and through the idle network connection.

And step 620, the computing node analyzes the second communication message and stores the analysis result.

Corresponding to the above method embodiment, the present specification further provides a data processing apparatus embodiment, and fig. 7 shows a schematic diagram of a data processing apparatus provided in an embodiment of the present specification. As shown in fig. 7, the apparatus includes:

an obtaining module 702, configured to obtain data to be stored, query a network connection in an idle state in a network connection pool, and determine a data buffer corresponding to a storage address of the network connection in a central node;

a generating module 704 configured to generate a first communication packet based on the data to be stored;

a write module 706 configured to invoke a remote direct memory access module to write the first communication packet into the data buffer through the network connection;

a receiving module 708, configured to receive a second communication packet returned by the central node, where the second communication packet is generated by analyzing the first communication packet by the central node and processing the data to be stored obtained through analysis.

Optionally, the data processing apparatus further includes:

a determining module configured to determine a first data buffer corresponding to a first storage address of the network connection and determine a second data buffer corresponding to a second storage address of the network connection in a central node;

accordingly, the writing module 706 is further configured to:

Optionally, the data processing apparatus further includes:

the sending module is configured to initiate a network connection request to the central node, wherein the network connection request comprises a first storage address of at least one first data buffer area;

a result receiving module, configured to receive a network connection establishment result returned by the central node, where the network connection establishment result includes a second storage address of at least one second data buffer in the central node and access right information;

the storage module is configured to put the network connection establishment result into the network connection pool and store the corresponding relation of the storage addresses of the data buffer areas in the network connection establishment result; the correspondence is used for sending a first communication packet to the central node based on the network connection.

Optionally, the data processing apparatus further includes:

the first transmission mode determining module is configured to determine a data transmission mode between the computing nodes and the central node according to the number of the computing nodes in the distributed network; or the like, or, alternatively,

and the second transmission mode determining module is configured to determine a data transmission mode between the computing node and the central node according to the number of network connections between the computing node and the central node in the distributed network.

Optionally, the first transmission mode determination module is further configured to:

Optionally, the generating module 704 is further configured to:

Optionally, the receiving module 708 is further configured to:

Optionally, the data processing apparatus further includes:

and the reading module is configured to call a remote direct memory access module to read a data processing result generated by processing the data to be stored from the data buffer area according to the storage address corresponding to the data buffer area in the central node.

The above is a schematic configuration of a data processing apparatus of the present embodiment. It should be noted that the technical solution of the data processing apparatus and the technical solution of the data processing method belong to the same concept, and details that are not described in detail in the technical solution of the data processing apparatus can be referred to the description of the technical solution of the data processing method.

FIG. 8 illustrates a block diagram of a computing device 800, according to one embodiment of the present description. The components of the computing device 800 include, but are not limited to, memory 810 and a processor 820. The processor 820 is coupled to the memory 810 via a bus 830, and the database 850 is used to store data.

Computing device 800 also includes access device 840, access device 840 enabling computing device 800 to communicate via one or more networks 860. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 840 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.

In one embodiment of the present description, the above-described components of computing device 800, as well as other components not shown in FIG. 8, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 8 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.

Computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 800 may also be a mobile or stationary server.

Wherein the memory 810 is configured to store computer-executable instructions and the processor 820 is configured to execute the following computer-executable instructions:

generating a first communication message based on the data to be stored;

The above is an illustrative scheme of a computing device of the present embodiment. It should be noted that the technical solution of the computing device and the technical solution of the data processing method belong to the same concept, and details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the data processing method.

An embodiment of the present specification also provides a computer readable storage medium storing computer instructions which, when executed by a processor, are used for implementing the steps of the data processing method.

The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the data processing method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the data processing method.

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of acts, but those skilled in the art should understand that the present embodiment is not limited by the described acts, because some steps may be performed in other sequences or simultaneously according to the present embodiment. Further, those skilled in the art should also appreciate that the embodiments described in this specification are preferred embodiments and that acts and modules referred to are not necessarily required for an embodiment of the specification.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

The preferred embodiments of the present specification disclosed above are intended only to aid in the description of the specification. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the embodiments and the practical application, to thereby enable others skilled in the art to best understand and utilize the embodiments. The specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method is applied to a computing node and comprises the following steps:

generating a first communication message based on the data to be stored;

2. The data processing method according to claim 1, further comprising, after querying the network connection pool for the network connection in the idle state:

3. The data processing method according to claim 1, wherein before querying the network connection pool for the network connection in the idle state, the method further comprises:

4. The data processing method according to claim 1, further comprising, after generating the first communication packet based on the data to be stored:

5. The data processing method of claim 4, wherein determining the data transmission mode between the compute nodes and the central node according to the number of the compute nodes in the distributed network comprises:

6. The data processing method according to claim 2, wherein the generating a first communication packet based on the data to be stored comprises:

7. The data processing method according to any one of claims 1 to 6, wherein the receiving the second communication packet returned by the central node includes:

8. The data processing method of claim 1, further comprising:

9. A data processing system comprising:

a central node and at least one compute node;

10. The data processing system of claim 9, the central node further configured to:

11. The data processing system of claim 9 or 10, the central node further configured to:

generating a second communication message based on the processing result;

12. The data processing system of claim 9, the central node further configured to:

13. A computing device, comprising:

a memory and a processor;

the memory is configured to store computer-executable instructions, and the processor is configured to implement the steps of the data processing method according to any one of claims 1 to 8 when executing the computer-executable instructions.

14. A computer readable storage medium storing computer instructions which, when executed by a processor, carry out the steps of the data processing method of any one of claims 1 to 8.