CN113434290A

CN113434290A - Data processing method and device based on RAFT protocol, and computer storage medium

Info

Publication number: CN113434290A
Application number: CN202110678920.9A
Authority: CN
Inventors: 张胜玉; 贾世萌
Original assignee: Lenovo Beijing Ltd
Current assignee: Lenovo Beijing Ltd
Priority date: 2021-06-18
Filing date: 2021-06-18
Publication date: 2021-09-24

Abstract

The embodiment of the invention discloses a data processing method based on a RAFT algorithm, which is applied to a main node, wherein the main node is configured with a first RDMA module, and the method comprises the following steps: receiving a current task request; wherein, the current task request carries current task data; and responding to the current task request, and sending a write instruction of the current task data to an SCM (local area network) module of the slave node by utilizing the first RDMA module according to a RAFT (random access field) protocol.

Description

Data processing method and device based on RAFT protocol, and computer storage medium

Technical Field

The present invention relates to the field of data processing, and in particular, to a data processing method and apparatus based on a RAFT protocol, and a computer storage medium.

Background

At present, the RAFT protocol is widely applied to many fields, and the main role is to maintain data consistency among multiple nodes, including application to a distributed network for distributed storage.

However, for data transmission in a distributed network, because the RAFT protocol needs to wait for a successful response message of the Follower when submitting data and indicates that the data has been successfully submitted only when more than half of the data passes through the RAFT protocol, if the Follower cannot timely return a successful response message, the throughput of the entire data transmission system is reduced, and the data transmission delay is high.

Disclosure of Invention

To solve the foregoing technical problems, embodiments of the present invention are directed to a data processing method and apparatus based on a RAFT protocol, and a computer storage medium.

The technical scheme of the invention is realized as follows:

the data processing method based on the RAFT algorithm is applied to a main node, the main node is provided with a first RDMA module, and the main node receives a current task request; wherein, the current task request carries current task data; and responding to the current task request, and sending a write instruction of the current task data to an SCM (local area network) module of the slave node by utilizing the first RDMA module according to a RAFT (random access field) protocol.

The data processing method based on the RAFT algorithm is applied to the slave node, the slave node is provided with a second RDMA module and an SCM module, and the slave node receives a writing instruction of task data sent by a master node by using the second RDMA module; writing the task data into the SCM module in response to the write instruction; and when the task data corresponding to the current data successful submission serial number exists in the SCM module, determining the task data as the applicable data.

The embodiment of the invention provides a data processing device based on a RAFT protocol, which is characterized in that the data processing device is provided with a first RDMA module and comprises a receiving unit and a sending unit,

the receiving unit is used for receiving a task request sent by a client; wherein the task request carries task data;

and the sending unit is used for responding to the current task request and sending a write instruction of the current task data to an SCM (local area network) module of a slave node by using the first RDMA module according to the RAFT protocol.

The embodiment of the invention provides data processing equipment based on a RAFT protocol, which is characterized in that the data processing equipment is provided with a second RDMA module and an SCM module, the data processing equipment comprises a receiving unit, a writing unit and a determining unit,

the receiving unit is used for receiving a writing instruction of the task data sent by the main node by using the second RDMA module;

the writing unit is used for responding to the writing instruction and writing the task data into the SCM module;

the determining unit is configured to determine the task data as applicable data when it is determined that the current data successful submission serial number corresponding to the task data exists in the SCM module.

An embodiment of the present invention provides a data processing device based on a RAFT protocol, including: a processor, a memory storing instructions executable by the processor, which when executed by the processor, implement the data processing method based on the RAFT protocol as described above.

An embodiment of the present invention provides a computer-readable storage medium, which stores a program, and when the program is executed by a processor, the program implements the data processing method based on the RAFT protocol.

According to the technical scheme provided by the embodiment of the invention, the main node can directly write data into the memory of the slave node by utilizing the RDMA module without waiting for the return response of the slave node and without the participation of the CPU of the slave node, so that the system throughput rate is further improved, and the data transmission efficiency is higher.

Drawings

Fig. 1 is a schematic flow chart of a data processing method based on a RAFT protocol in the related art;

fig. 2 is a schematic structural diagram of a distributed storage system according to an embodiment of the present invention;

fig. 3 is a first schematic flow chart illustrating an implementation of a data processing method based on a RAFT protocol according to an embodiment of the present invention;

fig. 4 is a schematic flow chart illustrating an implementation process of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 5 is a schematic flow chart illustrating an implementation of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 6 is a schematic flow chart of a fourth implementation flow of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 7 is a schematic flow chart of an implementation process of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 8 is a schematic flow chart of a sixth implementation process of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 9 is a seventh implementation flow diagram of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 10 is an implementation flow diagram eight of the data processing method based on the RAFT protocol according to the embodiment of the present invention;

fig. 11 is a flowchart illustrating a method for executing data processing by the distributed storage system according to an embodiment of the present invention;

fig. 12 is a first schematic structural diagram of a data processing apparatus based on a RAFT protocol according to an embodiment of the present invention;

fig. 13 is a schematic structural diagram of a data processing apparatus based on a RAFT protocol according to an embodiment of the present invention;

fig. 14 is a schematic structural diagram three illustrating a composition structure of a data processing apparatus based on a RAFT protocol according to an embodiment of the present invention;

fig. 15 is a schematic structural diagram of a data processing apparatus based on a RAFT protocol according to a fourth embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where appropriate, to enable embodiments of the invention described herein to be practiced in other than the order shown or described herein.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to be limiting of the invention.

Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.

1) Distributed storage: distributed storage is a data storage technology, which uses disk space on each machine in an enterprise through a network and forms a virtual storage device with these distributed storage resources, and data is distributed and stored in each corner of the enterprise. That is, a distributed storage system is a system in which data is stored in a distributed manner on a plurality of independent devices. A traditional network storage system is based on the fact that a centralized storage server is adopted to store all data, the storage server becomes the bottleneck of system performance, is also the focus of reliability and safety, and cannot meet the requirements of large-scale storage application. The distributed network storage system adopts an expandable system structure, utilizes a plurality of storage servers to share the storage load, and utilizes the position server to position the storage information, thereby not only improving the reliability, the availability and the access efficiency of the system, but also being easy to expand. It can be seen that distributed storage systems are typically fault tolerant by maintaining multiple copies, increasing the availability of the system.

2) RAFT protocol: the RAFT protocol is a distributed consistency protocol for maintaining consistency of multiple copies in a distributed storage system. For example, in a cluster with consistent properties, the RAFT protocol can be used to make all nodes at the same time have the same result on a certain value stored therein, i.e. to keep the storage shared for them consistent. The cluster has the property of automatic recovery, and when a few nodes fail, the normal operation of the cluster is not influenced, and when most nodes in the cluster fail, the cluster stops service (an error result is not returned). It can be seen that the RAFT protocol is used to ensure that the system will still be able to provide services to the outside normally even if a partial (or rather small) copy is down. The RAFT protocol is usually based on replicated state maps, i.e. all nodes start from the same state, go through the same sequence of operations (log), and finally reach the same state.

Each node of the RAFT protocol is in one of three states: leader, follow, and candiddate.

A Leader: all the request handlers, the Leader, receive the update request of the client, and synchronize to a plurality of other slave nodes after local processing;

follower: the passive updater of the request, accepts the update request from the Leader, and then writes to the local log file.

Candidate: if the Follower does not receive the heartbeat signal of the Leader within a period of time, judging that the Leader may have a fault, starting a master selection process, and changing the node into a Candidate state until the master selection is finished.

3) Remote Direct Memory Access (RDMA) technology: through RDMA, the home node may access the memory of the remote node "directly". The direct mode refers to that data can be directly transmitted to a memory of a remote node through a network like accessing a local memory, and a remote memory is read and written by bypassing a traditional complex Transmission Control Protocol (TCP)/Internet Protocol (IP) network Protocol stack of an ethernet, so that ultra-low-latency data Processing and ultra-high throughput Transmission can be easily realized, intervention of resources such as a Central Processing Unit (CPU) of the remote node is not required, and excessive resources are not consumed for data Processing and migration.

4) Persistent memory: also called Non-Volatile Memory (NVM) or Memory Class Memory (SCM), supports byte-addressing (byte-addressing), and can be directly operated by CPU instructions, and the data is not lost after power failure.

At present, the specific process of performing data distributed storage according to the RAFT protocol includes:

a Leader: the Leader sends data to the Follower through the application Message, and determines that the data is successfully submitted after receiving successful return responses of most of the Followers, and at the moment, the Leader can return a successful response to the client.

Follower: after receiving the message sent by the Leader, the Follower may perform persistent storage, return the result to the Leader, and perform log application after receiving the next message, that is, heartbeat data, of the Leader, that is, notify the upper layer application that the data has been stored, and the upper layer application may call the data to perform some related processing.

For example, fig. 1 is a schematic flow chart of a data processing method based on a RAFT protocol in the related art, and as shown in fig. 1, a Network Interface Controller (NIC) of a master node and an NIC of a slave node establish communication connection first. When the master node receives a client request (Sa1), the data is carried by the application Message and is sent to the NIC (Sa2) of the slave node through the NIC, and after the slave node receives the data (Sa3), the data is written into a persistent memory (Sa 4); and sending a response message of successful data writing to the master node through the NIC, and after receiving the success response (Sa6), the master node determines whether more than half of slave nodes respond successfully (Sa 7)? If not, the receipt of the response message continues until more than half of the total or the failure of the data submission to the slave node is indicated. If the data is successfully submitted from the node, the data consistency storage is successful. Further, the master node may update the data successful commit sequence number based on the number of the current data (Sa8), and transmit the heartbeat data to the slave node through the NIC (Sa 9).

Further, after receiving the heartbeat data, the slave node may determine whether there is a commit sequence number (Sa10) corresponding to the heartbeat data (Sa 10)? If not, continuing to wait for receiving the data, and if so, logging the data written into the persistent memory (S11).

It can be seen that, in the related art, a data processing method based on the RAFT protocol needs to wait for a successful response message of a Follower when submitting data, and indicates that the data has been successfully submitted only when more than half of the data passes through the Follower, and if the Follower cannot timely return a successful response message, the throughput of the entire data transmission system is reduced, so that the data transmission delay is high.

The embodiment of the invention provides a data processing method, a device and a computer storage medium based on a RAFT protocol, wherein a first RDMA module is configured as a data processing device of a main node, and the main node can receive a current task request; wherein, the current task request carries current task data; and responding to the current task request, and sending a write instruction of the current task data to an SCM (local area network) module of the slave node by utilizing the first RDMA module according to a RAFT (random access field) protocol. Correspondingly, the data processing equipment serving as the slave node is provided with a second RDMA module and an SCM module, and the slave node receives a writing instruction of task data sent by the master node by using the second RDMA module; responding to the writing instruction, and writing the task data into the SCM module; and when the task data corresponding to the current data successful submission serial number exists in the SCM module, determining the task data as the applicable data.

Therefore, the main node can directly write data into the memory of the slave node by using the RDMA module based on the RAFT protocol and the RDMA message, does not need to wait for the return response of the slave node and does not need the participation of the CPU of the slave node, thereby further improving the system throughput rate and having higher data transmission efficiency.

The data processing method based on the RAFT protocol provided by the embodiment of the invention is applied to data processing equipment based on the RAFT protocol. The following describes an exemplary application of the RAFT protocol-based data processing device provided in the embodiment of the present invention, and the RAFT protocol-based data processing device provided in the embodiment of the present invention may be implemented in any device having a data storage function, such as a mobile phone, a notebook computer, a tablet computer, a desktop computer, a server, a virtual machine, a smart television, a vehicle-mounted device, a wearable device, an industrial device, and a cloud device.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.

An embodiment of the present invention provides a data processing method based on a RAFT protocol, which is applied to a distributed storage system 100 shown in fig. 2, and the distributed storage system includes a data processing device 10 as a master node and a data processing device 20 as a slave node. The device 10 sets a first RDMA module 11; the device 20 sets a second RDMA module 21 and an SCM module 22. A first RDMA module 11 of the device 10 is adapted to send a write instruction of the data to the SCM module 22; the second RDMA module 21 of the device 20 is used to receive write commands for data sent by the device 10 and to write the data to the SCM module 22.

Fig. 3 is a first schematic flow chart illustrating an implementation process of a data processing method based on a RAFT protocol according to an embodiment of the present invention, as shown in fig. 3, in an embodiment of the present invention, a method for a primary node to perform data processing based on a RAFT protocol may include the following steps:

s101, receiving a current task request; and the current task request carries current task data.

In the embodiment of the present invention, the master node may receive a task request carrying task data.

It should be understood that, in the embodiment of the present invention, the current task request may be a request sent by a client received by a master node, and current task data is data values carried in the task request. The request can be regarded as a data write request or a data change request, and the data carried in the request is regarded as new data to be written or changed data. For example, a new data value is written from a client to a service node in the distributed storage system.

It should be noted that, in the embodiment of the present invention, when a client initiates a data change and needs to write a new value into a service node in the distributed storage system, the client may first connect to the host node and send a write request carrying changed data to the host node. That is, the client may send all requests carrying the changed data to the master node in the system first.

It should be understood that in embodiments of the present invention, the client may be other devices different from the master node, such as a mobile phone, a laptop, a tablet, a desktop, a server, a virtual machine, a smart tv, a vehicle device, a wearable device, an industrial device, and the like.

In some embodiments, the master node may receive a plurality of task requests sent by the client, and sequentially place the plurality of task requests in a buffer in a message queue in a time sequence.

In some embodiments, the master node may receive a set of task requests sent by the client, the set of task requests including a plurality of requests; the master node may combine the multiple requests and package them to generate a task request package. Or the main node can directly receive a task request packet sent by the client; wherein, the task request packet comprises a plurality of requests.

In some embodiments, the master node may receive a set of task requests comprising a plurality of requests sent by the client; the plurality of requests are corresponding to the same thing number, and are related to each other based on the same thing number, wherein the change data carried by one task request fails to be written into the service node in the distributed storage system, and all the task requests related to the task request fail at the same time. On the contrary, the data of one task request is successfully written, and the data of other task requests associated with the task request is also successfully written.

Further, in the embodiment of the present invention, after receiving the request carrying the task data sent by the client, the master node may further write the task data into the slave node.

And S102, responding to the current task request, and sending a write instruction of current task data to an SCM (local area network) module of the slave node by utilizing a first RDMA (remote direct memory access) module according to a RAFT (random access field) protocol.

In an embodiment of the present invention, after receiving the task request, the master node may further respond to the request to directly send a write instruction of data to the SCM of the slave node.

It will be appreciated that the RDMA module supports remote direct memory data access, and therefore, in embodiments of the invention, the host node may be configured with an RDMA module to enable the host node to support direct memory data access to the slave node for data sent by the client.

In addition, the slave node is also provided with an RDMA module, and after the communication is established based on the RDMA module, the master node and the slave node can directly write data into the memory of the slave node.

Furthermore, in order to prevent the data from being lost after power failure, the slave node may be provided with a persistent memory module, i.e., an SCM module, so that the master node can directly send a data write instruction to the SCM module of the slave node, that is, directly write data to the SCM of the slave node without participation of the CPU.

In summary, the RDMA module is configured by the master node and the slave node at the same time, and a communication connection is established based on the RDMA module to complete the direct writing of data by the master node into the SCM module of the slave node.

It should be understood that, in the embodiment of the present invention, data is written into the service node in the system according to the RAFT protocol, therefore, after receiving each task request (e.g., data change) from the client, in response to the request, the master node adds each change data as an entry to the log of the master node, where the entry corresponds to a log number, and the log is in an uncommitted state, and then the master node needs to copy the change data to all slave nodes by performing commit of the entry.

Specifically, the master node may implement the replication by means of an application Message according to the RAFT protocol. Further, the master node may perform the submission of entries in the form of an application Message and based on the RDMA Write technology supported by the RDMA module, i.e., each time the change data is written directly to the SCM module of the slave node, to achieve the copying of the data to the slave node.

Therefore, according to the data processing method based on the RAFT protocol applied to the master node, the master node can directly write data into the SCM module of the slave node by using the RDMA module according to the RAFT protocol without the participation of a CPU (Central processing Unit), so that the system throughput rate is further improved, and the data transmission efficiency is higher.

In other embodiments of the present invention, fig. 4 is a flowchart illustrating an implementation flow of a data processing method based on a RAFT protocol according to an embodiment of the present invention, and as shown in fig. 4, after sending a write instruction of task data to an SCM module of a slave node according to the RAFT protocol by using the first RDMA module according to the task request, that is, after step 102, the method for a master node to perform data processing based on the RAFT protocol may include the following steps:

s103, in response to the write instruction of successfully sending the task data to the slave nodes meeting the preset number, determining the task number corresponding to the task data as the successful data submission serial number of the current data.

It can be understood that, since the master node directly writes data into the memory of the slave node based on the RDMA Write technology supported by the RDMA module, the master node can determine that the data is successfully written into the slave node as long as it determines that the data Write instruction is successfully sent.

In addition, when the RDMA module performs spatial information interaction, if a task is completed, the completed task is placed in a Completion Queue (CQ), and the master node may query the Completion Queue CQ based on the bottom layer packet to know which tasks are currently successful.

In the embodiment of the invention, when the master node performs the write operation of the data to the slave nodes, the master node does not need to wait for the response message of the successful write of the data returned by the slave nodes, and the master node can confirm whether the number of the slave nodes successfully sending the write instruction of the current task data meets the preset number or not through the completion queue CQ, so as to determine whether the task data is successfully copied in the distributed storage system or not.

Specifically, after the master node successfully sends the write instruction of the task data to the slave nodes meeting the preset number, if the master node determines that the write instruction of the task data has been successfully sent to the slave nodes meeting the preset number based on the completion queue CQ, it indicates that the change data sent by the client has been successfully copied to the service node of the distributed storage system.

It should be noted that the preset number may refer to more than half of the nodes in the entire distributed storage system. After the master node determines that the write instruction of the task data is successfully sent to more than half of the slave nodes, the master node can determine that the task data is successfully copied to more than half of the slave nodes in the distributed system, namely the change data sent by the client at this time is successfully copied to the distributed storage system.

In the embodiment of the present invention, the task number corresponding to the task data refers to a log number corresponding to an entry of the task data in the log. The data successful commit sequence number is the sequence number of the entry that indicates that the data copy was successful.

Further, after determining that the changed data sent by the client is successfully copied to the distributed storage system, that is, the entry corresponding to the data of the current task is successfully submitted, the master node may determine the log number corresponding to the entry as the successful submission sequence number of the current data, that is, the commit number.

For example, the log includes a plurality of entries corresponding to a plurality of change data, each entry has its number, such as 1, 2, 3, 4, and 5, and then when the change data of 1 is successfully copied to more than half of the slave nodes, it indicates that the change data with the number of 1 is successfully copied to the distributed storage system, and then the current data successful commit sequence number is updated to 1; further, when the changed data of 2 is successfully copied to more than half of the slave nodes, the current data successful submission sequence number is updated to 2. In addition, if the changed data of 3 is not successfully copied to more than half of the slave nodes, the current data successful submission sequence number is still 3.

Further, in the embodiment of the present invention, after the master node determines that the task data is successfully copied to the plurality of service nodes in the distributed storage system, the master node determines that the entry corresponding to the change data is successfully submitted, and at this time, the log state corresponding to the entry is a submission state.

Further, in the embodiment of the present invention, after obtaining the current successful data submission sequence number, the master node may further send the successful data submission sequence number to each of the plurality of slave nodes to which the task data is successfully written.

And S104, sending a write instruction of the current data successful submission serial number to each SCM module of each slave node in the preset number of slave nodes by using the first RDMA module.

In the embodiment of the present invention, after obtaining the current successful data submission sequence number, the master node may send a write instruction of the successful data submission sequence number to the SCM module of each of the plurality of slave nodes that successfully write task data through the RDMA module, that is, directly write the successful data submission sequence number into the SCM module of the slave node by using the RDMA module.

In some embodiments, the master node may implement the SCM module writing the successful commit sequence number of the data to the slave node in the form of an application Message in the RAFT protocol and based on RDMA Write technology supported by the RDMA module.

The master node may separately send a Write instruction of the current successful data submission sequence number to the slave node, may also merge the current successful data submission sequence number with task data sent by the next client, and Write the merged content into an SCM module of the slave node in an application Message form by using an RDMA Write technology.

It will be appreciated that in a distributed storage system based on the RAFT protocol, the master node may periodically trigger heartbeat data, informing the slave nodes of the number of entries in the last committed log. Therefore, in the implementation of the invention, after the master node obtains the current successful data commit sequence number, the master node may provide the successful data commit sequence number to the slave node in the next heartbeat data based on the RAFT protocol, that is, the successful data commit sequence number of the current data is directly written to the SCM module of the slave node by using the RDMA Write technology, or is sent to the slave node by using the RDMA Send technology; wherein, the heartbeat data only contains the latest successful data submission serial number.

Further, in the embodiment of the present invention, the master node may return a prompt message indicating that the task is successful to the client.

Further, in the embodiment of the present invention, the master node may perform log application LogApply on current task data successfully written into the majority of slave nodes, that is, determine that the task data is applicable data, and notify that the upper layer application data has been effectively stored in the majority of service nodes in the distributed storage system, so that the upper layer application can perform some related processing.

Therefore, in the embodiment of the invention, the master node can directly write the data into the persistent memory of the slave node, and directly determines the success of data writing through the bottom layer message, and does not need to wait for the return response sent by the slave node, thereby reducing the communication content and reducing the data transmission delay.

Based on the foregoing embodiment, in a further embodiment of the present invention, fig. 5 is a schematic flow chart of an implementation process of a data processing method based on a RAFT protocol according to the embodiment of the present invention, and as shown in fig. 5, in the embodiment of the present invention, a method for a master node to perform data processing based on the RAFT protocol may include the following steps:

s201, receiving a plurality of task requests and buffering the task requests to a message queue.

S202, after a write instruction of at least one piece of task data carried by at least one historical task request in the message queue is sent to the SCM module of the slave node, the write instruction of the current task data is sent to the SCM module of the slave node by utilizing the first RDMA module in response to the current task request.

It should be understood that, in order to overcome the defect that the RAFT protocol needs strict order preservation and cannot implement concurrent submission, in the embodiment of the present invention, the master node sets a message queue buffer, and the buffer is located in the memory of the master node. When the master node receives a task request sent by the client, the master node can sequentially place the task request into the message queue buffer according to the time sequence.

Further, the master node may implement concurrent submission of task data carried in the task request based on the message queue.

Specifically, when the master node does not determine that the Write instruction for the task data carried by at least one task request in the Message queue is successfully sent to the SCM modules of more than a preset number (e.g., more than half) of the slave nodes, the master node may continue to send the Write instruction for the task data carried by the next task request to the SCM module of the slave node through the application Message format and based on the RDMA Write technology.

That is, in the entries of the task data sequentially arranged according to the numbers in the log, after the entry corresponding to at least one task data with the front number is submitted, the task is suspended, and before whether the at least one entry is successfully submitted is not determined, the master node can continue to submit the entry corresponding to the next task data.

Therefore, the master node caches the data change requests sent by the client according to the queue mode, and can continue to perform write operation from data to the memory of the slave node based on the next data change request initiated by the client without waiting for whether the task is successfully completed or not when executing the current data change request sent by the client, namely the master node can submit a plurality of requests in sequence, so that concurrent submission of data is realized, the system throughput rate is further improved, and the data transmission efficiency is higher.

Based on the foregoing embodiment, in yet another embodiment of the present invention, as shown in fig. 6, a fourth implementation flow diagram of the data processing method based on the RAFT protocol provided in the embodiment of the present invention is shown, and as shown in fig. 6, the method for the master node to execute data processing based on the RAFT protocol may include the following steps:

s301, receiving a plurality of data to be written in virtual addresses sent by a plurality of slave nodes.

S302, according to the length of the task data, a first virtual address is determined from the data to be written in the virtual address of each slave node.

S303, sending a writing instruction of the task data to a first physical address corresponding to the first virtual address in each SCM module of each slave node by using the first RDMA module.

In the embodiment of the invention, the master node can receive data to be written in a virtual address, namely a data available free space, which is a virtual address corresponding to a physical address of an SCM (service management unit) module in the slave node, sent by the slave node in a private message form or a custom command form in real time or periodically, and the master node can actively determine a corresponding storage position for current task data from the data available free space.

Specifically, the master node may determine a length corresponding to the current task data, and calculate a virtual address space, i.e., a first virtual address, corresponding to the current task data from the virtual address to be written in the data according to the data length.

Optionally, the master node may also determine the first virtual address corresponding to the current task data from the virtual addresses to be written in the data according to a preset address allocation rule, for example, the preset address allocation rule may be that each data is allocated with 1k space.

Further, after the first virtual address corresponding to the data is determined, the master node determines the virtual memory location where the current task data is written, and then the master node may write the task data into the SCM module and the physical address corresponding to the virtual address, that is, directly send the write instruction of the current task data to the SCM module and the physical address corresponding to the virtual address.

It can be seen that, in the embodiment of the present invention, when the master node performs a Write operation of task data based on the RDMA Write technology supported by the RDMA module, the master node may actively determine a memory Write location of the data, and perform a Write operation of the data to the SCM module of the slave node based on the memory Write location.

In other embodiments of the present invention, fig. 7 is a schematic flow chart illustrating an implementation flow of a data processing method based on a RAFT protocol according to an embodiment of the present invention, as shown in fig. 7, a method for a slave node to perform data processing based on a RAFT protocol may include the following steps:

s401, receiving a writing instruction of the task data sent by the main node by using a second RDMA module.

S402, responding to the writing instruction, and writing the task data into the SCM module.

And S403, when the current data corresponding to the task data in the SCM module is determined to be successfully submitted with the serial number, determining the task data as the applicable data.

It should be noted that, in the embodiment of the present invention, the RDMA module is configured at the same time for the master node and the slave node, and the establishment of the communication connection based on the RDMA module can implement that the master node directly writes data into the SCM module of the slave node, that is, directly writes the persistent memory of the task data based on the write instruction sent by the master node.

It should be appreciated that after the master node actively sends a write command for data to the SCM module of the slave node based on the RDMA module, the slave node may receive the write command for the data using the RDMA module, and may directly write the data to persistent memory, i.e., the SCM module, in response to the write command without involvement of the CPU.

Optionally, in the embodiment of the present invention, the task data may be task data corresponding to one task request of the client, or may be corresponding task data obtained by packaging a group of task requests including multiple requests; the task data can also be task data corresponding to any one task request in a group of task requests which have the same object number and have the association relationship.

Further, in this embodiment of the present invention, based on the fact that, in S103 to S104, if the master node determines that the task data has been successfully copied to more than half of the slave nodes, the master node may send a write instruction of successfully submitting a serial number to the SCM module of the slave node, and the slave node may periodically query the successfully submitted serial number written by the SCM module, and as long as the slave node queries the successfully submitted serial number corresponding to the latest written task data from the SCM module, the slave node may perform log application LogApply on the task data, that is, determine that the task data is determined to be applicable data, and the slave node may notify that the upper layer application data has been effectively stored to most service nodes in the distributed storage system, and may be taken by the upper layer application to perform some related processing.

In other embodiments, the slave node may further receive heartbeat data sent by the master node, the slave node may determine a successful data submission sequence number carried in the heartbeat data, and if the successful data submission sequence number corresponds to the task data written latest, the slave node may execute log application to the data, that is, determine that the task data is determined to be applicable data, and may be taken by an upper layer application to perform some related processing.

Therefore, in the embodiment of the invention, the slave node does not need to participate in a CPU, can directly write data into the persistent memory according to the write instruction sent by the master node, and does not need to return a response message to the master node, thereby reducing communication contents and reducing data transmission delay.

Based on the foregoing embodiment, fig. 8 is a sixth schematic flow chart illustrating an implementation flow of the data processing method based on the RAFT protocol according to the embodiment of the present invention, and as shown in fig. 8, the method for the slave node to execute data processing based on the RAFT protocol may include the following steps:

s601, performing memory mapping processing on the physical address corresponding to the SCM module to obtain a virtual address corresponding to the physical address.

S602, registering the virtual address to the second RDMA module so that the second RDMA module writes the task data into the physical address in the SCM module by accessing the virtual address.

It can be understood that, if the implementation of the present invention requires that the primary node directly write data into the persistent memory of the secondary node by using the RDMA module, and that the write persistent memory of the data needs to be implemented based on the communication between the RDMA module of the primary node and the RDMA module of the secondary node, the storage space corresponding to the SCM module needs to be mapped to the virtual memory of the device operating system, and the virtual memory needs to be registered on the RDMA module. Thus, the RDMA module may have access to this portion of the virtual memory region.

It can be seen that in the embodiments of the present invention, the memory mapping and the registration need to be completed before the data processing.

Specifically, in the embodiment of the present invention, the slave node may first perform memory mapping processing on the physical address corresponding to the SCM module, and map the physical address to the device local memory space, that is, the virtual address space, thereby establishing a mapping relationship between the physical address and the virtual address, such as an address mapping table, so as to obtain the virtual address corresponding to the physical address.

Alternatively, all or part of the space of the SCM module may be mapped to the virtual address space of the device by means of dev-dax or fs-dax.

Further, after mapping between the physical address space and the virtual address space is completed, the slave node may register all or part of the virtual address space to the native RDMA module, and after the registration is successful, the RDMA module may obtain the corresponding virtual address and mr returned by the registration (the mr may be regarded as a "key" for controlling and managing access rights of the memory), and further has a right to access the virtual address space.

The RDMA module can access the virtual address space based on mr returned by registration and realize the writing of data into the physical address in the SCM module; it should be appreciated that the RDMA module writes the log to a memory virtual address, in effect writing the data to the physical address corresponding to the virtual address, i.e., persistent memory.

It should be noted that, in the embodiment of the present invention, after the slave node completes the mapping of the memory and registers it to the RDMA module, the slave node may further provide the native corresponding virtual address and mr returned by registration to other nodes (not limited to the master node or other slave nodes) in the form of private messages through the RDMA module.

The slave node may set a timing module, such as a timer, and may send the current free space, that is, the data to be written in the virtual address, to the master node in a form of a custom command or a private message in real time or periodically.

As can be seen, the slave node may enable the RDMA module to write task data to a physical address in the SCM module through memory access by mapping the physical address space of the SCM module to virtual memory and registering onto the RDMA module.

Based on the foregoing embodiment, fig. 9 is a seventh schematic flow chart illustrating an implementation flow of the data processing method based on the RAFT protocol according to the embodiment of the present invention, and as shown in fig. 9, the method for the slave node to execute data processing based on the RAFT protocol may include the following steps:

s701, receiving the first write virtual address sent by the main node by using a second RDMA module.

S702, determining a first writing physical address corresponding to the first writing virtual address from the SCM module.

And S703, writing the task data into the first write physical address by using the second RDMA module.

It can be understood that the master node needs to receive a write instruction of data through the RDMA module in the slave node and write the persistent memory with the data, and therefore, the master node needs to send a memory address of the data in the slave node to the slave node, so that the slave node writes the persistent memory with the task data based on the memory address.

In the embodiment of the present invention, while the slave node receives the write instruction of the task data sent by the master node by using the RDMA module, the slave node may also receive the first virtual address corresponding to the task data sent by the master node by using the RDMA module.

Further, the RDMA module may determine, based on the mapping relationship between the virtual address and the physical address, such as an address mapping table, a first physical address corresponding to the first virtual address from the SCM module, and then write data into the first physical address, thereby completing writing data into the persistent memory.

Therefore, the slave node can receive the data sent by the master node and the virtual address determined by the master node for the data, and write the task data into the persistent memory based on the address mapping relationship.

Based on the foregoing embodiment, fig. 10 is an implementation flow diagram of an eighth implementation flow of the data processing method based on the RAFT protocol according to the embodiment of the present invention, and as shown in fig. 10, the method for the slave node to execute data processing based on the RAFT protocol may include the following steps:

s801, receiving a write instruction of a successful sequence number submission of current data sent by a main node by using a second RDMA module.

S802, responding to the writing instruction, and writing the successful submitting serial number of the current data into the SCM module.

In the embodiment of the present invention, based on S103 and S104, after the master node determines that the write instruction for successfully sending data to the slave nodes satisfying the preset number is successful, and sends the write instruction for successfully submitting the serial number to the slave nodes, the slave nodes may directly write the successfully submitted serial number into the SCM module of the slave nodes, that is, perform write persistence of the successfully submitted serial number of the data based on the write instruction sent by the master node.

Specifically, the slave node may receive, by using the RDMA module, a write instruction of the current data successful commit sequence number sent by the master node, and may also receive, by using the RDMA module, a target virtual address, that is, a second virtual address, determined by the master node for the sequence number from the data to be written virtual addresses, and the RDMA module may determine, based on a mapping relationship between the virtual address and the physical address, such as an address mapping table, a second physical address corresponding to the second virtual address from the SCM module, and then write the current data successful commit sequence number into the second physical address, thereby completing the commit of the sequence number to write the persistent memory.

Therefore, the slave node can receive the current data successfully submitted sequence number sent by the master node and the virtual address determined by the master node for the sequence number, and write the sequence number into the persistent memory based on the address mapping relationship.

For example, fig. 11 is a schematic flow chart of a method for processing data in a distributed storage system according to an embodiment of the present invention, where a master node and a slave node in the distributed system are both provided with an ethernet card with an RDMA engine module, that is, an RDMA Network Interface Controller (RNIC), and the RNIC in the master node establishes a communication connection with the RNIC in the slave node to implement interaction of spatial information.

As shown in fig. 11, the master node receives a client request (Sb1), carries data through an application Message and transmits the data to the RNIC of the slave node through the RNIC using RDMA Write technology (Sb 2). At this time, instead of waiting for a successful data response from the slave node, a message indicating whether data is successfully sent to the slave node is obtained based on the RDMA underlying packet, such as the query completion queue CQ (Sb3), and it is determined whether more than half of the slave nodes are successfully sent based on the completion queue CQ (Sb 4)? If not, continuously inquiring the CQ completion queue until more than half of the CQ completion queue is searched; or that the data submission to the slave failed. If the data is successfully submitted from the node, the data consistency storage is successful. Further, the master node may update the data successful commit sequence number based on the number of the current task data (Sb5) and send to the slave RNIC through the RNIC using RDMA Write technology in an application Message. At the same time, the master node may also perform LogApply on the data (Sb 6).

On the other hand, the RNIC of the slave node directly writes the data sent by the master node into the persistent memory (Sb7), without the participation of the CPU and the feedback of any response message by the master node, further, the slave node may periodically check the successful submission sequence number of the data written in the persistent memory (Sb8), and determine whether there is a submission sequence number corresponding to the newly submitted task data in the persistent memory (Sb9), if so, log application is performed on the corresponding data (Sb 10); if not, the persistent memory is continuously checked regularly.

In summary, by providing RDMA modules at the master node and the slave nodes in the distributed storage system, the master node can Write data directly to the persistent memory of the slave nodes using RDMA Write technology. The return response of the slave node is not required to wait, and the participation of a CPU of the slave node is not required, so that the system throughput rate is further improved, and the data transmission efficiency is higher.

Based on the foregoing embodiments, in an embodiment of the present invention, fig. 12 is a schematic structural diagram of a composition of a data processing device based on a RAFT protocol according to an embodiment of the present invention, as shown in fig. 12, the data processing device 10 based on a RAFT protocol includes a receiving unit 12, a sending unit 13, a buffering unit 14, and a determining unit 15,

the receiving unit 12 is configured to receive a current task request; wherein the current task request carries current task data;

the sending unit 13 is configured to send, in response to the current task request, a write instruction of the current task data to an SCM module of a slave node by using the first RDMA module according to the RAFT protocol.

In some embodiments, the receiving unit 12 is further configured to receive a plurality of task requests;

the buffer unit 14 is configured to buffer the plurality of task requests to a message queue;

the sending unit 13 is further configured to, after sending, to the SCM module of the slave node, a write instruction of at least one task data carried by at least one historical task request in the message queue, send, by using the first RDMA module, the write instruction of the current task data to the SCM module of the slave node in response to the current task request.

In some embodiments, the receiving unit 12 is further configured to receive a plurality of data to be written to virtual addresses sent by a plurality of slave nodes;

the determining unit 15 is configured to determine, according to the length of the task data, a first virtual address from the virtual addresses to be written in the data of each slave node;

the sending unit 13 is specifically configured to send, by using the first RDMA module, a write instruction of the task data to a first physical address corresponding to the first virtual address in each SCM module of each slave node.

In some embodiments, the determining unit 15 is further configured to, after sending the write instruction of the task data to the SCM module of the slave node according to the RAFT protocol by using the first RDMA module in response to the task request, determine, as the current data successful submission sequence number, a task number corresponding to the task data in response to the write instruction of the task data successfully sent to the slave node that satisfies a preset number;

the sending unit 13 is further configured to send, by using the first RDMA module, a write instruction of the current data successful commit sequence number to each SCM module of each of the slave nodes, where the write instruction satisfies the preset number of slave nodes.

Based on the foregoing embodiments, in an embodiment of the present invention, fig. 13 is a schematic structural diagram of a composition structure of a data processing device based on a RAFT protocol according to an embodiment of the present invention, as shown in fig. 13, the data processing device 20 based on a RAFT protocol includes a receiving unit 23, a writing unit 24, a determining unit 25, a memory mapping unit 26, and a registering unit 27,

the receiving unit 23 is configured to receive, by using the second RDMA module, a write instruction of task data sent by a host node;

the writing unit 24 is configured to respond to the write instruction, and write the task data into the SCM module;

the determining unit 25 is configured to determine the task data as applicable data when it is determined that the current data successful submission serial number corresponding to the task data exists in the SCM module.

In some embodiments, the receiving unit 23 is further configured to receive, by the second RDMA module, a write instruction of the current data successful commit sequence number sent by the master node;

the writing unit 24 is further configured to respond to the write instruction, and write the successful data submission serial number into the SCM module.

In some embodiments, the memory mapping unit 26 is configured to perform memory mapping processing on a physical address corresponding to the SCM module to obtain a virtual address corresponding to the physical address;

the registration unit is configured to register the virtual address with the second RDMA module, so that the second RDMA module writes the task data to the physical address in the SCM module by accessing the virtual address.

In some embodiments, the receiving unit 23 is further configured to receive, by using the second RDMA module, the first write virtual address sent by the master node;

the writing unit 24 is specifically configured to determine, from the SCM module, a first writing physical address corresponding to the first writing virtual address; and writing the task data to the first write physical address with the second RDMA module.

In an embodiment of the present invention, further, fig. 14 is a schematic diagram illustrating a composition structure of a data processing device based on a RAFT protocol, as shown in fig. 14, the data processing device 10 based on the RAFT protocol according to the embodiment of the present invention may further include a first RDMA module 11, a processor 16, and a memory 17 storing executable instructions of the processor 16, and further, the data processing device 10 based on the RAFT protocol may further include a communication interface 18, and a bus 19 for connecting the processor 16, the memory 17, and the communication interface 18.

In an embodiment of the present invention, the Processor 16 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a ProgRAMmable Logic Device (PLD), a Field ProgRAMmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It will be appreciated that the electronic devices used to implement the processor functions described above may be other devices, and embodiments of the present invention are not limited in particular. The data processing device 10 based on the RAFT protocol may further comprise a memory 17, which memory 17 may be connected to the processor 16, wherein the memory 17 is configured to store executable program code comprising computer operating instructions, and the memory 17 may comprise a high speed RAM memory, and may further comprise a non-volatile memory, such as at least two disk memories.

In an embodiment of the invention, a bus 19 is used to connect the communication interface 18, the processor 16 and the memory 17 and the intercommunication between these devices.

In an embodiment of the invention, the memory 17 is used for storing instructions and data.

Further, in an embodiment of the present invention, the processor 16 is configured to receive a current task request; wherein the current task request carries current task data; and responding to the current task request, and sending a writing instruction of the current task data to an SCM (local area network) module of a slave node by utilizing the first RDMA module according to the RAFT protocol.

In practical applications, the Memory 18 may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (Hard Disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor 16.

In the embodiment of the present invention, further, fig. 15 is a schematic diagram illustrating a composition structure of a data processing apparatus based on a RAFT protocol according to the present invention, as shown in fig. 15, the data processing apparatus 20 based on a RAFT protocol according to the present invention may further include a second RDMA module 21, an SCM module 22, a processor 28, and a memory 29 storing executable instructions of the processor 28, and further, the data processing apparatus 20 based on a RAFT protocol may further include a communication interface 210, and a bus 211 for connecting the processor 28, the memory 29, and the communication interface 210.

In an embodiment of the present invention, the Processor 28 may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a ProgRAMmable Logic Device (PLD), a Field ProgRAMmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It will be appreciated that the electronic devices used to implement the processor functions described above may be other devices, and embodiments of the present invention are not limited in particular. The data processing device 20 based on the RAFT protocol may further comprise a memory 29, which memory 29 may be connected to the processor 28, wherein the memory 29 is configured to store executable program code comprising computer operating instructions, and the memory 29 may comprise a high speed RAM memory, and may further comprise a non-volatile memory, such as at least two disk memories.

In an embodiment of the present invention, bus 211 is used to connect communication interface 210, processor 28, and memory 29, and the intercommunication among these devices.

In an embodiment of the present invention, memory 29 is used for storing instructions and data.

Further, in an embodiment of the present invention, the processor 28 is configured to receive, by using the second RDMA module, a write instruction of task data sent by the host node; writing the task data into the SCM module in response to the write instruction; and when the task data corresponding to the current data successful submission serial number exists in the SCM module, determining the task data as the applicable data.

In practical applications, the Memory 29 may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (Hard Disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to processor 28.

In addition, each functional module in this embodiment may be integrated into one recommendation unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware or a form of a software functional module.

Based on the understanding that the technical solution of the present embodiment essentially or a part contributing to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium, and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The embodiment of the invention provides data processing equipment based on a RAFT (reversible addition fragmentation chain transfer) protocol, which is used as the data processing equipment of a main node and is provided with a first RDMA (remote direct memory access) module, wherein the main node can receive a current task request; wherein, the current task request carries current task data; and responding to the current task request, and sending a write instruction of the current task data to an SCM (local area network) module of the slave node by utilizing the first RDMA module according to a RAFT (random access field) protocol. Correspondingly, the data processing equipment serving as the slave node is provided with a second RDMA module and an SCM module, and the slave node receives a writing instruction of task data sent by the master node by using the second RDMA module; responding to the writing instruction, and writing the task data into the SCM module; and when the task data corresponding to the current data successful submission serial number exists in the SCM module, determining the task data as the applicable data. Therefore, the RDMA module can be used for directly writing the data into the memory of the slave node, the return response of the slave node is not required to wait, and the CPU of the slave node is not required to participate, so that the system throughput rate is further improved, and the data transmission efficiency is higher.

An embodiment of the present invention provides a computer-readable storage medium on which a program is stored, which, when executed by a processor, implements the RAFT protocol-based data processing method applied to a master node or a slave node as described above.

Specifically, the program instructions corresponding to the data processing method based on the RAFT protocol in this embodiment may be stored on a storage medium such as an optical disc, a hard disk, or a usb disk.

When a program instruction corresponding to a data processing method based on a RAFT protocol applied to a master node in a storage medium is read or executed by an electronic device, the method includes the steps of:

receiving a current task request; wherein the current task request carries current task data;

and responding to the current task request, and sending a writing instruction of the current task data to an SCM (local area network) module of a slave node by utilizing the first RDMA module according to the RAFT protocol.

When program instructions in a storage medium corresponding to a data processing method based on a RAFT protocol applied to a slave node are read or executed by an electronic device, the method includes the steps of:

receiving a write instruction of task data sent by a main node by utilizing the second RDMA module;

writing the task data into the SCM module in response to the write instruction;

and when the task data corresponding to the current data successful submission serial number exists in the SCM module, determining the task data as the applicable data.

Accordingly, an embodiment of the present invention further provides a computer program product, where the computer program product includes computer-executable instructions for implementing steps in the data processing method based on the RAFT protocol, which is applied to the master node or the slave node and provided by the embodiment of the present invention.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of implementations of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or flowchart block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks in the flowchart and/or block diagram block or blocks. The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.

Claims

1. A data processing method based on a distributed consistent RAFT protocol, the data processing method being applied to a primary node, the primary node configuring a first remote direct data access, RDMA, module, the method comprising:

2. The method of claim 1, further comprising:

receiving a plurality of task requests and caching the task requests to a message queue;

after sending a write instruction of at least one task data carried by at least one historical task request in the message queue to the SCM module of the slave node, responding to the current task request, and sending the write instruction of the current task data to the SCM module of the slave node by using the first RDMA module.

3. The method of claim 1, further comprising:

receiving a plurality of data to be written in virtual addresses sent by a plurality of slave nodes;

determining a first virtual address from the data to be written virtual addresses of each slave node according to the length of the task data;

the sending, by the first RDMA module according to the RAFT protocol, a write instruction of the task data to an SCM module of a slave node in response to the task request includes:

sending a write instruction of the task data to a first physical address corresponding to the first virtual address in each SCM module of each slave node by using the first RDMA module.

4. The method of claim 3, wherein after sending a write instruction of the task data to a SCM module of a slave node using the first RDMA module according to the RAFT protocol in response to the task request, the method further comprises:

in response to a write instruction for successfully sending the task data to the slave nodes meeting the preset number, determining a task number corresponding to the task data as a successful data submission serial number of the current data;

sending, by the first RDMA module, a write instruction of the current data successful commit sequence number to each SCM module of each of the slave nodes that satisfy the preset number of slave nodes.

5. A data processing method based on a RAFT protocol, applied to a slave node, the slave node configuring a second RDMA module and a storage class memory SCM module, the method comprising:

writing the task data into the SCM module in response to the write instruction;

6. The method of claim 5, further comprising:

receiving a write instruction of the current data successful submission sequence number sent by a main node by using the second RDMA module;

and responding to the write instruction, and writing the successful data submission serial number into the SCM module.

7. The method of claim 5, further comprising:

performing memory mapping processing on a physical address corresponding to the SCM module to obtain a virtual address corresponding to the physical address;

register the virtual address with the second RDMA module to cause the second RDMA module to write the task data to the physical address in the SCM module by accessing the virtual address.

8. The method of claim 7, further comprising:

receiving a first virtual address sent by the master node by using the second RDMA module;

the writing the task data into the SCM module in response to the write instruction includes:

determining a first physical address corresponding to the first virtual address from the SCM module;

writing the task data to the first physical address with the second RDMA module.

9. A data processing device based on a RAFT protocol, characterized in that the data processing device is configured with a first RDMA module, the data processing device comprising a receiving unit and a sending unit,

the receiving unit is used for receiving a current task request; wherein the current task request carries current task data;

10. A data processing device based on a RAFT protocol, characterized in that the data processing device is configured with a second RDMA module and an SCM module, the data processing device comprising a receiving unit, a writing unit and a determining unit,