WO2024109239A1 - 集群数据同步方法、装置、设备及非易失性可读存储介质 - Google Patents

集群数据同步方法、装置、设备及非易失性可读存储介质 Download PDF

Info

Publication number
WO2024109239A1
WO2024109239A1 PCT/CN2023/116231 CN2023116231W WO2024109239A1 WO 2024109239 A1 WO2024109239 A1 WO 2024109239A1 CN 2023116231 W CN2023116231 W CN 2023116231W WO 2024109239 A1 WO2024109239 A1 WO 2024109239A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
synchronized
synchronization
node
cluster
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/116231
Other languages
English (en)
French (fr)
Inventor
黄炎
张璐
王见
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Suzhou Metabrain Intelligent Technology Co Ltd
Original Assignee
Suzhou Metabrain Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Suzhou Metabrain Intelligent Technology Co Ltd filed Critical Suzhou Metabrain Intelligent Technology Co Ltd
Publication of WO2024109239A1 publication Critical patent/WO2024109239A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
    • G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory

Definitions

  • the present application relates to the technical field of data synchronization, and in particular to a cluster data synchronization method, device, equipment and non-volatile readable storage medium.
  • Raft is a simpler, more convenient and easier to understand distributed algorithm.
  • RDMA Remote Direct Memory Access
  • TCP Transmission Control Protocol
  • UDP User Datagram Protocol
  • RDMA can be based on a variety of underlying protocols, with high flexibility. It does not need to switch between application state and kernel state and copy memory multiple times when sending data like TCP/UDP protocols, and has better performance.
  • the RDMA protocol comes with a lossless data network, which can solve the problems of sticky packets and retransmission of the TCP protocol and the problem of packet loss of the UDP protocol.
  • data sent based on the RDMA protocol does not need to be scheduled by the Central Processing Unit (CPU), which can save CPU resources and avoid interrupts.
  • CPU Central Processing Unit
  • Raft protocol is based on the basic sending and receiving logic, that is, RDMA is only used as a parallel alternative to TCP/UDP.
  • RDMA is only used as a parallel alternative to TCP/UDP.
  • this solution can implement the logic of the Raft protocol, it does not combine the unique characteristics of the RDMA network, resulting in the performance advantages of RDMA not being fully utilized, and the problem of low data synchronization efficiency still exists.
  • the purpose of the present application is to provide a cluster data synchronization method, device, equipment and non-volatile readable storage medium, which are configured to optimize the Raft protocol cluster data synchronization in combination with the RDMA network, and give full play to the performance advantages of RDMA in the Raft protocol cluster.
  • a cluster data synchronization method based on a master node in a distributed cluster, comprising:
  • the synchronization data receiving pool in the local memory is written by the source node based on the RDMA network calling communication device to be synchronized, the synchronization data is verified;
  • the synchronization task of the data to be synchronized is executed in the local memory, and the communication device is called based on the RDMA network in the form of broadcast to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized;
  • the source node is a slave node in the distributed cluster.
  • the data to be synchronized that is detected to be written by the source node into the synchronization data receiving pool in the local memory by calling the communication device based on the RDMA network may be:
  • the task completion callback function based on the RDMA network updates the synchronization request field in the CPU of the master node, confirming that the source node has written the data to be synchronized;
  • the communication device is called to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, which can be:
  • the synchronization request field is updated in the CPU of the slave node based on the task completion callback function, so that the slave node completes the verification of the data to be synchronized and executes the synchronization task of the data to be synchronized after monitoring the updated synchronization request field.
  • the data to be synchronized that is detected to be written by the source node into the synchronization data receiving pool in the local memory by calling the communication device based on the RDMA network may be:
  • the communication device is called to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, which can be:
  • the communication device is called to write the data to be synchronized into the synchronization data receiving pool of the slave node, so that the slave node can complete the verification of the data to be synchronized and execute the synchronization task of the data to be synchronized after polling and monitoring the data to be synchronized.
  • the data to be synchronized that is detected to be written by the source node into the synchronization data receiving pool in the local memory by calling the communication device based on the RDMA network may be:
  • the communication device is called to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, which can be:
  • the input and output ports are called to send a write completion flag to the slave node, so that after the slave node confirms that the data to be synchronized has been written, the verification of the data to be synchronized is completed and the synchronization task of the data to be synchronized is executed.
  • the data to be synchronized that is monitored to be written by the source node to the synchronization data receiving pool in the local memory by the communication device based on the RDMA network call may be:
  • the communication device is called to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, which can be:
  • the write completion flag is written to the agreed address of the write completion flag of each slave node, so that after the slave node confirms that the data to be synchronized has been written, the verification of the data to be synchronized is completed and the synchronization task of the data to be synchronized is executed.
  • the synchronization data receiving pool corresponds one-to-one to the remaining nodes in the distributed cluster except the node where the synchronization data receiving pool is located.
  • the synchronous data receiving pool is provided with an RDMA mapping array for maintaining the correspondence between the remaining nodes other than the local node and the synchronous data receiving pool, and an atomic index for allocating free space for the data to be synchronized.
  • the synchronization data receiving pool is provided with multiple sub-receiving pools corresponding to different types of data to be synchronized;
  • the synchronization task for the data to be synchronized is performed in the local memory, which can be:
  • the synchronization task of the data to be synchronized is executed in the local memory
  • the slave node performs synchronization tasks for the data to be synchronized, which can be:
  • the slave node executes the synchronization task for the data to be synchronized according to the type of the data to be synchronized.
  • the synchronization data receiving pool is pre-divided into data partitions of each type of sub-receiving pool according to the data length of each type of data to be synchronized.
  • types of sub-receiving pools include: a small data receiving pool, a response data receiving pool, and a large data receiving pool.
  • the data to be synchronized is a data type corresponding to the small data receiving pool
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the data to be synchronized into the first message header, it calls the communication device based on the RDMA network to write the first message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the communication device is called based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, which can be:
  • the communication device is called based on the RDMA network to write the first message header into the corresponding storage location.
  • the data to be synchronized is a data type corresponding to the response data receiving pool
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the data to be synchronized into the second message header, it calls the communication device based on the RDMA network to write the second message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the communication device is called based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, which can be:
  • the communication device is called based on the RDMA network to write the second message header into the corresponding storage location.
  • the first preset space of the response data receiving pool is used to store the heartbeat information of all nodes in the distributed cluster.
  • it also includes:
  • the second preset space of the response data receiving pool is used to store user-defined data.
  • the data to be synchronized is a data type corresponding to the large data receiving pool
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the synchronization task of the data to be synchronized into the third message header, it calls the communication device based on the RDMA network to write the third message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the synchronization task of the data to be synchronized is executed in the local memory, and the communication device is called based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster in the form of broadcast, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, including:
  • the communication device is called to write the third message header into the corresponding storage location, so that the slave node executes the logic contained in the third message header in the paging of the third preset space of the big data receiving pool of the slave node;
  • the local node and each slave node are controlled to copy the pages of the third preset space of the node to the cluster memory space of the node.
  • controlling the local node and each slave node to copy the pages of the third preset space of the node to the cluster memory space of the node may be:
  • a confirmation success response code is sent to each slave node, so that each slave node and the master node respectively perform the following steps after receiving the confirmation success response code:
  • the third message header only includes synchronization logic of the data to be synchronized
  • the third message header includes the synchronization logic of the data to be synchronized and the transaction callback function corresponding to the data to be synchronized, so that the master node or the slave node executes the logic corresponding to the transaction callback function when executing the synchronization task of the data to be synchronized;
  • the transaction callback function at least includes a data backup function for transaction data synchronization and a key data protection function.
  • the logic corresponding to executing the transaction callback function includes:
  • the logic of the synchronization task to execute the data to be synchronized is triggered;
  • a synchronization task execution completion message is returned to the master node
  • the key data protection is released and the data backup is deleted; otherwise, the callback operation after the error occurs is executed to release the key data protection and restore the cluster data to the initial state based on the data backup.
  • a cluster data synchronization device including:
  • a monitoring unit is configured to monitor a preset synchronous data receiving pool in a local memory
  • the verification unit is configured to verify the data to be synchronized when detecting that the synchronization data receiving pool in the local memory is written by the source node based on the RDMA network calling communication device;
  • the synchronization unit is configured to execute the synchronization task of the data to be synchronized in the local memory after the data to be synchronized passes the verification, and write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster in the form of broadcast based on the RDMA network, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized;
  • the source node is a slave node in the distributed cluster.
  • the present application also provides a cluster data synchronization device, including:
  • a memory arranged to store a computer program
  • the processor is configured to execute a computer program, and when the computer program is executed by the processor, the steps of any one of the above cluster data synchronization methods are implemented.
  • a non-volatile readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above cluster data synchronization methods are implemented.
  • the cluster data synchronization method provided by the present application provides a cluster data synchronization solution that optimizes the Raft protocol in combination with the RDMA network.
  • the source node that generates the data to be synchronized calls the communication device based on the RDMA network to directly write the data to be synchronized into the preset synchronization data receiving pool in the local memory of the master node.
  • the master node executes the synchronization task of the data to be synchronized in the local memory, and calls the communication device based on the RDMA network in the form of broadcast to directly write the data to be synchronized into the synchronization data receiving pool of the slave node, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized.
  • the data to be synchronized is transmitted in a direct writing manner, replacing the sending/receiving synchronization mode of the existing Raft protocol, which can avoid memory copying in the sending/receiving process, and the master node does not need to control IO (Input/Output) multiplexing in the cluster data synchronization work, and does not need flow control, which significantly reduces the amount of code for implementing synchronization and the CPU scheduling workload of the master node.
  • IO Input/Output
  • the present application also provides a cluster data synchronization device, equipment and non-volatile readable storage medium, which have the above-mentioned beneficial effects and are not described in detail here.
  • FIG1 is a flow chart of a cluster data synchronization method provided in an embodiment of the present application.
  • FIG2 is a schematic diagram of cluster memory distribution provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of the structure of a cluster data synchronization device provided in an embodiment of the present application.
  • FIG. 4 is a schematic diagram of the structure of a cluster data synchronization device provided in an embodiment of the present application.
  • the core of this application is to provide a cluster data synchronization method, device, equipment and non-volatile readable storage medium, which are configured to optimize the Raft protocol cluster data synchronization in combination with the RDMA network, and give full play to the performance advantages of RDMA in the Raft protocol cluster.
  • FIG1 is a flow chart of a cluster data synchronization method provided in an embodiment of the present application.
  • the cluster data synchronization method includes:
  • S101 monitoring a preset synchronous data receiving pool in a local memory.
  • the source node is a slave node in the distributed cluster.
  • the leader refers to the master node in the Raft protocol, which can be an arbitration node automatically elected in a distributed cluster that performs cluster data synchronization based on the Raft protocol.
  • the Raft protocol all follower nodes need to send messages to the leader when updating their status, and the leader broadcasts to the entire cluster.
  • the existing cluster data synchronization based on Raft protocol still adopts the IO sending and receiving mode similar to the traditional TCP/UDP protocol, resulting in that each time the cluster synchronization process sends data, the data to be synchronized needs to be copied locally to the CPU kernel, and then sent to the destination through IO, and the destination then copies it from the CPU kernel to the memory, which occupies a large amount of CPU resources and has low synchronization efficiency.
  • RDMA is only used as a parallel alternative to TCP/UDP.
  • the unique characteristics of the RDMA network are not combined, resulting in the performance advantages of RDMA not being fully utilized.
  • the cluster data synchronization method provided in the embodiment of the present application provides a more efficient write/read combination (write/read) method to replace the existing Raft protocol send/recv (send/recv) synchronization mode, for example, it is based on the communication device under the RDMA network, the slave node directly writes the data to be synchronized into the memory of the master node, and the master node then synchronizes the data to be synchronized to each slave node in the distributed cluster in the form of broadcast, and the master node and each slave node perform synchronization tasks according to the data to be synchronized, and complete the cluster data synchronization task of the entire distributed cluster.
  • write/read write/read
  • the memory distribution mechanism of the cluster In order to implement the above-mentioned cluster data synchronization solution based on RDMA network, the memory distribution mechanism of the cluster must be designed first.
  • the memory of the master node and the memory of each slave node in the distributed cluster in addition to the cluster memory space (responsible for storing the actual cluster data logic, each node has a copy, and the purpose of the consistency protocol is to ensure the data consistency of the cluster memory space on each node), there is also a preset synchronization data receiving pool for storing the data to be synchronized (responsible for processing the data to be synchronized sent by other nodes).
  • the synchronization data receiving pool can be designed to correspond one-to-one with the remaining nodes in the distributed cluster except the node where the synchronization data receiving pool is located.
  • an RDMA mapping array for maintaining the correspondence between the remaining nodes outside the node and the synchronization data receiving pool and an atomic index for allocating free space for the data to be synchronized can be provided in the synchronization data receiving pool.
  • the atomic index the key data in the distributed cluster can be protected to ensure that when the data to be synchronized is written and the synchronization task is executed, if a thread accesses the related data, the data that has not been synchronized will not be accessed.
  • Each synchronous data receiving pool consists of an RDMA mapping array and an atomic index, which are used to maintain the data in the synchronous data receiving pool.
  • each node pre-stores the memory region information of each node in the distributed cluster, that is, the correspondence between the synchronization data receiving pool preset in each node and other nodes.
  • the rules for cluster data synchronization are performed in accordance with the Raft protocol.
  • the node is defined as the source node for cluster data synchronization.
  • the source node sends the changed data as data to be synchronized to the master node, and the master node then sends the data to be synchronized in the form of a broadcast to each slave node in the distributed cluster for cluster data synchronization.
  • the master node monitors whether the synchronization data receiving pool in the local memory is written with the data to be synchronized according to the pre-deployed monitoring rules. For example, the master node can check the synchronization data receiving pool regularly, and if newly added data is found, it is determined as the data to be synchronized.
  • the source node can also use the traditional protocol to send a write completion mark through IO to inform the master node that the data to be synchronized has been written into the synchronization data receiving pool of the master node. Compared with sending the data to be synchronized in a completely IO manner, the IO occupancy time is reduced, but IO will still be occupied.
  • the source node can also write a write completion mark at the address after the data to be synchronized or another agreed address in the synchronization data receiving pool of the master node after completing the writing of the data to be synchronized, and wait for the master node to periodically poll whether the write completion mark appears in the synchronization data receiving pool.
  • the master node finds the write completion mark, it is determined as the data to be synchronized, but this requires the CPU of the master node to poll a large amount of memory data.
  • the CPU of the master node detects that the synchronization data receiving pool in the local memory is written by the source node to be synchronized, the cluster data synchronization work in the entire distributed cluster is started, and the information of the data to be synchronized is first verified on the master node.
  • the optional verification method can refer to the existing distributed cluster synchronization solution, which will not be repeated here.
  • the rules for cluster data synchronization are performed in accordance with the Raft protocol.
  • the master node After the master node has verified the information of the data to be synchronized, it calls the communication device based on the RDMA network in the form of broadcast to write the data to be synchronized directly into the synchronization data receiving pool of each slave node, so that each slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized.
  • the process in which the master node writes the data to be synchronized into the synchronization data receiving pool of the slave node is similar to the way in which the source node writes the data to be synchronized into the synchronization data receiving pool of the master node, both of which write the data to be synchronized into the synchronization data receiving pool of the destination end by calling the communication device.
  • the slave node monitors the synchronization data receiving pool in the local memory, starts verification when the data to be synchronized is detected, and executes the synchronization task of the data to be synchronized after the verification passes, which is similar to the process in which the master node executes the synchronization task of the data to be synchronized after monitoring the data to be synchronized, and the description of S101 above can be referred to.
  • the master node and each slave node After the master node and each slave node identify the data to be synchronized in the synchronization data receiving pool of the local memory and pass the verification, they each perform the local synchronization task according to the data to be synchronized. If the cluster data synchronization task is completed by directly performing the local synchronization task according to the data to be synchronized, the slave node does not need to return the confirmation code to the master node, thereby saving the CPU scheduling work required for cluster data synchronization.
  • the master node When the node where the cluster data change occurs is the master node, the master node directly executes S103 after completing the verification of the data to be synchronized.
  • the communication device may use a RoCE network card.
  • RoCE stands for RDMA over Converged Ethernet, i.e., RDMA based on converged Ethernet.
  • the cluster data synchronization method provided in the embodiment of the present application provides a cluster data synchronization solution that optimizes the Raft protocol in combination with the RDMA network.
  • the source node that generates the data to be synchronized directly writes the data to be synchronized into a preset synchronization data receiving pool in the local memory of the master node by calling the communication device based on the RDMA network.
  • the master node executes the synchronization task of the data to be synchronized in the local memory, and calls the communication device based on the RDMA network in the form of broadcast to directly write the data to be synchronized into the synchronization data receiving pool of the slave node, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, in a direct writing manner.
  • the transmission of the data to be synchronized replaces the sending/receiving synchronization mode of the existing Raft protocol, which can avoid memory copying during the sending/receiving process.
  • the master node does not need to control IO multiplexing during cluster data synchronization, and does not need flow control, which significantly reduces the amount of code to implement synchronization and the CPU scheduling workload of the master node.
  • the source node Regardless of whether the source node writes the data to be synchronized into the synchronization data receiving pool of the master node, or the master node writes the data to be synchronized into the synchronization data receiving pool of the slave node, that is, regardless of whether the source end is the source node and the destination end is the master node, or the source end is the master node and the destination end is the slave node, a mechanism needs to be set to enable the destination end to recognize the writing of the data to be synchronized.
  • the destination end can periodically check whether there is new data in the local synchronization data receiving pool to identify the data to be synchronized. Then in the cluster data synchronization method provided in the embodiment of the present application, in S102, the master node monitors the synchronization data receiving pool in the local memory that is written by the source node based on the RDMA network call communication device to be synchronized. It can be: when polling monitors that there is new data to be synchronized in the synchronization data receiving pool in the local memory, confirm that the source node has written the data to be synchronized.
  • the master node calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after monitoring the data to be synchronized and completing the verification of the data to be synchronized. It can be: based on the RDMA network call communication device to write the data to be synchronized into the synchronization data receiving pool of the slave node, so that after the slave node monitors the data to be synchronized through polling, it completes the verification of the data to be synchronized and performs the synchronization task of the data to be synchronized.
  • the destination end polls the local memory. When it detects that there is new data in the synchronization data receiving pool, it is considered to be data to be synchronized and then the synchronization operation is started.
  • the traditional protocol can be used to write the data to be synchronized at the source end through IO, and then send the write completion mark to the destination end through IO, so that the destination end knows that the local synchronization data receiving pool has been written into the data to be synchronized.
  • the master node detects that the synchronization data receiving pool in the local memory is written by the source node based on the RDMA network calling the communication device to be synchronized. It can be: when receiving the write completion mark sent by the input and output (IO) port after the source node has written the data to be synchronized, it is confirmed that the source node has written the data to be synchronized.
  • the master node calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after monitoring the data to be synchronized and completing the verification of the data to be synchronized.
  • the input and output (IO) port is called to send a write completion mark to the slave node, so that after the slave node confirms that the data to be synchronized is written, the verification of the data to be synchronized is completed and the synchronization task of the data to be synchronized is performed. That is, after the source end writes the data to be synchronized to the destination end, it sends a write completion flag by calling the destination end's IO, so that the destination end knows that the data to be synchronized has been written into the local synchronization data receiving pool.
  • the source end after the source end completes writing the data to be synchronized, it can continue to write the write completion mark to the address after the data to be synchronized or another agreed address of the synchronization data receiving pool of the destination end, and wait for the destination end to periodically poll and send a signal to determine the write completion mark as the data to be synchronized. Then in the cluster data synchronization method provided in the embodiment of the present application, in S102, the master node detects that the synchronization data receiving pool in the local memory is written by the source node based on the RDMA network call communication device.
  • the master node calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after monitoring the data to be synchronized and completing the verification of the data to be synchronized.
  • the source end calls the communication device based on the RDMA network to write the data to be synchronized into the destination end, it can continue to write the write completion mark into the destination end after the data to be synchronized or into another agreed address (i.e., the write completion mark agreed address) based on the RDMA network.
  • the destination end polls the agreed address of the write completion mark and recognizes the write completion mark, it confirms that the data to be synchronized has been written into the local synchronization data receiving pool.
  • the above methods either require IO to transmit the write completion flag, or the destination end cannot detect the write completion flag in time, causing the cluster data synchronization task to fail to start in time.
  • the master node detects that the synchronization data receiving pool in the local memory is written by the source node based on the RDMA network calling communication device to be synchronized. It can be: after the source node is detected that the task completion callback function based on the RDMA network updates the synchronization request field in the CPU of the master node after writing the data to be synchronized, it confirms that the source node writes Data to be synchronized has been entered.
  • the master node calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node in the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized.
  • the synchronization request field is updated in the CPU of the slave node based on the task completion callback function, so that the slave node completes the verification of the data to be synchronized and executes the synchronization task of the data to be synchronized after detecting the updated synchronization request field.
  • a synchronization request field (which can be defined as an available field) is added to the control program of the CPU of each node, and the completion IO callback function unique to RDMA technology (i.e., the task completion callback function defined above) is used to indicate that there is data written to the synchronization data receiving pool.
  • the completion IO callback function unique to RDMA technology i.e., the task completion callback function defined above
  • the task completion callback function uses the task completion callback function to update the synchronization request field of the destination end (such as +1), indicating that there is data to be sent and needs to wait for the destination end to process.
  • the process of checking the failure of the source end to send can be omitted, which also reduces the amount of code.
  • the synchronization request field corresponding to the local synchronization data receiving pool is reduced by 1 (available-1).
  • FIG. 2 is a schematic diagram of cluster memory distribution provided in an embodiment of the present application.
  • the synchronization data receiving pool on each node is designed to correspond to other nodes one by one to facilitate the management of the data to be synchronized from different source nodes.
  • different synchronization rules can be set for different types of data to be synchronized, and different types of data to be synchronized can be written into different storage areas for easy management.
  • a plurality of sub-receiving pools corresponding to different types of data to be synchronized are provided in the synchronization data receiving pool.
  • the master node executes the synchronization task for the data to be synchronized in the local memory, which may be: executing the synchronization task for the data to be synchronized in the local memory according to the type of the data to be synchronized.
  • the slave node executes the synchronization task for the data to be synchronized, which may be: the slave node executes the synchronization task for the data to be synchronized according to the type of the data to be synchronized.
  • the source end adds processing logic for different types of data to be synchronized based on the correspondence between the source node and the synchronization data receiving pool in other nodes in the distributed cluster stored locally, so that corresponding operations are performed according to the type of data to be synchronized during the process of writing at the source end and reading at the destination end to complete the synchronization task.
  • the type of sub-receiving pool can be designed according to the amount of data to be synchronized and the frequency of synchronization.
  • the types of sub-receiving pools can include: small data receiving pool (small pool), response data receiving pool (res pool) and big data receiving pool (big pool).
  • the small data receiving pool is used to receive short data packets, which are generally used for updating status and other data with small data interactions and frequent operations, and the CPU can complete data processing operations within a fixed time.
  • the response data receiving pool is used to receive confirmation data and update the cluster status.
  • the master node state it is responsible for processing response confirmation packets with complex logic, such as updating heartbeat information.
  • the big data receiving pool is used to process large amounts of updated data.
  • the corresponding memory pages of remote nodes may be directly overwritten, such as the snapshot (snapshot, a Raft protocol concept, which means that when a new node joins, the cluster status needs to be updated to the local node.
  • the operation process can be similar to memcpy) update process, and the process corresponding to processing cluster transactions.
  • the synchronization data receiving pool can also be designed to be pre-divided into data partitions of various types of sub-receiving pools according to the data length of each type of data to be synchronized.
  • Each data partition corresponds to a continuous memory address.
  • the synchronization request field and the type of the sub-receiving pool in the synchronization data receiving pool corresponding to other nodes can be designed to correspond one to one.
  • the source end After the source end writes the data to be synchronized to the destination end, it calls the task completion callback function, and updates the synchronization request field of the sub-receiving pool of the corresponding data type in the synchronization data receiving pool corresponding to the destination end and the source end according to the type of the data to be synchronized, so that the CPU of the destination end can directly determine the type of data to be synchronized after checking the update of the synchronization request field.
  • the master node is provided with a synchronization data receiving pool corresponding to the other three slave nodes, and each synchronization data receiving pool is divided into three types of sub-receiving pools: a small data receiving pool, a response data receiving pool, and a large data receiving pool. Then one synchronization data receiving pool corresponds to three synchronization request fields, and the master node maintains a total of nine synchronization request fields.
  • a synchronous data receiving pool corresponding to other nodes is set in the local memory of each node in the distributed cluster.
  • Each synchronous data receiving pool is divided into three types of sub-receiving pools: small data receiving pool, response data receiving pool and large data receiving pool.
  • Each type of sub-receiving pool contains multiple data partitions of this type of sub-receiving pool to realize storage management of the synchronous data receiving pool.
  • the memory space of its local memory is divided into a cluster memory space and a synchronization data receiving pool corresponding to the remaining nodes 201 (node 2, node 3, node 4).
  • the synchronization data receiving pool corresponds to node 2, node 3, and node 4 one by one.
  • Each synchronization data receiving pool is composed of an RDMA mapping array and an atomic index.
  • Each synchronization data receiving pool is further divided into a plurality of small data receiving pools, a plurality of response data receiving pools, and a plurality of large data receiving pools, which are used to receive the writing of different types of data to be synchronized and to perform synchronization tasks in a corresponding processing manner on the destination.
  • node 1, node 2, node 3, and node 4 are the serial numbers of each node in the distributed cluster.
  • the slave node When any slave node generates data to be synchronized, the slave node becomes the source node, and enters the process in which the source node sends the data to be synchronized to the master node, and the master node broadcasts the data to be synchronized to each slave node to complete the cluster data synchronization task.
  • the embodiments of the present application further illustrate the synchronization method of different types of data to be synchronized.
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which may be: after the source node assembles the data to be synchronized into a first message header, the source node calls the communication device based on the RDMA network to write the first message header into a corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the master node calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node in the distributed cluster. This can be done by: according to the storage area information of the synchronization data receiving pool in the slave node stored locally, calling the communication device based on the RDMA network to write the first message header into the corresponding storage location.
  • an embodiment of the present application provides a cluster data synchronization process as follows:
  • the source node (e.g., slave node 1 in the above embodiment) obtains the local cluster memory space and organizes the first message header (e.g., Raft protocol data, serialization processing of more complex data, etc.) according to the write request;
  • the first message header e.g., Raft protocol data, serialization processing of more complex data, etc.
  • the source node calls the write interface and directly writes the first message header into the corresponding memory area of the master node through the communication device (such as RoCE network card), instead of calling the send/recv function similar to the TCP protocol, avoiding memory copy in send/recv.
  • the master node also does not need to maintain IO multiplexing data, which significantly reduces the amount of code and CPU scheduling of the master node.
  • the source node After the source node completes writing, it uses the task completion callback function (completion IO) unique to RDMA technology to update synchronization on the master node.
  • the request field (available) indicates that there is data to be sent and needs to wait for the master node to process it.
  • the RDMA lossless data network is used, so the process of the source node checking for sending failures can be omitted, which also reduces the amount of code.
  • the process of the master node broadcasting to all slave nodes is similar to the process of the source node writing the data to be synchronized to the master node;
  • each slave node When each slave node receives the data to be synchronized, it updates the data in the local cluster memory space without returning a confirmation code.
  • the task completion callback function is also used to process the return value and update the synchronization request field.
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which may be: after the source node assembles the data to be synchronized into the second message header, the source node calls the communication device based on the RDMA network to write the second message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the communication device is called based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster. This can be done by: according to the storage area information of the synchronization data receiving pool in the slave node stored locally, the communication device is called based on the RDMA network to write the second message header into the corresponding storage location.
  • the writing method of the response data receiving pool can refer to the writing method of the small data receiving pool.
  • additional synchronization rules can be set for different types of response data.
  • the heartbeat signal of each node can be included in the response data receiving pool to complete the status monitoring of each node in the distributed cluster.
  • the first preset space of the response data receiving pool (such as res pool[0]) can be designed to store the heartbeat information of all nodes in the distributed cluster.
  • the heartbeat information can be synchronized regularly between the nodes through the cluster data synchronization method provided in the embodiment of the present application.
  • the cluster data synchronization method provided in the embodiment of the present application may further include:
  • the master node schedules a thread (defined as a heartbeat thread) to write the current timestamp to a specific RDMA memory at preset intervals.
  • the current timestamp will then be written to the first preset space of the response data receiving pool of all slave nodes via the RDMA lossless network.
  • Each slave node will then write the response timestamp to the first preset space of the response data receiving pool of the master node via the RDMA lossless network.
  • the master node can then check the response timestamps of all slave nodes.
  • the heartbeat packet processing method Compared with the traditional blocking mode send/receive (send/recv) write, the heartbeat packet processing method provided by the embodiment of the present application is similar to completely reading and writing local memory for upper-layer applications, and the whole process is non-blocking. Generally speaking, this means faster error detection.
  • the system bottom layer sends (send) process will be blocked, and the information packet carrying the message header is mixed with the heartbeat information, and reports upward together after the timeout occurs.
  • the master node this means that a large number of sudden error states need to be restored, and the logic is complicated and time-consuming.
  • the master node can detect the heartbeat packet timeout before other messages go wrong, so directly schedule the reconstruction cluster operation, which is quite helpful for the cluster recovery time.
  • the response data receiving pool can also be set up with a second preset space (such as res pool[1]) for storing user-defined data, that is, the type of data to be synchronized for user extension.
  • a second preset space such as res pool[1]
  • the embodiments of the present application further illustrate the synchronization method of different types of data to be synchronized.
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the synchronization task of the data to be synchronized into the third message header, it calls the communication device based on the RDMA network to write the third message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the synchronization task of the data to be synchronized is executed in the local memory, and the communication device is called based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster in the form of broadcast, so that the slave node executes the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, including:
  • the communication device is called to write the third message header into the corresponding storage location, so that the slave node executes the logic contained in the third message header in the paging of the third preset space of the big data receiving pool of the slave node;
  • the local node and each slave node are controlled to copy the pages of the third preset space of the node to the cluster memory space of the node.
  • the embodiment of the present application provides a cluster data synchronization method to design a big data receiving pool to receive the data to be synchronized, and during the update process, first receive the data to be synchronized, and update the data to be synchronized in the big data receiving pool in units of corresponding data pages in the cluster memory space, and then replace the updated data pages in the big data receiving pool with the corresponding data pages in the cluster memory space, thereby shortening the interruption to business services during the update process.
  • the present application embodiment provides a cluster data synchronization process as follows:
  • the process of the source node writing the third message header to the master node is the same as writing to the small data receiving pool;
  • each slave node After receiving the third message header sent by the master node, each slave node starts to process the logic of the synchronization task contained in the third message header locally;
  • the design of the data to be synchronized received by the big data receiving pool involves relatively complex logic, and the processing is relatively more time-consuming; in this process, in the big data receiving pool, according to the data involved in the third message header, the relevant logic of the synchronization task of the third message header is processed preferentially in the paging of the third preset space of the big data receiving pool (for example, if the synchronization data receiving pool has been divided into multiple data partitions of the big data receiving pool in advance, then in the data partition written in the third message header), and after the execution is completed, a confirmation success response code is sent to the master node;
  • the confirmation success response code belongs to a kind of confirmation data, so each slave node sends a confirmation success response code to the master node, and each slave node can call a communication device based on the RDMA network to write the confirmation success response code into the answer data receiving pool in the synchronization data receiving pool corresponding to each slave node in the master node;
  • the master node detects that the response data receiving pool in the local synchronization data receiving pool contains the confirmation success response codes of all slave nodes, and confirms that the relevant logic of the synchronization task of the third message header has been executed locally, and then broadcasts the confirmation execution message to all slave nodes;
  • each slave node After receiving the confirmation execution message, each slave node copies the pages in the third preset space to the corresponding pages in the cluster memory space of the node where it is located, completing the local synchronization task;
  • the cluster data synchronization task in this distributed cluster is completed, and each node reduces the synchronization request field corresponding to the local big data receiving pool by 1 (big pool available-1).
  • the cluster data synchronization method provided in the embodiment of the present application, after confirming that the local and each slave node have completed the execution of the logic contained in the third message header, the local and each slave node are controlled to copy the paging of the third preset space of the node to the cluster memory space of the node, which can be:
  • a confirmation success response code is sent to each slave node, so that each slave node and the master node respectively perform the following steps after receiving the confirmation success response code:
  • each slave node when each slave node replaces the paging of the third preset space with the corresponding paging in the cluster memory space after receiving the confirmation execution message sent by the master node, and when the master node replaces the paging of the third preset space with the corresponding paging in the cluster memory space locally, the corresponding address in the cluster memory space (assuming it corresponds to the fourth preset space) is exchanged to the third preset space through atomic logic, so as to process the service provided by this part of the paging in the cluster memory space based on the third preset space and execute the next processing logic; at the same time, each node copies the paging of the third preset space to the fourth preset space; after the copy is completed, the business service provided by this part of the data is briefly interrupted, and then the business service is resumed after exchanging it back to the original address using (atomic) logic.
  • the third preset space is the big pool[0] page in the large data receiving pool in the synchronization data receiving pool
  • the fourth preset space is page 2 of the cluster memory space
  • the above embodiments of the present application illustrate the synchronization process of big data in a distributed cluster.
  • the Raft protocol is not combined with transactions.
  • the node status can only be reset, which also affects the cluster data synchronization efficiency of the distributed cluster and the data recovery efficiency after synchronization failure.
  • cluster transactions all nodes are required to maintain data consistency, but for the Raft protocol, data only needs to be transmitted to more than half of the nodes to ensure the uniqueness of cluster data, and the few nodes that have not been successfully executed will directly exit the cluster and refresh the log information. In other words, all data needs to be cleared for nodes that fail to execute the synchronization task and then re-acquired from the master node.
  • Transaction is a mechanism, an operation sequence, and includes a set of operation commands.
  • the reason why the above problems occur when synchronizing transaction data is that the transaction submits or cancels operation requests to the system as a whole, that is, this set of database commands is either all executed or not executed, so the transaction is an indivisible work logic unit.
  • the third message header only includes the synchronization logic of the data to be synchronized
  • the third message header includes the synchronization logic of the data to be synchronized and the transaction callback function corresponding to the data to be synchronized, so that the master node or the slave node executes the logic corresponding to the transaction callback function when executing the synchronization task of the data to be synchronized;
  • the transaction callback function at least includes a data backup function for transaction data synchronization and a key data protection function.
  • non-transactional data refers to data to be synchronized that is large in volume and relatively concentrated, but allows for partial data synchronization failure.
  • Transactional data refers to the data to be synchronized in the transaction process mentioned above. During the execution process, only all of it can be executed successfully. If some of the data fails to execute, the synchronization task of the entire data to be synchronized will fail.
  • the data to be synchronized is transmitted, the relevant logic of the synchronization task is executed on each node, and the paging replacement steps and non-transactional data steps are performed, but a transaction callback function with corresponding functions is required to be provided in the third message header of the transaction data.
  • each transaction process has a unique corresponding identifier (ID), and a transaction callback function is designed in advance for the transaction process in the distributed cluster, so that each node provides a transaction callback function with a corresponding function in the registered third message header when initiating synchronization of transaction data.
  • the transaction callback function to be provided includes at least data backup before each node executes the synchronization task, and protection of key data, such as locking to suspend external business services to avoid errors during the synchronization process. These preparations are to return to the state before synchronization after the failure of transaction data synchronization, instead of exiting the node from the cluster and rejoining it once the synchronization fails as in the related art.
  • the key data can be the user data of the user facing the distributed cluster, and the user data needs to be protected when synchronizing transactional data.
  • the user's naming of the distributed cluster needs to be synchronized to each node in the distributed cluster, and when the user modifies the cluster name, the cluster name configured for all nodes needs to be locked and protected first to avoid other threads accessing the cluster name with an uncertain cluster name during the synchronization process (some access the cluster name before the modification, and some access the cluster name after the modification).
  • the transaction check function (evt_transaction_check) is used to check whether the prerequisites for executing the transaction on the data are reasonable;
  • the data backup function (evt_transaction_prepare) is used to back up the original data and protect key data when preparing to execute a transaction;
  • the slave node When the above two operations succeed or fail, the slave node will return a logical success message to the master node;
  • the transaction execution function (evt_transaction_commit) is used to execute the actual logic of the transaction. After the callback of this function is executed, the transaction data should be ready and can be read normally. After the synchronization task of the data to be synchronized is completed, all slave nodes will return the synchronization task execution completion message to the master node. If the master node checks that all slave nodes have successfully completed the synchronization task of the data to be synchronized, the scheduling transaction completion function (evt_transaction_finish) of all nodes will be called; otherwise, the transaction repair function (evt_transaction_restore) will be called.
  • the transaction completion function (evt_transaction_finish) is used to call back after the synchronization task of the data to be synchronized is successfully executed, release key data protection and delete backup operations, etc.
  • the transaction repair function (evt_transaction_restore) is used to call back after an unexpected execution error occurs, release data protection, and restore data to its original state.
  • the embodiment of the present application provides a logic corresponding to the execution transaction callback function, including:
  • the logic of the synchronization task to execute the data to be synchronized is triggered;
  • a synchronization task execution completion message is returned to the master node
  • the key data protection is released and the data backup is deleted; otherwise, the callback operation after the error occurs is executed to release the key data protection and restore the cluster data to the initial state based on the data backup.
  • the present application also discloses a cluster data synchronization device, equipment and non-volatile readable storage medium corresponding to the above method.
  • FIG3 is a schematic diagram of the structure of a cluster data synchronization device provided in an embodiment of the present application.
  • the cluster data synchronization device provided in the embodiment of the present application includes:
  • the monitoring unit 301 is configured to monitor a preset synchronous data receiving pool in the local memory
  • the synchronization unit 303 is configured to execute the synchronization task of the data to be synchronized in the local memory after the data to be synchronized passes the verification, and write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster in the form of broadcast based on the RDMA network call communication device, so that After the slave node detects the data to be synchronized and completes the verification of the data to be synchronized, it executes the synchronization task of the data to be synchronized;
  • the source node is a slave node in the distributed cluster.
  • the monitoring unit 301 monitors that the data to be synchronized written by the source node to the synchronization data receiving pool in the local memory by calling the communication device based on the RDMA network may be:
  • the task completion callback function based on the RDMA network updates the synchronization request field in the CPU of the master node, and confirms that the source node has written the data to be synchronized;
  • the synchronization unit 303 calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, which can be:
  • the synchronization request field is updated in the CPU of the slave node based on the task completion callback function, so that the slave node completes the verification of the data to be synchronized and executes the synchronization task of the data to be synchronized after monitoring the updated synchronization request field.
  • the synchronization data receiving pool corresponds one-to-one with the remaining nodes in the distributed cluster except the node where the data is located;
  • the synchronous data receiving pool is provided with an RDMA mapping array for maintaining the corresponding relationship between the remaining nodes outside the local node and the synchronous data receiving pool, and an atomic index for allocating free space for the data to be synchronized.
  • the synchronization data receiving pool is provided with multiple sub-receiving pools corresponding to different types of data to be synchronized;
  • the synchronization task for the data to be synchronized is performed in the local memory, which can be:
  • the synchronization task of the data to be synchronized is executed in the local memory
  • the slave node performs synchronization tasks for the data to be synchronized, which can be:
  • the slave node executes the synchronization task for the data to be synchronized according to the type of the data to be synchronized.
  • the synchronization data receiving pool is pre-divided into data partitions of each type of sub-receiving pool according to the data length of each type of data to be synchronized.
  • types of sub-receiving pools include: a small data receiving pool, a response data receiving pool, and a large data receiving pool.
  • the data to be synchronized is a data type corresponding to the small data receiving pool or a data type corresponding to the response data receiving pool
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the data to be synchronized into the first message header, it calls the communication device based on the RDMA network to write the first message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the synchronization unit 303 calls the communication device based on the RDMA network to write the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, which can be:
  • the communication device is called based on the RDMA network to write the first message header into the corresponding storage location.
  • the first preset space of the response data receiving pool is used to store the heartbeat information of all nodes in the distributed cluster.
  • the cluster data synchronization device provided in the embodiment of the present application further includes:
  • a timestamp writing unit is configured to periodically write a current timestamp into a first preset space in the local memory
  • a timestamp synchronization unit is configured to synchronize the current timestamp to a first preset space of each slave node through the RDMA network;
  • a timestamp reading unit is configured to read the response timestamp written into the local memory by each slave node
  • the status checking unit is configured to check the connection status of each slave node according to each response timestamp.
  • the second preset space of the response data receiving pool is used to store user-defined data.
  • the data to be synchronized is of a data type corresponding to the big data receiving pool
  • the source node calls the communication device based on the RDMA network to write the data to be synchronized, which can be:
  • the source node After the source node assembles the synchronization task of the data to be synchronized into the third message header, it calls the communication device based on the RDMA network to write the third message header into the corresponding storage location according to the storage area information of the synchronization data receiving pool in the master node stored locally;
  • the synchronization unit 303 executes the synchronization task of the synchronization data in the local memory, and calls the communication device based on the RDMA network in the form of broadcast.
  • the standby node writes the data to be synchronized into the synchronization data receiving pool of each slave node of the distributed cluster, so that the slave node performs the synchronization task of the data to be synchronized after detecting the data to be synchronized and completing the verification of the data to be synchronized, including:
  • the communication device is called to write the third message header into the corresponding storage location, so that the slave node executes the logic contained in the third message header in the paging of the third preset space of the big data receiving pool of the slave node;
  • the local node and each slave node are controlled to copy the pages of the third preset space of the node to the cluster memory space of the node.
  • the synchronization unit 303 controls the local and each slave node to copy the pages of the third preset space of the node to the cluster memory space of the node, which can be:
  • a confirmation success response code is sent to each slave node, so that each slave node and the master node respectively perform the following steps after receiving the confirmation success response code:
  • the third message header only includes the synchronization logic of the data to be synchronized
  • the third message header includes the synchronization logic of the data to be synchronized and the transaction callback function corresponding to the data to be synchronized, so that the master node or the slave node executes the logic corresponding to the transaction callback function when executing the synchronization task of the data to be synchronized;
  • the transaction callback function includes at least a data backup function for transaction data synchronization and a key data protection function.
  • the logic corresponding to the transaction callback function executed by the synchronization unit 303 includes:
  • the logic of the synchronization task to execute the data to be synchronized is triggered;
  • a synchronization task execution completion message is returned to the master node
  • the key data protection is released and the data backup is deleted; otherwise, the callback operation after the error occurs is executed to release the key data protection and restore the cluster data to the initial state based on the data backup.
  • the communication device may be a RoCE network card.
  • FIG. 4 is a schematic diagram of the structure of a cluster data synchronization device provided in an embodiment of the present application.
  • the cluster data synchronization device provided in the embodiment of the present application includes:
  • the memory 410 is arranged to store a computer program 411;
  • the processor 420 is configured to execute a computer program 411 , which, when executed by the processor 420 , implements the steps of the cluster data synchronization method according to any one of the above embodiments.
  • the processor 420 may include one or more processing cores, such as a 3-core processor, an 8-core processor, etc.
  • the processor 420 may be implemented in at least one of the following hardware forms: a digital signal processing DSP (Digital Signal Processing), a field programmable gate array FPGA (Field-Programmable Gate Array), and a programmable logic array PLA (Programmable Logic Array).
  • the processor 420 may also include a main processor and a coprocessor.
  • the main processor is a processor configured to process data in the wake-up state, also called a coprocessor.
  • Central Processing Unit (CPU); Coprocessor is a low-power processor configured to process data in standby mode.
  • processor 420 may be integrated with a graphics processor (GPU), which is configured to be responsible for rendering and drawing the content to be displayed on the display screen.
  • processor 420 may also include an artificial intelligence (AI) processor, which is configured to process computing operations related to machine learning.
  • AI artificial intelligence
  • the memory 410 may include one or more non-volatile readable storage media, which may be non-transitory.
  • the memory 410 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices.
  • the memory 410 is at least configured to store the following computer program 411, wherein the computer program 411, after being loaded and executed by the processor 420, can implement the relevant steps in the cluster data synchronization method disclosed in any of the aforementioned embodiments.
  • the resources stored in the memory 410 may also include an operating system 412 and data 413, etc., and the storage method may be temporary storage or permanent storage.
  • the operating system 412 may be Windows.
  • Data 413 may include, but is not limited to, the data involved in the above method.
  • the cluster data synchronization device may further include a display screen 430 , a power supply 440 , a communication interface 450 , an input/output interface 460 , a sensor 470 , and a communication bus 480 .
  • FIG. 4 does not constitute a limitation on the cluster data synchronization device, and may include more or fewer components than those shown in the figure.
  • the cluster data synchronization device provided in the embodiment of the present application includes a memory and a processor.
  • the processor executes the program stored in the memory, it can implement the above cluster data synchronization method, and the effect is the same as above.
  • the above-described device and equipment embodiments are merely schematic.
  • the division of modules is merely a logical function division. There may be other division methods in actual implementation, such as multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or modules, which may be electrical, mechanical or other forms.
  • the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
  • each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or software functional modules.
  • the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a non-volatile readable storage medium.
  • the technical solution of the present application, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the various embodiments of the present application.
  • an embodiment of the present application further provides a non-volatile readable storage medium, on which a computer program is stored.
  • a computer program is stored on which a computer program is stored.
  • the non-volatile readable storage medium may include: a U disk, a mobile hard disk, a read-only memory ROM (Read-Only Memory), a random access memory RAM (Random Access Memory), a magnetic disk or an optical disk, and other media that can store program codes.
  • the computer program included in the non-volatile readable storage medium provided in this embodiment can implement the steps of the above cluster data synchronization method when executed by the processor, and the effect is the same as above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Hardware Redundancy (AREA)

Abstract

本申请适用于数据同步技术领域，公开了一种集群数据同步方法、装置、设备及非易失性可读存储介质，产生待同步数据的源节点基于RDMA网络调用通信设备将待同步数据直接写入主节点的本地内存中预设的同步数据接收池，主节点在本地内存中执行对待同步数据的同步任务，并以广播的形式基于RDMA网络调用通信设备将待同步数据直接写入从节点的同步数据接收池，以使从节点执行对待同步数据的同步任务，以直接写入的方式进行待同步数据的传递，代替现有Raft协议的发送/接收的同步模式，可以避免发送/接收过程中的内存拷贝，主节点也无需在集群数据同步工作中控制I0复用，减轻了实现同步的代码量以及主节点的CPU调度工作。

Description

集群数据同步方法、装置、设备及非易失性可读存储介质
相关申请的交叉引用
本申请要求于2022年11月25日提交中国专利局，申请号为202211487122.9，申请名称为“集群数据同步方法、装置、设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据同步技术领域，特别是涉及一种集群数据同步方法、装置、设备及非易失性可读存储介质。
背景技术
分布式环境中需要大量的工作来保证数据一致性，避免不同节点间数据差异而导致的访问结果不同。这个过程需要大量的开发工作，且由于同步流程复杂，通常运行效率并不高。目前主流的数据一致性协议有两种，Paxos和Raft，其他所有协议均是这两种协议的变种。其中，Raft是一种更为简单方便易于理解的分布式算法，在解决分布式中的一致性问题时，相比传统的Paxos算法(一种基于消息传递的分布式一致性算法)，Raft将大量的计算问题分解成为了一些简单的相对独立的子问题。
远程直接数据存取(Remote Direct Memory Access，下文简称RDMA)是一种为解决网络传输中服务器端数据处理的延迟而产生的技术。RDMA通过网络将资料直接传入计算机的存储区，将数据从一个系统快速移动到远程系统存储器中，而不对操作系统造成任何影响。作为一种全新的网络数据传输机制，RDMA的实现需要软件、硬件的配合。而相较于传统的传输控制协议(Transmission Control Protocol，下文简称TCP)/用户数据报协议(User Datagram Protocol，下文简称UDP)来说，RDMA可以基于多种底层协议，灵活性较高，无需像TCP/UDP协议那样在发送数据时需要进行应用态/内核态切换以及多次内存拷贝，性能更优。且RDMA协议自带无损数据网络，可以解决TCP协议的粘包、重发问题以及UDP协议的丢包问题。且基于RDMA协议发送数据无需经过中央处理器(Central Processing Unit，下文简称CPU)调度，可以节约CPU资源，避免中断的产生。
然而，目前Raft协议与RDMA技术的结合，都是基于基础的发送、接收逻辑，即仅仅是将RDMA作为TCP/UDP的平行替代方案。这种方案虽然可以实现Raft协议的逻辑，但是没有结合RDMA网络独有的特性，导致RDMA的性能优势没有完全发挥出来，依然存在着数据同步效率低的问题。
对于上述问题，相关技术中并未提出有效的解决方案。
发明内容
本申请的目的是提供一种集群数据同步方法、装置、设备及非易失性可读存储介质，被设置为结合RDMA网络对Raft协议集群数据同步进行优化，充分发挥Raft协议集群中RDMA的性能优势。
为解决上述技术问题，在本申请的第一个方面，提供一种集群数据同步方法，基于分布式集群中的主节点，包括：
监测本地内存中预设的同步数据接收池；
当监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对待同步数据进行校验；
当待同步数据通过校验后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务；
其中，源节点为分布式集群中的一个从节点。
在一些实施例中，监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：
监测到源节点在写完待同步数据后基于RDMA网络的任务完成回调函数在主节点的CPU中更新的同步请求字段时，确认源节点写入了待同步数据；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：
基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，基于任务完成回调函数在从节点的CPU中更新同步请求字段，以使从节点在监测到更新的同步请求字段后完成对待同步数据的校验并执行对待同步数据的同步任务。
在一些实施例中，监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：
当轮询监测到本地内存中同步数据接收池有新增的待同步数据时，确认源节点写入了待同步数据；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：
基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池，以使从节点在轮询监测到待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。
在一些实施例中，监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：
接收到源节点在写完待同步数据后调用输入输出端口发送的写完成标志时，确认源节点写入了待同步数据；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：
基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，调用输入输出端口向从节点发送写完成标志，以使从节点确认被写入待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。
在一些实施例中，监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：
监测到源节点在写完待同步数据后在主节点的写完成标志约定地址写入的写完成标志时，确认源节点写入了待同步数据；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：
基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，在各从节点的写完成标志约定地址写入写完成标志，以使从节点确认被写入待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。
在一些实施例中，同步数据接收池与分布式集群中除所在节点外的其余节点一一对应。
在一些实施例中，同步数据接收池设有用于维护所在节点外的其余节点与同步数据接收池的对应关系的RDMA映射数组以及用于为待同步数据分配空余空间的原子索引。
在一些实施例中，同步数据接收池中设有多类与不同类型的待同步数据对应的子接收池；
相应的，在本地内存中执行对待同步数据的同步任务，可以为：
按照待同步数据的类型，在本地内存中执行对待同步数据的同步任务；
从节点执行对待同步数据的同步任务，可以为：
从节点按照待同步数据的类型执行对待同步数据的同步任务。
在一些实施例中，同步数据接收池中预先按照各类型待同步数据的数据长度划分为各类型子接收池的数据分区。
在一些实施例中，子接收池的类型包括：小数据接收池、应答数据接收池和大数据接收池。
在一些实施例中，当待同步数据为与小数据接收池对应的数据类型时，
源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据组装为第一消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，可以为：
根据本地存储的从节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置。
在一些实施例中，当待同步数据为与应答数据接收池对应的数据类型时，
源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据组装为第二消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第二消息头写入对应的存储位置；
基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，可以为：
根据本地存储的从节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第二消息头写入对应的存储位置。
在一些实施例中，应答数据接收池的第一预设空间用于存放分布式集群中所有节点的心跳信息。
在一些实施例中，还包括：
定时向本地内存的第一预设空间写入当前时间戳；
通过RDMA网络将当前时间戳同步至各从节点的第一预设空间；
读取各从节点写入本地内存的响应时间戳；
根据各响应时间戳检查各从节点的连接状态。
在一些实施例中，应答数据接收池的第二预设空间用于存放用户自定义数据。
在一些实施例中，当待同步数据为与大数据接收池对应的数据类型时，
源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据的同步任务组装为第三消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置；
在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，包括：
根据本地存储的从节点中同步数据接收池的存储区域信息，以广播的形式，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置，以使从节点在从节点的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在本地的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间。
在一些实施例中，在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间，可以为：
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，向各从节点发送确认成功响应码，以使各从节点在接收到确认成功响应码之后与主节点各自执行下述步骤：
将第三预设空间对应的集群内存空间中的第四预设空间的业务逻辑地址交换至第三预设空间，以基于第三预设空间的数据执行下一个处理逻辑，同时将第三预设空间的数据拷贝至第四预设空间；
在拷贝完成后，将业务逻辑地址交换回第四预设空间。
在一些实施例中，当待同步数据为与大数据接收池对应的数据类型且为非事务数据时，第三消息头仅包含待同步数据的同步逻辑；
当待同步数据为与大数据接收池对应的数据类型且为事务数据时，第三消息头包含待同步数据的同步逻辑以及与待同步数据对应的事务回调函数，以使主节点或从节点在执行对待同步数据的同步任务时执行事务回调函数对应的逻辑；
其中，事务回调函数至少包括针对事务数据同步的数据备份函数以及关键数据保护函数。
在一些实施例中，执行事务回调函数对应的逻辑包括：
检查待同步数据的同步任务的合理性；
执行数据备份及关键数据保护流程；
若待同步数据的同步任务合理且成功完成数据备份及关键数据保护流程，则向主节点返回准备逻辑执行成功消息；
在准备逻辑执行成功后，触发执行待同步数据的同步任务的逻辑；
在执行完毕待同步数据的同步任务的逻辑后，向主节点返回同步任务执行完毕消息；
若主节点及各从节点均成功执行待同步数据的同步任务的逻辑，则释放关键数据保护，删除数据备份；否则，执行发生错误后回调操作，释放关键数据保护并基于数据备份恢复集群数据至初始状态。
为解决上述技术问题，在本申请的第二个方面，还提供一种集群数据同步装置，包括：
监测单元，被设置为监测本地内存中预设的同步数据接收池；
校验单元，被设置为当监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对待同步数据进行校验；
同步单元，被设置为当待同步数据通过校验后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务；
其中，源节点为分布式集群中的一个从节点。
为解决上述技术问题，本申请还提供一种集群数据同步设备，包括：
存储器，被设置为存储计算机程序；
处理器，被设置为执行计算机程序，计算机程序被处理器执行时实现如上述任意一项集群数据同步方法的步骤。
为解决上述技术问题，在本申请的第三个方面，还提供一种非易失性可读存储介质，其上存储有计算机程序，计算机程序被处理器执行时实现如上述任意一项集群数据同步方法的步骤。
本申请所提供的集群数据同步方法，提供了一种结合RDMA网络对Raft协议优化的集群数据同步方案，产生待同步数据的源节点基于RDMA网络调用通信设备直接将待同步数据写入主节点的本地内存中预设的同步数据接收池，主节点在对待同步数据校验通过后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据直接写入从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，以直接写入的方式进行待同步数据的传递，代替现有Raft协议的发送/接收的同步模式，可以避免发送/接收过程中的内存拷贝，主节点也无需在集群数据同步工作中控制IO(Input/Output，输入/输出)复用，无需流控制，显著减轻了实现同步的代码量以及主节点的CPU调度工作量。
本申请还提供一种集群数据同步装置、设备及非易失性可读存储介质，具有上述有益效果，在此不再赘述。
附图说明
为了更清楚的说明本申请实施例或相关技术的技术方案，下面将对实施例或相关技术描述中所需要使用的附图作简单的介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种集群数据同步方法的流程图；
图2为本申请实施例提供的一种集群内存分布示意图；
图3为本申请实施例提供的一种集群数据同步装置的结构示意图；
图4为本申请实施例提供的一种集群数据同步设备的结构示意图。
具体实施方式
本申请的核心是提供一种集群数据同步方法、装置、设备及非易失性可读存储介质，被设置为结合RDMA网络对Raft协议集群数据同步进行优化，充分发挥Raft协议集群中RDMA的性能优势。
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
图1为本申请实施例提供的一种集群数据同步方法的流程图。
如图1所示，基于分布式集群中的主节点，本申请实施例提供的集群数据同步方法包括：
S101：监测本地内存中预设的同步数据接收池。
S102：当监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对待同步数据进行校验。
S103：当待同步数据通过校验后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务。
其中，源节点为分布式集群中的一个从节点。
在本申请实施例中，主节点(leader)指Raft协议中的主节点，可以为在基于Raft协议进行集群数据同步的分布式集群中通过自动选举产生的仲裁节点。根据Raft协议，所有的从节点(follower)进行状态更新时需要发送消息至主节点，由主节点广播至整个集群。
现有的基于Raft协议进行集群数据同步时还是采用类似传统的TCP/UDP协议的IO收发方式，导致集群同步过程每发送一次数据，都需要将待同步数据在本地拷贝至CPU内核，再通过IO发给目的端，目的端再从CPU内核拷贝到内存中，占用了大量CPU资源、同步效率较低，即使已经应用了Raft协议与RDMA结合的方案，也只是将RDMA当做TCP/UDP的平行替代方案，在进行数据同步时没有结合RDMA网络独有的特性，导致RDMA的性能优势没有完全发挥出来。为发挥RDMA网路的优势，本申请实施例提供的集群数据同步方法提供一种采用更高效率的写/读结合(write/read)的方式代替现有Raft协议的发送/接收(send/recv)的同步模式，比如，是基于RDMA网络下的通信设备，从节点将待同步数据直接写入主节点的内存，主节点再以广播的形式将待同步数据同步至分布式集群中各从节点，主节点和各从节点各自按照待同步数据执行同步任务，完成整个分布式集群的集群数据同步任务。
为实现上述基于RDMA网络的集群数据同步方案，首先要设计集群的内存分布机制。在本申请实施例中，在分布式集群中的主节点的内存和各从节点的内存中，除了集群内存空间(负责存储实际集群数据逻辑，每个节点都有一份，一致性协议目的就是要保证每个节点上集群内存空间的数据一致性)外，还预设有用于存储待同步数据的同步数据接收池(负责处理其他节点发送的待同步数据)。
为便于管理，在本申请实施例提供的集群数据同步方法中，可以设计同步数据接收池与分布式集群中除所在节点外的其余节点一一对应。
此外，可以在同步数据接收池设有用于维护所在节点外的其余节点与同步数据接收池的对应关系的RDMA映射数组以及用于为待同步数据分配空余空间的原子(atomic)索引。通过原子(atomic)索引，可以对分布式集群中的关键数据进行保护，以保证在待同步数据写入以及执行同步任务时，若有线程访问相关数据，不会访问到未完成同步的数据。
以四节点集群为例，对于任意节点，其内存空间划分为集群内存空间和同步数据接收池，且每个节点上同步数据接收池的数量均为三个，每个同步数据接收池均由一个RDMA映射数组和原子索引构成，用于对该同步数据接收池中的数据进行维护。
为确定要将待同步数据写入的位置，各节点中预先存储有分布式集群中各节点中的内存区域(memory region)信息，即在各节点中预设的同步数据接收池与其他节点的对应关系。
在本申请实施例中，按照Raft协议进行集群数据同步的规则，当某个节点中的集群数据发生变更时，定义该节点为集群数据同步的源节点，由源节点将发生变更的数据作为待同步数据发送至主节点，再由主节点将待同步数据以广播的形式发送给分布式集群中各从节点进行集群数据同步。
则在本申请实施例提供的集群数据同步方法中，对于S101来说，主节点根据预先部署的监测规则，监测本地内存中的同步数据接收池是否被写入待同步数据。例如，主节点可以定时检查同步数据接收池，如发现新增的数据，即确定为待同步数据。或者，源节点也可以采用传统协议的通过IO来发送写完成标志的方式，来告知主节点已将待同步数据写入主节点的同步数据接收池，这相较于完全采用IO的方式发送待同步数据，对IO的占用时间有所减少，但仍然会占用IO。另外，还可以采用源节点在完成待同步数据的写入后，在待同步数据之后的地址或主节点的同步数据接收池中的另一约定地址写入写完成标志，等待主节点定时轮询同步数据接收池中是否出现写完成标志，当主节点发现写完成标志时，确定为待同步数据，但这需要主节点的CPU轮询大量的内存数据。
对于S102，当主节点的CPU监测到本地内存中同步数据接收池被源节点写入待同步数据后，启动在整个分布式集群中的集群数据同步工作，首先在主节点上对待同步数据进行信息校验。可选的校验方式可以参考现有的分布式集群同步方案，在此不再赘述。
对于S103，按照Raft协议进行集群数据同步的规则，当主节点通过对待同步数据的信息校验后，以广播的形式，基于RDMA网络调用通信设备将待同步数据直接写入各从节点的同步数据接收池，以使各从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务。
其中，主节点将待同步数据写入从节点的同步数据接收池的过程，与源节点将待同步数据写入主节点的同步数据接收池的方式类似，均为通过调用通信设备将待同步数据写入目的端的同步数据接收池。相应的，从节点监测本地内存中的同步数据接收池、当检测到待同步数据后启动校验，并在校验通过后执行对待同步数据的同步任务，与主节点在监测到待同步数据后执行对待同步数据的同步任务的过程类似，可以参考上述S101的说明。
主节点与各从节点在识别到本地内存的同步数据接收池中的待同步数据并通过校验后，各自根据待同步数据执行本地的同步任务。如直接按照待同步数据进行本地的同步任务即完成集群数据同步任务，则从节点无需向主节点返回确认码，由此节约了集群数据同步所需的CPU调度工作。
当产生集群数据变更的节点为主节点，则主节点在完成对待同步数据的校验后，直接执行S103即可。
在本申请实施例提供的集群数据同步方法中，通信设备可以采用RoCE网卡。RoCE全称RDMA over Converged Ethernet，即基于融合以太网的RDMA。
本申请实施例提供的集群数据同步方法，提供了一种结合RDMA网络对Raft协议优化的集群数据同步方案，产生待同步数据的源节点基于RDMA网络调用通信设备直接将待同步数据写入主节点的本地内存中预设的同步数据接收池，主节点在对待同步数据校验通过后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据直接写入从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，以直接写入的方式进行 待同步数据的传递，代替现有Raft协议的发送/接收的同步模式，可以避免发送/接收过程中的内存拷贝，主节点也无需在集群数据同步工作中控制IO复用，无需流控制，显著减轻了实现同步的代码量以及主节点的CPU调度工作量。
不论是源节点将待同步数据写入主节点的同步数据接收池，还是主节点将待同步数据写入从节点的同步数据接收池，即不论是源端是源节点、目的端是主节点，还是源端是主节点、目的端是从节点，均需要设置使目的端识别到待同步数据的写入的机制。
在上述实施例中介绍了可以由目的端定时检查本地的同步数据接收池中是否有新增数据的方式来识别到待同步数据。则在本申请实施例提供的集群数据同步方法中，S102中主节点监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：当轮询监测到本地内存中同步数据接收池有新增的待同步数据时，确认源节点写入了待同步数据。S103中主节点基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池，以使从节点在轮询监测到待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。即源端在将待同步数据写入目的端后，无需通知目的端写入完成，而是由目的端轮询本地内存，当检查到同步数据接收池中有新增数据时则认为是待同步数据，再启动同步操作。
或者可以采用传统协议的通过IO的方式在源端写完待同步数据后，再通过IO将写完成标志发给目的端，以使目的端知晓本地的同步数据接收池被写入待同步数据。则在本申请实施例提供的集群数据同步方法中，S102中主节点监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：接收到源节点在写完待同步数据后调用输入输出(IO)端口发送的写完成标志时，确认源节点写入了待同步数据。S103中主节点基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，调用输入输出(IO)端口向从节点发送写完成标志，以使从节点确认被写入待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。即源端在将待同步数据写入目的端后，通过调用目的端的IO的方式发送写完成标志，以使目的端获悉本地同步数据接收池中被写入了待同步数据。
或者可以由源端将待同步数据写入完成后，将写完成标志继续写入待同步数据之后的地址或目的端的同步数据接收池的另一约定地址，等待目的端定时轮询发信写完成标志时确定为待同步数据。则在本申请实施例提供的集群数据同步方法中，S102中主节点监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：监测到源节点在写完待同步数据后在主节点的写完成标志约定地址写入的写完成标志时，确认源节点写入了待同步数据。S103中主节点基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，在各从节点的写完成标志约定地址写入写完成标志，以使从节点确认被写入待同步数据后，完成对待同步数据的校验并执行对待同步数据的同步任务。可选地，源端可以在基于RDMA网络调用通信设备将待同步数据写入目的端之后，继续基于RDMA网络调用通信设备将写完成标志写入目的端中待同步数据之后或另一约定地址(即写完成标志约定地址)，目的端轮询写完成标志约定地址识别到写完成标志后，确认本地同步数据接收池中被写入了待同步数据。
而上述方式，要么需要占用IO来传输写完成标志，要么目的端无法及时发现写完成标志、造成集群数据同步任务无法及时启动。
故在上述实施例的基础上，在本申请实施例提供的集群数据同步方法中，S102中主节点监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：监测到源节点在写完待同步数据后基于RDMA网络的任务完成回调函数在主节点的CPU中更新的同步请求字段后，确认源节点写 入了待同步数据。
S103中主节点基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，基于任务完成回调函数在从节点的CPU中更新同步请求字段，以使从节点在监测到更新的同步请求字段后完成对待同步数据的校验并执行对待同步数据的同步任务。
在可选实施中，在各节点的CPU的控制程序中增加同步请求字段(可以定义为available字段)，通过RDMA技术特有的completion IO回调功能(即上文定义的任务完成回调函数)来标识有同步数据接收池中有数据写入。源端在完成对待同步数据的写入后，利用任务完成回调函数更新目的端的同步请求字段(如+1)，表示有数据发送，需要等待目的端处理。在这个过程中，使用RDMA无损数据网络，则可以省略源端检查发送失败的流程，同样减少了代码量。在目的端中，增加一个循环遍历与同步数据接收池对应的同步请求字段的检查线程，遍历循环各源端对应的同步数据接收池对应的同步请求字段，例如当发现某个节点对应的同步请求字段有一个等待处理的请求(available＝1)，则确认监测到待同步数据，进入处理待同步数据的流程。
当各节点均根据待同步数据完成本地的同步任务后，再将本地的同步数据接收池对应的同步请求字段减1(available-1)。
图2为本申请实施例提供的一种集群内存分布示意图。
在上述实施例中提到，通过设计各节点上同步数据接收池与其他节点一一对应以便于对来自不同源节点的待同步数据的管理。除此以外，还可以针对不同类型的待同步数据设置不同的同步规则，并将不同类型的待同步数据写入不同的存储区域来便于管理。在本申请实施例提供的集群数据同步方法中，同步数据接收池中设有多类与不同类型的待同步数据对应的子接收池。
相应的，S103中主节点在本地内存中执行对待同步数据的同步任务，可以为：按照待同步数据的类型，在本地内存中执行对待同步数据的同步任务。
S103中从节点执行对待同步数据的同步任务，可以为：从节点按照待同步数据的类型执行对待同步数据的同步任务。
在可选实施例中，不论是源端是源节点、目的端是主节点，还是源端是主节点、目的端是从节点，在源端根据本地存储的分布式集群中其他节点中源节点与同步数据接收池的对应关系的基础上，增加对不同类型的待同步数据的处理逻辑，以在源端写入以及目的端读取完成同步任务的过程中均按照待同步数据的类型执行对应的操作。
如可以按照待同步数据的数据量大小和同步频繁程度设计子接收池的类型。如图2所示，子接收池的类型可以包括：小数据接收池(small pool)、应答数据接收池(res pool)和大数据接收池(big pool)。
其中，小数据接收池用于接收短数据包，一般用于更新状态等数据交互量少，操作频繁，且CPU可以固定时间内完成数据处理操作的数据。
应答数据接收池用于接收确认数据和更新集群状态，在主节点状态下负责处理复杂逻辑的响应确认包，例如更新心跳信息。
大数据接收池用于处理大更新量数据，在更新时可能会直接覆盖远端节点的对应内存页，如快照(snapshot，Raft协议概念，指新节点加入时需要将集群状态全部更新至本地节点，操作过程可以近似于memcpy)更新过程，以及处理集群事务对应的过程。
同时，还可以设计同步数据接收池中预先按照各类型待同步数据的数据长度划分为各类型子接收池的数据分区。每个数据分区对应一段连续的内存地址。在写入待同步数据时，将待同步数据写入对应类型的空白数据分区。
基于本申请实施例提供的内存架构设计，结合本申请实施例提供的任务完成回调机制，在各节点的同步数据接收池中，可以设计同步请求字段和与其他节点对应的同步数据接收池中的子接收池的类型一一对应，源端在将待同步数据写入目的端之后，调用任务完成回调函数，根据待同步数据的类型将目的端与源端对应的同步数据接收池中对应数据类型的子接收池的同步请求字段进行更新，以便目的端的CPU在查看到同步请求字段更新后能够直接确定待同步数据的类型。以四节点集群中的主节点为例，主节点设有与其他三个从节点对应的同步数据接收池，每个同步数据接收池中又分为小数据接收池、应答数据接收池和大数据接收池三类子接收池，则一个同步数据接收池对应三个同步请求字段，则主节点中共维护九个同步请求字段。
结合上述设计，设计分布式集群中每个节点的本地内存中设有与其他节点一一对应的同步数据接收池，在每个同步数据接收池内又划分为小数据接收池、应答数据接收池和大数据接收池三类子接收池，且每类子接收池均包含多个该类子接收池的数据分区，实现对同步数据接收池的存储管理。
如图2所示，在一个四节点的分布式集群中，对于任意节点201(例如节点1)，其本地内存的内存空间划分为集群内存空间和与其余节点201(节点2、节点3、节点4)对应的同步数据接收池，在节点1中，同步数据接收池与节点2、节点3、节点4一一对应，每个同步数据接收池都由RDMA映射数组和原子索引构成，每个同步数据接收池中又划分为多个小数据接收池、多个应答数据接收池和多个大数据接收池，用于接收不同类型的待同步数据的写入以及在目的端上采用对应的处理方式执行同步任务。其中，节点1、节点2、节点3、节点4为各节点在分布式集群中的序号，而在进行集群数据同步时，在Raft协议网络中，若选举节点1为主节点，则节点2、节点3、节点4可以依次对应为从节点1、从节点2、从节点3。当其中任意一个从节点产生待同步数据时，则该从节点即为源节点，并进入源节点将待同步数据发送至主节点、由主节点将待同步数据广播至各从节点以完成集群数据同步任务的流程。
在上述实施例的基础上，本申请实施例进一步对不同类型的待同步数据的同步方法进行说明。
在本申请实施例提供的集群数据同步方法中，当待同步数据为与小数据接收池对应的数据类型时，
S102中源节点基于RDMA网络调用通信设备写入待同步数据，可以为：源节点将待同步数据组装为第一消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置；
S103中主节点基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，可以为：根据本地存储的从节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置。
在可选实施中，当待同步数据为与小数据接收池对应的数据类型时，本申请实施例提供一种集群数据同步过程如下：
源节点(例如上述实施例中举例的从节点1)获取本地集群内存空间，根据写入请求组织第一消息头(如Raft协议数据、对较复杂数据的序列化处理等)；
源节点根据本地存储的主节点中源节点与同步数据接收池的对应关系，确定自己在主节点的本地内存中对应的同步数据接收池。例如，从节点1为图2所示的节点2，则从节点1根据主节点的内存区域(memory region)信息，将第一消息头写入主节点的内存中与节点2对应的同步数据接收池的小数据接收池中(如node 2 small pool[index＝0])，并更新主节点与节点2对应的同步数据接收池的小数据接收池可用消息数加1；
在源节点的写入过程中，源节点调用写(write)接口，直接通过通信设备(如RoCE网卡)将第一消息头直接写入主节点的对应内存区域中，而非调用类似TCP协议的发送/接收(send/recv)功能，避免在发送/接收(send/recv)中的内存拷贝，主节点也无需维护IO复用数据，显著减少了代码量与主节点的CPU调度；
源节点在完成写入后，通过RDMA技术特有的任务完成回调函数(completion IO)，在主节点更新同步 请求字段(available)，表示有数据发送，需要等待主节点处理；在这个过程中，使用RDMA无损数据网络，则可以省略源节点检查发送失败的流程，同样减少了代码量；
在主节点中，通过对同步请求字段的检查线程遍历本地与其余各节点对应的同步数据接收池时，发现与源节点对应的同步数据接收池中有一个待处理请求(available＝1)，则进行信息校验；当对第一消息头校验合法后，更新主节点集群内存空间中的对应数据，并将第一消息头广播至所有从节点；同时，由于采用无损数据网络，在主节点广播完成后，将Raft协议中定义的集群间消息编号logentry加1；
主节点广播至所有从节点的过程与源节点将待同步数据写入主节点的过程类似；
在各从节点接收到待同步数据时，各自更新本地集群内存空间的数据，无需返回确认码。
当待同步数据为与应答数据接收池对应的数据类型时，与小数据接收池类似，即同样是通过任务完成回调函数处理返回值并更新同步请求字段。则在本申请实施例提供的集群数据同步方法中，当待同步数据为与应答数据接收池对应的数据类型时，
S102中源节点基于RDMA网络调用通信设备写入待同步数据，可以为：源节点将待同步数据组装为第二消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第二消息头写入对应的存储位置；
S103中基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，可以为：根据本地存储的从节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第二消息头写入对应的存储位置。
应答数据接收池的写入方法可以参考小数据接收池的写入方法。不同的是，针对不同类型的应答数据，可以设置额外的同步规则。如在应答数据接收池中可以包括各节点的心跳信号，以完成分布式集群中各节点的状态监控。例如，可以设计应答数据接收池的第一预设空间(如res pool[0])用于存放分布式集群中所有节点的心跳信息。可选地，各节点之间可以定时通过本申请实施例提供的集群数据同步方法进行心跳信息的同步。
则在本申请实施例中，本申请实施例提供的集群数据同步方法还可以包括：
定时向本地内存的第一预设空间写入当前时间戳；
通过RDMA网络将当前时间戳同步至各从节点的第一预设空间；
读取各从节点写入本地内存的响应时间戳；
根据各响应时间戳检查各从节点的连接状态。
在可选实施中，主节点会调度一个线程(如定义为心跳线程)每隔预设时间向特定RDMA内存写入当前时间戳，则当前时间戳会通过RDMA无损网络写入所有从节点的应答数据接收池的第一预设空间，各从节点再通过RDMA无损网络将响应时间戳写入主节点的应答数据接收池的第一预设空间，主节点检查所有从节点的响应时间戳即可。
对比传统阻塞方式发送/接收(send/recv)写入，本申请实施例提供的心跳包处理方式对于上层应用来说类似于完全读写本地内存，整个过程是非阻塞的。通常来说，这意味着更快的错误检测。发送/接收(send/recv)在集群状态发生错误的时候，系统底层发送(send)过程会阻塞，携带消息头的信息包与心跳信息混杂在一起，在超时发生后一起向上报告。对主节点来说，这意味着突发性的大量错误状态需要恢复，逻辑复杂后耗时较长。而通过非阻塞的写(write)操作，主节点可以在其他消息发生错误前就检测到心跳包超时，于是直接调度重构集群操作，对于集群恢复时间有相当的帮助。
应答数据接收池除了用于存储确认数据和更新集群状态数据外，还可以设置有用于存放用户自定义数据的第二预设空间(如res pool[1])用于存放用户自定义数据，即用于用户扩展的待同步数据类型。
在上述实施例的基础上，本申请实施例进一步对不同类型的待同步数据的同步方法进行说明。
在本申请实施例提供的集群数据同步方法中，当待同步数据为与大数据接收池对应的数据类型时，
S102中源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据的同步任务组装为第三消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置；
S103中在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，包括：
根据本地存储的从节点中同步数据接收池的存储区域信息，以广播的形式，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置，以使从节点在从节点的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在本地的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间。
针对大更新量数据，各节点在执行同步任务时往往需要耗费较长时间，将对分布式集群提供的业务服务造成较长时间的中断。故针对大更新量数据，本申请实施例提供集群数据同步方法设计大数据接收池来接收待同步数据，并在更新过程中，先接收待同步数据，并在大数据接收池中以待同步数据在集群内存空间中对应数据页为单位进行更新后，再将大数据接收池中更新后的数据页替换集群内存空间中对应的数据页，从而在更新过程中缩短对业务服务的中断。
在可选实施中，本申请实施例提供一种集群数据同步过程如下：
源节点根据本地存储的主节点中源节点与同步数据接收池的对应关系，确定自己在主节点的本地内存中对应的同步数据接收池后，源节点发起对主节点的大数据接收池的写入请求，并组装的第三消息头，将第三消息头写入主节点中与源节点对应的同步数据接收池(例如上述实施例中举例的四节点集群中，节点2/从节点1对应的node 2 big pool[index＝0])中，并更新主节点中该同步数据接收池的可用消息数加1；
源节点将第三消息头写入主节点的过程与对小数据接收池的写入同理；
主节点通过对同步请求字段的检查线程遍历本地与其余各节点对应的同步数据接收池，发现与源节点对应的同步数据接收池中有一个待处理请求(available＝1)，则进行信息校验；当对第三消息头校验合法后，以广播形式基于RDMA网络调用通信设备将第三消息头写入各从节点对应主节点设置的同步数据接收池中的大数据接收池，同时在本地开始对第三消息头中包含的同步任务的逻辑进行处理；
各从节点在接收到主节点发送的第三消息头后，分别在本地开始对第三消息头中包含的同步任务的逻辑进行处理；
通常来说，设计由大数据接收池接收的待同步数据均涉及到较为复杂的逻辑，处理相对更加耗时；在这个过程中，在大数据接收池中，按照第三消息头涉及到的数据，优先在大数据接收池的第三预设空间的分页内(例如预先已经将同步数据接收池中划分了多个大数据接收池的数据分区，则在写入第三消息头的数据分区内)处理第三消息头的同步任务的相关逻辑，并在执行完毕后，向主节点发送确认成功响应码；
确认成功响应码属于一种确认数据，故各从节点向主节点发送确认成功响应码，可以为各从节点基于RDMA网络调用通信设备将确认成功响应码写入主节点中与各从节点对应的同步数据接收池中的应答数据接收池；
主节点检测到本地同步数据接收池中的应答数据接收池中包含所有从节点的确认成功响应码，并确认本地已经执行完毕第三消息头的同步任务的相关逻辑，则将确认执行消息广播至所有从节点；
各从节点收到确认执行消息后，将第三预设空间内的分页拷贝至所在节点的集群内存空间中的对应分页，完成本地的同步任务；
各节点均完成分页替换后，完成本次分布式集群中的集群数据同步任务，各节点将本地的大数据接收池对应的同步请求字段减1(big pool available-1)。
由于在进行分页替换时，也会对业务服务造成中断，为进一步减少对业务服务的影响，在本申请实施例提供的集群数据同步方法中，在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间，可以为：
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，向各从节点发送确认成功响应码，以使各从节点在接收到确认成功响应码之后与主节点各自执行下述步骤：
将第三预设空间对应的集群内存空间中的第四预设空间的业务逻辑地址交换至第三预设空间，以基于第三预设空间的数据执行下一个处理逻辑，同时将第三预设空间的数据拷贝至第四预设空间；
在拷贝完成后，将业务逻辑地址交换回第四预设空间。
在可选实施中，各从节点在接收到主节点发送的确认执行消息后进行第三预设空间的分页与集群内存空间内对应分页的替换时，以及主节点在本地进行第三预设空间的分页与集群内存空间内对应分页的替换时，均通过原子(atomic)逻辑将集群内存空间中的对应地址(假设对应第四预设空间)交换至第三预设空间，以基于第三预设空间处理集群内存空间中该部分分页提供的服务，执行下一个处理逻辑；与此同时，各节点将第三预设空间的分页拷贝至第四预设空间；在拷贝完成后，短暂中断该部分数据提供的业务服务，再利用(atomic)逻辑交换回原地址后，恢复业务服务。
以集群内存空间有四个分页为例，假设第三预设空间为同步数据接收池中的大数据接收池中的big pool[0]分页，第四预设空间为集群内存空间的分页2，则在big pool[0]分页中执行同步任务后，交换big pool[0]分页的指针与分页2的指针，即以big pool[0]分页提供原本由分页2提供的服务，并将big pool[0]分页的数据拷贝至分页2，在拷贝完成后，再将指针交换回来，而后释放big pool[0]分页的空间。
本申请上述实施例对分布式集群中大数据的同步过程进行了说明。目前在业界中，Raft协议没有与事务结合起来，对于某些节点数据执行失败只能重置节点状态，这同样影响分布式集群的集群数据同步效率和同步失败后的数据恢复效率。对于集群事务来说，要求全部节点保持数据一致性，但对于Raft协议来说，数据只需要传输到过半节点即可保证集群数据唯一性，而未执行成功的少数节点会直接退出集群，重新刷新日志信息。即是说，对同步任务执行失败的节点需要清除所有数据，再从主节点重新获取。
而对大更新量数据的处理相较于小数据的同步会更加耗时和复杂，各节点在执行同步任务的过程中更容易执行失败，则采用现有的Raft协议的同步办法即会出现同步失败后集群恢复时间较长的问题。
事务(Transaction)是一种机制、一个操作序列，包含了一组操作命令。在对于事务数据进行同步时之所以会出现上述问题，是事务把所有的命令作为一个整体一起向系统提交或撤销操作请求，即这一组数据库命令要么都执行，要么都不执行，因此事务是一个不可分割的工作逻辑单元。
故在上述实施例的基础上，在本申请实施例提供的集群数据同步方法中，当待同步数据为与大数据接收池对应的数据类型且为非事务数据时，第三消息头仅包含待同步数据的同步逻辑；
当待同步数据为与大数据接收池对应的数据类型且为事务数据时，第三消息头包含待同步数据的同步逻辑以及与待同步数据对应的事务回调函数，以使主节点或从节点在执行对待同步数据的同步任务时执行事务回调函数对应的逻辑；
其中，事务回调函数至少包括针对事务数据同步的数据备份函数以及关键数据保护函数。
需要说明的是，在本申请实施例中，非事务数据指需要同步的数据量较大、较为集中，但允许部分数据同步失败的待同步数据，例如针对部分节点上区分与其他节点的独有配置的数据同步，则无需使其他所有节点均对这一部分数据进行同步，也能视为完成了分布式集群的集群同步任务。事务数据则指上文提到的事务过程的待同步数据，在执行过程中只能全部执行成功，若其中部分数据执行失败，则整个待同步数据的同步任务均执行失败。
针对非事务数据的同步过程的说明请参考上述实施例，在此不再赘述。
而针对事务数据的同步过程，在进行待同步数据的传输、在各节点上执行同步任务的相关逻辑以及分页替换的步骤与非事务数据的步骤，但要求事务数据的第三消息头中提供对应功能的事务回调函数。
可选地，事务过程均有唯一对应的标识(ID)，预先针对分布式集群中的事务过程设计事务回调函数，以使各节点在发起对事务数据的同步时在注册的第三消息头中提供对应功能的事务回调函数。要提供的事务回调函数至少包括各节点在执行同步任务之前进行数据备份，并对关键数据进行保护，如加锁以暂停对外的业务服务，避免在同步过程中产生错误。这些准备工作是为了在对事务数据同步失败后可以回到同步前的状态，而不是像相关技术中那样一旦同步失败就要将节点退出集群再重新加入。其中，关键数据可以为分布式集群所面向的用户的用户数据，在进行事务性数据的同步时，需要对用户数据进行保护。例如，用户对分布式集群的命名需要同步至分布式集群中的各个节点，而当用户修改集群名称时，就需要对所有节点配置的集群名称先进行加锁保护，避免在同步过程中其他线程访问到集群名称为不确定的集群名称(有的访问到修改前的集群名称，有的访问到修改后的集群名称)。通过对关键数据进行保护，避免在分布式集群数据同步过程中接收业务访问时数据错乱。
在此基础上，本申请实施例提供一组事务回调函数的类型如下：
事务检查函数(evt_transaction_check)，用于检查该数据执行事务的前提条件是否合理；
数据备份函数(evt_transaction_prepare)，用于在准备执行事务时，备份原有数据，关键数据保护等流程；
以上两个操作成功或失败时，从节点会向主节点返回逻辑成功消息；
事务执行函数(evt_transaction_commit)，用于执行事务的实际逻辑，该函数回调执行后，事务数据应当准备完毕，并且可以正常读取；在待同步数据的同步任务执行完毕后，所有从节点会向主节点返回同步任务执行完毕消息；若主节点检查所有从节点均成功执行完了待同步数据的同步任务，则对所有节点的调度事务完成函数(evt_transaction_finish)；否则，调度事务修复函数(evt_transaction_restore)；
事务完成函数(evt_transaction_finish)，用于在待同步数据的同步任务执行成功后回调，释放关键数据保护和删除备份操作等过程；
事务修复函数(evt_transaction_restore)，用于在意外执行发生错误后回调，释放数据保护，恢复数据至原始状态。
则基于上述事务回调函数，本申请实施例提供一种执行事务回调函数对应的逻辑包括：
检查待同步数据的同步任务的合理性；
执行数据备份及关键数据保护流程；
若待同步数据的同步任务合理且成功完成数据备份及关键数据保护流程，则向主节点返回准备逻辑执行成功消息；
在准备逻辑执行成功后，触发执行待同步数据的同步任务的逻辑；
在执行完毕待同步数据的同步任务的逻辑后，向主节点返回同步任务执行完毕消息；
若主节点及各从节点均成功执行待同步数据的同步任务的逻辑，则释放关键数据保护，删除数据备份；否则，执行发生错误后回调操作，释放关键数据保护并基于数据备份恢复集群数据至初始状态。
上文详述了集群数据同步方法对应的各个实施例，在此基础上，本申请还公开了与上述方法对应的集群数据同步装置、设备及非易失性可读存储介质。
图3为本申请实施例提供的一种集群数据同步装置的结构示意图。
如图3所示，本申请实施例提供的集群数据同步装置包括：
监测单元301被设置为监测本地内存中预设的同步数据接收池；
校验单元302被设置为当监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对待同步数据进行校验；
同步单元303被设置为当待同步数据通过校验后，在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使 从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务；
其中，源节点为分布式集群中的一个从节点。
可选的，监测单元301监测到本地内存中同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据，可以为：
监测到源节点在写完待同步数据后基于RDMA网络的任务完成回调函数在主节点的CPU中更新的同步请求字段后，确认源节点写入了待同步数据；
同步单元303基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，可以为：
基于RDMA网络调用通信设备将待同步数据写入从节点的同步数据接收池后，基于任务完成回调函数在从节点的CPU中更新同步请求字段，以使从节点在监测到更新的同步请求字段后完成对待同步数据的校验并执行对待同步数据的同步任务。
可选的，同步数据接收池与分布式集群中除所在节点外的其余节点一一对应；
同步数据接收池设有用于维护所在节点外的其余节点与同步数据接收池的对应关系的RDMA映射数组以及用于为待同步数据分配空余空间的原子索引。
可选的，同步数据接收池中设有多类与不同类型的待同步数据对应的子接收池；
相应的，在本地内存中执行对待同步数据的同步任务，可以为：
按照待同步数据的类型，在本地内存中执行对待同步数据的同步任务；
从节点执行对待同步数据的同步任务，可以为：
从节点按照待同步数据的类型执行对待同步数据的同步任务。
可选的，同步数据接收池中预先按照各类型待同步数据的数据长度划分为各类型子接收池的数据分区。
可选的，子接收池的类型包括：小数据接收池、应答数据接收池和大数据接收池。
可选的，当待同步数据为与小数据接收池对应的数据类型或应答数据接收池对应的数据类型时，
源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据组装为第一消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置；
同步单元303基于RDMA网络调用通信设备将待同步数据写入分布式集群的各从节点的同步数据接收池，可以为：
根据本地存储的从节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第一消息头写入对应的存储位置。
可选的，应答数据接收池的第一预设空间用于存放分布式集群中所有节点的心跳信息。
可选的，本申请实施例提供的集群数据同步装置还包括：
时间戳写入单元，被设置为定时向本地内存的第一预设空间写入当前时间戳；
时间戳同步单元，被设置为通过RDMA网络将当前时间戳同步至各从节点的第一预设空间；
时间戳读取单元，被设置为读取各从节点写入本地内存的响应时间戳；
状态检查单元，被设置为根据各响应时间戳检查各从节点的连接状态。
可选的，应答数据接收池的第二预设空间用于存放用户自定义数据。
可选的，当待同步数据为与大数据接收池对应的数据类型时，
源节点基于RDMA网络调用通信设备写入待同步数据，可以为：
源节点将待同步数据的同步任务组装为第三消息头后，根据本地存储的主节点中同步数据接收池的存储区域信息，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置；
同步单元303在本地内存中执行对待同步数据的同步任务，并以广播的形式，基于RDMA网络调用通信设 备将待同步数据写入分布式集群的各从节点的同步数据接收池，以使从节点在监测到待同步数据并完成对待同步数据的校验后执行对待同步数据的同步任务，包括：
根据本地存储的从节点中同步数据接收池的存储区域信息，以广播的形式，基于RDMA网络调用通信设备将第三消息头写入对应的存储位置，以使从节点在从节点的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在本地的大数据接收池的第三预设空间的分页内执行第三消息头所包含的逻辑；
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间。
可选的，同步单元303在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，控制本地以及各从节点将所在节点的第三预设空间的分页拷贝至所在节点的集群内存空间，可以为：
在确认本地以及各从节点均完成执行第三消息头所包含的逻辑之后，向各从节点发送确认成功响应码，以使各从节点在接收到确认成功响应码之后与主节点各自执行下述步骤：
将第三预设空间对应的集群内存空间中的第四预设空间的业务逻辑地址交换至第三预设空间，以基于第三预设空间的数据执行下一个处理逻辑，同时将第三预设空间的数据拷贝至第四预设空间；
在拷贝完成后，将业务逻辑地址交换回第四预设空间。
可选的，当待同步数据为与大数据接收池对应的数据类型且为非事务数据时，第三消息头仅包含待同步数据的同步逻辑；
当待同步数据为与大数据接收池对应的数据类型且为事务数据时，第三消息头包含待同步数据的同步逻辑以及与待同步数据对应的事务回调函数，以使主节点或从节点在执行对待同步数据的同步任务时执行事务回调函数对应的逻辑；
其中，事务回调函数至少包括针对事务数据同步的数据备份函数以及关键数据保护函数。
进一步，同步单元303执行事务回调函数对应的逻辑包括：
检查待同步数据的同步任务的合理性；
执行数据备份及关键数据保护流程；
若待同步数据的同步任务合理且成功完成数据备份及关键数据保护流程，则向主节点返回准备逻辑执行成功消息；
在准备逻辑执行成功后，触发执行待同步数据的同步任务的逻辑；
在执行完毕待同步数据的同步任务的逻辑后，向主节点返回同步任务执行完毕消息；
若主节点及各从节点均成功执行待同步数据的同步任务的逻辑，则释放关键数据保护，删除数据备份；否则，执行发生错误后回调操作，释放关键数据保护并基于数据备份恢复集群数据至初始状态。
可选的，通信设备可以为RoCE网卡。
由于装置部分的实施例与方法部分的实施例相互对应，因此装置部分的实施例请参见方法部分的实施例的描述，这里暂不赘述。
图4为本申请实施例提供的一种集群数据同步设备的结构示意图。
如图4所示，本申请实施例提供的集群数据同步设备包括：
存储器410被设置为存储计算机程序411；
处理器420被设置为执行计算机程序411，该计算机程序411被处理器420执行时实现如上述任意一项实施例集群数据同步方法的步骤。
其中，处理器420可以包括一个或多个处理核心，比如3核心处理器、8核心处理器等。处理器420可以采用数字信号处理DSP(Digital Signal Processing)、现场可编程门阵列FPGA(Field-Programmable Gate Array)、可编程逻辑阵列PLA(Programmable Logic Array)中的至少一种硬件形式来实现。处理器420也可以包括主处理器和协处理器，主处理器是被设置为对在唤醒状态下的数据进行处理的处理器，也称 中央处理器CPU(Central Processing Unit)；协处理器是被设置为对在待机状态下的数据进行处理的低功耗处理器。在一些实施例中，处理器420可以集成有图像处理器GPU(Graphics Processing Unit)，GPU被设置为负责显示屏所需要显示的内容的渲染和绘制。一些实施例中，处理器420还可以包括人工智能AI(Artificial Intelligence)处理器，该AI处理器被设置为处理有关机器学习的计算操作。
存储器410可以包括一个或多个非易失性可读存储介质，该非易失性可读存储介质可以是非暂态的。存储器410还可包括高速随机存取存储器，以及非易失性存储器，比如一个或多个磁盘存储设备、闪存存储设备。本实施例中，存储器410至少被设置为存储以下计算机程序411，其中，该计算机程序411被处理器420加载并执行之后，能够实现前述任一实施例公开的集群数据同步方法中的相关步骤。另外，存储器410所存储的资源还可以包括操作系统412和数据413等，存储方式可以是短暂存储或者永久存储。其中，操作系统412可以为Windows。数据413可以包括但不限于上述方法所涉及到的数据。
在一些实施例中，集群数据同步设备还可包括有显示屏430、电源440、通信接口450、输入输出接口460、传感器470以及通信总线480。
本领域技术人员可以理解，图4中示出的结构并不构成对集群数据同步设备的限定，可以包括比图示更多或更少的组件。
本申请实施例提供的集群数据同步设备，包括存储器和处理器，处理器在执行存储器存储的程序时，能够实现如上的集群数据同步方法，效果同上。
需要说明的是，以上所描述的装置、设备实施例仅仅是示意性的，例如，模块的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个模块或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或模块的间接耦合或通信连接，可以是电性，机械或其它的形式。作为分离部件说明的模块可以是或者也可以不是物理上分开的，作为模块显示的部件可以是或者也可以不是物理模块，即可以位于一个地方，或者也可以分布到多个网络模块上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能模块可以集成在一个处理模块中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个非易失性可读存储介质中。基于这样的理解，本申请的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，执行本申请各个实施例方法的全部或部分步骤。
为此，本申请实施例还提供一种非易失性可读存储介质，该非易失性可读存储介质上存储有计算机程序，计算机程序被处理器执行时实现如集群数据同步方法的步骤。
该非易失性可读存储介质可以包括：U盘、移动硬盘、只读存储器ROM(Read-Only Memory)、随机存取存储器RAM(Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
本实施例中提供的非易失性可读存储介质所包含的计算机程序能够在被处理器执行时实现如上的集群数据同步方法的步骤，效果同上。
以上对本申请所提供的一种集群数据同步方法、装置、设备及非易失性可读存储介质进行了详细介绍。说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的装置、设备及非易失性可读存储介质而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以对本申请进行若干改进和修饰，这些改进和修饰也落入本申请权利要求的保护范围内。
还需要说明的是，在本说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括上述要素的过程、方法、物品或者设备中还存在另外的相同要素。

Claims (22)

  1. 一种集群数据同步方法，其特征在于，基于分布式集群中的主节点，包括：
    监测本地内存中预设的同步数据接收池；
    当监测到本地内存中所述同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对所述待同步数据进行校验；
    当所述待同步数据通过校验后，在本地内存中执行对所述待同步数据的同步任务，并以广播的形式，基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务；
    其中，所述源节点为所述分布式集群中的一个所述从节点。
  2. 根据权利要求1所述的集群数据同步方法，其特征在于，监测到本地内存中所述同步数据接收池被所述源节点基于RDMA网络调用通信设备写入的所述待同步数据，包括：
    监测到所述源节点在写完所述待同步数据后基于所述RDMA网络的任务完成回调函数在所述主节点的CPU中更新的同步请求字段时，确认所述源节点写入了所述待同步数据；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务，包括：
    基于所述RDMA网络调用通信设备将所述待同步数据写入所述从节点的所述同步数据接收池后，基于所述任务完成回调函数在所述从节点的CPU中更新所述同步请求字段，以使所述从节点在监测到更新的所述同步请求字段后完成对所述待同步数据的校验并执行对所述待同步数据的同步任务。
  3. 根据权利要求1所述的集群数据同步方法，其特征在于，监测到本地内存中所述同步数据接收池被所述源节点基于RDMA网络调用通信设备写入的所述待同步数据，包括：
    当轮询监测到本地内存中所述同步数据接收池有新增的所述待同步数据时，确认所述源节点写入了所述待同步数据；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务，包括：
    基于所述RDMA网络调用通信设备将所述待同步数据写入所述从节点的所述同步数据接收池，以使所述从节点在轮询监测到所述待同步数据后，完成对所述待同步数据的校验并执行对所述待同步数据的同步任务。
  4. 根据权利要求1所述的集群数据同步方法，其特征在于，监测到本地内存中所述同步数据接收池被所述源节点基于RDMA网络调用通信设备写入的所述待同步数据，包括：
    接收到所述源节点在写完所述待同步数据后调用输入输出端口发送的写完成标志时，确认所述源节点写入了所述待同步数据；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务，包括：
    基于所述RDMA网络调用通信设备将所述待同步数据写入所述从节点的所述同步数据接收池后，调用输入输出端口向所述从节点发送写完成标志，以使所述从节点确认被写入所述待同步数据后，完成对所述待同步数据的校验并执行对所述待同步数据的同步任务。
  5. 根据权利要求1所述的集群数据同步方法，其特征在于，监测到本地内存中所述同步数据接收池被所述源节点基于RDMA网络调用通信设备写入的所述待同步数据，包括：
    监测到所述源节点在写完所述待同步数据后在所述主节点的写完成标志约定地址写入的写完成标志时，确认所述源节点写入了所述待同步数据；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务，包括：
    基于所述RDMA网络调用通信设备将所述待同步数据写入所述从节点的所述同步数据接收池后，在各所述从节点的写完成标志约定地址写入写完成标志，以使所述从节点确认被写入所述待同步数据后，完成对所述待同步数据的校验并执行对所述待同步数据的同步任务。
  6. 根据权利要求1所述的集群数据同步方法，其特征在于，所述同步数据接收池与所述分布式集群中除所在节点外的其余节点一一对应。
  7. 根据权利要求1所述的集群数据同步方法，其特征在于，所述同步数据接收池设有用于维护所在节点外的其余节点与所述同步数据接收池的对应关系的RDMA映射数组以及用于为所述待同步数据分配空余空间的原子索引。
  8. 根据权利要求1所述的集群数据同步方法，其特征在于，所述同步数据接收池中设有多类与不同类型的所述待同步数据对应的子接收池；
    相应的，所述在本地内存中执行对所述待同步数据的同步任务，包括：
    按照所述待同步数据的类型，在本地内存中执行对所述待同步数据的同步任务；
    所述从节点执行对所述待同步数据的同步任务，包括：
    所述从节点按照所述待同步数据的类型执行对所述待同步数据的同步任务。
  9. 根据权利要求8所述的集群数据同步方法，其特征在于，所述同步数据接收池中预先按照各类型所述待同步数据的数据长度划分为各类型所述子接收池的数据分区。
  10. 根据权利要求8所述的集群数据同步方法，其特征在于，所述子接收池的类型包括：小数据接收池、应答数据接收池和大数据接收池。
  11. 根据权利要求10所述的集群数据同步方法，其特征在于，当所述待同步数据为与所述小数据接收池对应的数据类型时，
    所述源节点基于所述RDMA网络调用通信设备写入所述待同步数据，包括：
    所述源节点将所述待同步数据组装为第一消息头后，根据本地存储的所述主节点中所述同步数据接收池的存储区域信息，基于所述RDMA网络调用通信设备将所述第一消息头写入对应的存储位置；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，包括：
    根据本地存储的所述从节点中所述同步数据接收池的存储区域信息，基于所述RDMA网络调用通信设备将所述第一消息头写入对应的存储位置。
  12. 根据权利要求10所述的集群数据同步方法，其特征在于，当所述待同步数据为与所述应答数据接收池对应的数据类型时，
    所述源节点基于所述RDMA网络调用通信设备写入所述待同步数据，包括：
    所述源节点将所述待同步数据组装为第二消息头后，根据本地存储的所述主节点中所述同步数据接收池的存储区域信息，基于所述RDMA网络调用通信设备将所述第二消息头写入对应的存储位置；
    所述基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，包括：
    根据本地存储的所述从节点中所述同步数据接收池的存储区域信息，基于所述RDMA网络调用通信设备将所述第二消息头写入对应的存储位置。
  13. 根据权利要求10所述的集群数据同步方法，其特征在于，所述应答数据接收池的第一预设空间用 于存放所述分布式集群中所有节点的心跳信息。
  14. 根据权利要求13所述的集群数据同步方法，其特征在于，还包括：
    定时向本地内存的所述第一预设空间写入当前时间戳；
    通过所述RDMA网络将所述当前时间戳同步至各所述从节点的所述第一预设空间；
    读取各所述从节点写入本地内存的响应时间戳；
    根据各所述响应时间戳检查各所述从节点的连接状态。
  15. 根据权利要求10所述的集群数据同步方法，其特征在于，所述应答数据接收池的第二预设空间用于存放用户自定义数据。
  16. 根据权利要求10所述的集群数据同步方法，其特征在于，当所述待同步数据为与所述大数据接收池对应的数据类型时，
    所述源节点基于所述RDMA网络调用通信设备写入所述待同步数据，包括：
    所述源节点将所述待同步数据的同步任务组装为第三消息头后，根据本地存储的所述主节点中所述同步数据接收池的存储区域信息，基于所述RDMA网络调用通信设备将所述第三消息头写入对应的存储位置；
    所述在本地内存中执行对所述待同步数据的同步任务，并以广播的形式，基于所述RDMA网络调用通信设备将所述待同步数据写入所述分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务，包括：
    根据本地存储的所述从节点中所述同步数据接收池的存储区域信息，以广播的形式，基于所述RDMA网络调用通信设备将所述第三消息头写入对应的存储位置，以使所述从节点在所述从节点的所述大数据接收池的第三预设空间的分页内执行所述第三消息头所包含的逻辑；
    在本地的所述大数据接收池的所述第三预设空间的分页内执行所述第三消息头所包含的逻辑；
    在确认本地以及各所述从节点均完成执行所述第三消息头所包含的逻辑之后，控制本地以及各所述从节点将所在节点的所述第三预设空间的分页拷贝至所在节点的集群内存空间。
  17. 根据权利要求16所述的集群数据同步方法，其特征在于，所述在确认本地以及各所述从节点均完成执行所述第三消息头所包含的逻辑之后，控制本地以及各所述从节点将所在节点的所述第三预设空间的分页拷贝至所在节点的集群内存空间，包括：
    在确认本地以及各所述从节点均完成执行所述第三消息头所包含的逻辑之后，向各所述从节点发送确认成功响应码，以使各所述从节点在接收到所述确认成功响应码之后与所述主节点各自执行下述步骤：
    将所述第三预设空间对应的所述集群内存空间中的第四预设空间的业务逻辑地址交换至所述第三预设空间，以基于所述第三预设空间的数据执行下一个处理逻辑，同时将所述第三预设空间的数据拷贝至所述第四预设空间；
    在拷贝完成后，将所述业务逻辑地址交换回所述第四预设空间。
  18. 根据权利要求16所述的集群数据同步方法，其特征在于，当所述待同步数据为与所述大数据接收池对应的数据类型且为非事务数据时，所述第三消息头仅包含所述待同步数据的同步逻辑；
    当所述待同步数据为与所述大数据接收池对应的数据类型且为事务数据时，所述第三消息头包含所述待同步数据的同步逻辑以及与所述待同步数据对应的事务回调函数，以使所述主节点或所述从节点在执行所述对所述待同步数据的同步任务时执行所述事务回调函数对应的逻辑；
    其中，所述事务回调函数至少包括针对事务数据同步的数据备份函数以及关键数据保护函数。
  19. 根据权利要求18所述的集群数据同步方法，其特征在于，执行所述事务回调函数对应的逻辑包括：
    检查所述待同步数据的同步任务的合理性；
    执行数据备份及关键数据保护流程；
    若所述待同步数据的同步任务合理且成功完成所述数据备份及关键数据保护流程，则向所述主节点返 回准备逻辑执行成功消息；
    在准备逻辑执行成功后，触发执行所述待同步数据的同步任务的逻辑；
    在执行完毕所述待同步数据的同步任务的逻辑后，向所述主节点返回同步任务执行完毕消息；
    若所述主节点及各所述从节点均成功执行所述待同步数据的同步任务的逻辑，则释放关键数据保护，删除数据备份；否则，执行发生错误后回调操作，释放所述关键数据保护并基于数据备份恢复集群数据至初始状态。
  20. 一种集群数据同步装置，其特征在于，包括：
    监测单元，被设置为监测本地内存中预设的同步数据接收池；
    校验单元，被设置为当监测到本地内存中所述同步数据接收池被源节点基于RDMA网络调用通信设备写入的待同步数据时，对所述待同步数据进行校验；
    同步单元，被设置为当所述待同步数据通过校验后，在本地内存中执行对所述待同步数据的同步任务，并以广播的形式，基于所述RDMA网络调用通信设备将所述待同步数据写入分布式集群的各从节点的所述同步数据接收池，以使所述从节点在监测到所述待同步数据并完成对所述待同步数据的校验后执行对所述待同步数据的同步任务；
    其中，所述源节点为所述分布式集群中的一个所述从节点。
  21. 一种集群数据同步设备，其特征在于，包括：
    存储器，被设置为存储计算机程序；
    处理器，被设置为执行所述计算机程序，所述计算机程序被所述处理器执行时实现如权利要求1至19任意一项所述集群数据同步方法的步骤。
  22. 一种非易失性可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至19任意一项所述集群数据同步方法的步骤。
PCT/CN2023/116231 2022-11-25 2023-08-31 集群数据同步方法、装置、设备及非易失性可读存储介质 Ceased WO2024109239A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211487122.9 2022-11-25
CN202211487122.9A CN115550384B (zh) 2022-11-25 2022-11-25 集群数据同步方法、装置、设备及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2024109239A1 true WO2024109239A1 (zh) 2024-05-30

Family

ID=84719739

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/116231 Ceased WO2024109239A1 (zh) 2022-11-25 2023-08-31 集群数据同步方法、装置、设备及非易失性可读存储介质

Country Status (2)

Country Link
CN (1) CN115550384B (zh)
WO (1) WO2024109239A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119441355A (zh) * 2024-11-05 2025-02-14 上海沄熹科技有限公司 一种基于rdma共享内存池的分布式数据库系统
CN119576690A (zh) * 2024-11-14 2025-03-07 中国通信建设第四工程局有限公司 一种数据中心集群监控方法、装置及系统
CN121029869A (zh) * 2025-10-31 2025-11-28 南开大学 基于数据一致性拷贝和共识算法的空管分布式存储中间件

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115988005B (zh) * 2022-11-08 2023-07-18 北京太格时代电气股份有限公司 一种采用分布式集群进行信号同步的方法
CN115550384B (zh) * 2022-11-25 2023-03-10 苏州浪潮智能科技有限公司 集群数据同步方法、装置、设备及计算机可读存储介质
CN116647571B (zh) * 2023-06-25 2026-04-10 济南浪潮数据技术有限公司 一种ha应用缓存同步方法、装置、设备及介质
CN116915676B (zh) * 2023-07-13 2024-09-06 南方电网科学研究院有限责任公司 一种电磁暂态实时仿真通信组件的评测方法、装置和设备
CN117857660B (zh) * 2024-03-07 2024-08-02 济南浪潮数据技术有限公司 数据传输协议自适应方法、系统、设备及介质
CN119728719A (zh) * 2025-02-27 2025-03-28 苏州元脑智能科技有限公司 数据同步方法、存储介质、电子设备及程序产品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140047263A1 (en) * 2012-08-08 2014-02-13 Susan Coatney Synchronous local and cross-site failover in clustered storage systems
CN110691062A (zh) * 2018-07-06 2020-01-14 浙江大学 一种数据写入方法、装置及其设备
CN112597251A (zh) * 2020-12-29 2021-04-02 天津南大通用数据技术股份有限公司 数据库集群日志同步方法、装置、服务器及存储介质
CN113360577A (zh) * 2021-06-18 2021-09-07 苏州浪潮智能科技有限公司 一种mpp数据库数据处理方法、装置、设备及存储介质
CN115550384A (zh) * 2022-11-25 2022-12-30 苏州浪潮智能科技有限公司 集群数据同步方法、装置、设备及计算机可读存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10061743B2 (en) * 2015-01-27 2018-08-28 International Business Machines Corporation Host based non-volatile memory clustering using network mapped storage
CN110019093A (zh) * 2017-12-28 2019-07-16 中国移动通信集团安徽有限公司 数据写入方法、装置、设备及介质
CN109639777B (zh) * 2018-11-28 2021-12-10 优刻得科技股份有限公司 数据同步的方法、装置、系统和非易失性存储介质
CN111259072B (zh) * 2020-01-08 2023-11-14 广州虎牙科技有限公司 数据同步方法、装置、电子设备和计算机可读存储介质
CN111368002A (zh) * 2020-03-05 2020-07-03 广东小天才科技有限公司 一种数据处理方法、系统、计算机设备和存储介质
CN113905054B (zh) * 2021-08-30 2023-08-08 苏州浪潮智能科技有限公司 基于RDMA的Kudu集群数据同步方法、装置、系统
CN114172916A (zh) * 2021-11-26 2022-03-11 苏州浪潮智能科技有限公司 一种Spark集群中的数据传输方法、系统及相关装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140047263A1 (en) * 2012-08-08 2014-02-13 Susan Coatney Synchronous local and cross-site failover in clustered storage systems
CN110691062A (zh) * 2018-07-06 2020-01-14 浙江大学 一种数据写入方法、装置及其设备
CN112597251A (zh) * 2020-12-29 2021-04-02 天津南大通用数据技术股份有限公司 数据库集群日志同步方法、装置、服务器及存储介质
CN113360577A (zh) * 2021-06-18 2021-09-07 苏州浪潮智能科技有限公司 一种mpp数据库数据处理方法、装置、设备及存储介质
CN115550384A (zh) * 2022-11-25 2022-12-30 苏州浪潮智能科技有限公司 集群数据同步方法、装置、设备及计算机可读存储介质

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119441355A (zh) * 2024-11-05 2025-02-14 上海沄熹科技有限公司 一种基于rdma共享内存池的分布式数据库系统
CN119576690A (zh) * 2024-11-14 2025-03-07 中国通信建设第四工程局有限公司 一种数据中心集群监控方法、装置及系统
CN121029869A (zh) * 2025-10-31 2025-11-28 南开大学 基于数据一致性拷贝和共识算法的空管分布式存储中间件

Also Published As

Publication number Publication date
CN115550384B (zh) 2023-03-10
CN115550384A (zh) 2022-12-30

Similar Documents

Publication Publication Date Title
CN115550384B (zh) 集群数据同步方法、装置、设备及计算机可读存储介质
CN103827829B (zh) 在中间件机器环境中提供和管理用于多节点应用的消息队列的系统及方法
US8346719B2 (en) Multi-node replication systems, devices and methods
CN107771321B (zh) 数据中心中的恢复
FI101432B (fi) Vikasietoinen tietokonejärjestelmä
JP3790589B2 (ja) 分散データベーストランザクションのコミットメント方法
US8495266B2 (en) Distributed lock
CN111368002A (zh) 一种数据处理方法、系统、计算机设备和存储介质
CN102265277A (zh) 数据存储系统的操作方法和装置
CN101383690A (zh) 一种基于socket的容错计算机系统的网络同步方法
CN1770110B (zh) 对I/O完成进行无锁InfiniBand轮询的方法和系统
CN101814077A (zh) 一种基于oci 9的数据库访问中间件
US6594709B1 (en) Methods and apparatus for transferring data using a device driver
WO2025195152A1 (zh) 数据备份系统、方法、装置、设备、存储介质及程序产品
Camargos et al. Multicoordinated paxos
US20210218827A1 (en) Methods, devices and systems for non-disruptive upgrades to a replicated state machine in a distributed computing environment
CN120723558B (zh) 一种芯片原型验证方法、设备、介质和产品
US8301750B2 (en) Apparatus, system, and method for facilitating communication between an enterprise information system and a client
US6487580B1 (en) Method and system for managing concurrently executable computer processes
Bondavalli et al. State restoration in a COTS-based N-modular architecture
CN112153114A (zh) 一种分时访问共享器件的方法、系统、设备及介质
US12386854B2 (en) Data synchronization method and apparatus
CN118331644A (zh) 一种交互控制方法、装置、设备及介质
CN112069160B (zh) 一种基于cap数据清洗同步方法
CN111752911A (zh) 一种基于Flume的数据传输方法、系统、终端及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23893332

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23893332

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 28.11.2025)