CN115292407A

CN115292407A - Synchronization method, apparatus and storage medium

Info

Publication number: CN115292407A
Application number: CN202210836771.9A
Authority: CN
Inventors: 徐加伟; 周杰; 李飞飞; 章颖强; 黄贵
Original assignee: Alibaba China Co Ltd
Current assignee: Alibaba China Co Ltd
Priority date: 2022-07-15
Filing date: 2022-07-15
Publication date: 2022-11-04

Abstract

The embodiment of the application provides a synchronization method, synchronization equipment and a storage medium. In the synchronization method, a main node acquires a physical log and synchronizes the physical log to a standby node based on a consistency protocol in the process of executing a user request, and when a consistency persistence site of the physical log of a cluster covers the physical log generated by a transaction requested by the user, a processing result requested by the user can be returned. In the embodiment, the logic logs are not required to be generated and synchronized after the transaction is submitted, the physical logs can be accurately synchronized across the nodes based on the consistency protocol, meanwhile, the delay generated in the log synchronization process is reduced, and the high availability of the database and the transaction processing performance of the database are improved.

Description

Synchronization method, apparatus and storage medium

Technical Field

The present application relates to the field of database technologies, and in particular, to a synchronization method, device, and storage medium.

Background

With the rising of user data scale and the development of the demand for data security, the demand of high available performance of the database for users is higher and higher. In order to improve the high availability of the database, the existing database product may synchronize the corresponding logical logs of the transaction operation based on a Master-Slave replication mode (Master-Slave) so as to perform data backup on different nodes. However, the method for realizing data replication based on the logic log is in a transaction level, needs to wait for the transaction to be synchronized after being submitted, has higher delay, and reduces the transaction processing performance of the database. Therefore, a new solution is yet to be proposed.

Disclosure of Invention

Aspects of the present disclosure provide a synchronization method, device, and storage medium, which are used to achieve high availability of a database and improve transaction performance of the database.

The embodiment of the application provides a synchronization method, which comprises the following steps: receiving a user request; acquiring a physical log generated in the process of executing the user request; synchronizing the physical logs to at least one standby node based on a consistency protocol to cause the at least one standby node to store the physical logs in respective file storage systems; according to the storage success message returned by the at least one standby node, determining a consistent storage location of the physical logs of the distributed database cluster; and if the consistent storage location covers the physical log generated by the user request, returning the processing result of the user request.

The embodiment of the application also provides a synchronization method, which is used for receiving the physical log sent by the main node based on the consistency protocol; the physical log is generated by the main node in the process of executing the user request; storing the physical log in a file storage system, and returning a storage success message to the main node; responding to an instruction of upgrading to a new main node, and if the storage location of the physical log in the file storage system is ahead of the current consistent storage location of the distributed database cluster, playing back the physical log according to the current consistent storage location of the distributed database cluster.

An embodiment of the present application further provides a server, including: a memory and a processor; the memory is to store one or more computer instructions; the processor is to execute the one or more computer instructions to: the steps in the method provided by the embodiments of the present application are performed.

The embodiment of the present application further provides a synchronization system, which includes: a master node and at least one backup node; the master node is configured to: receiving a user request; acquiring a physical log generated in the process of executing the user request; synchronizing the physical log to at least one standby node based on a coherence protocol; according to the storage success message returned by the at least one standby node, determining a consistent storage location of the physical logs of the distributed database cluster; if the consistency storage site covers the physical log generated by the user request, returning the processing result of the user request; any standby node is configured to: receiving a physical log sent by a main node based on a consistency protocol, storing the physical log in a file storage system of the standby node, and returning a storage success message to the main node; responding to an instruction of upgrading to a new main node, if the storage location of the physical log in the file storage system is ahead of the current consistent storage location of the distributed database cluster, and playing back the physical log from the file storage system according to the current consistent storage location of the distributed database cluster.

The embodiments of the present application further provide a computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the synchronization method provided in the embodiments of the present application can be implemented.

In the synchronization method provided by the embodiment of the application, the main node acquires the physical logs and synchronizes the physical logs to the standby node based on the consistency protocol in the process of executing the user request, and when the consistency persistence site of the physical logs of the cluster covers the physical logs generated by the transaction requested by the user, the processing result of the user request can be returned. In the embodiment, the logic logs are not required to be generated and synchronized after the transaction is submitted, the physical logs can be accurately synchronized across the nodes based on the consistency protocol, meanwhile, the delay generated in the log synchronization process is reduced, and the high availability of the database and the transaction processing performance of the database are improved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a database synchronization system according to an exemplary embodiment of the present application;

FIG. 2 is a schematic diagram of a distributed database cluster from the perspective of a database kernel as provided by an exemplary embodiment of the present application;

FIG. 3 is a flowchart illustrating an implementation of a user request in a distributed database cluster according to an exemplary embodiment of the present application;

FIG. 4 is a schematic diagram of physical log cross-node synchronization provided by an exemplary embodiment of the present application;

FIG. 5 is a flowchart illustrating a process for demoting a primary node to a standby node according to an exemplary embodiment of the present application;

fig. 6 is a flowchart illustrating a process of upgrading a standby node to a primary node according to an exemplary embodiment of the present application;

fig. 7 is a schematic flowchart of a data synchronization method executed on a master node side according to an exemplary embodiment of the present application;

fig. 8 is a schematic flowchart of a data synchronization method executed on a standby node side according to an exemplary embodiment of the present application;

fig. 9 is a schematic structural diagram of a server according to an exemplary embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following specific embodiments of the present application and the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "the plural" typically includes at least two, but does not exclude the presence of at least one.

It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrases "comprising one of \8230;" does not exclude the presence of additional like elements in an article or system comprising the element.

In the computer field, a transaction refers to a unit of program execution that accesses and possibly updates various data items in a database. The transaction provides a mechanism for incorporating all operations involved in an activity into an indivisible execution unit, and all operations that make up the transaction can only commit if all operations can execute normally, and if any one of the operations fails to execute, rollback (Rollback) of the entire transaction will result.

Database transactions typically have characteristics such as Atomicity (Atomicity), consistency (Consistency), isolation (Isolation), and persistence (Durability), which are abbreviated as ACID.

The atomicity means that all operations in one transaction cannot end in a middle link. When an error occurs in the transaction during execution, the transaction is rolled back to the state before the transaction starts.

Where coherency means that the database must be in a coherency state both before and after execution of a transaction. If the transaction completes successfully, then all changes in the system will apply correctly and the system is in a valid state. If an error occurs in the transaction, all changes in the system will automatically roll back, returning the system to the original state.

Isolation refers to, in a concurrent environment, when different transactions manipulate the same data at the same time, each transaction has its own complete data space. The modifications performed by a concurrent transaction must be isolated from the modifications performed by any other concurrent transaction. When the transaction checks the data updating result, the state of the data is the state before the data is modified by another transaction, or the state of the data is the state after the data is modified by another transaction, and the transaction cannot check the data in the intermediate state.

The persistence means that the update of the database by the transaction can be permanently saved after the transaction is successfully finished. Even if the system is crashed, the database can be recovered to the state when the transaction is successfully finished after the database system is restarted.

Aiming at the technical problem that the transaction processing performance of the database is reduced by a mode of realizing data replication based on a logic log in the prior art, in some embodiments of the application, a solution is provided, and the solution is based on a physical log and a consistency protocol, so that the high availability of the database is improved. The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Fig. 1 is a schematic structural diagram of a database synchronization system according to an exemplary embodiment of the present application, and as shown in fig. 1, the database synchronization system 100 includes a database client 10 and a distributed database cluster. The database may be implemented as a distributed database such as MySQL, mongoDB, redis, or the like, which is not limited in this embodiment.

The computing nodes in the distributed database cluster are deployed according to a master-slave structure, and the master-slave structure is composed of a master node (Leader) capable of executing read operation and write operation and at least one standby node (follow) capable of executing read operation. Such as primary node 201, backup node 202, backup node 203, and backup node 204 shown in fig. 1. The number of the standby nodes in the distributed database cluster may also be 1, 2, or more than 3, and may be specifically set according to requirements, and the 3 standby nodes illustrated in fig. 1 are merely used for exemplary illustration.

Any computing node may be implemented based on a server device, and the server device may be a conventional server, a cloud server, a virtualized data center, or an elastic computing instance on the cloud, and the present embodiment is not limited. The server device mainly includes a processor, a hard disk, a memory, a system bus, and the like, and is similar to a general computer architecture, and is not described in detail.

In the database synchronization system 100, the computing nodes may communicate with each other in a wired or wireless manner. The WIreless communication mode includes short-distance communication modes such as bluetooth, zigBee, infrared, wiFi (WIreless-Fidelity), long-distance WIreless communication modes such as LORA, and WIreless communication mode based on a mobile network. When the mobile network is connected through communication, the network standard of the mobile network may be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G + (LTE +), 5G, 5.5G, wiMax, and the like.

In the embodiment of the present application, as shown in fig. 1, a file storage system is deployed in any computing node in a computing layer, and furthermore, a physical log generated by any computing node in the computing layer and a received physical log do not need to be stored in a storage layer, which is convenient for executing a synchronization operation of the physical logs between the computing nodes and a playback operation of the physical logs of the computing nodes.

Alternatively, the file storage system may be a general file storage system. In addition to providing storage and playback operations of physical logs, data storage services may also be provided to other applications. For example, in some embodiments, the file storage System of the Computing node may be a file storage System conforming to a POSIX (Portable Operating System Interface for Computing Systems) protocol. The POSIX protocol standardizes a service interface called by the operating system, so that the file storage system has universality, and the multiplexing performance of the file storage system is improved.

In this embodiment, the master node 201 is mainly configured to receive a user request sent by the client 10, and initiate a corresponding transaction to execute the user request. The user request may be a read request or a write request to a file in the distributed database. In performing the user request, the master node 201 may generate a physical log. Wherein the physical log is used for recording actual data change of the database system on each data page in the file storage system. The master node 201 may continuously generate and store the physical log during the execution of the user request. The storage method may be persistent storage, and the following embodiments will be described by taking persistent storage as an example.

In the process of executing the user request, the master node 201 may obtain a physical log generated in the process of executing the transaction corresponding to the user request, and may synchronize the physical log to at least one standby node in the cluster based on the consistency protocol. As shown in fig. 2, from the perspective of the database kernel, a protocol layer in the database kernel may perform consistency negotiation of the physical log based on a consistency protocol, and may perform state switching on a database server layer.

In the distributed database cluster, the at least one standby node may store the received physical log, and the storage may be implemented as persistent processing of the physical log so as to perform disaster recovery on the physical log of the master node. After receiving the physical log sent by the main node, any standby node can persistently store the physical log in a file storage system of the standby node and return a persistence success message to the main node. When the main node is abnormal, the standby node can be quickly switched to a new main node, the service of the database is quickly recovered based on the persisted physical logs in the local file storage system, and the high availability of the database is improved.

Wherein, the consistency protocol is used for leading each node of the distributed system to reach the same data value. In a distributed database cluster, the consistency protocol may be implemented as either Paxos (a consistency algorithm for message passing) protocol or Raft (a consensus algorithm) protocol. That is, the primary node 201 may synchronize the physical logs that are continuously generated to the at least one standby node based on the Paxos protocol or the Raft protocol. The physical logs generated uninterruptedly in the transaction processing process may be synchronized in batches or one by one, which is not limited in this embodiment.

Next, the synchronization process of the physical logs will be exemplified by using Paxos protocol as an example. Wherein, the Paxos protocol contains the following roles: client, proposal initiator (promoter), decider (Acceptor), and Learner (Learner). The client is used for sending a user request to the distributed database cluster and waiting for a response. Wherein the proposal initiator is used to propose a proposal (i.e. data value). The decision maker is used for receiving the proposal and voting the proposal. If the proposal is approved by a majority of the clusters, the proposal is approved. Wherein the majority assignment in a cluster is more than half of the total decision makers in the cluster, i.e.: if there are N decisioners in the cluster, an agreement is approved when the agreement is agreed on by N/2+1 decisioners. Wherein, the learner is used for reading the selection results of the proposal initiator and the decision maker on the proposal.

When the physical logs are synchronized based on the Paxos protocol, the master node can be used as a proposal initiator for proposing a proposal for synchronizing the physical logs generated in the transaction execution process in the process of executing the corresponding transaction requested by the user. Each standby node in the cluster acts as a decision maker for accepting the proposal, copying the physical log from the primary node, and persistently storing the copied physical log in a local file storage system. And if the persistence is finished, returning the persistence success message. If a backup node returns a persistence success message, the backup node may be considered to approve the synchronization operation of the physical log. When more than half of the standby nodes in the cluster return the persistence success message of the physical log, the synchronization operation of the physical log can be considered to be consistent in most dispatches of the cluster. Based on the consistency protocol, zero data loss can be realized in the cross-node synchronization process of the physical logs, and the disaster recovery backup capability of the database cluster is improved.

The master node may determine a consistent persistent location (i.e., a consistent storage location) of a physical log of the distributed database cluster according to the persistent storage success message returned by the at least one standby node. The persistence site of the physical log is used for describing the persistence progress of the physical log. When most (more than half) of the pie in the cluster returns the same persisted site, the master node may treat the persisted site as a cluster-consistent persisted site. Typically, the persisted location of a distributed database cluster may lag behind the persisted location of the master node. When the persistence site of the distributed data cluster covers all physical logs generated by the transaction requested by the user, the master node can consider that the persistence operation of the physical logs generated in the execution process of the user request achieves cluster majority dispatching, at the moment, the transaction can be submitted, and the processing result of the user request is returned to the client.

The following description will be made with reference to the accompanying drawings.

As shown in fig. 3, after the master node generates a physical log i, a physical log j, and a physical log k (i, j, and k are integers) in the process of executing a transaction requested by a user, the master node may initiate consistency synchronization on the generated physical logs in sequence and wait for a consistency synchronization result of the physical logs.

Wherein, the persistence sites of different standby nodes may be different under the influence of the transmission speed or the persistence processing progress. If the cluster comprises 6 computing nodes, if the persistence site of 4 standby nodes in the cluster is a physical log i, the persistence site of another standby node is a physical log j, and the persistence site of another standby node is a physical log k, it can be determined that the cluster consistency persistence site is the physical log i.

As shown in FIG. 3, the master node may wait for a consistent synchronization process of all physical logs generated by a transaction, and when the last physical log generated by the transaction (e.g., physical log k in FIG. 3) agrees among the plurality of dispatches of the cluster, the master node may commit the corresponding transaction. If the last physical log generated by a transaction does not agree among the majority of dispatches of the cluster, the master node may roll back the corresponding transaction.

In this embodiment, in a computation layer in a distributed database cluster, a file storage system is deployed in any one of computation nodes. And the master node acquires the physical logs and synchronizes the physical logs to the standby node based on a consistency protocol in the process of executing the user request, and can return the processing result of the user request after the consistency persistence site of the physical logs of the cluster covers the physical logs generated by the transaction requested by the user. In the embodiment, the logic logs are not required to be generated and synchronized after the transaction is submitted, the physical logs can be accurately synchronized across the nodes based on the consistency protocol, meanwhile, the delay generated in the log synchronization process is reduced, and the high availability of the database and the transaction processing performance of the database are improved.

In some alternative embodiments, to reduce the time consumption for synchronizing physical logs across nodes, the master node 201 may perform the operations of synchronizing physical logs across nodes in parallel while storing the physical logs persistently in the local file storage system. The following description is made by way of example with reference to the accompanying drawings.

Alternatively, the primary node 201 may employ an asynchronous manner to persist the physical logs generated by the transaction in a file storage system of the primary node 201. As shown in fig. 3, the asynchronous mode refers to that after the master node 201 continuously generates the physical log, the physical log may be flushed to a cache (e.g., a memory cache), and a persistent process is called to write the physical log into a disk for persistent storage; after the physical log is flushed to the cache, the main node can continue to execute the subsequent operation of the transaction without waiting for the persistence to return a persistence result. The persistence process may write the corresponding physical logs to the disk in the order of the call. In the above process, after the physical log is flushed to the cache, it is usually necessary to wait for the persistence process to read it and write it to the disk. The primary node may synchronize the physical log to at least one standby node based on a consistency protocol while the physical log is asynchronously awaiting persistent storage.

As shown in fig. 4, after the physical log is generated, it is flushed to the cache and waits for a log SYNC command. The SYNC command is used for synchronizing the physical log to a file storage system of the main node so as to perform persistent storage on the physical log. In the process of waiting for the log SYNC command, the primary node may initiate a synchronization request for the physical log to the standby node based on a consistency protocol, i.e., paxos synchronization illustrated in fig. 4. After receiving the synchronization request, the standby node can acquire the physical log to be synchronized, write the physical log into the cache of the standby node, and wait for the SYNC command of persisting the log to the log file. For the master node, in the waiting process, after receiving the log SYNC command, the master node can perform persistence processing on the physical log. The master node may determine that the physical log write was successful when the persistence operation of the physical log agreed in the cluster majority dispatch.

Assuming that the time length required by the standby node for synchronizing and persisting the physical log is t1, and the time length for persistently storing the physical log by the main node is t2, the time required by the persistent storage and cross-node consistency synchronization operation of the physical log in the main node is max { t1, t2}. Compared with a mode of performing local persistence and then performing cross-node synchronous operation (the required time duration is t1+ t 2), the method provided by the embodiment can shorten the cluster persistence time duration of the physical log.

In the embodiment, the cross-node synchronization operation of the log and the log can be performed in parallel at the SYNC stage of the log of the local file storage system, and the consistency synchronization operation of the physical log to the standby node after the persistence of the main node is finished is not required to be started, so that the cluster persistence time consumption of the log data is reduced, and the influence of the cluster persistence time consumption of the log data on the database system is reduced.

In some optional embodiments, the master node in the database cluster may actively downgrade to a standby node when no anomaly has occurred. In the following embodiment, the downgraded master node is described as compute node a. After the degradation, the computing node a may ensure consistency of the transaction on the distributed nodes and consistency between the response received by the client and the actual transaction state of the database based on the persisted physical log. As will be exemplified below in connection with fig. 5.

Alternatively, in response to an instruction to demote to a backup node, computing node A may determine that the persisted site did not arrive in a cluster-consistent physical log at the time of demotion. The condition that the clusters are not consistent means that the physical logs are not synchronized and persistently stored in a majority of the clusters. As shown in FIG. 5, compute node A may truncate (truncate) the portion of the physical log that is not consistent with the cluster. For example, continuing with the example of FIG. 3, if physical log k has not reached cluster agreement when the primary node is demoted to a backup node, then computing node A may truncate physical log k. Typically, the persistency progress of the physical log on the primary node prior to destaging is greater than the cluster-consistent persistency progress. After the degradation, the computing node A can delete the physical logs with the persistence progress larger than the cluster consistency based on a truncation mode, so that log synchronization with a new main node is realized.

Optionally, as shown in FIG. 5, computing node A may also determine that it is in an executed done state when downgraded and awaits a submitted user request. The computing node a may detect the result of the physical log submission of the user request to the persistence process. The submitted result is used for describing whether the computing node A submits all physical logs generated by the user request to the persistence process. If the computing node A does not submit all the physical logs generated by the user request, the computing node A can return a transaction submission error prompt (namely: a handler error, HA error) to the client and clear the data in the intermediate execution state of the transaction. Furthermore, when the master node is degraded, the consistency of the response received by the client to the user request and the actual transaction state of the database can be ensured.

Optionally, the computing node a may also determine a user request in an execution state at the time of the degradation and cut off a user link of the user request to interrupt a transaction of the user request. As shown in fig. 5, for a user request in an execution state during demotion, the computing node a cannot perform a rollback operation of a transaction corresponding to the user request, and a new master node may rollback an uncommitted transaction. Thus, compute node A may flush the cache of dirty data pages and data dictionaries associated with the transaction in the cache pool to maintain atomicity for the transaction.

In some embodiments, the cluster consistency site of the physical log is smaller than the persisted site of the primary node. Optionally, after demoting to a standby node, the computing node a may also determine a physical log checkpoint (checkpoint) after demoting. Generally, the physical log checkpoint lags behind the current consistency persistency site of the distributed database cluster, or the physical log checkpoint is the same as the current consistency persistency site of the distributed database cluster, thereby ensuring the security and accuracy of the persistency stored data in the database. The computing node A may play back the physical log from the file storage system of the computing node A based on the checkpoint for the physical log.

For example, before the computing node a is degraded, the log persistence site of the local file storage system is the physical log 6; after destaging, the checkpoint determined by the new master node is physical log 4, and then the compute node a may put back the physical logs of the local file storage system to physical log 4 to ensure the accuracy and consistency of the cluster physical logs after destaging.

Alternatively, as shown in fig. 6, for any standby node, the master node may be upgraded to the master node when the master node is abnormal, or the master node may be upgraded actively. In response to the instruction of upgrading to a new master node, if the persistence site of the physical log in the file storage system of the new master node is ahead of the current consistency site of the cluster, the physical log can be played back from the local file storage system according to the current consistency persistence site of the cluster, so that the persistence progress of the upgraded new master node is the same as the cluster consistency persistence progress before upgrading. Furthermore, the probability that the log data are lost by the standby nodes with part of persistence progress lags can be reduced, and the safety and consistency of the log data are improved.

Optionally, the new master node may further determine, from the received physical logs, that the persistency site does not reach the cluster consistency during the upgrade, and truncate the physical log in which the persistency site does not reach the cluster consistency, so that the persistency progress of the new master node after the upgrade is the same as the cluster consistency persistency progress before the upgrade.

Alternatively, since the physical log is not a transaction-level log, the standby node may rollback uncommitted transactions after upgrading to a new primary node. The rollback refers to a behavior of restoring the program or the data to a last correct state when the program or the data is abnormally processed. Based on the rollback operation of the transaction, the atomicity of the transaction can be kept when the standby node is upgraded, and the influence of the copying operation of the physical log on the performance of the transaction is reduced. Therefore, the advantages of low physical log delay and good playback performance can be fully utilized, and meanwhile the database is guaranteed to have high transaction processing performance and high availability across nodes.

In addition to the database synchronization system described in the foregoing embodiments, the embodiments of the present application also provide a synchronization method, which will be described below with reference to the accompanying drawings.

Fig. 7 is a flowchart illustrating a synchronization method applied to a master node in a distributed database cluster according to an exemplary embodiment of the present application. In the distributed database cluster, a file storage system is deployed on any computing node of a computing layer. The method, when executed on the master node side of the computing layer, may include the steps of:

step 701, receiving a user request.

Step 702, obtaining the physical log generated in the process of executing the user request.

Step 703, synchronizing the physical logs to at least one standby node based on a consistency protocol, so that the at least one standby node stores the physical logs in respective file storage systems.

Step 704, determining a consistent storage location of the physical logs of the distributed database cluster according to the storage success message returned by the at least one standby node.

Step 705, if the consistency storage location covers the physical log generated by the user request, returning the processing result of the user request.

Optionally, when the primary node and any standby node store the physical log, the physical log may be persistently stored in the file storage system of the standby node.

In some exemplary embodiments, synchronizing the physical log to at least one standby node based on a consistency protocol includes: submitting the physical log to a storage process in an asynchronous mode, and waiting for the storage process to store the log in a file storage system of the main node; in the process of asynchronously waiting for storage, the physical logs are synchronized in parallel to the at least one standby node based on a coherency protocol.

In some exemplary embodiments, the method further comprises: responding to an instruction that the computing node is degraded into a standby node, and determining a physical log of which the storage sites do not reach the cluster agreement during degradation; truncating physical logs for which the storage sites do not agree on a cluster.

In some exemplary embodiments, the method further comprises: determining a first user request which is executed and waits to be submitted when the downgrading is finished; detecting a result submitted by a physical log requested by the first user to a storage process; and if the physical log requested by the first user is not completely submitted to the storage process, returning a transaction execution error prompt requested by the first user, and deleting the data in the intermediate execution state of the transaction.

In some exemplary embodiments, the method further comprises: determining a degraded physical log checkpoint; the physical log checkpoint lags behind a current consistent storage location of the distributed database cluster; and playing back the physical log from the file storage system of the main node according to the checkpoint of the physical log.

In this embodiment, in a computation layer in a distributed database cluster, a file storage system is deployed at any computation node. And the main node acquires the physical logs and synchronizes the physical logs to the standby node based on a consistency protocol in the process of executing the user request, and can return the processing result of the user request after the consistency storage site of the physical logs of the cluster covers the physical logs generated by the transaction requested by the user. In the embodiment, the logic logs are not required to be generated and synchronized after the transaction is submitted, the physical logs can be accurately synchronized across the nodes based on the consistency protocol, meanwhile, the delay generated in the log synchronization process is reduced, and the high availability of the database and the transaction processing performance of the database are improved.

Fig. 8 is a flowchart illustrating a synchronization method applied to a standby node in a distributed database cluster according to an exemplary embodiment of the present application. In the distributed database cluster, a file storage system is deployed at any computing node of a computing layer. When the method is executed on the side of the standby node of the computation layer, the method may include the steps as shown in fig. 8:

step 801, receiving a physical log sent by a master node based on a consistency protocol; the physical log is generated by the master node in the course of executing the user request.

And step 802, storing the physical log in a file storage system of a standby node, and returning a storage success message to the main node.

Step 803, responding to an instruction for upgrading to a new master node, if the storage location of the physical log in the file storage system is ahead of the current consistent storage location of the distributed database cluster, playing back the physical log from the file storage system according to the current consistent storage location of the distributed database cluster.

In some exemplary embodiments, the method further comprises: determining physical logs of which the storage sites do not reach cluster consistency during upgrading from the received physical logs; truncating physical logs for which the storage sites do not achieve cluster consistency.

In some exemplary embodiments, the method further comprises: determining a user request in an uncommitted state during upgrading; and rolling back the transaction requested by the user and deleting the data in the intermediate execution state of the transaction.

In this embodiment, in a computation layer in a distributed database cluster, a file storage system is deployed in any one of computation nodes. The standby node can receive the physical log sent by the main node through the consistency protocol and store the received physical log. When the standby node is upgraded to a new main node, the physical log can be played back from the file storage system according to the cluster consistency storage site, so that the storage site of the new standby node is consistent with the consistency storage site of the cluster after the main node and the standby node are switched.

It should be noted that the execution subjects of the steps of the methods provided in the above embodiments may be the same device, or different devices may be used as the execution subjects of the methods. For example, the execution subjects of steps 701 to 704 may be device a; for another example, the execution main bodies of

steps

701 and 702 may be device a, and the execution main body of step 703 may be device B; and so on.

In addition, in some of the flows described in the above embodiments and the drawings, a plurality of operations are included in a specific order, but it should be clearly understood that the operations may be executed out of the order presented herein or in parallel, and the sequence numbers of the operations, such as 701, 702, etc., are merely used for distinguishing different operations, and the sequence numbers themselves do not represent any execution order. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel. It should be noted that, the descriptions of "first", "second", etc. in this document are used for distinguishing different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit the types of "first" and "second".

Fig. 9 illustrates a schematic structural diagram of a server provided in an exemplary embodiment of the present application, where the server is suitable for the database synchronization system provided in the foregoing embodiment. As shown in fig. 9, the server includes: memory 901, processor 902, and communications component 903.

A memory 901 for storing computer programs and may be configured to store other various data to support operations on the server. Examples of such data include instructions for any application or method operating on the server.

The memory 901 may be implemented by any type or combination of volatile or non-volatile memory devices, such as Static Random Access Memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

Wherein the processor 902 is coupled with the memory 901 for executing the computer program in the memory 901 for performing the corresponding synchronization method.

In some exemplary embodiments, when the server shown in fig. 9 is implemented as a master node in a distributed cluster of databases, the processor 902 is configured to: receiving a user request; acquiring a physical log generated in the process of executing the user request; synchronizing the physical logs to at least one standby node based on a consistency protocol to cause the at least one standby node to store the physical logs in respective file storage systems; according to the storage success message returned by the at least one standby node, determining a consistent storage location of the physical logs of the distributed database cluster; and if the consistency storage site covers the physical log generated by the user request, returning the processing result of the user request.

Optionally, when the primary node and any backup node store the physical log, the physical log may be persistently stored in a local file storage system.

Optionally, when synchronizing the physical log to at least one standby node based on a consistency protocol, the processor 902 is specifically configured to: submitting the physical log to a storage process in an asynchronous mode, and waiting for the storage process to store the log in a file storage system of the main node; synchronizing the physical logs in parallel to the at least one standby node based on a consistency protocol during the asynchronous waiting for storage.

Optionally, the processor 902 is further configured to: responding to an instruction that the computing node is degraded into a standby node, and determining a physical log of which the storage sites do not reach the cluster agreement during degradation; truncating physical logs for which the storage sites do not agree on a cluster.

Optionally, the processor 902 is further configured to: determining a first user request which is executed and waits to be submitted when the downgrading is finished; detecting a result submitted by a physical log requested by the first user to a storage process; and if the physical log requested by the first user is not completely submitted to the storage process, returning a transaction execution error prompt requested by the first user, and deleting the data in the intermediate execution state of the transaction.

Optionally, the processor 902 is further configured to: determining a degraded physical log checkpoint; the physical log checkpoint lags behind a current consistent storage location of the distributed database cluster; and playing back the physical log from the file storage system of the main node according to the checkpoint of the physical log.

In other exemplary embodiments, when the server shown in fig. 9 is implemented as a standby node in a distributed cluster of databases, the processor 902 is specifically configured to: receiving a physical log sent by a main node based on a consistency protocol; the physical log is generated by the main node in the process of executing the user request; storing the physical log in a file storage system of the standby node, and returning a storage success message to the main node; responding to an instruction of upgrading to a new main node, if the storage position of the physical log in the file storage system is ahead of the current consistent storage position of the distributed database cluster, playing back the physical log from the file storage system according to the current consistent storage position of the distributed database cluster.

Optionally, the processor 902 is further configured to: determining physical logs of which the storage sites do not reach cluster consistency during upgrading from the received physical logs; truncating physical logs for which the storage sites do not achieve cluster consistency.

Optionally, the processor 902 is further configured to: determining a user request in an uncommitted state during upgrading; and rolling back the transaction requested by the user and deleting the data in the intermediate execution state of the transaction.

Further, as shown in fig. 9, the server further includes: communication component 903, power component 904, and the like. Only some of the components are schematically shown in fig. 9, and the server is not meant to include only the components shown in fig. 9.

Wherein the communication component 903 is configured to facilitate wired or wireless communication between the device in which the communication component is located and other devices. The device in which the communication component is located may access a wireless network based on a communication standard, such as WiFi,2G, 3G, 4G, or 5G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC) technology, radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra Wideband (UWB) technology, bluetooth (BT) technology, and other technologies.

The power supply 904 provides power to various components of the device in which the power supply is located. The power components may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device in which the power component is located.

In this embodiment, in a computing layer in a distributed database cluster, a file storage system is deployed at any computing node, a master node acquires a physical log and synchronizes the physical log to a standby node based on a consistency protocol in the process of executing a user request, and when a consistency persistence site of the physical log of the cluster covers the physical log generated by a transaction requested by the user, a processing result of the user request can be returned. In such an embodiment, the logical logs are not required to be generated and synchronized after the transaction is submitted, and the delay generated in the log synchronization process can be reduced and the high availability of the database and the transaction processing performance of the database can be improved while the physical logs are accurately synchronized across the nodes based on the consistency protocol.

Accordingly, the present application further provides a computer readable storage medium storing a computer program, where the computer program is capable of implementing the steps that can be executed by the server in the foregoing method embodiments when executed.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises that element.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A synchronization method, comprising:

receiving a user request;

acquiring a physical log generated in the process of executing the user request;

synchronizing the physical logs to at least one standby node based on a consistency protocol to cause the at least one standby node to store the physical logs in respective file storage systems;

according to the storage success message returned by the at least one standby node, determining a consistent storage location of a physical log of the distributed database cluster;

and if the consistency storage site covers the physical log generated by the user request, returning the processing result of the user request.

2. The method of claim 1, wherein synchronizing the physical logs to at least one standby node based on a coherence protocol comprises:

submitting the physical log to a storage process in an asynchronous mode, and waiting for the storage process to store the log in a file storage system;

synchronizing the physical logs in parallel to the at least one standby node based on a consistency protocol during the asynchronous waiting for storage.

3. The method of claim 1, further comprising:

responding to an instruction of degrading into a standby node, and determining a physical log of which the storage site does not reach cluster consistency during degrading;

truncating physical logs for which the storage sites do not agree on a cluster.

4. The method of claim 3, further comprising:

determining a first user request which is executed and waits to be submitted when the downgrading is finished;

detecting a result submitted by a physical log requested by the first user to a storage process;

and if the physical log requested by the first user is not completely submitted to the storage process, returning a transaction execution error prompt requested by the first user, and deleting the data in the intermediate execution state of the transaction.

5. The method of claim 4, further comprising:

determining a degraded physical log checkpoint; the physical log checkpoint lags behind a current consistent storage location of the distributed database cluster;

and playing back the physical log from the file storage system according to the checkpoint of the physical log.

6. A method of synchronization, the method comprising:

receiving a physical log sent by a main node based on a consistency protocol; the physical log is generated by the main node in the process of executing the user request;

storing the physical log in a file storage system, and returning a storage success message to the main node;

responding to an instruction of upgrading to a new main node, if a storage location of a physical log in the file storage system is ahead of a current consistent storage location of a distributed database cluster, and playing back the physical log from the file storage system according to the current consistent storage location of the distributed database cluster.

7. The method of claim 6, further comprising:

determining physical logs of which the storage sites do not reach cluster consistency during upgrading from the received physical logs;

truncating physical logs for which the storage sites do not achieve cluster consistency.

8. The method of claim 6, further comprising:

determining a user request in an uncommitted state during upgrading;

and rolling back the transaction requested by the user and deleting the data in the intermediate execution state of the transaction.

9. A server, comprising: a memory and a processor;

the memory is to store one or more computer instructions;

the processor is to execute the one or more computer instructions to: performing the steps of the method of any one of claims 1-8.

10. A synchronization system, comprising: a master node and at least one backup node;

the master node is configured to: receiving a user request; acquiring a physical log generated in the process of executing the user request; synchronizing the physical log to at least one standby node based on a coherence protocol; according to the storage success message returned by the at least one standby node, determining a consistent storage location of a physical log of the distributed database cluster; if the consistent storage location point covers the physical log generated by the user request, returning the processing result of the user request;

any standby node is configured to: receiving a physical log sent by a main node based on a consistency protocol, storing the physical log in a file storage system of the standby node, and returning a storage success message to the main node; responding to an instruction of upgrading to a new main node, if the storage position of the physical log in the file storage system is ahead of the current consistent storage position of the distributed database cluster, playing back the physical log from the file storage system according to the current consistent storage position of the distributed database cluster.

11. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, is able to carry out the synchronization method of any one of claims 1 to 8.