CN116974825A - Backup method, device, equipment and storage medium - Google Patents

Backup method, device, equipment and storage medium Download PDF

Info

Publication number
CN116974825A
CN116974825A CN202310997797.6A CN202310997797A CN116974825A CN 116974825 A CN116974825 A CN 116974825A CN 202310997797 A CN202310997797 A CN 202310997797A CN 116974825 A CN116974825 A CN 116974825A
Authority
CN
China
Prior art keywords
node
backup
nodes
log
transaction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310997797.6A
Other languages
Chinese (zh)
Inventor
刘新宇
王海龙
王蒙蒙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Dameng Database Co Ltd
Original Assignee
Shanghai Dameng Database Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Dameng Database Co Ltd filed Critical Shanghai Dameng Database Co Ltd
Priority to CN202310997797.6A priority Critical patent/CN116974825A/en
Publication of CN116974825A publication Critical patent/CN116974825A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1458—Management of the backup or restore process
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1446—Point-in-time backing up or restoration of persistent data
    • G06F11/1448—Management of the data involved in backup or backup restore
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/466—Transaction processing
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention discloses a backup method, a backup device, backup equipment and a storage medium. The method comprises the following steps: after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, suspending a transaction submitting request and acquiring a current time stamp; generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp; waking up a pending transaction commit request; backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes; after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, the invention sends the backup completion information to the SP nodes.

Description

Backup method, device, equipment and storage medium
Technical Field
The embodiment of the invention relates to the technical field of computers, in particular to a backup method, a backup device, backup equipment and a storage medium.
Background
The distributed database cluster is a database cluster formed by a plurality of application nodes, adopts a framework mode of calculation, storage and log separation, has the characteristics of high availability and transaction processing, and along with the wide application of the distributed database cluster, in order to ensure the data security, the backup operation needs to be supported for the distributed cluster, but the traditional single-machine backup mode is not applicable to the distributed cluster, and mainly has the following problems:
(1) If the data in the distributed cluster are distributed on a plurality of nodes, according to the traditional stand-alone backup mode, backup needs to be executed on each node storing the data in turn, and when the number of the nodes is large, the execution is complicated.
(2) For the online backup mode, if each node executes backup, the backup sets generated after the backup are independent, and the data consistency of each backup set cannot be ensured, for example: a certain transaction is submitted in a backup set of one node but not yet submitted in a generated backup set of another node, and the state of the same transaction in the backup sets of different nodes cannot be guaranteed to be consistent, so that the whole cluster restored by using the backup sets cannot be guaranteed to be consistent in data of all nodes.
(3) For the offline backup mode, although the offline backup can be performed after the cluster is normally closed to ensure that the data of each node is consistent, the cluster is not available during the backup period, and the high availability of the cluster is affected.
Disclosure of Invention
The embodiment of the invention provides a backup method, a device, equipment and a storage medium, which ensure that each node in a cluster can generate a backup set with consistent transactions in an online state by adding a transaction consistent log in a backup flow, and can complete the backup of all data nodes of the whole cluster by executing only one backup instruction, and the cluster can normally provide service during the backup.
According to an aspect of the present invention, there is provided a backup method including:
after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, suspending a transaction submitting request and acquiring a current time stamp;
generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp;
waking up a pending transaction commit request;
Backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes;
and after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, sending the backup completion information to the SP nodes.
According to another aspect of the present invention, there is provided a backup method including:
receiving broadcast information sent by an MP node, wherein the broadcast information carries a current time stamp;
generating a transaction consistent log corresponding to the BP node according to the current timestamp;
backing up the log file according to LSN of the transaction consistent log corresponding to the BP node;
and after the log file backup is completed, the log file backup completion information is sent to the MP node, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
According to another aspect of the present invention, there is provided a backup apparatus configured in an MP node, the backup apparatus comprising:
the current timestamp acquisition module is used for suspending a transaction submitting request and acquiring a current timestamp after receiving data file backup completion information sent by all BP nodes in the distributed database cluster;
The transaction consistent log generation module is used for generating a transaction consistent log corresponding to the MP node according to the current time stamp, and broadcasting the current time stamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current time stamp;
a pending transaction commit request wakeup module configured to wakeup a pending transaction commit request;
the first log file backup module is used for backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes;
and the first backup completion information sending module is used for sending the backup completion information to the SP nodes after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
According to another aspect of the present invention, there is provided a backup apparatus configured in a BP node, the backup apparatus comprising:
the broadcast information receiving module is used for receiving broadcast information sent by the MP node, wherein the broadcast information carries a current time stamp;
the transaction consistent log generating module is used for generating a transaction consistent log corresponding to the BP node according to the current timestamp;
The second log file backup module is used for backing up the log files according to the LSN of the transaction consistent log corresponding to the BP node;
the second backup completion information sending module is used for sending the log file backup completion information to the MP node after the log file backup is completed, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the backup method of any one of the embodiments of the present invention.
According to another aspect of the present invention, there is provided a computer readable storage medium storing computer instructions for causing a processor to execute a backup method according to any embodiment of the present invention.
After receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, suspending a transaction submitting request and acquiring a current time stamp; generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp; waking up a pending transaction commit request; backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes; after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, sending the backup completion information to the SP nodes, and ensuring that each node in the cluster can generate a backup set with consistent transaction in an online state by adding a transaction consistent log in a backup process, wherein the backup of all data nodes of the whole cluster can be completed by executing only one backup instruction, and the cluster can normally provide service during the backup.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a backup method in an embodiment of the invention;
FIG. 2 is a flow chart of another backup method in an embodiment of the invention;
FIG. 3 is a schematic diagram of a backup device according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of another backup device according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device in an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.
Example 1
Fig. 1 is a flowchart of a backup method provided in an embodiment of the present invention, where the embodiment is applicable to a backup situation, the method may be performed by a backup device in the embodiment of the present invention, and the device may be implemented in a software and/or hardware manner, as shown in fig. 1, and the method specifically includes the following steps:
S110, after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, suspending the transaction commit request and obtaining the current timestamp.
The specific way to suspend the transaction commit request is: the MP node suspends all transaction commit requests from the SP node.
It should be noted that the embodiment of the present invention is applied to a distributed database cluster, where the distributed database cluster includes: the system comprises an SP node, an MP node and at least one BP node, wherein the SP node (computing node, SQL processor) responds to a database request to generate and schedule a plan, the SP node is hereinafter called SP node for short, the SP node does not store data, and a backup set is not required to be generated during backup. BP node (plan execution node, backend Processor) stores actual data, and accepts and executes the transmitted sub-plan. Hereinafter, BP node is abbreviated. MP nodes (metadata nodes, metadata Processor) provide metadata services, hereinafter referred to as MP nodes.
The current time stamp is the time when the data file backup completion information sent by all BP nodes in the distributed database cluster is received. Namely, the timestamp of the completion data file is backed up by all BP nodes and MP nodes in the distributed database cluster.
Specifically, after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, the implementation manner of suspending the transaction commit request and obtaining the current timestamp may be: MP node receives backup request sent by SP node; and the MP node backs up the data file according to the backup request and broadcasts the backup request to BP nodes in the distributed database cluster so that the BP nodes back up the data file according to the backup request. After receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, the MP node suspends the transaction commit request and acquires the current timestamp.
It should be noted that, suspending the transaction commit request and acquiring the current timestamp are performed simultaneously.
S120, generating a transaction consistent log corresponding to the MP node according to the current time stamp, and broadcasting the current time stamp to BP nodes in the distributed database cluster, so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current time stamp.
Transaction a Transaction (Transaction) is a program execution unit that accesses and possibly updates items of data in a database. Transaction consistency-a transaction must be a change in a database from one consistency state to another, and for a distributed cluster, a transaction consistency is manifested in that for the same transaction, either all nodes within the cluster are considered committed or all are considered uncommitted, at which point the cluster may be considered to be in a transaction consistent state. If at some point the same transaction is considered committed at a node in the cluster and not committed at another node, then the nodes of the cluster are considered not in a transaction consistent state.
Specifically, the method for generating the transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to the BP nodes in the distributed database cluster, so that the method for generating the transaction consistent log corresponding to the BP node by the BP node according to the current timestamp may be: broadcasting the current time stamp to BP nodes in the distributed database cluster, so that the BP nodes generate transaction consistent logs corresponding to the BP nodes according to the current time stamp; and after log generation completion information sent by all BP nodes in the distributed database cluster is received, generating a transaction consistent log corresponding to the MP nodes.
S130, waking up the pending transaction commit request.
The manner of waking up the pending transaction commit request may be: a pre-suspended transaction commit request from the SP node is awakened.
It should be noted that, when the MP node and the BP node generate the transaction consistent log, the transaction commit request from the SP node is suspended, so that it can be ensured that when the MP node and each BP node generate the transaction consistent log in the distributed database cluster, no new transaction commit occurs, and the state of any transaction on each node is consistent, so that the data of each node in the cluster is also consistent.
And S140, backing up the log file according to the LSN of the transaction consistent log corresponding to the MP node.
Where LSN (log sequence number ) represents a log sequence number, which is incremented with log generation, hereinafter LSN.
Specifically, the backup mode of the log file according to the LSN of the transaction consistent log corresponding to the MP node may be: determining the LSN of the transaction consistent log corresponding to the MP node as the log file backup end LSN corresponding to the MP node; and backing up the log files according to the log file backup end LSN corresponding to the MP node.
And S150, after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, sending the backup completion information to the SP nodes.
Specifically, after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, the manner of sending the backup completion information to the SP node may be: when the MP node log file backup is completed, and log file backup completion information sent by all BP nodes in the distributed database cluster is received, a backup result is generated according to the backup completion conditions of all BP nodes and MP nodes in the distributed database cluster, the backup result is sent to the SP node, the SP node receives the final backup result of the MP node, and the backup result is returned to the user.
It should be noted that, if all BP nodes and MP nodes in the distributed database cluster complete log file backup, a backup result is generated according to the backup completion conditions of all BP nodes and MP nodes in the distributed database cluster, and the backup result is sent to the SP node, so that after the SP node receives the final backup result of the MP node, the backup result is returned to the user.
Optionally, after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, before suspending the transaction commit request, the method further includes:
receiving a backup request sent by an SP node;
and backing up the data file according to the backup request, and broadcasting the backup request to BP nodes in the distributed database cluster so that the BP nodes backup the data file according to the backup request.
Specifically, the SP node receives a backup request input by a user, the SP node forwards the backup request to the MP node, and after receiving the backup request, the MP node backs up the data file according to the backup request, and broadcasts the backup request to all BP nodes in the distributed database cluster, so that all BP nodes in the distributed database cluster back up the data file according to the backup request.
Optionally, generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster, so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp, including:
broadcasting the current time stamp to BP nodes in the distributed database cluster, so that the BP nodes generate transaction consistent logs corresponding to the BP nodes according to the current time stamp;
and after log generation completion information sent by all BP nodes in the distributed database cluster is received, generating a transaction consistent log corresponding to the MP nodes.
Optionally, backing up the log file according to the LSN of the transaction consistent log corresponding to the MP node, including:
determining the LSN of the transaction consistent log corresponding to the MP node as the log file backup end LSN corresponding to the MP node;
and backing up the log files according to the log file backup end LSN corresponding to the MP node.
It should be noted that, determining the LSN of the transaction consistent log corresponding to the MP node as the log file backup end LSN corresponding to the MP node, that is, obtaining the LSN of the currently backed up log file, if the LSN of the currently backed up log file is the log file backup end LSN, ending the backup of the log file.
The embodiment of the invention provides an execution method of online collaborative backup, which can complete online backup of each data node of a whole distributed cluster by executing a backup instruction on an SP node. The execution method is as follows: after receiving the backup request (online coordinated backup) from the user, the SP node forwards the backup request to the MP node. After receiving the backup request, the MP node broadcasts the backup request to all BP nodes in the distributed database cluster, so that the BP nodes execute the backup request, the MP node and the BP nodes respectively generate respective backup sets, and after the backup of all BP nodes and MP nodes in the distributed database cluster is completed, the SP responds to the end of the backup execution of the user, thereby achieving the purpose of backing up the whole distributed cluster.
In order to ensure that MP nodes and BP nodes can generate a backup set with consistent transactions, a special type of log, namely a transaction consistent log, is added in a backup process, the transaction consistent log is generated by the MP nodes coordinating the BP nodes, and a current time stamp (the time when the MP nodes receive data file backup completion information sent by all BP nodes in a distributed database cluster) is recorded in the transaction consistent log. And then broadcasting the message to other BP nodes in the cluster, and notifying each BP node to generate a transaction consistent log (comprising the current time stamp).
The flow of generating the transaction consistent log by the MP node and each BP node is as follows:
1. the MP node suspends transaction commit requests from the SP (the purpose of suspending SP transaction commit requests is to ensure that the transaction states within all nodes during the journaling are uniquely determined, and there will be no cases where one transaction is not committed when the MP node journals but has been committed when the BP node journals).
2. MP nodes cooperate with BP nodes to generate a transaction consistent log respectively, and the LSN value corresponding to the log is used as the backup end LSN of each node, so that the backup end positions of each node are ensured to be consistent in transaction, namely the data are consistent.
3. After the transaction consistent logs of the MP node and each BP node are generated, the suspended SP transaction submitting request is awakened.
As can be seen from the above steps, when the MP node and each BP node generate a transaction coincidence log, no new transaction is submitted, and the state of any transaction on each node is coincided, so that the data of each node in the cluster is coincided.
During collaborative backup, the restored cluster can reach the state of consistent transaction by means of the identification that the cluster at the position of the consistent transaction log is the state of consistent transaction. By taking the LSN corresponding to the transaction consistent log as the LSN of which the backup is finished, the backup end positions generated by all nodes of the cluster are consistent, and when the backup sets are used for restoring, each node can be restored to the LSN position of the transaction consistent log, so that all nodes of the restored cluster are in a consistent state.
According to the technical scheme, after receiving data file backup completion information sent by all BP nodes in the distributed database cluster, a transaction submitting request is suspended, and a current time stamp is acquired; generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp; waking up a pending transaction commit request; backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes; after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, sending the backup completion information to the SP nodes, and ensuring that each node in the cluster can generate a backup set with consistent transaction in an online state by adding a transaction consistent log in a backup process, wherein the backup of all data nodes of the whole cluster can be completed by executing only one backup instruction, and the cluster can normally provide service during the backup.
Example two
Fig. 2 is a flowchart of a backup method provided by an embodiment of the present invention, where the embodiment is applicable to a backup situation, the method may be performed by a backup device in the embodiment of the present invention, and the device may be implemented in a software and/or hardware manner, as shown in fig. 2, and the method specifically includes the following steps:
S210, receiving broadcast information sent by an MP node, wherein the broadcast information carries a current time stamp.
It should be noted that the embodiment of the present invention is applied to a distributed database cluster, where the distributed database cluster includes: the system comprises an SP node, an MP node and at least one BP node.
The current time stamp is the time when the data file backup completion information sent by all BP nodes in the distributed database cluster is received. Namely, the timestamp of the completion data file is backed up by all BP nodes and MP nodes in the distributed database cluster.
Specifically, the manner of receiving the broadcast information sent by the MP node may be: the SP node receives a backup request input by a user, forwards the backup request to the MP node, and after receiving the backup request, the MP node backs up the data file according to the backup request and broadcasts the backup request to all BP nodes in the distributed database cluster, and the BP node receives the backup request broadcasted by the MP node; the BP node backs up the data file according to the backup request; the BP nodes send the data file backup completion information to the MP nodes, so that the MP nodes broadcast the current time stamp to the BP nodes in the distributed database cluster after receiving the data file backup completion information sent by all the BP nodes in the distributed database cluster, and the BP nodes receive the broadcast information sent by the MP nodes.
S220, generating a transaction consistent log corresponding to the BP node according to the current timestamp.
It should be noted that, after the MP node suspends the transaction commit request and the BP node receives the broadcast information sent by the MP node, a transaction consistent log corresponding to the BP node is generated according to the current timestamp.
S230, backing up the log file according to the LSN of the transaction consistent log corresponding to the BP node.
Specifically, the backup mode of the log file according to the LSN of the transaction consistent log corresponding to the BP node may be: determining the LSN of the transaction consistent log corresponding to the BP node as the log file backup end LSN corresponding to the BP node; and backing up the log files according to the log file backup end LSN corresponding to the BP node.
And S240, after the log file backup is completed, the log file backup completion information is sent to the MP node, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
It should be noted that, after the log file backup is completed, each BP node in the distributed database cluster sends log file backup completion information to the MP node, and after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, the MP node sends the backup completion information to the SP node.
Optionally, before receiving the broadcast information sent by the MP node, the method further includes:
receiving a backup request broadcasted by an MP node;
backing up the data file according to the backup request;
and sending the data file backup completion information to the MP node so that the MP node broadcasts the current timestamp to BP nodes in the distributed database cluster after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster.
Specifically, the manner of receiving the backup request broadcasted by the MP node may be: the SP node receives the backup request input by the user, forwards the backup request to the MP node, and after receiving the backup request, the MP node backs up the data file according to the backup request and broadcasts the backup request to all BP nodes in the distributed database cluster, and the BP node receives the backup request broadcast by the MP node.
Specifically, the method for sending the data file backup completion information to the MP node, so that the MP node broadcasts the current timestamp to the BP nodes in the distributed database cluster after receiving the data file backup completion information sent by all the BP nodes in the distributed database cluster may be: after the BP node finishes backing up the data file, the back-up completion information of the data file is sent to the MP node. And after the MP node receives the data file backup completion information sent by all BP nodes in the distributed database cluster, broadcasting the current timestamp to the BP nodes in the distributed database cluster.
Optionally, backing up the log file according to the LSN of the transaction consistent log corresponding to the BP node, including:
determining the LSN of the transaction consistent log corresponding to the BP node as the log file backup end LSN corresponding to the BP node;
and backing up the log files according to the log file backup end LSN corresponding to the BP node.
Specifically, the method for determining the LSN of the transaction consistent log corresponding to the BP node as the log file backup end LSN corresponding to the BP node may be: and acquiring the LSN of the current backup log file, and ending the backup of the log file if the LSN of the current backup log file is the log file backup ending LSN.
It should be noted that, the LSN of the log file backup corresponding to each BP node is the same as the LSN of the log consistent with the transaction corresponding to the BP node, that is, the LSNs of the log file backup corresponding to different BP nodes may be the same or different.
Optionally, after generating the transaction consistent log corresponding to the BP node according to the current timestamp, the method further includes:
and sending the log generation completion information to the MP node so that the MP node generates a transaction consistent log corresponding to the MP node after receiving the log generation completion information sent by all BP nodes in the distributed database cluster.
It should be noted that, after suspending the transaction commit request and acquiring the current timestamp, the MP node broadcasts the current timestamp to BP nodes in the distributed database cluster, and after all BP nodes in the distributed database cluster generate the transaction consistent log, the MP node generates the transaction consistent log corresponding to the MP node.
In one specific example, the backup flow is as follows:
1. the SP node receives the backup request input by the user and forwards the backup request to the MP node for processing.
2. After receiving the backup request, the MP node broadcasts the backup request to all BP nodes for processing, and simultaneously, the MP node starts a backup flow and waits for the reply of completion of all BP backups.
3. And after the BP node receives the backup request broadcast by the MP node. And after the data file backup is completed, sending data file backup completion information to the MP node, and waiting for the notification of the MP node to generate a transaction consistent log.
4. After the MP node receives the data file backup completion information sent by all BP nodes, the MP node starts a round of processes for cooperatively generating a transaction consistent log.
5. The MP node suspends all transaction submitting requests from the SP node, acquires the local time, determines the acquired local time as a current time stamp, adds the current time stamp into a message for notifying the BP node to generate a transaction consistent log, and generates a transaction consistent log after the BP node receives the notification message of the MP node, wherein the log content is the current time stamp broadcasted by the MP node.
6. After the BP node generates the transaction consistent log, taking the LSN of the transaction consistent log as a log file backup end LSN corresponding to the BP node, and returning a result to the MP node, and the BP node backups the log file according to the log file backup end LSN corresponding to the BP node. For example, the REDO log file may be backed up to generate the meta file.
7. After receiving the log generation completion information sent by all BP nodes, the MP node generates a transaction consistent log with the same content, and then takes the LSN of the transaction consistent log as the log file backup end LSN corresponding to the MP node to wake up the transaction submitting request of the SP node. So far, the generation flow of the transaction consistent log of a round is ended. ( Remarks: the LSN value of MP node and the LSN value of BP node are not related, and the LSN value of MP node and the LSN value of BP node with the same business are different in big probability. )
8. The MP node continues to perform backup work on the log file (until the log file backup corresponding to the MP node ends LSN), for example: and backing up REDO log files to generate meta files. And waiting for all BP nodes to finish backup of the log files. At this time, the backup end LSNs in the backup sets of all nodes of the cluster are LSN values corresponding to the transaction consistency logs generated by all nodes, and no transaction is submitted during the generation of the transaction consistency logs, so that the transaction at the LSN positions of the backup ends of all nodes is consistent, and the data is consistent, thereby ensuring that the cluster data restored by using the backup sets is consistent.
9. And after the MP node receives the message of finishing the back-up of all BP nodes, returning a final back-up result to the SP node.
10. And the SP node receives the final backup result of the MP node and returns the backup result to the user.
According to the technical scheme, broadcast information sent by an MP node is received through the BP node, wherein the broadcast information carries a current time stamp; generating a transaction consistent log corresponding to the BP node according to the current timestamp; backing up the log file according to LSN of the transaction consistent log corresponding to the BP node; after the log file is backed up, the log file backup completion information is sent to the MP node, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, and the backup set with consistent transaction can be generated by each node in the cluster in an online state by adding a transaction consistent log in a backup process, so that the backup of all data nodes of the whole cluster can be completed by executing only one backup instruction, and the cluster can normally provide service during the backup.
Example III
Fig. 3 is a schematic structural diagram of a backup device according to an embodiment of the present invention. The embodiment may be applicable to the case of backup, and the device may be implemented in a software and/or hardware manner, and the device may be integrated in any device that provides a backup function, as shown in fig. 3, where the backup device specifically includes: the current timestamp obtaining module 310, the transaction consistent log generating module 320 corresponding to the MP node, the pending transaction commit request wakeup module 330, the first log file backup module 340 and the first backup completion information sending module 350.
The system comprises a current timestamp acquisition module, a data file backup completion module and a current timestamp acquisition module, wherein the current timestamp acquisition module is used for suspending a transaction submission request and acquiring a current timestamp after receiving data file backup completion information sent by all BP nodes in a distributed database cluster;
the transaction consistent log generation module is used for generating a transaction consistent log corresponding to the MP node according to the current time stamp, and broadcasting the current time stamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current time stamp;
a pending transaction commit request wakeup module configured to wakeup a pending transaction commit request;
The first log file backup module is used for backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes;
and the first backup completion information sending module is used for sending the backup completion information to the SP nodes after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
The product can execute the method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 4 is a schematic structural diagram of a backup device according to an embodiment of the present invention. The embodiment may be applicable to the case of backup, and the device may be implemented in a software and/or hardware manner, and the device may be integrated in any device that provides a backup function, as shown in fig. 4, where the backup device specifically includes: the broadcast information receiving module 410, the transaction consistent log generating module 420, the second log file backup module 430 and the second backup completion information transmitting module 440 corresponding to the BP node.
The system comprises a broadcast information receiving module, a data processing module and a data processing module, wherein the broadcast information receiving module is used for receiving broadcast information sent by an MP node, and the broadcast information carries a current time stamp;
the transaction consistent log generating module is used for generating a transaction consistent log corresponding to the BP node according to the current timestamp;
The second log file backup module is used for backing up the log files according to the LSN of the transaction consistent log corresponding to the BP node;
the second backup completion information sending module is used for sending the log file backup completion information to the MP node after the log file backup is completed, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
The product can execute the method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example five
Fig. 5 shows a schematic diagram of the structure of an electronic device 10 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 5, the electronic device 10 includes at least one processor 11, and a memory, such as a Read Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., communicatively connected to the at least one processor 11, in which the memory stores a computer program executable by the at least one processor, and the processor 11 may perform various appropriate actions and processes according to the computer program stored in the Read Only Memory (ROM) 12 or the computer program loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 may also be stored. The processor 11, the ROM 12 and the RAM 13 are connected to each other via a bus 14. An input/output (I/O) interface 15 is also connected to bus 14.
Various components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16 such as a keyboard, a mouse, etc.; an output unit 17 such as various types of displays, speakers, and the like; a storage unit 18 such as a magnetic disk, an optical disk, or the like; and a communication unit 19 such as a network card, modem, wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as the backup method.
In some embodiments, the backup method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the backup method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the backup method in any other suitable way (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (12)

1. A backup method applied to a distributed database cluster, the distributed database cluster comprising: the backup method is executed by the MP node and comprises the following steps:
after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, suspending a transaction submitting request and acquiring a current time stamp;
Generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp;
waking up a pending transaction commit request;
backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes;
and after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster, sending the backup completion information to the SP nodes.
2. The method of claim 1, further comprising, after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster, before suspending the transaction commit request:
receiving a backup request sent by an SP node;
and backing up the data file according to the backup request, and broadcasting the backup request to BP nodes in the distributed database cluster so that the BP nodes backup the data file according to the backup request.
3. The method of claim 1, wherein generating a transaction consistent log corresponding to the MP node according to the current timestamp, and broadcasting the current timestamp to BP nodes in the distributed database cluster, so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current timestamp, comprises:
Broadcasting the current time stamp to BP nodes in the distributed database cluster, so that the BP nodes generate transaction consistent logs corresponding to the BP nodes according to the current time stamp;
and after log generation completion information sent by all BP nodes in the distributed database cluster is received, generating a transaction consistent log corresponding to the MP nodes.
4. The method of claim 1, wherein backing up the log file according to LSNs of the transaction consistent log corresponding to the MP node, comprises:
determining the LSN of the transaction consistent log corresponding to the MP node as the log file backup end LSN corresponding to the MP node;
and backing up the log files according to the log file backup end LSN corresponding to the MP node.
5. A backup method applied to a distributed database cluster, the distributed database cluster comprising: the backup method is executed by the BP node and comprises the following steps:
receiving broadcast information sent by an MP node, wherein the broadcast information carries a current time stamp;
generating a transaction consistent log corresponding to the BP node according to the current timestamp;
backing up the log file according to LSN of the transaction consistent log corresponding to the BP node;
And after the log file backup is completed, the log file backup completion information is sent to the MP node, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
6. The method of claim 5, further comprising, prior to receiving the broadcast information transmitted by the MP node:
receiving a backup request broadcasted by an MP node;
backing up the data file according to the backup request;
and sending the data file backup completion information to the MP node so that the MP node broadcasts the current timestamp to BP nodes in the distributed database cluster after receiving the data file backup completion information sent by all BP nodes in the distributed database cluster.
7. The method of claim 5, wherein backing up the log file according to the LSN of the transaction consistent log corresponding to the BP node comprises:
determining the LSN of the transaction consistent log corresponding to the BP node as the log file backup end LSN corresponding to the BP node;
and backing up the log files according to the log file backup end LSN corresponding to the BP node.
8. The method of claim 5, further comprising, after generating a transaction consistent log corresponding to the BP node from the current timestamp:
and sending the log generation completion information to the MP node so that the MP node generates a transaction consistent log corresponding to the MP node after receiving the log generation completion information sent by all BP nodes in the distributed database cluster.
9. A backup apparatus configured in an MP node, the backup apparatus comprising:
the current timestamp acquisition module is used for suspending a transaction submitting request and acquiring a current timestamp after receiving data file backup completion information sent by all BP nodes in the distributed database cluster;
the transaction consistent log generation module is used for generating a transaction consistent log corresponding to the MP node according to the current time stamp, and broadcasting the current time stamp to BP nodes in the distributed database cluster so that the BP nodes generate the transaction consistent log corresponding to the BP nodes according to the current time stamp;
a pending transaction commit request wakeup module configured to wakeup a pending transaction commit request;
the first log file backup module is used for backing up log files according to LSNs of the transaction consistent logs corresponding to the MP nodes;
And the first backup completion information sending module is used for sending the backup completion information to the SP nodes after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
10. A backup apparatus configured in a BP node, the backup apparatus comprising:
the broadcast information receiving module is used for receiving broadcast information sent by the MP node, wherein the broadcast information carries a current time stamp;
the transaction consistent log generating module is used for generating a transaction consistent log corresponding to the BP node according to the current timestamp;
the second log file backup module is used for backing up the log files according to the LSN of the transaction consistent log corresponding to the BP node;
the second backup completion information sending module is used for sending the log file backup completion information to the MP node after the log file backup is completed, so that the MP node sends the backup completion information to the SP node after receiving the log file backup completion information sent by all BP nodes in the distributed database cluster.
11. An electronic device, the electronic device comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the backup method of any one of claims 1-8.
12. A computer readable storage medium storing computer instructions for causing a processor to perform the backup method of any one of claims 1-8.
CN202310997797.6A 2023-08-08 2023-08-08 Backup method, device, equipment and storage medium Pending CN116974825A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310997797.6A CN116974825A (en) 2023-08-08 2023-08-08 Backup method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310997797.6A CN116974825A (en) 2023-08-08 2023-08-08 Backup method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116974825A true CN116974825A (en) 2023-10-31

Family

ID=88479472

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310997797.6A Pending CN116974825A (en) 2023-08-08 2023-08-08 Backup method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116974825A (en)

Similar Documents

Publication Publication Date Title
CN115994053A (en) Parallel playback method and device of database backup machine, electronic equipment and medium
CN115617908A (en) MySQL data synchronization method, device, database terminal, medium and system
CN112948081B (en) Method, device, equipment and storage medium for processing tasks in delayed mode
CN115525666A (en) Real-time data updating method and device, electronic equipment and storage medium
CN116540938A (en) Data reading method, device, distributed storage system, equipment and storage medium
CN116974825A (en) Backup method, device, equipment and storage medium
CN114579260A (en) Transaction processing method and system
CN115934742A (en) Fault processing method, device, equipment and storage medium
CN114691781A (en) Data synchronization method, system, device, equipment and medium
CN115237426A (en) Method, device and equipment for determining database difference and storage medium
US20210240698A1 (en) Asynchronous remote calls with undo data structures
CN113656239A (en) Monitoring method and device for middleware and computer program product
CN115269128A (en) Distributed transaction processing method, device, equipment and storage medium
CN114924806B (en) Dynamic synchronization method, device, equipment and medium for configuration information
CN117112695A (en) Database synchronization method and device, electronic equipment and storage medium
CN114401275B (en) Cloud mobile phone data backup and restore method and device, electronic equipment and storage medium
CN115237968A (en) Node management method, device, equipment and storage medium in database system
CN117742899A (en) Distributed transaction processing method, device and medium in distributed database
CN117520055A (en) Data recovery method, device, equipment and medium based on data cluster
CN115858104A (en) Method, device and equipment for acquiring snapshot of transaction commit sequence number and storage medium
CN114443773A (en) Distributed system data synchronization method, device, equipment and storage medium
CN116405544A (en) Micro-service data processing method and device, electronic equipment and medium
CN117520456A (en) Data synchronization method and device, electronic equipment and storage medium
CN117579632A (en) Block consensus method, device, electronic equipment and storage medium
CN117609171A (en) Authority data synchronization method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination