CN115328696A - Data backup method in database - Google Patents
Data backup method in database Download PDFInfo
- Publication number
- CN115328696A CN115328696A CN202210901272.3A CN202210901272A CN115328696A CN 115328696 A CN115328696 A CN 115328696A CN 202210901272 A CN202210901272 A CN 202210901272A CN 115328696 A CN115328696 A CN 115328696A
- Authority
- CN
- China
- Prior art keywords
- metadata
- backup
- database
- data
- stored
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
Abstract
The specification provides a data backup method and device in a database, wherein the method comprises the following steps: responding to a backup event aiming at a database, acquiring metadata to be backed up stored in the local part of the database, and carrying out backup processing on the acquired metadata; deleting the target metadata in response to a deletion instruction of a user for the target metadata; and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
Description
Technical Field
The embodiment of the specification relates to the technical field of databases, in particular to a data backup method and device in a database.
Background
This section is intended to provide a background or context to the embodiments of the specification that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
In the field of data security, data security is generally secured by a backup recovery function to cope with situations where data needs to be recovered, such as data loss. In a database, the amount of data stored is usually very large, and the requirement on data security is also high, so that frequent backups of data in the database are required. Therefore, the backup and recovery of the database usually consume large storage resources and take a long time.
Disclosure of Invention
To overcome the problems in the related art, the present specification provides the following methods and apparatuses.
In a first aspect of embodiments of the present specification, there is provided a method for backing up data in a database, the method including:
responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the local part of the database, and carrying out backup processing on the acquired metadata;
deleting the target metadata in response to a deletion instruction of a user for the target metadata; and the number of the first and second groups,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
In a second aspect of embodiments of the present specification, there is provided an apparatus for backing up data in a database, the apparatus comprising:
a data backup unit: the system comprises a database, a backup server and a backup server, wherein the database is used for responding to a backup event aiming at the database, acquiring metadata to be backed up which is stored in the local part of the database, and backing up the acquired metadata;
a data archiving unit: the target metadata is deleted in response to a deletion instruction of a user for the target metadata; and the number of the first and second groups,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
In a third aspect of embodiments of the present specification, there is provided a storage medium; the storage medium has stored thereon a computer program which, when executed, implements the steps of the method as described below:
responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the local part of the database, and carrying out backup processing on the acquired metadata;
deleting the target metadata in response to a deletion instruction of a user for the target metadata; and the number of the first and second groups,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
In a fourth aspect of embodiments herein, there is provided a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the following method when executing the program:
responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the local part of the database, and carrying out backup processing on the acquired metadata;
deleting the target metadata in response to a deletion instruction of a user for the target metadata; and the number of the first and second groups,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished. The above embodiments of the present specification have at least the following advantages:
in the above technical solution, because a storage architecture is adopted in the database, in which only the metadata is locally stored in the database node, and the service data corresponding to the metadata is stored in the remote storage system, when performing data backup on the database, only the metadata locally stored in the database node needs to be backed up, only the corresponding service data is archived when the metadata is deleted, and the archived technical data is restored when the data is restored, instead of performing full backup on the service data stored in the remote storage system, so that the storage cost and time cost consumed when performing data backup on the database are greatly reduced.
Drawings
In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present disclosure, and other drawings can be obtained by those skilled in the art according to the drawings.
FIG. 1 is a schematic diagram illustrating an architecture of a data backup method in a database according to an embodiment of the present specification;
FIG. 2 is a flow chart that schematically illustrates a method for backing up data in a database, in accordance with an embodiment of the present specification;
FIG. 3 schematically illustrates a flow chart of another method for backing up data in a database according to an embodiment of the present specification;
FIG. 4 is a diagram schematically illustrating a metadata backup process in a data backup method in a database according to an embodiment of the present specification;
FIG. 5 is a diagram schematically illustrating a data recovery process in a data backup method in a database according to an embodiment of the present specification;
fig. 6 is a schematic diagram illustrating a data archiving process in a data backup method in a database according to an embodiment of the present specification;
FIG. 7 is a block diagram schematically illustrating a data backup apparatus in a database according to an embodiment of the present specification;
fig. 8 schematically shows a hardware structure diagram of a computer device in which a data backup method in a database according to an embodiment of the present specification is provided.
In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
Detailed Description
The principles and spirit of the present description will be described with reference to a number of exemplary embodiments. It is understood that these embodiments are given solely to enable those skilled in the art to better understand and to implement the present description, and are not intended to limit the scope of the present description in any way. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the description to those skilled in the art.
As will be appreciated by one skilled in the art, embodiments of the present description may be embodied as a system, apparatus, device, method, or computer program product. Thus, the present description may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
In the field of data security, it is often necessary to periodically backup important data to restore it to a backup-time state when needed. In the field of databases, the requirement on data security is generally high, and therefore, data in the database needs to be backed up more frequently. However, the amount of data contained in a database is generally large, and frequent backup of large amounts of data consumes large storage resources and time costs.
In view of the above, the present specification provides a method for backing up data in a database.
The embodiments of the present description will be described in detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating an architecture of a data backup system in a database according to an exemplary embodiment. As shown in fig. 1, the system may include a network 10, a server 11, a number of electronic devices, such as a cell phone 12, a cell phone 13, a cell phone 14, and so on.
The server 11 may be a physical server comprising a separate host, or the server 11 may be a virtual server, a cloud server, etc. carried by a cluster of hosts. Handsets 12-14 are just one type of electronic device that a user may use. In fact, it is obvious that the user can also use electronic devices of the type such as: tablet devices, notebook computers, personal Digital Assistants (PDAs), wearable devices (e.g., smart glasses, smart watches, etc.), etc., to which one or more embodiments of the present disclosure are not limited. The network 10 may comprise various types of wired or wireless networks.
In one embodiment, the server 11 may cooperate with handsets 12-14; wherein, can accept user's operation by the mobile phone 12-14, and upload order and file received to the server 11 through the network 10, then process the file by the server 11 on the basis of the scheme of this specification. In another embodiment, the handsets 12-14 may independently implement aspects of the present description; the mobile phones 12-14 receive user operations, and process the received commands and files based on the scheme of the specification so as to realize data backup in the database.
Referring to fig. 2, fig. 2 is a flowchart illustrating a method for backing up data in a database according to an exemplary embodiment, where the method is applied to a processing device, and the processing device may be the server 11 or the mobile phones 12 to 14 shown in fig. 1.
The method comprises the following steps:
step 202, in response to a backup event for a database, obtaining metadata to be backed up stored locally in the database, and performing backup processing on the obtained metadata.
In the embodiments illustrated in the present specification, the data stored in the database includes business data stored in a remote storage system, and metadata corresponding to the business data stored locally in the database.
The remote storage system is used for storing business data, and the database can access the business data stored in the remote storage system.
It should be noted that the service data may specifically include any type of user data stored in the database by the user based on specific service requirements.
The data type of the service data is generally related to the service type installed in the processing device in which the database is located, and is not particularly limited in this specification.
For example, in an example, taking the service as a cloud service as an example, the service data may specifically be cloud data related to the cloud service, which is stored in a database on a cloud server by a user. For example, the cloud Service may specifically be an Object Storage Service (OSS) Service provided by the cloud Service platform for the user, and the database may be an OSS database on the cloud Service platform, in which case, the cloud data may specifically include unstructured data such as a photo and a video uploaded to the OSS database by the user for Storage.
The metadata includes data for describing details of the database and data for describing business data corresponding to the metadata.
The database may obtain a backup event for the database to backup data in the database.
The backup event for the database may be a backup event triggered at regular time based on a preset backup period, or a backup event triggered by a database management program or a user, or a backup event triggered by detecting that a specific change occurs to data in the database, which is not specifically limited in this specification.
After the backup event is acquired, the database may acquire locally stored metadata to be backed up.
The metadata to be backed up refers to a portion of the metadata that needs to be backed up, and may be generally specified by the backup event, may be preset as all metadata, or may specify a portion of metadata, and the like, which is not specifically limited in this specification.
The present specification does not specifically limit the target location of the backup storage of the metadata to be backed up. For example, the backup may be stored locally in the computing node, in the remote storage system, or in a designated storage device according to actual needs.
In general, when the metadata is backed up, the time of backup is recorded.
By backing up the metadata to be backed up, the metadata can be restored to a state corresponding to the designated restoration time when the metadata needs to be restored.
Due to the adoption of the architecture of separately storing the metadata and the business data, when some metadata local to the database are deleted, the corresponding business data stored in the remote storage system of the metadata are not deleted synchronously.
Generally, after the metadata is deleted, the corresponding service data can not be used any more; therefore, when the local target metadata is deleted, the corresponding service data in the remote storage system can be deleted.
However, since the local target metadata may be restored after being deleted, and the corresponding service data needs to be reused, the service data corresponding to the target metadata cannot be directly deleted when the target metadata is deleted.
Before deleting the service data corresponding to the target metadata, archiving the service data to be deleted in the remote storage system. Referring to fig. 3, fig. 3 is a flowchart of a data backup method in a database according to an exemplary embodiment, where the method is applied to a processing device, and the processing device may be the server 11 or the mobile phones 12 to 14 shown in fig. 1. The method comprises the following steps:
In the embodiments illustrated herein, a database comprises a node cluster comprised of a plurality of database nodes. The data stored in the database includes service data stored in a remote storage system, and metadata corresponding to the service data stored in the database node.
In one illustrative embodiment shown in the present specification, the database employs a storage-computing separation architecture, and the database includes a cloud native data warehouse; the database node includes a computing node that performs computation based on the service data stored in the remote storage system. The computing nodes can perform large-scale parallel processing on the data in the database.
A cloud native data warehouse is a data warehouse implemented based on shared resources on the cloud, such as an OSS (object Storage Service) Storage system, a Simple Storage Service (S3), and the like.
The remote storage system is used for storing service data, the service data are shared by a plurality of database nodes in the node cluster, and each database node can access the service data stored in the remote storage system.
In an exemplary embodiment shown in the present specification, the remote storage system includes an OSS storage system deployed in a cloud.
The metadata includes data for describing details of the database and data for describing business data corresponding to the metadata.
The database may obtain a backup event for the database to backup data in the database.
The backup event for the database may be a backup event triggered at regular time based on a preset backup period, or a backup event triggered by a database management program or a backup instruction issued by a user, or a backup event triggered by detecting that data in the database has a specific change, which is not specifically limited in this specification.
In an exemplary embodiment shown in the present specification, the backup event is a backup event triggered based on a preset backup cycle timing.
In an exemplary embodiment shown in this specification, the backup event includes a first backup event for the metadata triggered based on a preset first backup cycle, and a second backup event for a log file corresponding to the metadata triggered based on a preset second backup cycle; generally, the second backup period is smaller than the first backup period.
In the data node, the data volume of the metadata to be backed up may be large, and the frequency of the backup is high, so as to further reduce the storage space and time consumed by the backup, the frequency of the backup of the metadata may be reduced, and the backup of the metadata is replaced by backing up the log file corresponding to the metadata with a significantly smaller data volume.
For example, an RPO (recovery point objective) index in a certain database is 1 hour, that is, the maximum backup period is 1 hour, and if the metadata of the computing node of the database is backed up 1 time every 1 hour, more resources are consumed; if the period of metadata backup is prolonged to 48 hours, the log file corresponding to the metadata is backed up every 1 hour, and because the data volume of the log file is far less than that of the corresponding metadata, the resource consumed by backup is greatly reduced.
After the first backup event is acquired, each database node may acquire locally stored metadata to be backed up, respectively.
The metadata to be backed up refers to a portion of the metadata that needs to be backed up, and may be generally specified by the backup event, may be preset as all metadata, or may specify a portion of metadata, and the like, which is not specifically limited in this specification.
The log file records modification detail information corresponding to modification of corresponding metadata after the last backup of the corresponding metadata is completed, wherein the modification can comprise various operations of adding, rewriting, deleting and the like of the metadata, and the operations can change the metadata.
After the metadata to be backed up is acquired, the metadata can be backed up. The present specification does not specifically limit the target location of the backup storage of the metadata to be backed up. For example, the backup may be stored locally in the computing node, in the remote storage system, or in a designated storage device according to actual needs.
After the second backup event is acquired, each database node may acquire a log file corresponding to locally stored metadata to be backed up.
The metadata to be backed up refers to a portion of the metadata that needs to be backed up, and may be generally specified by the backup event, preset as all metadata, or specified as partial metadata, which is not specifically limited in this specification.
The log file records modification detail information corresponding to modification of corresponding metadata after the last backup of the corresponding metadata is completed, wherein the modification can comprise adding, rewriting, deleting and other operations which can change the metadata.
After the metadata to be backed up and the log file corresponding to the metadata are obtained, the metadata can be backed up. The metadata to be backed up and the target location of the backup storage of the log file corresponding to the metadata are not specifically limited in this specification. For example, the backup may be stored locally in the computing node, in the remote storage system, or in a designated storage device according to actual needs.
By backing up the metadata to be backed up and the log files corresponding to the metadata to be backed up, the metadata can be restored to the state corresponding to the designated restoration time when the metadata needs to be restored.
In an embodiment shown in this specification, the backup of the obtained metadata is stored in the remote storage system, for example, an OSS storage system.
In an embodiment shown in this specification, the storing the obtained metadata backup in the remote storage system includes: writing the metadata into a data pipeline; and reading the metadata from the data pipeline, and uploading the metadata fragments to the remote storage system in parallel by using a thread pool.
As shown in fig. 4, fig. 4 is a schematic diagram of a metadata backup process in a data backup method in a database according to an embodiment of the present specification.
The backup of the metadata in each data node can be written into a data pipeline, and the process of transmitting the backup of the metadata to a remote storage system can be accelerated by adopting a thread pool because the data volume of the metadata can be relatively large.
The thread pool can upload the backup of the metadata acquired from the data pipeline to a remote storage system in a concurrent and fragmentation mode, and the backup uploading speed can be accelerated in a reasonable multi-thread resource allocation mode.
In order to save the storage space required by backup, the backup of metadata with too long storage time can be deleted;
for example, a preset first storage duration may be set, and after the metadata backup is completed, whether the storage duration of the existing metadata backup exceeds the first storage duration may be respectively determined, and the metadata backup whose storage duration exceeds the first storage duration may be deleted.
For another example, the saving time period of the metadata may be controlled by the number of backups of the saved metadata; for example, the maximum number of the stored metadata backups may be preset, and when the number of the stored metadata backups exceeds the maximum number, the metadata backup having the longest storage time may be deleted.
For another example, the storage time of the metadata can be controlled by the storage space occupied by the backup of the stored metadata; for example, the maximum storage space occupied by the backup of the stored metadata may be preset, and when the storage space occupied by the backup of the stored metadata exceeds the maximum storage space, the backup of the metadata having the longest storage duration may be deleted.
Generally, after the metadata is deleted, the corresponding service data can not be used any more; therefore, when the local target metadata is deleted, the corresponding service data in the remote storage system can be deleted.
However, since the local target metadata may need to be restored after being deleted, and the corresponding service data needs to be reused, the service data corresponding to the target metadata cannot be directly deleted when the target metadata is deleted.
Before deleting the service data corresponding to the target metadata, archiving processing may be performed on the service data stored in the remote storage system corresponding to the database and stored in the remote storage system corresponding to the target metadata.
The archiving process may be to directly archive the service data to be archived, or to archive the service data to be archived after performing specific processing, and this specification is not limited specifically.
In one embodiment shown in this specification, the service data to be archived may be compressed and then archived, so as to save space required for archiving the original output.
When the data in the database needs to be restored, the user may send a backup restoration instruction to restore the metadata to the state at the designated restoration time.
The recovery instruction may include a recovery time, or referred to as a recovery point; in particular, when the recovery instruction does not include the recovery time or the recovery time is a default value, the recovery time may be specified as a preset recovery time, such as the latest recovery time or the time when the metadata is backed up last time, and the like, and this specification does not specifically limit this.
The recovery time includes a consistency recovery time. When a plurality of database nodes in the node cluster process transactions, the sequence of different database nodes submitting and processing the transactions may be different, so that the states of the nodes are synchronized at the time after the transaction submission processing is completed, and the recovery time of the state synchronization among the nodes is the consistency recovery time. In this description, unless otherwise specified, the recovery time is generally the consistency recovery time.
And step 314, based on the acquired log file, replaying the acquired metadata until the metadata is restored to the data version corresponding to the restoration time.
After receiving the backup recovery instruction, each database node may obtain a backup of the metadata corresponding to the recovery time and a backup of the corresponding log file. The backup of the metadata corresponding to the restoration time may be metadata that was backed up last before or after the restoration time, and the log file may be a log file that is backed up last after the restoration time.
And replaying the metadata according to the log file until the metadata is restored to the data version corresponding to the restoration time.
In particular, when there is a backup of metadata having the same backup time as the recovery time, there is no need to replay the metadata based on a log file.
In one illustrative embodiment shown in this specification, the restore instruction instructs to restore the metadata to a new cluster;
before the playing back the acquired backup of the metadata based on the acquired backup of the log file, the method further includes:
according to the number of the database nodes included in the node cluster, creating a recovery cluster consisting of the same number of new database nodes, wherein the new database nodes in the recovery cluster correspond to the database nodes in the node cluster one by one;
and respectively transmitting the backup of the metadata and the backup of the log files in each database node in the node cluster to a corresponding new database node in the recovery cluster.
As shown in fig. 5, fig. 5 is a schematic diagram of a data recovery process in a data backup method in a database according to an embodiment of the present disclosure.
When the metadata needs to be restored to a new cluster, firstly, a restoration cluster corresponding to the node cluster can be created; the number of the database nodes in the recovery cluster is equal to the number of the database nodes in the node cluster, and the database nodes in the recovery cluster correspond to the database nodes in the node cluster one to one.
After the creation of the recovery cluster is completed, according to the recovery time, each database node in the recovery cluster can respectively obtain the metadata corresponding to the database node in the node cluster corresponding to the database node and the backup of the log file corresponding to the metadata.
And replaying the metadata based on the log file corresponding to the backup of the metadata in each database node of the recovery cluster, recovering the metadata to the data version corresponding to the recovery time, and recovering the metadata to the version at the recovery time in each database node of the recovery cluster.
And step 316, restoring the deleted service data to the state corresponding to the restoration time based on the archived service data.
And based on the archived service data, restoring the deleted service data to the state corresponding to the restoration time, so that the restored metadata can correctly access the corresponding service data.
In one exemplary implementation shown in this specification, since the service data corresponding to the metadata deleted before the recovery time is no longer needed, only the service data archived after the recovery time may be recovered.
For example, if the service data to be restored is compressed during the archiving process, the service data may be decompressed first.
As shown in fig. 6, fig. 6 is a schematic diagram of a data archiving process in a data backup method in a database according to an embodiment of the present disclosure.
The service data corresponding to a certain metadata is table t1, and table t1 initially contains 8 oss files f1, f2, … …, f8. This time is the recovery time rp1.
Thereafter, when the user performs some operation, for example, a vacuun merge operation, on table t1, the partial data in table 1 is deleted, and 8 oss files are merged into 4 new oss files f '1, f'2, f '3, f'4, which is recovery time rp2. Since part of the traffic data is deleted, it is necessary to archive the traffic data, that is, the state of table t1 before the data is deleted, that is, the state at the recovery time rp1. After each oss file in table t1 is renamed to be f1.Done.. F8.Done, the original oss file f1, f2, … …, f8 is deleted.
After that, the user performs some operation on table t1, for example, after insert operation, oss files f '5 and f'6 are inserted into table 1, but no data is deleted, so that it is not necessary to archive the previous service data, which is recovery time rp3.
After that, after the user performs some operation on table t1, for example, a vacuun merge operation, part of the data in table 1 is deleted, which is the recovery time rp4. Since part of the traffic data is deleted, it is necessary to archive the traffic data, i.e., the state of table t1 before the data is deleted, i.e., the state at the recovery time rp3. After each oss file in table t1 was renamed f '1.done.. F '6.done, the original oss file f '1, f '2, … …, f '6 was deleted.
When a backup recovery instruction is acquired and the metadata is recovered to the recovery time rp1, the service data corresponding to the recovery time rp1, that is, the archived file f1.done.
Step 318, periodically and respectively determining whether the archive of the existing service data exceeds the preset second storage duration according to the preset determination period, and deleting the archive of the service data exceeding the preset second storage duration.
Similarly, in order to save the storage space required for archiving, the archiving of the service data with too long storage time can be deleted;
for example, a preset second storage duration may be set, and at every preset determination period, it may be determined whether the storage duration of the archive of the existing service data exceeds the second storage duration, and the archive of the service data whose storage duration exceeds the second storage duration is deleted.
For another example, the duration of the business data storage may be controlled by the number of archives of the stored business data; for example, the maximum number of archives of the stored service data may be preset, and when the archives of the stored service data exceed the maximum number, the archive of the service data with the longest storage duration may be deleted.
For another example, the storage time of the service data can be controlled by the storage space occupied by the archive of the stored service data; for example, the maximum storage space occupied by archiving the stored service data may be preset, and when the storage space occupied by archiving the stored service data exceeds the maximum storage space, the archive of the service data with the longest storage duration may be deleted.
In an exemplary embodiment of the present specification, a method of backing up data in a database is also provided. Referring to fig. 7, fig. 7 is a block diagram of a data backup method in a database according to an embodiment of the present disclosure.
The database comprises a node cluster consisting of a plurality of database nodes; the data stored in the database comprises business data stored in a remote storage system and metadata which is stored locally in the database node and corresponds to the business data; the method is applied to the database node; the device comprises:
data backup unit 710: the backup method comprises the steps of responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the database locally, and carrying out backup processing on the acquired metadata;
data archive unit 720: the device comprises a deleting module, a judging module and a judging module, wherein the deleting module is used for deleting target metadata in response to a deleting instruction of a user for the target metadata; and (c) a second step of,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
Optionally, the apparatus further comprises:
the data recovery unit 730: the system comprises a backup recovery instruction, a storage unit and a processing unit, wherein the backup recovery instruction is used for responding to a backup recovery instruction, acquiring a recovery moment included in the backup recovery instruction, acquiring metadata corresponding to a backup which is the latest time away from the recovery moment, and acquiring a log file corresponding to the latest backup after the recovery moment;
and replaying the acquired metadata based on the acquired log file until the metadata is restored to the data version corresponding to the restoration time.
Optionally, the data backup unit 710 is specifically configured to obtain locally stored metadata to be backed up; and a log file corresponding to the metadata, and performing backup processing on the acquired metadata and the log file; wherein the log file records modification detail information corresponding to modifications of the metadata that occur after the last backup is completed.
Optionally, the backup event is a backup event triggered periodically based on a preset backup period; the backup event comprises a first backup event aiming at the metadata, which is triggered based on a preset first backup period; and a second backup event for the log file triggered based on a preset second backup period; the period duration corresponding to the second backup period is smaller than the period duration corresponding to the first backup period;
correspondingly, the data backup unit 710 is specifically configured to, in response to the acquired first backup event, acquire locally stored metadata to be backed up, and perform backup processing on the acquired metadata; and the number of the first and second groups,
and responding to the acquired second backup event, acquiring a log file corresponding to the metadata, and performing backup processing on the acquired log file.
Optionally, the data backup unit 710 is specifically configured to backup the metadata to the remote storage system.
Optionally, the metadata backup unit 710 is specifically configured to write the metadata into a data pipeline; and reading the metadata from the data pipeline, and uploading the metadata fragments to the remote storage system in parallel by using a thread pool.
Optionally, the apparatus further comprises:
the first cleaning unit 740 is configured to respectively determine whether the storage duration of the existing metadata backup exceeds a preset first storage duration after the metadata backup is completed, and delete the metadata backup exceeding the preset first storage duration.
The service data is selectable, and the device further comprises:
a second cleaning unit 750, configured to periodically and respectively determine whether archiving of existing service data exceeds a preset second storage duration according to a preset determination period;
and if so, deleting the archive of the service data exceeding the preset second storage time length.
Optionally, the database includes a node cluster composed of a plurality of database nodes; the metadata is stored in the database node; the restore instruction indicates to restore the metadata to a new cluster; the device further comprises:
a recovery cluster unit 760, configured to create a recovery cluster composed of new database nodes of the same number according to the number of database nodes included in the node cluster, where the new database nodes in the recovery cluster correspond to the database nodes in the node cluster one to one; and respectively transmitting the backup of the metadata and the backup of the log files in each database node in the node cluster to a corresponding new database node in the recovery cluster.
Optionally, the database adopts a storage and calculation separation architecture; the database comprises a cloud native data repository; the database nodes include compute nodes that perform computations based on the traffic data stored by the remote storage system.
Optionally, the remote storage system includes an OSS storage system deployed in a cloud.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on at least one network unit. Some or all of the elements can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement without inventive effort.
In exemplary embodiments of the present description, embodiments of an apparatus and a terminal applied thereto are also provided.
The embodiment of the device in the specification can be applied to computer equipment, such as a server or terminal equipment. The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and as a logical device, the device is formed by reading corresponding computer program instructions in the nonvolatile memory into the memory through the processor where the device is located and running the computer program instructions. From a hardware aspect, as shown in fig. 8, fig. 8 is a hardware structure diagram of a computer device 80 in which an apparatus according to an embodiment of the present disclosure is located, and besides the processor 810, the memory 830, the network interface 820, and the nonvolatile memory 840 shown in fig. 8, a server or an electronic device in which the apparatus is located in an embodiment may also include other hardware according to an actual function of the computer device, which is not described again.
In an exemplary embodiment of the present specification, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the present description may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present description described in the "exemplary methods" section above of the present description, when the program product is run on the terminal device.
A program product for implementing the above method according to an embodiment of the present specification may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present specification is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or at least one readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for this specification may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + +, or the like, as well as conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of at least one embodiment can also be implemented in combination in a single embodiment. In another aspect, various features that are described in the context of a single embodiment can also be implemented in at least one embodiment separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system elements and components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into at least one software product.
Thus, particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. Further, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.
Claims (14)
1. A method of data backup in a database, the method comprising:
responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the local part of the database, and carrying out backup processing on the acquired metadata;
deleting the target metadata in response to a deletion instruction of a user for the target metadata; and (c) a second step of,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
2. The method according to claim 1, wherein the obtaining of the metadata to be backed up stored locally in the database and the backup processing of the obtained metadata include:
obtaining metadata to be backed up stored locally in the database; and a log file corresponding to the metadata, and performing backup processing on the acquired metadata and the log file; wherein the log file records modification detail information corresponding to modifications of the metadata that occur after completion of a last backup.
3. The method as set forth in claim 2, wherein,
the backup event is a backup event triggered regularly based on a preset backup period; the backup event comprises a first backup event aiming at the metadata, which is triggered based on a preset first backup period; the second backup event aiming at the log file is triggered based on a preset second backup period; the period duration corresponding to the second backup period is smaller than the period duration corresponding to the first backup period;
the metadata to be backed up stored locally is acquired in response to the acquired backup event; and a log file corresponding to the metadata, and performing backup processing on the acquired metadata and the log file, including:
in response to the acquired first backup event, acquiring locally stored metadata to be backed up, and performing backup processing on the acquired metadata; and (c) a second step of,
and responding to the acquired second backup event, acquiring a log file corresponding to the metadata, and performing backup processing on the acquired log file.
4. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,
the backup processing of the acquired metadata includes:
and carrying out fragment processing on the metadata, and backing up the metadata fragments obtained after the fragment processing to the remote storage system in a parallel uploading manner.
5. The method of claim 2, further comprising:
after the metadata backup is completed, whether the storage time length of the existing metadata backup exceeds a preset first storage time length or not is judged, and the metadata backup exceeding the preset first storage time length is deleted.
6. The method of claim 2, further comprising:
responding to a backup recovery instruction, acquiring a recovery time included in the backup recovery instruction, and acquiring metadata corresponding to a backup which is the latest time away from the recovery time and a log file corresponding to the latest backup after the recovery time;
and replaying the acquired metadata based on the acquired log file until the metadata is restored to the data version corresponding to the restoration time.
7. The method of claim 6, further comprising:
and the obtained backup of the metadata is replayed based on the obtained backup of the log file until the deleted service data is restored to the state corresponding to the restoration time based on the archived service data after the metadata is restored to the data version corresponding to the restoration time.
8. The method of claim 1, further comprising:
according to a preset judgment period, periodically and respectively judging whether the archiving of the existing service data exceeds a preset second storage time length;
and if so, deleting the archived business data exceeding the preset second storage time length.
9. The method of claim 6, wherein the first and second light sources are selected from the group consisting of,
the database comprises a node cluster consisting of a plurality of database nodes; the metadata is stored locally at the database node; the restore instruction indicates to restore the metadata to a new cluster;
the method further comprises the following steps:
the obtained metadata is replayed until the metadata is restored to a data version corresponding to the restoration moment, and a restoration cluster composed of new database nodes with the same number is created according to the number of the database nodes included in the node cluster, wherein the new database nodes in the restoration cluster correspond to the database nodes in the node cluster one to one;
and respectively transmitting the backup of the metadata and the backup of the log files in each database node in the node cluster to a corresponding new database node in the recovery cluster.
10. The method of claim 1, wherein the first and second light sources are selected from the group consisting of,
the remote storage system comprises an OSS storage system deployed at the cloud.
11. The method as set forth in claim 1, wherein,
the database adopts a storage and calculation separation architecture; the database comprises a cloud native data repository; the database includes computing nodes that compute based on the traffic data stored by the remote storage system.
12. An apparatus for backing up data in a database, the apparatus comprising:
a data backup unit: the backup method comprises the steps of responding to a backup event aiming at a database, acquiring metadata to be backed up which is stored in the database locally, and carrying out backup processing on the acquired metadata;
a data archiving unit: the device comprises a deleting module, a judging module and a judging module, wherein the deleting module is used for deleting target metadata in response to a deleting instruction of a user for the target metadata; and the number of the first and second groups,
and archiving the business data which is stored in the remote storage system corresponding to the database and corresponds to the target metadata, and deleting the business data after the archiving processing is finished.
13. A storage medium having stored thereon a computer program which, when executed, carries out the steps of the method according to any one of claims 1-11.
14. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the following method when executing the program: wherein the processor, when executing the program, implements the method of any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210901272.3A CN115328696A (en) | 2022-07-28 | 2022-07-28 | Data backup method in database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210901272.3A CN115328696A (en) | 2022-07-28 | 2022-07-28 | Data backup method in database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115328696A true CN115328696A (en) | 2022-11-11 |
Family
ID=83920442
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210901272.3A Pending CN115328696A (en) | 2022-07-28 | 2022-07-28 | Data backup method in database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115328696A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501552A (en) * | 2023-06-21 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Data backup method, device, system and storage medium |
-
2022
- 2022-07-28 CN CN202210901272.3A patent/CN115328696A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116501552A (en) * | 2023-06-21 | 2023-07-28 | 苏州浪潮智能科技有限公司 | Data backup method, device, system and storage medium |
| CN116501552B (en) * | 2023-06-21 | 2023-09-08 | 苏州浪潮智能科技有限公司 | Data backup method, device, system and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8250033B1 (en) | Replication of a data set using differential snapshots | |
| US8904125B1 (en) | Systems and methods for creating reference-based synthetic backups | |
| US10310951B1 (en) | Storage system asynchronous data replication cycle trigger with empty cycle detection | |
| JP6521462B2 (en) | Improving backup system performance | |
| US9558072B1 (en) | Block-level incremental recovery of a storage volume | |
| US9377964B2 (en) | Systems and methods for improving snapshot performance | |
| US9424137B1 (en) | Block-level backup of selected files | |
| US20150213100A1 (en) | Data synchronization method and system | |
| US10628298B1 (en) | Resumable garbage collection | |
| US10656860B2 (en) | Tape drive library integrated memory deduplication | |
| CN112114999A (en) | Data backup method, data recovery method, data backup device, data recovery device and electronic equipment | |
| US10223038B2 (en) | File management in a file system | |
| US10168920B2 (en) | Duplexing file system data | |
| US20220091876A1 (en) | System and method of selectively restoring a computer system to an operational state | |
| US9785641B2 (en) | Reducing a backup time of a backup of data files | |
| US20150149415A1 (en) | Rollback Mechanism for Linear Tape File Systems | |
| US9606749B2 (en) | Asynchronous tape backup and restore from tape backup in a disk storage environment | |
| CN113076220A (en) | Data processing method and device, electronic equipment and computer readable medium | |
| CN115328696A (en) | Data backup method in database | |
| US11314732B2 (en) | Database migration technique | |
| US8595271B1 (en) | Systems and methods for performing file system checks | |
| US11513701B2 (en) | Storage management system and method | |
| US7865472B1 (en) | Methods and systems for restoring file systems | |
| US10388304B1 (en) | Storing data on unused areas of a storage medium | |
| US11625184B1 (en) | Recalling files from tape |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |