CN109597717B - Data backup and recovery method and device, electronic equipment and storage medium - Google Patents

Data backup and recovery method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN109597717B
CN109597717B CN201811498098.2A CN201811498098A CN109597717B CN 109597717 B CN109597717 B CN 109597717B CN 201811498098 A CN201811498098 A CN 201811498098A CN 109597717 B CN109597717 B CN 109597717B
Authority
CN
China
Prior art keywords
data
backup
backed
target
recovered
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811498098.2A
Other languages
Chinese (zh)
Other versions
CN109597717A (en
Inventor
薛治芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Beijing Kingsoft Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd, Beijing Kingsoft Cloud Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN201811498098.2A priority Critical patent/CN109597717B/en
Publication of CN109597717A publication Critical patent/CN109597717A/en
Application granted granted Critical
Publication of CN109597717B publication Critical patent/CN109597717B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore

Abstract

The embodiment of the invention provides a data backup and recovery method, a data backup and recovery device, a server and a storage medium. The data backup method is applied to a server and comprises the following steps: receiving a data backup instruction aiming at a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk; and backing up the data to be backed up to a public cloud data center according to the backup type. The invention can reduce the equipment cost of data backup.

Description

Data backup and recovery method and device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data backup method, a data recovery method, a data backup apparatus, an electronic device, and a storage medium.
Background
Private Clouds (Private Clouds) are cloud storage spaces that are built for individual use by a user. Currently, in a private cloud scenario, when hardware of a cloud disk fails or data in the cloud disk has a logical error, irreversible data loss is caused. The reasons for the data logical errors include: the operation of deleting data by mistake, the attack of hacker or virus damage to the cloud disk, and the like occur.
In order to prevent data in the cloud disk from being lost, the current method is as follows: and backing up the data in the cloud disk to a local data center of the user. Specifically, the local data center may be implemented by deploying some storage devices, that is, data in the cloud disk may be backed up to some storage devices.
However, the storage devices in the local data center are typically purchased by the user, which increases the cost of the equipment used to store the data.
Disclosure of Invention
Embodiments of the present invention provide a data backup method, a data recovery method, a data backup device, an electronic device, and a storage medium, so as to reduce the device cost of data backup. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a data backup method, which is applied to a server, and the method includes:
receiving a data backup instruction aiming at a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk;
and backing up the data to be backed up to a public cloud data center according to the backup type.
Optionally, backing up the data to be backed up to the public cloud data center according to the backup type includes:
under the condition that the backup type is full backup, the data to be backed up is backed up to the public cloud data center in full;
and under the condition that the backup type is incremental backup, determining a data block for incremental backup in the data to be backed up, and backing up the data block for incremental backup to the public cloud data center.
Optionally, determining the data block to be subjected to incremental backup in the data to be backed up includes:
for each data block in the data to be backed up, executing the following steps:
comparing the hash value corresponding to the data block with hash values of parent backup data blocks stored in the public cloud data center and corresponding to the data block;
and determining the data block as the data block for incremental backup under the condition that the comparison results are different.
Optionally, after receiving a data backup instruction for the target cloud disk, the method further includes:
dividing the data to be backed up into a first preset number of data blocks;
calculating a hash value corresponding to each data block in the first preset number of data blocks;
and storing the hash value corresponding to each data block to the public cloud data center.
Optionally, after the data to be backed up is backed up to the public cloud data center, the method further includes:
recording backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up, wherein the backup chain is stored in the server or the public cloud data center;
wherein the backup information includes: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
Optionally, after receiving a data backup instruction for the target cloud disk, the method further includes:
storing the metadata of the data to be backed up and the metadata of each data block of the data to be backed up to the public cloud data center, wherein the metadata of the data to be backed up at least comprises: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk, and the metadata of each data block of the data to be backed up at least comprises: the name and size of the data block and the location of the data block in the data to be backed up.
In a second aspect, an embodiment of the present invention provides a data recovery method, which is applied to a server, and the method includes:
receiving a data recovery instruction aiming at a target cloud disk;
determining a recovery type for the data to be recovered in the target cloud disk;
under the condition that the recovery type is full data recovery, reading full backup content corresponding to the data to be recovered from a public cloud data center, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk;
under the condition that the recovery type is incremental data recovery, reading all backup contents corresponding to the data to be recovered from the public cloud data center, performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk;
wherein, all backup contents of the data to be restored comprise: the data recovery method comprises the steps of obtaining full backup contents corresponding to the data to be recovered and each incremental backup content corresponding to the data to be recovered.
Optionally, the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identifier of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identifier of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored;
reading all backup contents corresponding to the data to be restored from the public cloud data center comprises:
reading the full backup content from the public cloud data center based on the identification of the full backup content, and reading the incremental backup content from the public cloud data center based on the identification of each incremental backup content.
Optionally, performing data recovery on the data to be recovered based on all the backup contents, and obtaining target backup data corresponding to the data to be recovered includes:
and restoring the full backup content corresponding to the data to be restored, and sequentially restoring the incremental backup content corresponding to the data to be restored according to the sequence of incremental backups on the basis of the restored full backup content to obtain the target backup data corresponding to the data to be restored.
Optionally, restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk, including:
acquiring the position and the size of the data to be recovered in the target cloud disk;
and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
In a third aspect, an embodiment of the present invention provides a data backup apparatus, which is applied to a server, and the apparatus includes:
the first receiving module is used for receiving a data backup instruction aiming at a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk;
and the backup module is used for backing up the data to be backed up to the public cloud data center according to the backup type.
Optionally, the backup module includes:
the first backup unit is used for backing up the data to be backed up to the public cloud data center in a full amount under the condition that the backup type is the full amount backup;
and the second backup unit is used for determining the data blocks to be subjected to incremental backup in the data to be backed up and backing up the data blocks to be subjected to incremental backup to the public cloud data center under the condition that the backup type is incremental backup.
Optionally, the second backup unit is specifically configured to, for each data block in the data to be backed up, perform the following steps: comparing the hash value corresponding to the data block with hash values of parent backup data blocks stored in the public cloud data center and corresponding to the data block; and determining the data block as the data block for incremental backup under the condition that the comparison results are different.
Optionally, the apparatus further comprises:
the first storage module is used for dividing the data to be backed up into a first preset number of data blocks after the first receiving module receives a data backup instruction for a target cloud disk; calculating a hash value corresponding to each data block in the first preset number of data blocks; and storing the hash value corresponding to each data block to the public cloud data center.
Optionally, the apparatus further comprises:
the recording module is used for recording backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up after the backup module backs up the data to be backed up to a public cloud data center, and the backup chain is stored in the server or the public cloud data center;
wherein the backup information includes: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
Optionally, the apparatus further comprises:
a second storage module, configured to store, after the first receiving module receives a data backup instruction for a target cloud disk, metadata of the data to be backed up and metadata of each data block of the data to be backed up to the public cloud data center, where the metadata of the data to be backed up at least includes: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk, and the metadata of each data block of the data to be backed up at least comprises: the name and size of the data block and the location of the data block in the data to be backed up.
In a fourth aspect, an embodiment of the present invention provides a data recovery apparatus, which is applied to a server, and the apparatus includes:
the second receiving module is used for receiving a data recovery instruction aiming at the target cloud disk;
the determining module is used for determining the recovery type of the data to be recovered in the target cloud disk;
the first recovery module is used for reading full backup content corresponding to the data to be recovered from a public cloud data center under the condition that the recovery type is full data recovery, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
the second recovery module is configured to, when the recovery type is incremental data recovery, read all backup contents corresponding to the data to be recovered from the public cloud data center, perform data recovery on the data to be recovered based on all the backup contents, obtain target backup data corresponding to the data to be recovered, and recover the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
wherein, all backup contents of the data to be restored comprise: the data recovery method comprises the steps of obtaining full backup contents corresponding to the data to be recovered and each incremental backup content corresponding to the data to be recovered.
Optionally, the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identification of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identification of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored;
the second recovery module is specifically configured to read the full-volume backup content from the public cloud data center based on the identifier of the full-volume backup content, and read the incremental backup content from the public cloud data center based on the identifier of each incremental backup content.
Optionally, the second restoring module is specifically configured to restore full-volume backup content corresponding to the data to be restored, and sequentially restore incremental backup content corresponding to the data to be restored according to a sequence of incremental backups on the basis of the restored full-volume backup content, so as to obtain target backup data corresponding to the data to be restored.
Optionally, the first recovery module and the second recovery module are specifically configured to obtain a position and a size of the data to be recovered in the target cloud disk; and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
In a fifth aspect, an embodiment of the present invention provides a server, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the steps of the data backup method according to the first aspect when executing the program stored in the memory.
In a sixth aspect, an embodiment of the present invention provides a server, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
the memory is used for storing a computer program;
the processor is configured to implement the steps of the data recovery method according to the second aspect when executing the program stored in the memory.
In a seventh aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the data backup method according to the first aspect.
In an eighth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data recovery method according to the second aspect.
According to the scheme provided by the embodiment of the invention, when data in the target cloud disk needs to be backed up, the server receives a data backup instruction aiming at the target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk; and backing up the data to be backed up to a public cloud data center according to the backup type. According to the embodiment of the invention, the data to be backed up in the target cloud disk is backed up to the public cloud data center in a full backup or incremental backup mode, so that the equipment cost of data backup can be reduced.
Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a flowchart of a data backup method according to an embodiment of the present invention;
fig. 2 is a flowchart of a data recovery method according to an embodiment of the present invention;
fig. 3 is a structural diagram of a data backup device according to an embodiment of the present invention;
fig. 4 is a structural diagram of a data recovery apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a server according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of another server according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Currently, in a private cloud scenario, when hardware of a cloud disk fails or data in the cloud disk has a logical error, irreversible data loss is caused. In order to prevent data in the cloud disk from being lost, the current method is as follows: and backing up the data in the cloud disk to a local data center of the user. In particular, the local data center may be implemented by deploying some storage devices as storage clusters of a predetermined type. The predetermined type of storage cluster includes: swift (object Storage service cluster), Glusterfs (distributed File System), NFS (Network File System), Posix (local File System), TSM (Tivoli Storage Manager, comprehensive maintenance System), Ceph (distributed Storage System), and the like.
However, the storage devices of the local data center are typically purchased by the user, which increases the cost of the equipment used to store the data. Moreover, the storage devices of the local data center require user-self maintenance, such as: when the storage capacity of the local data center is insufficient, a user is required to expand the capacity, and the like, so that the cost of manual operation and maintenance is increased.
In order to solve the problem of the prior art, embodiments of the present invention provide a data backup method, a data recovery method, a data backup device, a data recovery device, a server, and a storage medium.
First, a data backup method provided in an embodiment of the present invention is described below.
It should be noted that the data backup method provided in the embodiment of the present invention may be applied to a server. The server is used for backing up data to be backed up in a target cloud disk in a local data center to a preset public cloud. Specifically, the server may receive a data backup instruction for the target cloud disk sent by a client device, and perform backup on data to be backed up in the target cloud disk in response to the data backup instruction.
As shown in fig. 1, a data backup method provided in an embodiment of the present invention may include the following steps:
s101, receiving a data backup instruction for a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk.
When data in a cloud disk needs to be backed up, the cloud disk can be used as a target cloud disk. The data backup instruction received by the server and directed to the target cloud disk may be a data backup instruction sent by a client device. The data backup instruction sent by the client device may be generated by a user performing corresponding operation in an interface of the client device, or may be a data backup instruction that is preset by the user and is automatically generated by the client device when a certain predetermined condition is reached. For example, if the user presets that data in the target cloud disk is backed up at regular intervals, the client device may periodically generate a data backup instruction and send the data backup instruction to the server.
In this embodiment, the data to be backed up refers to: and one backup data block needing to be backed up in the target cloud disk. Before backup, data in the target cloud disk may be divided into a second preset number of backup data blocks. Each backup data block is data to be backed up, so that the data in the target cloud disk can be backed up conveniently by taking the data block as a unit. That is to say, the target cloud disk may be divided into a plurality of intervals in advance, and when the target cloud disk is backed up, for each of the divided intervals, current data in the interval is used as data to be backed up.
The second preset number may be set as desired. In a specific application, the second preset number may be determined according to the type of the public cloud data center used for backing up the data to be backed up. For example, if the data to be backed up is backed up in amazon public cloud S3, the size of one backup data block is set to 50M, and then if the size of the data of the target cloud disk is 500M, the second preset number is 10, that is, the data of the target cloud disk is divided into 10 backup data blocks.
Further, after receiving a data backup instruction for the target cloud disk, the method may further include the following steps:
dividing data to be backed up into a first preset number of data blocks;
calculating a hash value corresponding to each data block in a first preset number of data blocks;
and storing the hash value corresponding to each data block to the public cloud data center.
In this embodiment, the data to be backed up may be divided into a first preset number of data blocks, and the hash value of each data block in the data to be backed up is calculated. Therefore, when the incremental backup is performed on the target cloud disk, which data blocks in the data to be backed up are determined to be backed up. The unit of the size of the data block may be a byte, and for example, the size of one data block may be 32 bytes.
The present embodiment is not limited to the method of calculating the hash value of each data block. For example, the MD5(Message-Digest Algorithm) may be used to calculate the hash value of each data block.
After the hash value corresponding to each data block is obtained through calculation, the calculated hash value can be stored in a public cloud data center, so that incremental backup can be performed on a target cloud disk subsequently.
And S102, backing up the data to be backed up to the public cloud data center according to the backup type.
In this embodiment, the backup type of the data to be backed up in the target cloud disk may include: full backup and incremental backup. Specifically, the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk. The specific form of the backup type may be a backup type parameter. According to the backup type parameter, the backup type of the data to be backed up in the target cloud disk can be determined. In practical applications, the backup type parameter and the value form thereof may be preset and may be various. Illustratively, the backup type parameter is represented as: the method comprises the following steps that increment indicates that the backup type is incremental backup when the value of the increment is True, namely when the increment is True; when increment takes a value of False, that is, when increment is False, the backup type is represented as full backup.
In practical application, if the target cloud disk is backed up for the first time, all data in the target cloud disk needs to be backed up, that is, the target cloud disk is subjected to full backup, and then the backup type in the data backup instruction is full backup.
If the target cloud disk is not backed up for the first time, in one mode, changed data in the target cloud disk can be backed up, namely incremental backup is performed on the target cloud disk, and then the backup type in the data backup instruction is incremental backup; in another mode, a full backup may also be performed on the target cloud disk, and then the backup type in the data backup instruction is a full backup.
In this embodiment, backing up the data to be backed up to the public cloud data center according to the backup type may include the following steps:
step 1021, under the condition that the backup type is full backup, fully backing up the data to be backed up to the public cloud data center;
and step 1022, determining a data block to be subjected to incremental backup in the data to be backed up under the condition that the backup type is incremental backup, and backing up the data block subjected to incremental backup to the public cloud data center.
Step 1021 is described in detail below:
a public cloud is a cloud that third party providers offer to users to be able to use. Backing up data to a public cloud may have the following advantages:
1. the durability of public cloud storage data is higher than that of a local data center, the durability of 99.99999999% can be usually realized, and the integrity of the data can be still ensured even if a large disaster occurs;
2. when the data is backed up to the public cloud, once a disaster occurs in the local data center, the backup data stored in the public cloud can be used for completing the recovery of the data.
In this embodiment, the public cloud data center may be selected by the user. Illustratively, the public cloud data center may be amazon public cloud S3.
If the target cloud disk is backed up for the first time, the data to be backed up can be backed up to the public cloud data center. If the target cloud disk is not backed up for the first time, the target cloud disk can be fully backed up, that is, the data to be backed up can be backed up to the public cloud data center. Meanwhile, in order to facilitate subsequent incremental backup of the target cloud disk, the hash value of each data block in the data to be backed up can be stored in the public cloud data center.
It is understood that the structure of different public cloud data centers may be different. Then, for different public cloud data centers, the manner of backing up the data to be backed up to the public cloud data center is different. Specifically, when the data to be backed up is backed up to the public cloud data center, the selected cloud computing service, the storage component and the drive to the public cloud data center are different. In the following, taking the public cloud data center as amazon public cloud S3 as an example, the process of backing up the data to be backed up to S3 is briefly described:
in this example, the open source community OpenStack is selected to provide cloud computing services, and the block storage component Cinder in the open source community OpenStack is utilized as a storage component. In the sender component, a Backup service may provide a Backup service for a storage volume, and support Backup of the storage volume to a backend device supported by an open source community OpenStack, such as a Swift cluster, a Ceph cluster, and the like. Currently, S3 is not supported in the backend device supported by the open source community OpenStack. Therefore, in order to backup the data to be backed up to S3, it is necessary to first implement the driving of S3. In the present example, the driving of S3 may be realized by the following configuration information:
connf file configuration the following:
backup_driver=cinder.backup.drivers.s3
backup_s3_endpoint_url=http://localhost:82
backup_s3_access_key_id=**
backup_s3_secret_access_key=***
backup_s3_KMS=****
backup_s3_bucket=volumebackups
backup_s3_object_size=52428800
wherein "ring.conf" represents the name of the configuration file; "backup _ driver _ drivers.s 3" indicates that the drive realized with the backup service is the drive to S3; "backup _ s3_ endpoint _ url ═ http: // localhost: 82 "indicates that the URL (Uniform Resource Locator) corresponding to S3 is http: // localhost: 82; "backup _ S3_ access _ key _ id ═ indicates that the user name accessed S3 is ×; "backup _ S3_ secret _ access _ key" indicates that the password for accessing S3 is x; "backup _ S3_ KMS ═ indicates whether to encrypt the data to be backed up to S3, if" True, indicates an encrypted backup, otherwise, indicates an unencrypted backup; "backup _ S3_ bucket" indicates that data to be backed up is backed up into a bucket named as volume backup in S3; indicating that the size of the data to be backed up is 52428800 bytes, i.e., the size of the data to be backed up is 50M.
When the target cloud disk is backed up for the first time, the driving of S3 may be implemented through the above configuration information. And executing the command after the configuration is completed: and the service openstack-client-backup start starts the backup service, and backs up the data block to be backed up to S3. Specifically, the data to be backed up is backed up into the selected S3 bucket. Thus, the full backup of the target cloud disk is realized. Moreover, the hash value of each data chunk may also be stored in the S3 bucket to facilitate subsequent incremental backups of the target cloud disk.
Step 1022 is described in detail below:
if the target cloud disk is not backed up for the first time, incremental backup can be performed on the target cloud disk in order to reduce backup time and occupy the storage space of the public cloud data center as little as possible. That is, for the data to be backed up, only the data blocks which are changed with respect to the last backup are backed up to the public cloud data center.
In one implementation, determining a data block to be incrementally backed up in the data to be backed up may include:
for each data block in the data to be backed up, executing the following steps:
comparing the hash value corresponding to the data block with the hash value of the parent backup data block corresponding to the data block stored in the public cloud data center;
and determining the data block as the data block for incremental backup under the condition that the comparison results are different.
Specifically, for each data block in the data to be backed up, the hash value of the data block may be compared with the hash value of the parent backup data block corresponding to the data block in the public cloud data center. If the comparison result is different, which indicates that the data block is changed relative to the parent backup data block, the data block may be backed up to the public cloud data center. On the contrary, if the comparison results of the data blocks are the same, which indicates that the data block is unchanged from the parent backup data block, the data block does not need to be backed up to the public cloud data center.
Wherein, the parent backup data block of any data block means: in the public cloud data center, the backup content which is backed up last time in all the backup contents corresponding to the data block. For example: a data block is a, and all backup contents corresponding to the data block are a0, a1 and a2, wherein a2 is the backup content of the latest backup, and then the parent backup data block of the data block a is a 2.
It is to be appreciated that for any data chunk, whether or not the data chunk needs to be backed up to the public cloud data center, the hash value of the data chunk can be stored to the public cloud data center. Therefore, when the next incremental backup is performed, the hash value of each data block stored in the public cloud data center during the current incremental backup is conveniently used for determining which data blocks in the data to be backed up are backed up during the next incremental backup.
According to the scheme provided by the embodiment of the invention, when data in the target cloud disk needs to be backed up, the server receives a data backup instruction aiming at the target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk; and backing up the data to be backed up to a public cloud data center according to the backup type. According to the embodiment of the invention, the data to be backed up in the target cloud disk is backed up to the public cloud data center in a full backup or incremental backup mode, so that the equipment cost of data backup can be reduced. Meanwhile, the labor operation and maintenance cost of a user needing to backup data can be reduced. According to the scheme provided by the embodiment of the invention, the data to be backed up in the target cloud disk is backed up to the public cloud data center. Therefore, when the target cloud disk fails or the data in the target cloud disk has a logic error, the data to be recovered in the target cloud disk can be recovered from the public cloud data center.
Optionally, on the basis of the embodiment shown in fig. 1, after the data to be backed up is backed up to the public cloud data center, the method may further include: and recording the backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up, wherein the backup chain is stored in a server or a public cloud data center.
Wherein, the backup information comprises: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
Because the data to be backed up refers to current data in a certain interval in the target cloud disk, the backup chain corresponding to the data to be backed up may be a backup chain for the interval. Here, the backup chain corresponding to the data to be backed up may be regarded as a straight line, and for the interval where the data to be backed up is located, the time for performing full backup on the data in the interval and the identifier of the corresponding full backup content, and the time for performing incremental backup on the data in the interval each time and the identifier of the corresponding incremental backup content are both recorded in the straight line. And the backup information of the data to be backed up in each backup process is connected in the straight line according to the sequence of the backup time. In this way, in the continuous backup process, a backup chain corresponding to the data to be backed up is constructed. The backup chain constructed for the data to be backed up can be used for performing data recovery on the data to be backed up.
Moreover, the constructed backup chain can be stored in a server or a public cloud data center, which is reasonable.
Optionally, on the basis of the embodiment shown in fig. 1, after receiving a data backup instruction for a target cloud disk, the method further includes:
storing the metadata of the data to be backed up and the metadata of each data block of the data to be backed up to a public cloud data center, wherein the metadata of the data to be backed up at least comprises: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk are determined, and the metadata of each data block of the data to be backed up at least comprises the following components: the name, size, and location of the data block in the data to be backed up.
The position of the data to be backed up in the target cloud disk may be: offset of data to be backed up. The offset is: and the offset of the data to be backed up relative to the preset position in the target cloud disk.
In this embodiment, the metadata of the data to be backed up may be stored in the public cloud data center. Therefore, when the data to be backed up is restored in full, the full backup content corresponding to the data to be backed up can be read from the public cloud data center according to the name of the data to be backed up, and then the data to be backed up is restored to the corresponding position in the target cloud disk according to the position and the size of the data to be backed up in the target cloud disk.
In specific application, when the data to be backed up is backed up to the public cloud data center, the data to be backed up can be compressed by adopting a preset compression algorithm and then backed up to the public cloud data center, so that the transmission quantity of the data can be reduced. Accordingly, the metadata of the data to be backed up may further include: name of the preset compression algorithm. Therefore, when the data to be backed up is restored, the corresponding decompression algorithm is adopted to decompress the backup content in the compressed form corresponding to the data to be backed up in the public cloud data center according to the preset compression algorithm, and then the backup content is restored to the corresponding position in the target cloud disk.
The position of any data block of the data to be backed up may be: and the offset of the data block to be detected. The offset is: the offset of the data block relative to the predetermined position in the data to be backed up.
In this embodiment, the metadata of each data block of the data to be restored may also be stored in the public cloud data center. Therefore, when any data block is subjected to incremental restoration, the data block can be read from the public cloud data center according to the name of the data block, namely, the incremental backup content corresponding to the data block is read, and then the data block is restored to the corresponding position in the data to be backed up according to the position and the size of the data block in the data to be backed up. Thus, the recovery of the data to be backed up is realized. Further, according to the position and size of the data to be backed up in the target cloud disk, the data to be backed up can be restored to the corresponding position in the target cloud disk.
In specific application, when any data block is backed up to the public cloud data center, the data block can be compressed by adopting a preset compression algorithm and then backed up to the public cloud data center, so that the transmission quantity of data can be reduced. Accordingly, the metadata of the data block may further include: name of the preset compression algorithm. Therefore, when the data block is restored, the backup content in the compressed form corresponding to the data block in the public cloud data center is decompressed by adopting the corresponding decompression algorithm according to the preset compression algorithm, and then the data block is restored to the corresponding position in the data to be backed up.
Certainly, there is no strict sequence between the step of storing the metadata of the data to be backed up and the metadata of each data block of the data to be backed up in the public cloud data center and the step of backing up the data to be backed up in the public cloud data center.
Optionally, on the basis of the embodiment shown in fig. 1, the backing up the data blocks subjected to incremental backup in step S1022 to the public cloud data center may include the following steps:
and backing up at least two continuous data blocks for incremental backup to the public cloud data center in a merging and sending mode.
If at least two continuous data blocks for incremental backup exist in the data to be backed up, the at least two continuous data blocks for incremental backup can be merged, and the merged data blocks are backed up to the public cloud data center. Therefore, the times of network connection can be reduced, and the efficiency of data backup is improved.
The following describes a data backup method according to an embodiment of the present invention with a specific embodiment, which includes the following steps:
step 21, the client device creates a backup request for the target cloud disk according to the operation of the user;
step 22, the client device checks and judges the parameters of the backup request; wherein the requested parameters include: the identification of the target cloud disk and the number of times of backup requests; the content of the examination includes: checking whether the number of backup requests is greater than 1; if the backup request is not greater than 1, determining that the backup request is a full backup request for the target cloud disk, wherein the judgment content comprises: judging whether the target cloud disk is in backup, if not, turning to the step 23, otherwise, ending the process; if the backup request is larger than 1, determining that the backup request is an incremental backup request for the target cloud disk, wherein the judgment content comprises the following steps: judging whether the target cloud disk is in backup or not, judging whether the current size of the target cloud disk is the same as that of the target cloud disk in full backup or not, if the target cloud disk is not in backup and the size of the target cloud disk is the same, turning to the step 23, and if the target cloud disk is not in backup and the size of the target cloud disk is the same, ending the process;
step 23, the client device stores the backup basic information into a local database; the backup basic information includes: the backup state of whether the target cloud disk is backing up or not, the last backup time of the target cloud disk and the identification of the target cloud disk;
step 24, the client device sends a data backup instruction for the target cloud disk to the server; the form of the data backup instruction is represented as: < < signalrecipt RPC > >;
step 25, if the full backup is performed on the target cloud disk, the client device sends a configuration file for driving S3 to the server, and a shader component in the server configures S3 through the configuration file and accesses S3; if incremental backup is performed on the target cloud disk, the client device determines that the public cloud data center (namely the back end) is S3, and accesses S3 through the server;
step 26, the server circulates the whole target cloud disk, and generates data to be backed up after reading the data with the size of chunk _ size; taking each sha _ block _ size byte of each data to be backed up as a data block, calculating the hash value of each data block, and storing the hash value as a sha256 file; if the full backup is carried out on the target cloud disk, the step 27 is carried out; if the incremental backup is performed on the target cloud disk, turning to step 28;
step 27, the server reads each data to be backed up in the target cloud disk, compresses the data to be backed up, stores the compressed data in an S3 bucket, and goes to step 29;
step 28, the server compares the sha256 file value (i.e. hash value) of each data block with the sha256 file value of the parent backup data block of the data block in S3; if not, backing up the data block to an S3 bucket, and if the data block is the same, storing the sha256 file value of the data block to an S3 bucket;
step 29, the server saves the metadata file and the sha256 file containing the metadata of each data to be backed up to an S3 bucket; wherein, the metadata file comprises: the length and offset of each data to be backed up, the name of the corresponding backup file and the name of a compression algorithm for compressing each data to be backed up.
A data recovery method provided in an embodiment of the present invention is described below.
It should be noted that the data recovery method provided in the embodiment of the present invention may be applied to a server. The server is used for recovering data to be recovered in a target cloud disk in a local data center from a public cloud data center. Specifically, the server may receive a data recovery instruction for the target cloud disk sent by a client device, and recover the data to be recovered in the target cloud disk in response to the instruction.
As shown in fig. 2, a data recovery method provided in an embodiment of the present invention may include the following steps:
s201, receiving a data recovery instruction aiming at a target cloud disk.
When data in a cloud disk needs to be recovered, the cloud disk can be used as a target cloud disk. The data recovery instruction received by the server and directed to the target cloud disk may be a data recovery instruction issued by a client device. The data recovery instruction sent by the client device may be generated by a user performing a corresponding operation in an interface of the client device. The data recovery instruction may also be automatically generated when the client device detects that the target cloud disk fails or detects that data in the target cloud disk is abnormal, such as a logic error.
S202, determining the recovery type of the data to be recovered in the target cloud disk.
After a data recovery instruction for a target cloud disk is received, a recovery type for data to be recovered in the target cloud disk can be determined, so that different recovery operations can be executed according to different recovery types. In the embodiment of the present invention, the data to be recovered refers to: and one recovery data block needing to be recovered in the target cloud disk. As described in step S101, the target cloud disk may be divided into a plurality of sections in advance, and when backing up the target cloud disk, current data in each of the divided sections may be used as data to be backed up. Then correspondingly, the data to be recovered is the data in a certain divided interval, which is up to a certain time. For example, the data to be recovered is in interval a in the target cloud disk, by 21: 00.
In this embodiment, the recovery type may include: both types of full data recovery and incremental data recovery.
Specifically, the server may determine the above recovery type from the backup record for the target cloud disk. If the server only performs full backup on the target cloud disk before the data recovery, the recovery type of the data to be recovered for the target cloud disk at this time is as follows: and recovering the full data. If the server performs incremental backup on the target cloud disk before the data recovery, the recovery type of the data to be recovered for the target cloud disk at this time is as follows: incremental data recovery.
And S203, under the condition that the recovery type is full data recovery, reading full backup content corresponding to the data to be recovered from the public cloud data center, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk.
When the server performs full backup on the target cloud disk, the full backup content corresponding to the data to be restored in the target cloud disk is backed up in the public cloud data center. Therefore, when the target cloud disk is restored in a full amount, the full-amount backup content corresponding to the data to be restored can be directly read from the public cloud data center, and the full-amount backup content is restored to the corresponding position of the cloud disk. This means that: not only the full backup content corresponding to the data to be restored needs to be restored, but also the full backup content corresponding to the data to be restored needs to be restored to: and the position of the data to be recovered in the target cloud disk.
Specifically, when reading the full backup content corresponding to the data to be restored, the server may obtain metadata of the data to be restored in advance, where the metadata may include: the name, size, and offset of the data to be recovered. And searching the full backup content corresponding to the data to be restored in the public cloud data center according to the name of the data to be restored in the metadata, and then reading the full backup content. Wherein the metadata of the data to be recovered is stored in the server.
After the full backup content is used as the target backup data corresponding to the data to be restored, the target backup data can be restored to the corresponding position of the target cloud disk.
In one implementation manner, restoring target backup data corresponding to data to be restored to a corresponding position of a target cloud disk may include:
acquiring the position and the size of data to be recovered in a target cloud disk;
and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
The position and the size of the data to be recovered in the target cloud disk can be obtained by obtaining metadata of the data to be recovered. Then, the position of the data to be restored in the target cloud disk may be used as the position of the target backup data corresponding to the data to be restored in the target cloud disk, and by combining the size of the data to be restored, the position interval of the target backup data corresponding to the data to be restored in the target cloud disk may be determined, and the target backup data corresponding to the data to be restored may be restored to the corresponding position of the target cloud disk.
And S204, under the condition that the recovery type is incremental data recovery, reading all backup contents corresponding to the data to be recovered from the public cloud data center, performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk.
Wherein, all backup contents of the data to be restored comprise: the method comprises the steps of obtaining full backup contents corresponding to data to be restored and incremental backup contents corresponding to the data to be restored.
It can be understood that when the data in the interval where the data to be restored in the target cloud disk is located is fully backed up, each data block of the data is backed up in the public cloud data center. When the data of the interval where the data to be restored is located is subjected to incremental backup, only the data blocks in the data, which are subjected to incremental backup, are backed up in the public cloud data center, wherein the hash value of any data block subjected to incremental backup is different from the hash value of the corresponding parent backup data block of the data block in the public cloud data center. If the data to be restored, which is subjected to full backup and incremental backup before the data of the corresponding interval, needs to be restored according to the sequence of backup, before restoration, all backup contents corresponding to the data to be restored need to be read.
Because the data to be restored is data in a certain interval in the target cloud disk, the full backup content corresponding to the data to be restored refers to: and carrying out corresponding full backup content when the data in the interval is backed up for the first time. Each incremental backup content corresponding to the data to be restored refers to: and performing incremental backup on the data in the interval according to the corresponding incremental backup content.
In this embodiment, the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identification of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identification of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored.
Accordingly, reading all backup contents corresponding to the data to be restored from the public cloud data center may include:
reading the full backup content from the public cloud data center based on the identification of the full backup content, and reading the incremental backup content from the public cloud data center based on the identification of each incremental backup content.
Specifically, the server may query, in the local or public cloud data center, a backup chain corresponding to the data to be restored according to the name of the data to be restored in the metadata of the data to be restored. In the backup chain, the server may query the identifier of the full backup content corresponding to the data to be restored and the identifier of each corresponding incremental backup content. And further, based on the inquired identification of the full backup content and the identification of each incremental backup content, the server reads the full backup content and each incremental backup content corresponding to the data to be restored from the public cloud data center. Any incremental backup content identification can include: and the name of each data block corresponding to the incremental backup content. Because the incremental backup content is only one data block which changes in the data to be backed up, each data block corresponding to the incremental backup content refers to: and each changed data block in the data to be backed up. Therefore, each data block can be read from the public cloud data center according to the name of each data block, namely the incremental backup content corresponding to the data to be restored is read.
In a specific embodiment, when reading all backup contents, the backup contents may be read according to the sequence of the last backup content, the parent backup content, the total backup content, that is, the backup contents are read according to the sequence from the back to the first backup time, and the backup contents are stored in a list according to the sequence of reading. Therefore, when the data is restored, the data restoration can be performed step by step from the tail part of the list, namely, the data restoration is performed from the full backup content which is backed up firstly.
After reading all the backup contents of the data block, the data block can be restored based on all the backup contents.
In an implementation manner, performing data recovery on data to be recovered based on all backup contents to obtain target backup data corresponding to the data to be recovered may include:
and recovering the full backup content corresponding to the data to be recovered, and sequentially recovering the incremental backup content corresponding to the data to be recovered according to the sequence of incremental backup on the basis of the recovered full backup content to obtain the target backup data corresponding to the data to be recovered.
In the implementation manner, when performing data recovery on data to be recovered, the data recovery may be performed according to a sequence from the first to the last of the time to be backed up of the backup content, that is, the full-amount backup content backed up first is restored first, and the incremental backup content backed up last is restored last.
For example, the data to be restored includes 3 data blocks b1, b2 and b3, and the data in the section where the data to be restored is located has been subjected to one full backup and two incremental backups. Wherein, the corresponding full backup content during the full backup is as follows: 1, 2, 3, where 1 represents the backup data value for data chunk b1, 2 represents the backup data value for data chunk b2, and 3 represents the backup data value for data chunk b 3; the corresponding backup contents during the first incremental backup are as follows: # 4, # 1 indicating null, i.e., no backup of data block b1, 4 indicating a backup data value of data block b2, and # 2 indicating null, i.e., no backup of data block b 3; the corresponding backup contents during the second incremental backup are as follows: #, 5, the 1 st # indicates null, i.e., data block b1 was not backed up, the 2 nd # indicates null, i.e., data block b2 was not backed up, and 5 indicates the backup data value of data block b 3.
When the data to be restored is restored, restoring the full amount of backup content corresponding to the data to be restored first, and then the current restored content is: 1, 2, 3; then, restoring the corresponding backup content during the first incremental backup, and then the current restored content is: 1, 4, 3, namely updating the backup data value of the 2 nd data block; and finally, restoring the corresponding backup content during the second incremental backup, wherein the current restored content is as follows: 1, 4, 5, namely updating the backup data value of the 3 rd data block. The restored content after all the backup content is restored is: 1,4,5. Then, the backup data value of the target backup data corresponding to the data to be restored is: 1, 4, 5, wherein 1, 4, 5 are backup data values of three data chunks b1, b2, and b3, respectively, which are target backup data.
It should be noted that, in practical applications, when any incremental backup content is read, only the corresponding data blocks are read, and the data blocks are not restored to the corresponding positions of the data to be restored. For example, in the above example, when the first incremental backup content is read, only the backup data value 4 of data block b2 is read, and data block b2 is not restored to the corresponding position of the data to be restored. Therefore, before any incremental backup content is restored, each data block can be restored to a corresponding position in the data to be restored according to the position and size of each data block corresponding to the incremental backup content in the data to be backed up corresponding to the incremental backup content.
After the target backup data corresponding to the data to be restored is obtained, the target backup data can be restored to the corresponding position of the target cloud disk.
In one implementation manner, restoring target backup data corresponding to data to be restored to a corresponding position of a target cloud disk includes:
acquiring the position and the size of data to be recovered in a target cloud disk;
and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
The position and the size of the data to be recovered in the target cloud disk can be obtained by obtaining metadata of the data to be recovered. Then, the position of the data to be restored in the target cloud disk may be used as the position of the target backup data corresponding to the data to be restored in the target cloud disk, and by combining the size of the data to be restored, the position interval of the target backup data corresponding to the data to be restored in the target cloud disk may be determined, and the target backup data corresponding to the data to be restored may be restored to the corresponding position of the target cloud disk.
And for the data to be restored, restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk, namely realizing the data restoration of the data to be restored.
According to the scheme provided by the embodiment of the invention, when the data in the target cloud disk needs to be recovered, the server receives a data recovery instruction aiming at the target cloud disk; determining a recovery type for the data to be recovered in the target cloud disk; under the condition that the recovery type is full data recovery, reading full backup content corresponding to the data to be recovered from a public cloud data center, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk; and under the condition that the recovery type is incremental data recovery, reading all backup contents corresponding to the data to be recovered from the public cloud data center, performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk. According to the embodiment of the invention, the target backup data corresponding to the data to be recovered is determined from the public cloud data center in a full recovery or incremental recovery mode, and the target backup data is recovered to the corresponding position of the target cloud disk, so that when the target cloud disk fails or the data in the target cloud disk has a logic error, the data can be recovered from the public cloud data center more conveniently.
The following describes a data recovery method provided in an embodiment of the present invention with a specific embodiment, where the method is a method for performing incremental recovery on data to be recovered in a target cloud disk, and may include the following steps:
step 41, the client device creates a backup recovery request for the target cloud disk according to the operation of the user;
step 42, the client device checks and judges the parameters of the backup recovery request; wherein the requested parameters include: identification of a target cloud disk; the content of the examination includes: checking the type of backup of the target cloud disk before the recovery, if the backup is incremental backup, determining that the recovery is incremental recovery, and judging that: if the target cloud disk is in backup, turning to step 43, otherwise, ending the process;
step 43, the client device checks whether a target cloud disk to be restored exists; if yes, go to step 45; if not, go to step 44;
step 44, creating a new disk as a target cloud disk, and turning to step 45;
step 45, the client device sends a data recovery instruction for the target cloud disk to the server; the data recovery instruction is represented in the form of: < < signal receiver RPC > >;
step 46, the client device determines that the public cloud data center (i.e. the back end) is S3, and accesses S3 through the server;
step 47, the server queries a backup chain corresponding to the data to be restored of the target cloud disk;
and 48, for the data to be restored, the server reads the metadata of the data to be restored from the tail part to the head part of the backup chain of the data to be restored, reads the name of each backup file on the backup chain from the metadata, reads the backup file from the S3 bucket, reads the name of a compression algorithm from the metadata, decompresses the backup file by a corresponding decompression algorithm, and restores the decompressed backup file to the target cloud disk.
In step 48, in the case that a new disk is created as a target cloud disk, after data recovery is performed on the target cloud disk, the server may further store the basic information of the target cloud disk stored locally in the database of the client device; the basic information of the target cloud disk comprises: name, size, etc. of the target cloud disk.
Corresponding to the embodiment shown in fig. 1, an embodiment of the present invention provides a data backup apparatus, which is applied to a server, and as shown in fig. 3, the apparatus includes:
a first receiving module 301, configured to receive a data backup instruction for a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk;
and the backup module 302 is configured to backup the data to be backed up to the public cloud data center according to the backup type.
According to the scheme provided by the embodiment of the invention, when data in the target cloud disk needs to be backed up, the server receives a data backup instruction aiming at the target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk; and backing up the data to be backed up to a public cloud data center according to the backup type. According to the embodiment of the invention, the data to be backed up in the target cloud disk is backed up to the public cloud data center in a full backup or incremental backup mode, so that the equipment cost of data backup can be reduced. Meanwhile, the labor operation and maintenance cost of a user needing to backup data can be reduced.
Optionally, the backup module 302 includes:
the first backup unit is used for backing up the data to be backed up to the public cloud data center in a full amount under the condition that the backup type is the full amount backup;
and the second backup unit is used for determining the data blocks to be subjected to incremental backup in the data to be backed up and backing up the data blocks to be subjected to incremental backup to the public cloud data center under the condition that the backup type is incremental backup.
Optionally, the second backup unit is specifically configured to, for each data block in the data to be backed up, perform the following steps: comparing the hash value corresponding to the data block with hash values of parent backup data blocks stored in the public cloud data center and corresponding to the data block; and determining the data block as the data block for incremental backup under the condition that the comparison results are different.
Optionally, the apparatus further comprises:
the first storage module is configured to divide the data to be backed up into a first preset number of data blocks after the first receiving module 301 receives a data backup instruction for a target cloud disk; calculating a hash value corresponding to each data block in the first preset number of data blocks; and storing the hash value corresponding to each data block to the public cloud data center.
Optionally, the apparatus further comprises:
the recording module is used for recording backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up after the backup module backs up the data to be backed up to a public cloud data center, and the backup chain is stored in the server or the public cloud data center;
wherein the backup information includes: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
Optionally, the apparatus further comprises:
a second storage module, configured to, after the first receiving module 301 receives a data backup instruction for a target cloud disk, store metadata of the data to be backed up and metadata of each data block of the data to be backed up to the public cloud data center, where the metadata of the data to be backed up at least includes: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk, and the metadata of each data block of the data to be backed up at least comprises: the name and size of the data block and the location of the data block in the data to be backed up.
Corresponding to the embodiment shown in fig. 2, an embodiment of the present invention provides a data recovery apparatus, which is applied to a server, and as shown in fig. 4, the apparatus includes:
a second receiving module 401, configured to receive a data recovery instruction for a target cloud disk;
a determining module 402, configured to determine a recovery type for data to be recovered in the target cloud disk;
a first recovery module 403, configured to, when the recovery type is full data recovery, read full backup content corresponding to the data to be recovered from a public cloud data center, use the full backup content as target backup data corresponding to the data to be recovered, and recover the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
a second recovery module 404, configured to, when the recovery type is incremental data recovery, read all backup contents corresponding to the data to be recovered from the public cloud data center, perform data recovery on the data to be recovered based on all the backup contents, obtain target backup data corresponding to the data to be recovered, and recover the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
wherein, all backup contents of the data to be restored comprise: the data recovery method comprises the steps of obtaining full backup contents corresponding to the data to be recovered and each incremental backup content corresponding to the data to be recovered.
According to the scheme provided by the embodiment of the invention, when the data in the target cloud disk needs to be recovered, the server receives a data recovery instruction aiming at the target cloud disk; determining a recovery type for the data to be recovered in the target cloud disk; under the condition that the recovery type is full data recovery, reading full backup content corresponding to the data to be recovered from a public cloud data center, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk; and under the condition that the recovery type is incremental data recovery, reading all backup contents corresponding to the data to be recovered from the public cloud data center, performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk. According to the embodiment of the invention, the target backup data corresponding to the data to be recovered is determined from the public cloud data center in a full recovery or incremental recovery mode, and the target backup data is recovered to the corresponding position of the target cloud disk, so that when the target cloud disk fails or the data in the target cloud disk has a logic error, the data can be recovered from the public cloud data center more conveniently.
Optionally, the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identification of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identification of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored;
the second restoring module 404 is specifically configured to read the full amount of backup content from the public cloud data center based on the identifier of the full amount of backup content, and read the incremental backup content from the public cloud data center based on the identifier of each incremental backup content.
Optionally, the second restoring module 404 is specifically configured to restore full-amount backup content corresponding to the data to be restored, and sequentially restore incremental backup content corresponding to the data to be restored according to a sequence of incremental backups on the basis of the restored full-amount backup content, so as to obtain target backup data corresponding to the data to be restored.
Optionally, the first recovery module 403 and the second recovery module 404 are specifically configured to obtain a position and a size of the data to be recovered in the target cloud disk; and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
The embodiment of the present invention further provides a server, as shown in fig. 5, including a processor 501, a communication interface 502, a memory 503 and a communication bus 504, where the processor 501, the communication interface 502 and the memory 503 complete mutual communication through the communication bus 504,
a memory 503 for storing a computer program;
the processor 501 is configured to implement any of the above-described data backup methods when executing the program stored in the memory 503, so as to obtain the same technical effects.
The communication bus mentioned in the above server may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the server and other devices.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
Another server according to an embodiment of the present invention is provided, as shown in fig. 6, and includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, where the processor 601, the communication interface 602, and the memory 603 complete mutual communication through the communication bus 604,
a memory 603 for storing a computer program;
the processor 601 is configured to implement any of the above-described data recovery methods when executing the program stored in the memory 603, so as to obtain the same technical effects.
The communication bus mentioned in the above server may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the server and other devices.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the data backup method in any of the above embodiments to achieve the same technical effects.
In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements the data recovery method in any of the above embodiments to obtain the same technical effect.
In another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the data backup method of any of the above embodiments to achieve the same technical effects.
In yet another embodiment provided by the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the data recovery method of any of the above embodiments to achieve the same technical effect.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The procedures or functions according to the embodiments of the invention are brought about in whole or in part when the computer program instructions are loaded and executed on a computer. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. The term "comprising" is used to specify the presence of stated features, integers, steps, operations, elements, components, and/or groups thereof, but does not exclude the presence of other similar features, integers, steps, operations, components, or groups thereof.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the device/server/storage medium/computer program product embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (18)

1. A data backup method is applied to a server, and the method comprises the following steps:
receiving a data backup instruction aiming at a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk;
according to the backup type, backing up the data to be backed up to a public cloud data center;
wherein, according to the backup type, backing up the data to be backed up to a public cloud data center comprises:
under the condition that the backup type is full backup, the data to be backed up is backed up to the public cloud data center in full;
under the condition that the backup type is incremental backup, determining data blocks for incremental backup in the data to be backed up, and backing up the data blocks for incremental backup to the public cloud data center;
determining a data block for incremental backup in the data to be backed up includes:
for each data block in the data to be backed up, executing the following steps:
comparing the hash value corresponding to the data block with hash values of parent backup data blocks stored in the public cloud data center and corresponding to the data block;
determining the data block as the data block for incremental backup under the condition that the comparison results are different;
after receiving the data backup instruction for the target cloud disk, the method further includes:
dividing the data to be backed up into a first preset number of data blocks;
calculating a hash value corresponding to each data block in the first preset number of data blocks;
and storing the hash value corresponding to each data block to the public cloud data center.
2. The method of claim 1,
after the data to be backed up is backed up to the public cloud data center, the method further comprises the following steps:
recording backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up, wherein the backup chain is stored in the server or the public cloud data center;
wherein the backup information includes: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
3. The method of claim 1, wherein after receiving the data backup instruction for the target cloud disk, the method further comprises:
storing the metadata of the data to be backed up and the metadata of each data block of the data to be backed up to the public cloud data center, wherein the metadata of the data to be backed up at least comprises: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk, and the metadata of each data block of the data to be backed up at least comprises: the name and size of the data block and the location of the data block in the data to be backed up.
4. A data recovery method is applied to a server, and the method comprises the following steps:
receiving a data recovery instruction aiming at a target cloud disk;
determining a recovery type for the data to be recovered in the target cloud disk;
under the condition that the recovery type is full data recovery, reading full backup content corresponding to the data to be recovered from a public cloud data center, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk;
under the condition that the recovery type is incremental data recovery, reading all backup contents corresponding to the data to be recovered from the public cloud data center, performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to the corresponding position of the target cloud disk;
wherein, all backup contents of the data to be restored comprise: the data recovery method comprises the steps of obtaining full backup contents corresponding to the data to be recovered and each incremental backup content corresponding to the data to be recovered.
5. The method according to claim 4, wherein the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identifier of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identifier of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored;
reading all backup contents corresponding to the data to be restored from the public cloud data center comprises:
reading the full backup content from the public cloud data center based on the identification of the full backup content, and reading the incremental backup content from the public cloud data center based on the identification of each incremental backup content.
6. The method according to claim 4, wherein performing data recovery on the data to be recovered based on all the backup contents to obtain target backup data corresponding to the data to be recovered comprises:
and restoring the full backup content corresponding to the data to be restored, and sequentially restoring the incremental backup content corresponding to the data to be restored according to the sequence of incremental backups on the basis of the restored full backup content to obtain the target backup data corresponding to the data to be restored.
7. The method according to claim 4, wherein restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk comprises:
acquiring the position and the size of the data to be recovered in the target cloud disk;
and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
8. A data backup device, applied to a server, the device comprising:
the first receiving module is used for receiving a data backup instruction aiming at a target cloud disk; the data backup instruction carries a backup type for backing up data to be backed up of the target cloud disk;
the backup module is used for backing up the data to be backed up to a public cloud data center according to the backup type;
wherein the backup module comprises:
the first backup unit is used for backing up the data to be backed up to the public cloud data center in a full amount under the condition that the backup type is the full amount backup;
the second backup unit is used for determining a data block for performing incremental backup in the data to be backed up and backing up the data block for performing incremental backup to the public cloud data center under the condition that the backup type is incremental backup;
the second backup unit is specifically configured to, for each data block in the data to be backed up, execute the following steps: comparing the hash value corresponding to the data block with hash values of parent backup data blocks stored in the public cloud data center and corresponding to the data block; determining the data block as the data block for incremental backup under the condition that the comparison results are different;
the device further comprises:
the first storage module is used for dividing the data to be backed up into a first preset number of data blocks after the first receiving module receives a data backup instruction for a target cloud disk; calculating a hash value corresponding to each data block in the first preset number of data blocks; and storing the hash value corresponding to each data block to the public cloud data center.
9. The apparatus of claim 8, further comprising:
the recording module is used for recording backup information corresponding to the data to be backed up in a backup chain corresponding to the data to be backed up after the backup module backs up the data to be backed up to a public cloud data center, and the backup chain is stored in the server or the public cloud data center;
wherein the backup information includes: and backing up the data to be backed up to the backup time of the public cloud data center and the identification of the corresponding backup content.
10. The apparatus of claim 8, further comprising:
a second storage module, configured to store, after the first receiving module receives a data backup instruction for a target cloud disk, metadata of the data to be backed up and metadata of each data block of the data to be backed up to the public cloud data center, where the metadata of the data to be backed up at least includes: the name and the size of the data to be backed up and the position of the data to be backed up in the target cloud disk, and the metadata of each data block of the data to be backed up at least comprises: the name and size of the data block and the location of the data block in the data to be backed up.
11. A data recovery apparatus, applied to a server, the apparatus comprising:
the second receiving module is used for receiving a data recovery instruction aiming at the target cloud disk;
the determining module is used for determining the recovery type of the data to be recovered in the target cloud disk;
the first recovery module is used for reading full backup content corresponding to the data to be recovered from a public cloud data center under the condition that the recovery type is full data recovery, taking the full backup content as target backup data corresponding to the data to be recovered, and recovering the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
the second recovery module is configured to, when the recovery type is incremental data recovery, read all backup contents corresponding to the data to be recovered from the public cloud data center, perform data recovery on the data to be recovered based on all the backup contents, obtain target backup data corresponding to the data to be recovered, and recover the target backup data corresponding to the data to be recovered to a corresponding position of the target cloud disk;
wherein, all backup contents of the data to be restored comprise: the data recovery method comprises the steps of obtaining full backup contents corresponding to the data to be recovered and each incremental backup content corresponding to the data to be recovered.
12. The apparatus according to claim 11, wherein the data to be restored corresponds to a backup chain, the backup chain is stored in the server or the public cloud data center, and the backup chain records: the identifier of the full backup content and the backup time of the full backup content corresponding to the data to be restored, and the identifier of each incremental backup content and the backup time of each incremental backup content corresponding to the data to be restored;
the second recovery module is specifically configured to read the full-volume backup content from the public cloud data center based on the identifier of the full-volume backup content, and read the incremental backup content from the public cloud data center based on the identifier of each incremental backup content.
13. The apparatus of claim 11,
the second recovery module is specifically configured to recover full-volume backup contents corresponding to the data to be recovered, and sequentially recover incremental backup contents corresponding to the data to be recovered according to an order of incremental backups on the basis of the recovered full-volume backup contents, so as to obtain target backup data corresponding to the data to be recovered.
14. The apparatus of claim 11,
the first recovery module and the second recovery module are specifically configured to obtain a position and a size of the data to be recovered in the target cloud disk; and restoring the target backup data corresponding to the data to be restored to the corresponding position of the target cloud disk according to the position and the size.
15. A server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor, when executing the program stored in the memory, implementing the method steps of any of claims 1-3.
16. A server is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
the memory is used for storing a computer program;
the processor, when executing the program stored in the memory, implementing the method steps of any of claims 4-7.
17. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 1-3.
18. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, which computer program, when being executed by a processor, carries out the method steps of any one of claims 4-7.
CN201811498098.2A 2018-12-07 2018-12-07 Data backup and recovery method and device, electronic equipment and storage medium Active CN109597717B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811498098.2A CN109597717B (en) 2018-12-07 2018-12-07 Data backup and recovery method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811498098.2A CN109597717B (en) 2018-12-07 2018-12-07 Data backup and recovery method and device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN109597717A CN109597717A (en) 2019-04-09
CN109597717B true CN109597717B (en) 2020-12-25

Family

ID=65961529

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811498098.2A Active CN109597717B (en) 2018-12-07 2018-12-07 Data backup and recovery method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN109597717B (en)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110147380B (en) * 2019-05-23 2023-11-24 腾讯科技(成都)有限公司 Data updating method, device, server and storage medium
CN110445841B (en) * 2019-07-12 2022-04-22 苏州浪潮智能科技有限公司 Cloud disk mounting method and device for cloud physical machine and storage medium
CN112306745B (en) * 2019-07-26 2023-04-07 中移(苏州)软件技术有限公司 Backup method and device and storage medium
CN112631826B (en) * 2019-10-09 2023-04-07 中移(苏州)软件技术有限公司 Backup processing method and device and computer readable storage medium
CN111240855A (en) * 2019-12-31 2020-06-05 中国建设银行股份有限公司 API debugging system, method, storage medium and computer device
CN111309522A (en) * 2020-02-14 2020-06-19 星辰天合(北京)数据科技有限公司 Object storage data protection method and device based on snapshot technology
CN111597074A (en) * 2020-04-07 2020-08-28 国家卫星气象中心(国家空间天气监测预警中心) Carbon satellite multi-center data backup method based on digital fingerprints
CN112269690A (en) * 2020-11-17 2021-01-26 珠海大横琴科技发展有限公司 Data backup method and device
CN112363873A (en) * 2020-11-27 2021-02-12 上海爱数信息技术股份有限公司 Distributed consistent backup and recovery system and backup method thereof
CN113722150B (en) * 2021-07-23 2023-08-22 苏州浪潮智能科技有限公司 Cloud hard disk data compression backup and recovery method, device, equipment and storage medium
WO2023151547A1 (en) * 2022-02-14 2023-08-17 华为技术有限公司 Data processing system and method, and device
CN114726718A (en) * 2022-03-23 2022-07-08 新华三技术有限公司 Configuration information synchronization method and device
CN115168111B (en) * 2022-09-07 2022-12-13 南京云信达科技有限公司 Object storage mass data backup method and system based on cache
CN115437849A (en) * 2022-09-21 2022-12-06 梁力 Data processing method, device, equipment and computer readable storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104461785A (en) * 2014-12-15 2015-03-25 中电长城网际系统应用有限公司 Data backup method, data recovery method and device of virtual machine and physic machine
CN104932956B (en) * 2015-06-19 2018-02-27 华南理工大学 A kind of cloud disaster-tolerant backup method towards big data
CN105373452B (en) * 2015-12-11 2019-02-01 上海爱数信息技术股份有限公司 A kind of data back up method
CN107454161A (en) * 2017-07-31 2017-12-08 郑州云海信息技术有限公司 A kind of data back up method and device
CN107493330A (en) * 2017-08-16 2017-12-19 北京新网数码信息技术有限公司 A kind of cloud service method and Cloud Server

Also Published As

Publication number Publication date
CN109597717A (en) 2019-04-09

Similar Documents

Publication Publication Date Title
CN109597717B (en) Data backup and recovery method and device, electronic equipment and storage medium
US20210382791A1 (en) Data Backup Technique for Backing Up Data to an Object Storage Service
US8874520B2 (en) Processes and methods for client-side fingerprint caching to improve deduplication system backup performance
US10552640B2 (en) In-situ data verification for the cloud
US9396073B2 (en) Optimizing restores of deduplicated data
US9075532B1 (en) Self-referential deduplication
CN107710164B (en) Disaster recovery as a service
CN111262822B (en) File storage method, device, block link point and system
JP2000200208A (en) Method and device for file backup, and program recording medium
US20230133533A1 (en) Snapshot copy operation between endpoints
US11580015B2 (en) Garbage collection for a deduplicated cloud tier using functions
CN113157487A (en) Data recovery method and apparatus thereof
CN114020522A (en) Data backup method and device, electronic equipment and system
EP3396554A1 (en) Backup control method and backup control device
CN110807000B (en) File repair method and device, electronic equipment and storage medium
CN107846429B (en) File backup method, device and system
US20190026304A1 (en) Container metadata separation for cloud tier
US11435930B2 (en) Intelligent recovery from multiple clouds copies
US10684922B2 (en) Enhanced data storage using compressed data
JP2017208113A (en) Data storage method, data storage apparatus, and storage device
US20200341891A1 (en) Garbage collection for a deduplicated cloud tier
CN112733210B (en) Equipment identifier obtaining method and device and computer readable storage medium
CN111625186B (en) Data processing method, device, electronic equipment and storage medium
GB2496258A (en) Encrypting data objects to back-up
WO2017039538A1 (en) Systems and methods for unified storage services

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant