CN116048874A

CN116048874A - Data backup method and system based on cloud environment

Info

Publication number: CN116048874A
Application number: CN202211392952.3A
Authority: CN
Inventors: 卢士达; 刘逸逸; 陈书渊
Original assignee: State Grid Shanghai Electric Power Co Ltd
Current assignee: State Grid Shanghai Electric Power Co Ltd
Priority date: 2022-11-08
Filing date: 2022-11-08
Publication date: 2023-05-02

Abstract

The application discloses a data backup method and system based on cloud environment, wherein the data backup system comprises a shared storage pool, more than one backup service node and more than one access service node. The method comprises the following steps: the backup service node obtains data to be backed up from a data production system and stores the data to be backed up into a file system of a local disk; accessing a preset cloud storage access interface, and backing up data to be backed up in a file system of a local disk to the shared storage pool in an object storage mode; and the data backed up in the shared storage pool is provided for the more than one access service nodes to access. The cloud backup scheme with separated calculation and storage is provided, and backup data backed up in a shared storage pool can be used by a plurality of access service nodes, so that the sharing of the backup data stored in the cloud is realized.

Description

Data backup method and system based on cloud environment

[ field of technology ]

The application relates to the field of information technology, in particular to a data backup method and system based on a cloud environment.

[ background Art ]

This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not to be taken as an admission of prior art as including in this section.

With the continuous and deep use of new generation information technologies such as cloud computing and big data technologies in informatization construction, service systems need to provide instantly available data anytime and anywhere, so that the operation and maintenance of the whole service system are more efficient and safer. In order to ensure the data reliability of the service system, the data backup system takes the service system to be backed up as a data production system, and the data backup is carried out from the data production system. When the data production system is in fault, the data is destroyed and other abnormal conditions, the data recovery of the data production system can be carried out by using the backed-up data.

With the explosive growth of data volume of data production systems and the increasing number of data production systems connected, the pressure on data backup systems increases. The traditional data backup system is used for backing up the data acquired from the data production system to the local disk, but the mode is easy to cause the whole collapse of the data backup system under the condition of high backup processing pressure, and causes the data backed up to the local disk to be influenced, so that the problems of data loss, damage and the like are caused when the data is serious, and the disaster tolerance performance is poor. In addition, conventional data backup systems do not facilitate access sharing of backup data.

[ invention ]

In view of this, the present application provides a data backup method and system based on cloud environment, so as to improve disaster tolerance performance of data backup, and facilitate access sharing of backup data.

The specific technical scheme is as follows:

in a first aspect, a data backup method based on a cloud environment is provided, and the data backup method is applied to a data backup system of a cloud, wherein the data backup system comprises a shared storage pool, more than one backup service node and more than one access service node; the method comprises the following steps:

the backup service node obtains data to be backed up from a data production system and stores the data to be backed up into a file system of a local disk; accessing a preset cloud storage access interface, and backing up data to be backed up in a file system of a local disk to the shared storage pool in an object storage mode;

and the data backed up in the shared storage pool is provided for the more than one access service nodes to access.

According to an implementation manner in an embodiment of the present application, the method further includes:

the access service node receives a write request and acquires disk offset information of the data requested to be written from the write request;

And storing the data requested to be written into a local cache through a local file system and the disk offset information correspondingly.

According to an implementation manner of the embodiment of the present application, storing, by the local file system, the data requested to be written and the disk offset information in a local cache includes:

the virtual file system VFS in the access service node transmits the disk offset information to a read-write module through a user space file system FUSE, and the read-write module correspondingly stores the data requested to be written and the disk offset information in a local cache.

According to an implementation manner in an embodiment of the present application, the method further includes: the backup service node stores metadata of the corresponding object to a metadata server;

the access service node receives a read request and acquires disk offset information of the data requested to be read from the read request; judging whether the local cache contains data corresponding to the disk offset information, if so, reading the data corresponding to the disk offset information from the local cache; and otherwise, acquiring metadata from a metadata server according to the disk offset information, reading data from the object data stored in the shared storage pool according to the metadata, storing the data in a local cache, and providing the read data to a local file system of the access service node for responding to the read request.

According to an implementation manner in the embodiments of the present application, obtaining, from the read request, disk offset information where the requested read data is located is performed by the VFS in the access service node, where the VFS transfers, to the read-write module, the disk offset information where the requested read data is located through the FUSE; and executing the steps of judging, reading the data corresponding to the disk offset information from the local cache, acquiring metadata, reading the data and storing the data in the local cache by the read-write module, and transmitting the read data to the VFS through FUSE to respond to the read request.

According to an implementation manner of the embodiment of the present application, the name of the object in the shared storage pool is obtained from disk information of the data to be backed up in the shared storage pool, information of the data production system, information of the backup data, and information of the object.

In a second aspect, a data backup system based on a cloud environment is provided, where the data backup system includes a shared storage pool, more than one backup service node, and more than one access service node;

the backup service node is used for acquiring data to be backed up from a data production system and storing the data to be backed up into a file system of a local disk; accessing a preset cloud storage access interface, and backing up data to be backed up in a file system of a local disk to the shared storage pool in an object storage mode;

And the shared storage pool is used for storing the data to be backed up for the access of the more than one access service nodes.

According to an implementation manner in the embodiments of the present application, the access service node is configured to receive a write request, and obtain, from the write request, disk offset information where data requested to be written is located; and storing the data requested to be written into a local cache through a local file system and the disk offset information correspondingly.

According to an implementation manner in the embodiments of the present application, the access service node is configured to receive a read request, and obtain, from the read request, disk offset information where data requested to be read is located; judging whether the local cache contains data corresponding to the disk offset information, if so, reading the data corresponding to the disk offset information from the local cache; and otherwise, acquiring metadata from a metadata server according to the disk offset information, reading data from the object data stored in the shared storage pool according to the metadata, storing the data in a local cache, and providing the read data to a local file system of the access service node for responding to the read request.

According to a third aspect, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the first aspects described above.

According to a fourth aspect, there is provided an electronic device comprising:

one or more processors; and

a memory associated with the one or more processors, the memory for storing program instructions that, when read for execution by the one or more processors, perform the steps of the method of any of the first aspects above.

According to a specific embodiment provided by the application, the application discloses the following technical effects:

1) The application provides a cloud backup scheme with separated calculation and storage, wherein a backup service node of the cloud is used for completing a backup processing function, and a shared storage pool of the cloud is used for providing a backup data sharing and storage function of each data production system. According to the scheme, even if the backup service node is reduced in performance or damaged due to high processing pressure, the storage function is not affected, the whole breakdown of the data backup system is avoided, and the disaster recovery performance is improved. The backup data backed up in the shared storage pool can be used by a plurality of access service nodes, so that the sharing of the backup data stored in the cloud is realized.

2) When receiving a write request, the access service node provided by the application stores only data to be written and disk offset information in a local cache, but does not write the data to be written and the disk offset information into an object storage in a shared storage pool. In this way, the writing process of other systems to the backup data will not affect the backup data itself, so that multiple service nodes can access the backup data at the same time, and collision is avoided.

3) When the access service node reads data, if the data to be read exists in the local cache, the data in the local cache is directly used for responding to a read request; otherwise, the data is read from the object data stored in the shared storage pool and stored in the local cache, and the read data is provided for the local file system to respond to the read request. This approach allows for faster response speeds and reduces the performance consumption of accessing the shared storage pool by the access service node.

4) The naming mode of the object names can effectively distinguish the data production system, quickly locate the magnetic disk and the object in the data backup system, and ensure the uniqueness of each storage path.

5) The distributed architecture of the multi-backup service node and the shared storage pool can flexibly expand computing capacity and storage capacity.

Of course, not all of the above-described advantages need be achieved at the same time in practicing any one of the products of the present application.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

[ description of the drawings ]

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a data backup system based on a cloud environment according to an embodiment of the present application;

FIG. 2 is a flowchart of a data backup method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an object storage scheme of a data backup system according to an embodiment of the present application;

FIG. 4 is a schematic diagram of backup of full data and incremental data according to an embodiment of the present application;

Fig. 5 is a structural block diagram of an electronic device provided in an embodiment of the present application.

[ detailed description ] of the invention

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.

Exemplary embodiments of the present application are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

The terminology used in the embodiments of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

Depending on the context, the word "if" as used herein may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to detection". Similarly, the phrase "if determined" or "if detected (stated condition or event)" may be interpreted as "when determined" or "in response to determination" or "when detected (stated condition or event)" or "in response to detection (stated condition or event), depending on the context.

Fig. 1 is a schematic diagram of a data backup system based on a cloud environment according to an embodiment of the present application, where the data backup system is disposed at a cloud end. As shown in fig. 1, the data backup system may include a shared storage pool, one or more backup service nodes, and one or more access service nodes. Only one backup service node and one access service node are exemplified in fig. 1.

The backup service node has the main function of backing up the data of the data production system to the shared storage pool, and completing the processing related to calculation in the whole storage process.

The main function of the shared storage pool is to store the data backed up from the data production system and complete the function of data storage.

The primary function of the access service node is to access the backed up data in the shared storage pool. The access may be in response to the data consumption system.

The backup service nodes, the shared storage pool and the access service nodes are uniformly distributed on the cloud. The backup server and the access service node can adopt cloud servers, and the cloud servers are also called cloud computing servers or cloud hosts, and are host products in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and virtual special server (VPs, virtual Private Server) service are overcome. The shared memory pool may employ a virtual memory resource pool. The virtual Storage resource is to centrally manage a plurality of Storage medium modules (e.g., hard disk, RAID) by a certain means, and all the Storage modules are centrally managed in one Storage Pool (Storage Pool). From the perspective of the compute node, it is seen that not a plurality of hard disks, but a partition or volume, appears as a very large capacity (e.g., greater than 1T) hard disk. The storage system which can uniformly manage a plurality of storage devices and provides a large-capacity and high-data transmission performance for users is called virtual storage.

A backup service node may interface with a data production system and be responsible for backing up the data of the data production system at the cloud. One backup service node can also be connected with a plurality of data production systems in a butt joint mode and is responsible for backing up the data of the plurality of data production systems in the cloud.

An access service node may interface with a data consumption system and be responsible for accessing data backed up in a shared storage pool in response to an access request from the data consumption system. An access service node may interface with multiple data consumption systems and be responsible for accessing backed up data in a shared storage pool in response to access requests from the multiple data consumption systems.

In addition, other types of service nodes, such as recovery service nodes, desensitization service nodes, etc., may be included in the system framework, and the specific functions of these nodes will be illustrated in the following embodiments.

It should be appreciated that the numbers of backup service nodes, access service nodes, restore service nodes, desensitization service nodes, and shared storage pools in fig. 1 are merely illustrative. There may be any number of backup service nodes, access service nodes, restore service nodes, desensitization service nodes, and shared storage pools, as desired by the implementation.

Fig. 2 is a flowchart of a data backup method according to an embodiment of the present application, where the method is implemented based on the cloud data backup system shown in fig. 1. As shown in fig. 2, the method may include the steps of:

step 202: the backup service node obtains data to be backed up from the data production system and stores the data to be backed up into a file system of the local disk.

Step 204: the backup service node accesses a preset cloud Storage access interface, and backs up data to be backed up in a file system of a local disk to a shared Storage pool in the form of Object-based Storage.

Step 206: the data backed up in the shared storage pool is provided to more than one access service node for access.

It can be seen that the present application provides a cloud backup scheme with separate computation and storage, where a backup service node of the cloud completes a backup processing function, and a shared storage pool of the cloud provides a backup data sharing storage function of each data production system. According to the scheme, even if the backup service node is reduced in performance or damaged due to high processing pressure, the storage function is not affected, the whole breakdown of the data backup system is avoided, and the disaster recovery performance is improved.

In addition, the distributed architecture of the multi-backup service node and the shared storage pool can flexibly expand the computing capacity and the storage capacity. And the backup data backed up in the shared storage pool can be used by a plurality of access service nodes, so that the sharing of the backup data stored in the cloud is realized.

The above data backup method is described in detail with reference to the embodiments. First, the step 202, namely, "the backup service node obtains the data to be backed up from the data production system, and stores the data to be backed up in the file system of the local disk" will be described in detail.

As one of the realizations, the backup service node provided in the embodiments of the present application may obtain the data to be backed up from the data production system in response to a backup request of the data production system. For example, when a data production system has a data backup requirement, or periodically, a backup request is sent to a backup service node. The backup request can carry the storage position information of the data to be backed up in the data production system, and the backup service node obtains the data to be backed up according to the storage position information.

As another implementation, the data production system may provide address information of a database in the data production system to the backup service node in advance. The backup service node may actively obtain the data to be backed up from the data production system according to the address information, e.g. periodically obtain the data to be backed up from the data production system.

In this embodiment of the present application, a form of full-volume backup and incremental backup combined with log backup may be adopted, for example, full-volume backup is adopted for the first time, and then incremental backup is performed periodically, and log backup is performed simultaneously. Full-back-up, also known as base back-up, refers to making one full copy of all data of a database in a data production system. Incremental backup refers to backup of data in a data production system that has changed as compared to the previous backup. The log backup is to backup log files generated by each data node in the data production system. The data recovery of the data production system can be performed by using the full-volume backed-up data, the incremental backed-up data and the backed-up log file, namely, the data recovery is performed to a certain historical time point, so that the reliability of the data production system is ensured.

The following describes the step 204 in detail, that is, "the backup service node accesses the preset cloud storage access interface, and backs up the data to be backed up in the file system of the local disk to the shared storage pool in the form of object storage".

In the embodiment of the application, the backup data of each data production system are maintained in the shared storage pool of the cloud, so that the backup service node realizes the backup of the data through a preset cloud storage access interface.

Data production systems typically employ file storage formats that are different from the object storage formats employed by the cloud's shared storage pool. There are currently some tools such as S3FS that can perform file system simulations in the object store in the cloud, but after simulation, the files in the object store can only be accessed in a single-shared manner by these tools. In view of this, the embodiments of the present application provide a more preferable implementation manner, that is, the backup service node may first store the data to be backed up obtained from the data production system in the file system of the local disk, and then store the data to be backed up in the file system of the local disk in the shared storage pool in the form of object storage, and store the metadata of the corresponding object to the metadata server. In this way, the data in the shared storage pool adopts the form of object storage, and can be used by a plurality of service nodes at the same time, so that the sharing of the backup data in cloud storage is realized.

Specifically, the backup service node generates a file system in a local disk of the backup service node. The backup service node adopts a storage space in a block storage form, and the local disk can be a physical disk or a virtual disk.

As one of the possible ways, when a file system is generated for a data production system in a local disk, the storage space size of the file system may initially take a fixed value, for example 1TB. And when the storage space corresponding to the file system is insufficient, increasing the storage space for the file system.

As another implementation manner, when a file system is generated for a data production system in a local disk, since a full amount of data is backed up for the first time, an initial storage space can be allocated for the file system of the data production system according to the data amount of the full amount of data. For example, allocating 1.5 times the total amount of data. And when the storage space corresponding to the file system is insufficient, increasing the storage space for the file system.

When adding storage space to a file system, storage space may be added to the file system depending on the size of the initial storage space of the file system. For example, 0.5 times the initial storage space is increased each time. In principle, is adjusted according to the storage space actually used by the data production system.

The backup service node stores the data to be backed up obtained from the data production system in a file system of the local disk.

Then, the backup service node stores the data to be backed up in the file system of the local disk in the shared storage pool in the form of object storage.

As one of the possible ways, the backup service node may read data from the file system of the local disk in the addressing space order from beginning to end in blocks of a preset size (e.g. 1 MB), and write the data in the form of objects in the object storage in the shared storage pool.

In writing to object stores, embodiments of the present application provide a preferred naming convention to name objects. I.e., the name of the object is derived from the disk information of the data to be backed up in the shared storage pool, the information of the data production system, the information of the backup data, and the information of the object. For example, the name of an object takes the form:

"disk id_data production System id_backup data id_object sequence number"

Wherein the disk id is a disk identification of the data to be backed up in the shared storage pool, and the id is globally unique. The backup data id is used to distinguish what backup data the data production system is. As shown in fig. 3, the object sequence number may be a fragment sequence number of the object in a disk of the backup service node. For example, 1TB is split into 1 MB-sized blocks, and then there are 1000000 blocks in total. The object number corresponding to the first block is 0000000, and the object number corresponding to the last block is 0999999.

The naming mode of the object names can effectively distinguish the data production system, quickly locate the magnetic disk and the object in the data backup system, and ensure the uniqueness of each storage path.

And finally, the backup service node stores the metadata (Meta) data of the corresponding object to the metadata server. The metadata server is responsible for maintaining metadata of object stores for use in queries when accessing those object stores.

Furthermore, in order to ensure the security of the data, the backup service node encrypts the data to be backed up in the data transmission process when acquiring the data to be backed up from the data production system. In addition, when the backup service node stores data in the shared storage pool, the backup service node also encrypts the data to be backed up, for example, an encryption algorithm such as SM2 (an algorithm of national density) is adopted, so as to ensure the data security. Accordingly, subsequent other service nodes decrypt the data after reading the backup data.

The above step 206, i.e. "data backed up in a shared storage pool is provided to more than one access service node for access" is described in detail below in connection with an embodiment.

As shown in fig. 1, an access service node in a data backup system is configured to access a shared storage pool at the request of a data consumption system and to provide data for use by the data consumption system. The data consumption system can utilize the acquired data to perform data analysis, mining, decision making, testing and the like.

When the data consuming system needs to consume data, an access request for the data producing system may be sent to the access service node, as shown in fig. 3, the access request mainly comprising a write request or a read request.

The access service node generates a file system in the local disk. For example, a file system may be generated using FUSE (File System in UserSpace, user space file system) and interfaced to the VFS of Linux (virtual File System ). So that the VFS can see the file directory and use all file operation commands of Linux.

For a write request, an access service node acquires disk offset information where data requested to be written is located from the write request; and storing the data requested to be written into the local cache through the local file system and the disk offset information correspondingly. For example, when the access service node needs to perform a write operation upon a write request, the VFS transfers the disk offset to be written to the read-write module through the FUSE, the read-write module stores the data requested to be written in the local cache (cache), and the local cache stores the correspondence between the written data and the disk offset. When the subsequent read operation is related to the data of the disk offset, the read-write module directly acquires the data from the local cache and transmits the data to the VFS through the FUSE, and the data is not read from the object storage of the shared storage pool.

For a read request, an access service node acquires offset information of a disk where data requested to be read are located from the read request; judging whether the local cache contains data corresponding to the disk offset information, and if so, reading the data corresponding to the disk offset information from the local cache; otherwise, obtaining metadata from the metadata server according to the disk offset information, reading data from the object data stored in the shared storage pool according to the metadata, and storing the data in the local cache. The access service node then provides the read data to the local file system of the access service node in response to the read request.

For example, the VFS transmits the offset information where the requested data is located to the read-write module through the FUSE, and the read-write module first determines whether the data corresponding to the offset exists in the local cache, if so, directly reads the data corresponding to the offset from the cache, and transmits the data corresponding to the offset to the read-write module through the FUSE to respond to the read request. If the data does not exist, the read-write module acquires metadata from the metadata server according to the offset information, determines an object of the offset in the shared storage pool, reads the data from the object, stores the read data in the local cache and returns the read data to the VFS through the FUSE to respond to the read request.

It can be seen that except for backups, only read operations are performed for data accesses in the shared storage pool, while write operations only occur in the local cache of the access service node. When the written data is read later, the shared storage pool is not required to be accessed repeatedly, and the data in the local cache is read directly. According to the method, on one hand, a plurality of service nodes can access backup data at the same time, so that collision is avoided, on the other hand, the method has a faster response speed, and the performance consumption caused by accessing the service nodes to the shared storage pool is reduced.

The local cache of the access service node can be realized by using an SSD (Solid State Disk) so as to improve the overall read-write performance.

Further, the backup data read by the access service node from the shared storage pool may be the backup data after the desensitization processing. The desensitization of backup data in the shared storage pool may be performed by the desensitization service node.

As shown in fig. 1, the desensitization service node accesses a preset cloud storage access interface to perform desensitization processing on the data backed up in the shared storage pool, and stores the backed up data after the desensitization processing for providing data for a data consumption system.

The desensitization refers to data deformation of certain sensitive data through a desensitization rule, so that the sensitive data is reliably protected. The sensitive data can be information needing protection such as an identity card number, a mobile phone number, a card number, a client number, a mailbox, a client address and the like.

The desensitization treatment mode adopted by the desensitization service node can include, but is not limited to: substitution, encryption, generalization, truncation, masking, and the like. The desensitization service node can additionally store the backup data after the desensitization processing, so that the subsequent access service node can access the backup data in response to the access request of the data consumption system.

The manner in which the backup service node stores data to be backed up in the local file system is described in detail below in connection with embodiments.

In an embodiment of the present application, the file system generated by the backup service node for the data production system may include a first folder, a second folder, and a third folder. The first folder is used for storing full-volume data copies corresponding to different backup times, the second folder is used for storing copy snapshots corresponding to the full-volume data copies, and the third folder is used for storing log files.

It should be noted that the limitations of "first", "second", "third", and the like in the embodiments of the present application are not limited in size, order, and number, and are merely used to distinguish between the names, for example, "first folder", "second folder", and "third folder" are used to distinguish between three folders in terms of names. The concept of "folders" referred to in the embodiments of the present application is a data structure used to organize and manage storage data, each folder corresponding to a piece of storage space.

The processing manner of the backup service node provided in the embodiment of the present application for processing various data to be backed up is described herein.

The backup service node obtains the full data from the data production system for the first time, and then the data to be backed up is the full data. The backup service node stores the acquired full data as a copy in a first folder, generates a copy snapshot for the copy, and stores the copy snapshot in a second folder.

For non-primary backup, incremental data of the current data backup compared with the last data backup can be obtained from the data production system, full-quantity data is synthesized by using the obtained incremental data and the copy snapshot generated last time, the copy stored in the first folder is updated by using the synthesized full-quantity data, and the copy snapshot is generated for the copy and stored in the second folder.

As an example, as shown in fig. 4. For the first data backup, acquiring the full data of the distributed database (each data node), marking the full data as full data 1, storing the acquired full data as a copy in a first folder, generating a copy snapshot 1, and storing the copy snapshot 1 in a second folder.

And for the second data backup, obtaining incremental data 2 of the distributed database, synthesizing full-volume data by utilizing the incremental data and the current copy, storing the obtained full-volume data 2 as a copy in a first folder, generating a copy snapshot 2, wherein the copy snapshot 2 is a snapshot of the full-volume data 2, and storing the copy snapshot 2 in a second folder.

And for the third data backup, obtaining incremental data 3 of the distributed database, synthesizing the full-volume data by utilizing the incremental data and the current copy, then storing the obtained full-volume data 3 as a copy in a first folder, generating a copy snapshot 3, wherein the copy snapshot 3 is a snapshot of the full-volume data 3, and storing the copy snapshot 3 in a second folder.

The subsequent data backups are all incremental data backups, and are similar to the processing of the second data backup and the third data backup, and so on.

The data backup mode only needs to carry out full-volume backup when the data is backed up for the first time, and backup data is obtained from the distributed database in the subsequent incremental backup mode.

It should be noted that, both the full data and the incremental data are data states of a distributed database at a certain time point, so that the finally formed duplicate snapshot may also correspond to the time point, and the time point information corresponding to the duplicate snapshot may be recorded.

In addition to storing the full and incremental data described above, in embodiments of the present application, the backup service node may periodically obtain log files from the data production system. The periodicity employed herein may be a fixed periodicity, such as daily acquisition and backup of log files of the data production system. Non-fixed periods may also be employed, such as shorter periods for periods where data changes frequently, and longer periods for periods where data changes infrequently. Additionally, the retrieval and backup of log files may be triggered based on specific events, such as log backup instructions. Other mechanism triggers are also possible, and are not exhaustive herein.

The log records mainly the operation information of the transaction recorded by the data node in the data production system, the state of the transaction and other information. Because in some data production systems employing distributed data, multiple data nodes execute different sub-transactions of the same transaction under the scheduling of the coordinating node, respectively, and write local logs. Thus, the log file obtained by the backup service node may be a sub-transaction log of a plurality of data nodes.

The backup service node mainly stores the copy snapshot and the log file in a shared storage pool.

Based on the above-described manner of data backup, a data restoration service can be performed for the data production system by a restoration service node as shown in fig. 1. The recovery service node may, after obtaining the data recovery instruction for the data production system, recover, on the target device, the data of the data production system at the specified point in time using the data in the shared storage pool that has been backed up for the data production system. For convenience of description, this designated time point is referred to as a recovery time point. Wherein the information of the recovery time point may be carried in a data recovery instruction, for example, the user specifies the recovery time point. In addition, the recovery time point may be determined by a preset rule or a default manner based on the data recovery instruction, for example, before the default recovery is 1 day, before the default recovery is 1 hour, or the like.

The sending of the data recovery instruction may be event-based, for example, in the event of a failure of the data production system or the like, automatically triggering the sending of the data recovery instruction. The sending of the data recovery instruction can also be triggered manually by a manager, for example, in the event of a failure of the data production system, data corruption, etc.

When the data is recovered, the recovery service node firstly accesses a preset cloud storage access interface, acquires metadata corresponding to the data production system from the metadata server, and reads corresponding object data from the shared storage pool according to the metadata. Because the read data in the object storage format cannot be directly accessed in a file mode, the recovery service node can generate file data by using the read object data and store the file data into a file system of a local disk; and then, the generated file data is utilized to carry out data recovery on the target equipment.

When the generated file data is used for carrying out data recovery on the target equipment, judging whether copy snapshots corresponding to recovery time points exist in the file data, and if so, directly mounting (mount) the copy snapshots corresponding to the recovery time points on each data node of the target equipment.

The target device in the embodiment of the application may be a device where the database of the data production system is originally located, or may be a standby device outside the data production system. For example, if the data in the data production system is damaged, the copy snapshot corresponding to the recovery time point may be correspondingly mounted on the device where the database of the data production system is originally located (such a scenario is shown in fig. 1). If the equipment where the database of the data production system is originally located fails and the equipment cannot be used normally, the data state can be recovered on other standby equipment to the data state at the recovery time point.

If the copy snapshot corresponding to the recovery time point does not exist in the file data, the copy snapshot which is before the recovery time point and is closest to the recovery time point is firstly obtained, a log file between the determined copy snapshot corresponding time point and the recovery time point is taken as a log to be reworked, the determined copy snapshot is mounted to the target device, and the log to be reworked is respectively reworked on the target device according to the time sequence of the transaction until the recovery time point.

By means of the data recovery mode of mounting the copy snapshot and the redo log, data can be recovered to any time point in a corresponding period of backup data.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

As shown in fig. 5, a block diagram of an electronic device of a data backup method according to an embodiment of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, servers, blade servers, mainframes, and other appropriate computers. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 5, the apparatus 500 includes a computing unit 501 that can perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

Various components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, etc.; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508 such as a magnetic disk, an optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the method of data backup. For example, in some embodiments, the method of data backup may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508.

In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When a computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the method of data backup described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method of data backup in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here can be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of large management difficulty and weak service expansibility in the traditional physical host and virtual special server (VPs, VI irtual Private Server) service. The server may also be a server of a distributed system or a server that incorporates a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present application may be performed in parallel or sequentially or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. The data backup method based on the cloud environment is applied to a data backup system of a cloud, and is characterized in that the data backup system comprises a shared storage pool, more than one backup service node and more than one access service node; the method comprises the following steps:

2. The method according to claim 1, wherein the method further comprises:

3. The method of claim 1, wherein storing the requested written data in the local cache corresponding to the disk offset information via a local file system comprises:

4. A method as claimed in claim 1, 2 or 3, wherein the method further comprises: the backup service node stores metadata of the corresponding object to a metadata server;

5. The method of claim 4, wherein obtaining disk offset information of the requested read data from the read request is performed by a VFS in the access service node, the VFS transferring the disk offset information of the requested read data to a read-write module through a FUSE; and executing the steps of judging, reading the data corresponding to the disk offset information from the local cache, acquiring metadata, reading the data and storing the data in the local cache by the read-write module, and transmitting the read data to the VFS through FUSE to respond to the read request.

6. The method of claim 1, wherein the names of the objects in the shared storage pool are derived from disk information of the data to be backed up in the shared storage pool, information of the data production system, information of the backup data, and information of the objects.

7. A data backup system based on a cloud environment, which is characterized by comprising a shared storage pool, more than one backup service node and more than one access service node;

8. The data backup system of claim 7 wherein the access service node is configured to receive a write request and obtain disk offset information from the write request for the data requested to be written; and storing the data requested to be written into a local cache through a local file system and the disk offset information correspondingly.

9. The data backup system according to claim 7 or 8, wherein the access service node is configured to receive a read request, and obtain, from the read request, disk offset information where the requested read data is located; judging whether the local cache contains data corresponding to the disk offset information, if so, reading the data corresponding to the disk offset information from the local cache; and otherwise, acquiring metadata from a metadata server according to the disk offset information, reading data from the object data stored in the shared storage pool according to the metadata, storing the data in a local cache, and providing the read data to a local file system of the access service node for responding to the read request.

10. The data backup system of claim 7, wherein the names of the objects in the shared storage pool are derived from disk information of the data to be backed up in the shared storage pool, information of the data production system, information of the backup data, and information of the objects.