CN116737466A

CN116737466A - Backup processing method, device, system, electronic equipment and readable storage medium

Info

Publication number: CN116737466A
Application number: CN202311022207.4A
Authority: CN
Inventors: 孙亚军
Original assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Current assignee: China Mobile Communications Group Co Ltd; China Mobile Suzhou Software Technology Co Ltd
Priority date: 2023-08-15
Filing date: 2023-08-15
Publication date: 2023-09-12
Anticipated expiration: 2043-08-15
Also published as: CN116737466B

Abstract

The invention provides a backup processing method, a backup processing device, a backup processing system, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring a target request, wherein the target request carries processing operation information; determining a node identifier of a first distributed storage cluster for carrying out backup storage on the first backup data based on the target request, wherein the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units carry out mutual access on the backup data through the same object used by the backup data; and performing target processing on the first backup data in an object storage system of the first distributed storage cluster based on the node identification and the target request, wherein the target processing is indicated by the processing operation information.

Description

Backup processing method, device, system, electronic equipment and readable storage medium

Technical Field

The embodiment of the invention relates to the technical field of distributed storage, in particular to a backup processing method, a device, a system, electronic equipment and a readable storage medium.

Background

With the rapid development of the cloud storage technology field, the distributed storage technology is widely applied, and the block storage based on the distributed storage system such as Ceph can provide block equipment for a cloud operating system such as Openstack, and can be used as a system disk and a data disk of virtual machines for storing data.

At present, the backup of the volume of the OpenStack based on Ceph is to create a disk by using a storage pool, and the backup data is stored in the disk (Image) newly created in the backup storage pool, so that the backup processing is realized, and the backup processing has the problem of poor backup flexibility.

Disclosure of Invention

The embodiment of the invention provides a backup processing method, a device, a system, electronic equipment and a readable storage medium, which are used for solving the problem of poor backup flexibility in backup processing in the prior art.

In a first aspect, an embodiment of the present invention provides a backup processing method, where the method includes:

acquiring a target request, wherein the target request carries processing operation information and is used for processing first backup data;

Determining a node identifier of a first distributed storage cluster for carrying out backup storage on the first backup data based on the target request, wherein the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units carry out mutual access on the backup data through the same object used by the backup data;

and performing target processing on the first backup data in an object storage system of the first distributed storage cluster based on the node identification and the target request, wherein the target processing is indicated by the processing operation information.

In a second aspect, an embodiment of the present invention provides a backup processing apparatus, including:

the system comprises an acquisition module, a processing module and a storage module, wherein the acquisition module is used for acquiring a target request, the target request carries processing operation information, and the target request is used for processing first backup data;

the determining module is used for determining a node identifier of a first distributed storage cluster for carrying out backup storage on the first backup data based on the target request, wherein the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units carry out mutual access on the backup data through the same object used by the backup data;

And the processing module is used for carrying out target processing on the first backup data in the object storage system of the first distributed storage cluster based on the node identification and the target request, wherein the target processing is indicated by the processing operation information.

In a third aspect, an embodiment of the present invention provides a backup processing system, where the system includes: front-end equipment, control equipment and node equipment; wherein, the liquid crystal display device comprises a liquid crystal display device,

the front-end equipment is used for sending a target request to the control equipment, wherein the target request carries processing operation information and is used for processing first backup data;

the control device is used for receiving a target request sent by the front-end device; determining a node identifier of a first distributed storage cluster for carrying out backup storage on the first backup data based on the target request, wherein the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units carry out mutual access on the backup data through the same object used by the backup data; scheduling the target request to node equipment corresponding to the node identifier;

And the node equipment is used for carrying out target processing on the first backup data in the object storage system of the first distributed storage cluster based on the target request, wherein the target processing is indicated by the processing operation information.

In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, where the computer program implements the steps of the backup processing method described above when executed by the processor.

In a fifth aspect, an embodiment of the present invention provides a readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the backup processing method described above.

In the embodiment of the invention, the distributed storage clusters of each area unit are distributed in the form of object storage, so that the distributed storage clusters of each area unit can access each other, the node identification of the first distributed storage cluster for carrying out backup storage on the first backup data is determined under the condition that the target request for carrying out processing on the first backup data is acquired, and the target processing is carried out on the first backup data in the object storage system of the first distributed storage cluster based on the node identification and the target request, thereby realizing backup processing. In addition, under the condition that the first backup data is backup data in a distributed storage cluster of other area units different from the first area unit, backup processing among the different area units can be realized, so that the flexibility of the backup processing can be improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic flow chart of a backup processing method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a complete flow of a backup processing method under a scene in an embodiment of the invention;

FIG. 3 is a schematic diagram of a complete flow of a backup processing method under scenario two in an embodiment of the present invention;

FIG. 4 is a schematic diagram of a complete flow of a backup processing method under scenario III in an embodiment of the present invention;

FIG. 5 is a schematic diagram of a complete flow of a backup processing method under scenario IV in an embodiment of the invention;

FIG. 6 is a schematic diagram of a backup processing device according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a backup processing system according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

It should be noted that, the backup processing method provided by the embodiment of the invention relates to the technical field of cloud storage, in particular to the technical field of distributed storage, and can be widely applied to cloud operating systems.

Currently, a backup of a volume based on Ceph OpenStack is to create a disk using a storage pool, and the backup process is implemented by storing backup data in a disk (Image) newly created in the backup storage pool. The backup processing of the technology needs the same Ceph cluster, so that backup data cannot be restored to a volume newly built by a distributed storage cluster of other Area units (Area Zone, AZ).

That is, in the related art, the main limitation is that the backup recovery across AZ cannot be performed, for example, the area a and the area B are two independent area units, and the backup data cannot be accessed mutually, so that the backup recovery of the backup data across AZ cannot be performed, and the flexibility of the backup processing is poor.

And the backup recovery of the cross-AZ can recover the data backup of the cloud host and the cloud hard disk to other AZ environments when one AZ is in a downtime condition, so that the flexibility of backup processing is improved, and the robustness and the high availability are enhanced.

For example, a volume backup created in the distributed storage cluster of the area unit of area a can restore the backup data to the volume in the distributed storage cluster of the area unit of area B. For example, 3 AZs in the B region, the a region and the C region deploy cross-AZ access object storage systems, each AZ has an access entry of the object storage system, backup data is stored in the three AZ distributed storage clusters in the three AZ object storage systems in a three-copy manner through erasure codes, and each AZ distributed storage cluster can read and write data through the access entry of the AZ. Therefore, it is possible to design a distributed storage cluster that restores the backup data of AZ in the a area to AZ in the B area.

The purpose of the embodiment of the invention is to enable the distributed storage cluster to perform the backup recovery of the cross AZ. The backup processing method of the embodiment of the invention can be executed by the backup processing device of the embodiment of the invention. And the backup processing apparatus may be configured in the backup processing system to perform the backup processing method. The backup processing method provided by the embodiment of the invention is first described below.

Referring to fig. 1, a flowchart of a backup processing method provided by an embodiment of the present invention is shown. For application to a backup processing system, as shown in fig. 1, the method may include the steps of:

step 101, obtaining a target request, wherein the target request carries processing operation information, and the target request is used for processing the first backup data.

The backup processing system may include a front-end device, which may correspond to a client, and a control device, which may correspond to a backend, which may be referred to as a console.

The front-end device may send the target request to the control device when the front-end device obtains the target request of the user, and correspondingly, the control device may receive the target request sent by the front-end device, thereby obtaining the target request.

The target request may include processing operation information, where the processing operation information may indicate a backup process required by a user, for example, create backup data, i.e., create backup, restore backup data, i.e., restore backup, and delete backup data, i.e., delete backup.

Other information may be included in the target request, such as a volume identifier when the target request is a request to create a backup, to backup the first backup data to the volume indicated by the volume identifier. When the target request is a request for backup restoration, the target request may include an identifier of the first backup data, and if the backup restoration is a backup restoration of cross-AZ, the target request further needs to include a source volume identifier and a target volume identifier, where the source volume and the target volume are located in distributed storage clusters of different area units. When the target request is a request for backup deletion, it may include an identification of the first backup data.

Step 102, determining a node identifier of a first distributed storage cluster for performing backup storage on the first backup data based on the target request, where the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units perform mutual access of the backup data through the same object used by the backup data.

It should be noted that, before the backup processing is performed, a distributed storage cluster needs to be deployed for each area unit, and the distributed storage clusters of the deployed distributed storage clusters may include the first area unit and the distributed storage clusters of other area units. Correspondingly, when the backup processing is performed, the area units can be selected from the area units, and the corresponding processing is performed in the distributed storage clusters of the selected area units.

For example, when performing backup recovery, the first distributed storage cluster of the first area unit may be selected to store the first backup data, and if the first backup data is backed up in the distributed storage clusters of other area units before the first backup data is backed up, the backup recovery across AZ may be implemented.

The Ceph object storage can be used for backing up data, the object storage systems of the distributed storage clusters are cross-regional units, and the distributed storage clusters supporting different AZ can access the object storage systems, so that the distributed storage clusters of different AZ can acquire the data in the corresponding object storage systems, and the cross-AZ backup recovery can be performed.

In an implementation, a volume in a distributed storage cluster may be created by a block storage service (cinder) in the cloud operating system OpenStack, and the created volume may be based on a Ceph cluster, i.e., the storage backend of the volume created by the cinder is Ceph, so that the volume created by the cinder corresponds to an image in a Ceph block storage. Among other things, a client may include an application program interface (client-api), a scheduler (client-scheduler), and a storage node (client-buccup) for a block storage service.

In addition, the backup data can be stored in an object storage environment based on Ceph clusters, the object storage system can be cross-AZ, and backup data can be acquired under different area units, namely, the data can be read and written on node equipment of different AZ through an object storage interface (such as an S3 interface). The node device may be referred to as a client-backup node, i.e., a node in a distributed storage cluster that stores backup data, on which a client is deployed.

Meanwhile, since the block storage uses a three-backup mechanism and the object storage uses an erasure code mechanism, the space used by the object storage relative to the block storage is smaller, so that the stored data volume is small.

In addition, the object storage used for backup in the distributed storage clusters under different AZ is the same, so that the environment which is not communicated originally can be enabled to carry out data interaction through backup, namely, the distributed storage clusters of different AZ can carry out the mutual access of backup data through the same object used for backup data.

In step 102, the first distributed storage cluster may be a distributed storage cluster in which the first backup data is finally stored, that is, after the backup processing, the storage location of the first backup data is located in a node device of the first distributed storage cluster, where the node device is a device corresponding to the node identifier determined by the control device.

The control device may determine the node identifier by combining the processing operation information in the target request, the identifier of the first backup data, and the volume identifier, where the processing indicated by the processing operation information is different, and the determination manner of the node identifier is different, which will be described in detail below.

And step 103, performing target processing on the first backup data in the object storage system of the first distributed storage cluster based on the node identification and the target request, wherein the target processing is indicated by the processing operation information.

In this step, the control device may schedule the target request to the node device corresponding to the node identifier based on the node identifier, and accordingly, the node device may perform target processing on the first backup data in the object storage system of the first distributed storage cluster based on the target request, and may perform backup processing on the first backup data in the object storage system of the first distributed storage cluster according to the processing manner indicated by the processing operation information based on the target request.

The target process may be a process of creating first backup data, a process of recovering the first backup data from the same AZ distributed storage cluster, a process of recovering the first backup data from the AZ distributed storage cluster, and a process of deleting the first backup data.

In the implementation process, the client-api is executed on the client-base node after receiving the target request of the user, and the client-base node is scheduled to the dependent client-base node based on the target request. The dependent cander-bakcup node refers to a storage node for the backup processing of the first backup data.

In this embodiment, by arranging the distributed storage clusters of each area unit in the form of object storage, the distributed storage clusters of each area unit may access each other, and determine, when a target request for performing processing of the first backup data is obtained, a node identifier of a first distributed storage cluster in which the first backup data is backed up and stored, and perform target processing on the first backup data in an object storage system of the first distributed storage cluster based on the node identifier and the target request, so that backup processing may be implemented. In addition, under the condition that the first backup data is backup data in a distributed storage cluster of other area units different from the first area unit, backup processing among the different area units can be realized, so that the flexibility of the backup processing can be improved.

The following describes the backup processing method according to the embodiment of the present invention in detail for different backup processing scenarios.

Scene one: the backup process is a creation process of backup data.

And in the case that the processing operation information indicates that the creation of the first backup data is performed, determining, based on the target request, a node identifier of a first distributed storage cluster in which the first backup data is backup-stored, including:

and determining the node identifier of the node to which the corresponding volume of the first volume identifier in the target request belongs as the node identifier of the first distributed storage cluster.

In this scenario, the target request carries a first volume identifier, i.e. volume_id, so as to store the first backup data sent by the front-end device into a volume corresponding to the volume_id of the distributed storage cluster.

The control device may determine a node identifier of a node to which the volume_id corresponds to the node to which the volume belongs as a node identifier of the first distributed storage cluster, and schedule the target request to execute on a server-bakcup node corresponding to the node identifier, that is, schedule the target request to a backup backend of a host (host) to which the volume corresponding to the volume_id belongs.

Scene II: the backup processing is recovery processing of the backup data under the same AZ.

And when the processing operation information indicates that the first backup data is restored in the same distributed storage cluster, and the backup record of the first backup data is queried in the first database, determining, based on the target request, a node identifier of the first distributed storage cluster in which the first backup data is backed up and stored, where the node identifier includes:

determining the node identification of the node to which the first backup data indicated by the backup record belongs as the node identification of the first distributed storage cluster; wherein the first database is accessible to a distributed storage cluster of at least two area units, the at least two area units comprising the first area unit.

Scene III: the backup process is a deletion process of the backup data.

And when the processing operation information indicates that the first backup data is deleted and the backup record of the first backup data is queried in the first database, determining, based on the target request, a node identifier of a first distributed storage cluster in which the first backup data is backed up and stored, including:

In the second and third scenarios, the target request carries the identifier of the first backup data, namely backup_id, so as to restore or delete the backup of the same AZ for the first backup data designated by the user.

The control device may access the first database to query backup records of the first backup data based on the identification of the first backup data. The first database is used for storing backup records of backup data in the distributed storage clusters of at least two area units, and the backup records of the same backup data in the distributed storage clusters of different area units are stored by adopting the same object.

The backup record may include basic information of the backup data, such as backup time, backup location, etc. Under the condition that the control equipment inquires that the backup record of the first backup data exists in the first database, the node identifier of the node where the first backup data indicated by the backup record of the backup_id belongs can be determined to be the node identifier of the first distributed storage cluster, and the target request is scheduled to be executed on a cander-backup node corresponding to the node identifier, namely, the target request is scheduled to the backup back end of the host where the first backup data corresponding to the backup_id belongs.

Scene four: the backup process is a restore process of the backup data under the cross-AZ condition.

And when the processing operation information indicates that the first backup data is restored in the distributed storage cluster of the cross-regional unit, determining, based on the target request, a node identifier of the first distributed storage cluster in which the first backup data is backed up and stored, including:

and determining a node identifier of a node to which a second volume identifier corresponding to the volume belongs in the target request as the node identifier of the first distributed storage cluster, wherein the second volume identifier is the identifier of the target volume recovered by the first backup data.

In this scenario, the target request carries a second volume identifier, where the second volume identifier is a target volume identifier recovered by the first backup data, so as to recover the first backup data in the source volume to a volume indicated by the target volume identifier.

The control device may determine the node identifier of the node to which the target volume identifier corresponds to the volume belongs as the node identifier of the first distributed storage cluster, and schedule the target request to execute on the server-bakcup node corresponding to the node identifier, that is, schedule the target request to the backup backend where the host corresponding to the volume_id is the same.

That is, for a target request for backup restoration across AZ, the control device can normally receive the target request sent by the front-end device, and send the target request to the server-backcup node of the distributed storage cluster of the specified other available area unit.

Correspondingly, after the target request is scheduled to the server-bakcup node corresponding to the node identifier, the server-bakcup node performs target processing on the first backup data in the object storage system of the first distributed storage cluster based on the target request.

Optionally, the object storage system of the first distributed storage cluster includes a first object storage system and a second object storage system; wherein, the liquid crystal display device comprises a liquid crystal display device,

the first object storage system is a first type object storage system, the first type indicates that the object storage system and the distributed storage clusters of other area units can perform mutual access of backup data, the second object storage system is a second type object storage system, and the second type indicates that the object storage system and the distributed storage clusters of other area units cannot perform mutual access of backup data.

That is, the first distributed storage cluster may retain the original object storage system (i.e., the second type object storage system) and the backup data, on the basis of which a new object storage system (i.e., the first type object storage system) may be deployed, the first type object storage system may be a cross-regional unit, and distributed storage systems capable of supporting different AZs may all access the object storage system, so that the distributed storage systems of different AZs may all acquire the backup data in the corresponding object storage system, and backup recovery across AZs may be performed.

In the implementation process, a cross-region support flag may be set for each piece of backup data, which may be referred to as a support_cross_az flag, where the support_cross_az flag of the backup data is first information, indicates that the backup data may be accessed by the distributed storage clusters of other region units, that is, supports the cross-AZ backup recovery, and where the support_cross_az flag of the backup data is second information, indicates that the backup data may not be accessed by the distributed storage clusters of other region units, that is, does not support the cross-AZ backup recovery. Wherein the first information may be set to True and the second information may be set to False.

The control device may determine, according to the support_cross_az flag, a target type of the object storage cluster performing the backup processing, and perform the target processing on the first backup data in an object storage system corresponding to the target type. Wherein the target type may be one of the first type and the second type.

The support_cross_az flag may be stored in a metadata (metadata) field of a backup data table of a server database, which may be used to store metadata of backup data in a distributed storage cluster. In addition, in order to realize the backup recovery across AZ, the support_cross_az flag may be stored in the first database after the backup processing is completed, that is, the support_cross_az flag is stored in the corresponding object of the backup data.

In the implementation process, the CInder-backup service needs to support the butt joint of a new object storage system, and is compatible with the functions of creating, restoring and deleting the backup of the stock backup data, and the backup restoration of the backup data which can cross AZ is carried out. That is, the configuration file needs to have connection parameters of the cross-AZ object storage system, and the connection parameters of the original object storage system need to be reserved.

Optionally, in a case where the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, performing, based on the node identifier and the target request, target processing on the first backup data in an object storage system of the first distributed storage cluster includes:

determining target configuration information based on the target request, wherein the target configuration information is as follows: configuration information of the first object storage system or configuration information of the second object storage system;

and scheduling a first interface of the node corresponding to the node identifier, performing target processing on the first backup data in an object storage system corresponding to the target configuration information, and determining the first interface based on the processing operation information.

Wherein, when the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, the corresponding scenes may include a first scene, a second scene and a third scene.

That is, in the first, second or third scenario, the target configuration information may be determined based on the target request, so as to target the first backup data for which type of object storage system, so that the back ends of the two object storage systems (onest) may be adapted during the backup processing, and further improve the flexibility of the backup processing.

Correspondingly, the client-bakcup node can load target configuration information and load the backup drive by using the target configuration information so as to schedule the first interface to perform target processing on the first backup data in the corresponding object storage system by using the loaded backup drive.

When the target configuration information is the configuration information of the first type of object storage system, the loaded backup driver can enable the client-bakcup node to perform target processing on the first backup data in the first type of object storage system. When the target configuration information is the configuration information of the second type of object storage system, the loaded backup driver can enable the client-bakcup node to perform target processing on the first backup data in the second type of object storage system.

The processing operation information is different and the first interface is also different. When the scene is a scene one, the first interface can be a backup interface and can be called a backup interface; when the scene is a scene two, the first interface can be a backup recovery interface with AZ, and can be called as a restore interface; when the scene is scene three, the first interface can be a deletion interface and can be called a delete interface; when the scene is scene four, the first interface may be a backup recovery interface crossing AZ, the backup recovery interface may be an interface newly developed on the original function of the distributed storage cluster, and the cander-bakcup node may access the first database based on the interface and acquire backup data from the distributed storage clusters of other area units to realize backup recovery crossing AZ.

In the case that the scenario is scenario one, optionally, in the case that the processing operation information indicates that the creation of the first backup data is performed, determining, based on the target request, target configuration information includes:

under the condition that backup data does not exist in a volume corresponding to a first volume identifier in the target request, determining the target configuration information as the configuration information of the first object storage system;

Under the condition that backup data exists in a volume corresponding to a first volume identifier in the target request and a cross-region support mark in metadata of second backup data is first information, determining that the target configuration information is the configuration information of the first object storage system, wherein the second backup data is backup data of last backup in time in the volume corresponding to the first volume identifier, and the first information indicates that the backup data can be accessed by a distributed storage cluster of other region units;

and under the condition that backup data exists in the volume corresponding to the first volume identifier in the target request and the cross-region support mark in the metadata of the second backup data is second information, determining the target configuration information as the configuration information of the second object storage system, wherein the second information indicates that the backup data can not be accessed by the distributed storage clusters of other region units.

In scenario one, the determination of the target configuration information may be implemented in combination with the first volume identifying whether backup data exists in the corresponding volume and the cross-region support flag of metadata of the second backup data.

When the first volume identifies that backup data is not present in the corresponding volume, configuration information for the new object storage system (i.e., the first type of object storage system) may be loaded by default. When backup data exists in the corresponding volume of the first volume identifier, the determination of the target configuration information can be further realized based on the last backup in the volume, namely, the cross-region support flag of the second backup data, when the cross-region support flag of the second backup data is Ture, the configuration information of the new object storage system is loaded, and when the cross-region support flag of the second backup data is False, the configuration information of the old object storage system (namely, the object storage system of the second type) is loaded.

In a first scenario, optionally, the scheduling the node identifier to identify a first interface of a corresponding node, performing target processing on the first backup data in an object storage system corresponding to the target configuration information, including:

and scheduling a first interface of the node corresponding to the node identifier to store the first backup data into a volume corresponding to a first volume identifier in an object storage system corresponding to the target configuration information under the condition that the processing operation information indicates to create the first backup data.

The backup interface is scheduled to upload the first backup data to a volume corresponding to the first volume identifier in the object storage system corresponding to the target configuration information by using the loaded backup drive.

In addition, when the backup data uploading is completed, metadata of the first backup data and metadata of a source volume of the first backup data may also be uploaded to the server database. And under the condition of backing up the first backup data to the new object storage system, the cross-region support flag of the metadata of the first backup data can be updated to True, and the backup is completed, wherein the metadata can comprise information such as an encryption key and the like.

The complete flow of the backup processing method under the scene in the embodiment of the present invention may be shown in fig. 2, where in the implementation process of the backup processing method under the scene, a D-Subminiature interface (DB interface) for querying metadata details of the last backup in the volume needs to be provided on an Edge Computing (EC) side, and on an elastic block storage (Elastic Block Store, EBS) side, backup driver loading needs to adapt multiple back ends, metadata of the backup data newly uploaded in the backup creation logic, metadata of the source volume into onest, and a span region support flag is newly added in metadata of the backup data.

In the case that the scenes are scene two and scene three, determining the target configuration information based on the target request includes:

acquiring metadata of the first backup data based on the identification of the first backup data in the target request;

determining that the target configuration information is the configuration information of the first object storage system under the condition that a cross-region support mark in metadata of the first backup data is first information;

and determining that the target configuration information is the configuration information of the second object storage system under the condition that the cross-region support mark in the metadata of the first backup data is the second information.

In scenario two and scenario three, the target configuration information may be determined based on the cross-region support flag in the metadata of the current backup.

Metadata of the first backup data can be obtained from the client database based on the identification of the first backup data, when the cross-region support flag of the first backup data is True, configuration information of a new object storage system is loaded, and when the cross-region support flag of the first backup data is False, configuration information of an old object storage system is loaded.

In the second scenario, optionally, the scheduling the node identifier to identify a first interface of a corresponding node, performing target processing on the first backup data in an object storage system corresponding to the target configuration information, where the target processing includes:

and under the condition that the processing operation information indicates that the first backup data is restored in the same distributed storage cluster, a first interface of a node corresponding to the node identifier is scheduled, the first backup data is downloaded from an object storage system corresponding to the target configuration information to a volume indicated by a third volume identifier in the target request, and the volume indicated by the third volume identifier is located in the first distributed storage cluster.

The server-bakcup node uses the loaded backup drive to schedule the restore interface to download the first backup data from the object storage system corresponding to the target configuration information to the volume indicated by the third volume identifier in the target request.

And if the target configuration information is the configuration information of the new object storage system, downloading the first backup data from the new object storage system to the volume indicated by the third volume identifier in the target request, and if the target configuration information is the configuration information of the old object storage system, downloading the first backup data from the old object storage system to the volume indicated by the third volume identifier in the target request.

In the implementation process of the backup processing method under the second scenario in the embodiment of the present invention, as shown in fig. 3, in the implementation process of the backup processing method under the second scenario, on the EBS side, the backup driver increases verification of the cross-region support flag of metadata of the backup data, and the backup restored database lock is replaced from the original cander database using single AZ to the multiple AZ mirror service (angle) database, so that when the backup restoration is performed, the backup data except the first backup data cannot be processed in the volume indicated by the source volume identifier.

In a third scenario, optionally, the scheduling the node identifier to identify a first interface of a corresponding node, performing target processing on the first backup data in an object storage system corresponding to the target configuration information, where the target processing includes:

and under the condition that the processing operation information indicates that the first backup data is deleted, the first interface of the node corresponding to the node identifier is scheduled, and the first backup data is deleted from the object storage system corresponding to the target configuration information.

Namely, the client-bakcup node uses the loaded backup drive to schedule the delete interface to delete the first backup data from the object storage system corresponding to the target configuration information.

And deleting the first backup data from the new object storage system if the target configuration information is the configuration information of the new object storage system, and deleting the first backup data from the old object storage system if the target configuration information is the configuration information of the old object storage system.

In the implementation process of the backup processing method under the third scenario in the embodiment of the present invention, as shown in fig. 4, in the implementation process of the backup processing method under the third scenario, on the EBS side, the backup driver increases verification of the cross-region support flag of metadata of the backup data, and the database lock for backup deletion is replaced from the original cander database using single AZ to the glance database with multiple AZ, so that when the backup deletion is performed, the backup data except the first backup data cannot be processed in the volume indicated by the source volume identifier.

In the case where the scenario is scenario four, the default loading is the configuration information of the first type of object storage system. Optionally, in the case where the processing operation information indicates that the recovery of the first backup data in the distributed storage cluster of the cross-regional unit is performed, performing, based on the node identifier and the target request, target processing on the first backup data in an object storage system of the first distributed storage cluster, where the target processing includes:

scheduling a first interface of a node corresponding to the node identifier, and downloading the first backup data from an object storage system of which the source volume identifier indicates the source volume in the target request to a target volume indicated by a target volume identifier in the target request;

the source volume is a volume created by the distributed storage cluster based on the second area unit, the target volume is a volume created by calling a second interface for the first distributed storage cluster in advance based on the source volume identifier, the target volume identifier and the identifier of the first backup data, and object storage systems of the source volume and the target volume are both object storage systems of a first type.

When the backup processing method supports backup restoration across AZs, a new volume (i.e., target volume) needs to be created in the selectable available AZ distributed storage clusters. The target request may carry the identifier of the first backup data and the identifier of the target volume, and the front-end device may query the source volume identifier through the database based on the identifier of the first backup data and send the source volume identifier to the control device, where the target request obtained by the control device carries the identifier of the first backup data, the source volume identifier and the target volume identifier.

The server-api may newly add a second interface to the backup recovery across AZ, where the entry parameters of the second interface need to include the target volume identifier, the source volume identifier, and the identifier of the first backup data. The control device may invoke the second interface to create a target volume for the first distributed storage cluster based on the source volume identification, the target volume identification, and the identification of the first backup data in advance.

Correspondingly, the client-bakcup node can schedule the newly deployed cross-AZ backup recovery interface to download the first backup data from the object storage system of which the source volume is indicated by the source volume identifier to the target volume indicated by the target volume identifier by using the loaded backup drive so as to realize the cross-AZ backup recovery.

Optionally, the method further comprises:

generating lock data based on a source volume identification of first backup data, wherein the lock data indicates that a volume indicated by the source volume identification cannot process backup data except the first backup data;

storing the lock data in a second database, the second database being accessible to a distributed storage cluster of at least two region units;

and deleting the lock data in the second database under the condition that the first backup data processing is completed.

The related operations for backups of the same volume are all completely mutually exclusive, and thus require locking of the volume that is being backed up. The reason for locking is to ensure that the underlying drive is serially backed up (e.g., backup creation, backup deletion, backup restoration) for one volume.

The backup across AZ is restored to a new volume and does not support concurrent operation, and under the condition of supporting the backup recovery function across AZ, a lock capable of crossing AZ needs to be used, and the lock can be accessed by a distributed storage cluster of each area unit, so that the action range of the lock can be expanded to a plurality of area units, and thus each backup process with AZ and backup processes among different AZ have mutual exclusion effects, and each backup process of the same volume is serial operation instead of concurrent operation.

A glance database may be used, i.e. the second database may be a glance database, and a table may be added to the glance database, specifically for use as a lock. A table is newly added to the glance database, and a field may include volume-id, time (atime), and processing operation information (status), which may indicate one of backup creation, backup deletion, and backup restoration.

The related operation of each backup is to insert the generated lock data into the table, wherein the insertion of the lock data successfully indicates that the locking is successful, namely, the lock is successfully acquired, and if the lock acquisition fails, the lock acquisition can be tried again at intervals. The record of the lock data is deleted from the glance database after the backup process is complete, i.e., the lock is released.

If the backup processing causes blocking, the record in the glance database cannot be deleted, and if the glance database is disconnected, the record in the glance database cannot be deleted normally, the size of the table in the glance database can be scanned by a client-backup node in backup creation to realize whether the backup is blocked or not, and when the fact that the table is unchanged within one hour is detected, the backup is judged to be blocked, and if the record in the glance database is cleared. And when the backup is deleted, the backup is restored to be blocked, and the glance database is disconnected, all records in the scanning table can be timed by adding a task, and if the preset time is exceeded from the time when the records are inserted to the present time, the records in the glance database are cleared.

In the implementation process, the configuration file needs to include the connection of the cander database and the glance database at the same time.

Optionally, before the downloading the first backup data from the object storage system of the source volume indicated by the source volume identifier in the target request to the target volume indicated by the target volume identifier in the target request, the method further includes:

acquiring metadata of the first backup data and metadata of the source volume;

checking the source volume attribute based on the metadata of the first backup data and the metadata of the source volume;

the downloading the first backup data from the object storage system of the source volume identification indication source volume in the target request to the target volume indicated by the target volume identification in the target request includes:

and under the condition that the source volume attribute test is passed, downloading the first backup data from the object storage system of which the source volume is indicated by the source volume identifier in the target request to a target volume indicated by the target volume identifier in the target request.

The verification of the source volume attribute may include verification of a size, a type, an encryption attribute, and the like, and the first backup data is downloaded from the object storage system of the target request, in which the source volume identifier indicates the source volume, into the target volume indicated by the target volume identifier in the target request only when the source volume size, the type, and the encryption attribute pass the verification.

The verification of the source volume attributes may be performed based on the metadata of the first backup data and the metadata of the source volume using existing or new verification means. The encryption volume does not support the creation of the volume under the backup recovery of the cross AZ, so that in the case that the volume is encrypted, the source volume attribute test is not passed, and the backup recovery of the cross AZ cannot be performed at the moment.

Optionally, the method further comprises:

when the first backup data is in a backup recovery process, the state of the target volume is marked as a first state, and the first state indicates that other processing of the backup data cannot be performed based on the target volume;

and under the condition that the backup and recovery of the first backup data are completed, identifying the state of the target volume as a second state, wherein the second state indicates that the backup data can be processed based on the target volume.

In addition, in the backup recovery process of the cross-AZ, the state of the target volume may be identified as a first state, where the first state may be a backup recovery state, indicating that other processes of backing up data cannot be performed based on the target volume, and the state of the source volume may be the backup state without modification. And in the case that the backup recovery is completed, identifying the state of the target volume as a second state, wherein the second state can be an available state, which means that the processing of the backup data can be performed based on the target volume.

In addition, if the backup processing is the backup recovery across AZ, since the source volume state and the backup information cannot be changed, the control device usually scans the target volume state during polling to ensure the normal running of the backup recovery across AZ.

In the implementation process of the backup processing method under the scene four in the embodiment of the invention, as shown in fig. 5, a set of interfaces of new volumes needs to be newly added from top to bottom in the backup recovery process of spanning AZ on the EC side, the entries of the interfaces include the identification of backup data, the identification of source volumes and the identification of target volumes, checksum state maintenance of the target volumes is added, a set of new interfaces can be developed for the backup of spanning AZ on the EBS side, the locking of the source volumes in the backup process is performed by using a glance database which is accessible by a plurality of AZs, metadata of backup data and metadata of the source volumes are obtained, and verification of size (size), type (type), encryption attribute and the like is performed based on the metadata of the backup data and the metadata of the source volumes.

The backup processing device provided by the embodiment of the invention is described below.

Referring to fig. 6, a schematic structural diagram of a backup processing device according to an embodiment of the present invention is shown. As shown in fig. 6, the backup processing apparatus 600 includes:

The obtaining module 601 is configured to obtain a target request, where the target request carries processing operation information, and the target request is used to process the first backup data;

a determining module 602, configured to determine, based on the target request, a node identifier of a first distributed storage cluster that performs backup storage on the first backup data, where the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units perform mutual access of the backup data through the same object used by the backup data;

and the processing module 603 is configured to perform, in the object storage system of the first distributed storage cluster, target processing on the first backup data based on the node identifier and the target request, where the target processing is indicated by the processing operation information.

Optionally, the determining module 602 is specifically configured to:

determining a node identifier of a node to which a first volume identifier corresponding to a volume in the target request belongs as the node identifier of the first distributed storage cluster under the condition that the processing operation information indicates that the first backup data is created;

Determining a node identifier of a node to which the first backup data indicated by the backup record belongs as a node identifier of the first distributed storage cluster under the condition that the processing operation information indicates a target backup operation and the backup record of the first backup data is queried in a first database; the distributed storage clusters of at least two area units can access the first database, the at least two area units comprise the first area unit, and the target backup operation is to restore the first backup data in the same distributed storage cluster or delete the first backup data;

and under the condition that the processing operation information indicates that the first backup data is restored in the distributed storage cluster of the cross-regional unit, determining the node identifier of the node to which the second volume identifier corresponding to the volume belongs in the target request as the node identifier of the first distributed storage cluster, wherein the second volume identifier is the identifier of the target volume restored by the first backup data.

Optionally, the first database is configured to store backup records of backup data in the distributed storage clusters of at least two area units, where the backup records in the distributed storage clusters of different area units indicating the same backup data are stored by using the same object.

Optionally, in a case where the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, the processing module 603 includes:

the determining unit is used for determining target configuration information based on the target request, wherein the target configuration information is as follows: configuration information of the first object storage system or configuration information of the second object storage system;

and the processing unit is used for scheduling a first interface of the node corresponding to the node identification, performing target processing on the first backup data in the object storage system corresponding to the target configuration information, and determining the first interface based on the processing operation information.

Optionally, in a case where the processing operation information indicates that the creation of the first backup data is performed, the determining unit is specifically configured to:

Optionally, in a case where the processing operation information indicates that the target backup operation is performed, the determining unit is specifically configured to:

Optionally, the processing unit is specifically configured to:

scheduling a first interface of a node corresponding to the node identifier to store the first backup data to a volume corresponding to a first volume identifier in an object storage system corresponding to the target configuration information under the condition that the processing operation information indicates to create the first backup data;

when the processing operation information indicates that the first backup data is deleted, a first interface of a node corresponding to the node identifier is scheduled, and the first backup data is deleted from an object storage system corresponding to the target configuration information;

Optionally, in a case where the processing operation information indicates that the restoration of the first backup data in the distributed storage cluster of the cross-regional unit is performed, the processing module 603 includes:

a downloading unit, configured to schedule the node to identify a first interface of a corresponding node, and download the first backup data from an object storage system of a source volume indicated by a source volume identifier in the target request to a target volume indicated by a target volume identifier in the target request;

Optionally, the apparatus further includes:

the generation module is used for generating lock data based on a source volume identification of the first backup data, wherein the lock data indicates that a volume indicated by the source volume identification cannot process backup data except the first backup data;

a storage module for storing the lock data into a second database, the second database being accessible by a distributed storage cluster of at least two region units;

and the deleting module is used for deleting the lock data in the second database under the condition that the first backup data processing is completed.

Optionally, the processing module further includes:

an obtaining unit, configured to obtain metadata of the first backup data and metadata of the source volume;

a checking unit, configured to perform a check of a source volume attribute based on metadata of the first backup data and metadata of a source volume;

the downloading unit is specifically configured to:

Optionally, the processing module further includes an identification unit; the identification unit is specifically configured to:

The backup processing device 600 can implement each process implemented in the foregoing backup processing method embodiment, and achieve the same technical effects, and for avoiding repetition, a detailed description is omitted herein.

The backup processing system provided by the embodiment of the invention is explained below.

Referring to fig. 7, a schematic structural diagram of a backup processing system provided by an embodiment of the present invention is shown, where, as shown in fig. 7, a backup processing system 700 includes a front-end device 701, a control device 702, and a node device 703; wherein the control device 702 establishes communication connection with the front-end device 701 and the node device 703 respectively;

the front-end device 701 is configured to send a target request to the control device 702, where the target request carries processing operation information, and the target request is used to process the first backup data;

The control device 702 is configured to receive a target request sent by the front-end device 701; determining a node identifier of a first distributed storage cluster for carrying out backup storage on the first backup data based on the target request, wherein the first distributed storage cluster is a distributed storage cluster of a first area unit, the backup data of the first area unit is stored in an object storage system based on the first distributed storage cluster, and the first distributed storage cluster and the distributed storage clusters of other area units carry out mutual access on the backup data through the same object used by the backup data; scheduling the target request to the node device 703 corresponding to the node identifier;

the node device 703 is configured to perform, based on the target request, target processing on the first backup data in the object storage system of the first distributed storage cluster, where the target processing is the processing indicated by the processing operation information.

Optionally, the control device 702 is specifically configured to:

Optionally, in the case where the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, the node apparatus 703 is specifically configured to:

Optionally, in the case where the processing operation information indicates that the creation of the first backup data is performed, the node apparatus 703 is further configured to:

Optionally, in the case where the processing operation information indicates that the target backup operation is performed, the node apparatus 703 is further configured to:

Optionally, the node apparatus 703 is further configured to:

Optionally, in a case where the processing operation information indicates that the restoration of the first backup data in the distributed storage cluster of the cross-regional unit is performed, the node apparatus 703 is further configured to:

Optionally, the node apparatus 703 is further configured to:

acquiring metadata of the first backup data and metadata of the source volume;

Optionally, the node apparatus 703 is further configured to:

Referring to fig. 8, a schematic structural diagram of an electronic device according to an embodiment of the present invention is shown. As shown in fig. 8, the electronic device 800 includes: a processor 801, memory 802, a user interface 803, and a bus interface 804.

A processor 801 for reading the program in the memory 802, performs the following procedures:

In fig. 8, a bus architecture may be comprised of any number of interconnected buses and bridges, and in particular one or more processors represented by the processor 801 and various circuits of the memory represented by the memory 802. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, power management circuits, etc., which are well known in the art and, therefore, will not be described further herein. Bus interface 804 provides an interface. The user interface 803 may also be an interface capable of interfacing with an inscribed desired device for a different user device, including but not limited to a keypad, display, speaker, microphone, joystick, etc.

The processor 801 is responsible for managing the bus architecture and general processing, and the memory 802 may store data used by the processor 801 in performing operations.

Optionally, the processor 801 is further configured to:

Optionally, in a case where the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, the processor 801 is further configured to:

Optionally, in a case where the processing operation information indicates that creation of the first backup data is performed, the processor 801 is further configured to:

Optionally, in a case where the processing operation information indicates that the target backup operation is performed, the processor 801 is further configured to:

Optionally, the processor 801 is further configured to:

Optionally, in a case where the processing operation information indicates that the restoration of the first backup data in the distributed storage cluster of the cross-regional unit is performed, the processor 801 is further configured to:

Optionally, the processor 801 is further configured to:

acquiring metadata of the first backup data and metadata of the source volume;

Optionally, the processor 801 is further configured to:

Preferably, the embodiment of the present invention further provides an electronic device, including a processor 801, a memory 802, and a computer program stored in the memory 802 and capable of running on the processor 801, where the computer program when executed by the processor 801 implements each process of the foregoing embodiment of the backup processing method, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.

The embodiment of the invention also provides a readable storage medium, on which a computer program is stored, which when executed by a processor, implements the processes of the above backup processing method embodiment, and can achieve the same technical effects, and in order to avoid repetition, the description is omitted here. Wherein the readable storage medium is selected from Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), magnetic disk or optical disk.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other manners. For example, the system embodiments described above are merely illustrative, e.g., the division of the elements is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely illustrative of the present invention, and the present invention is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.

Claims

1. A backup processing method, the method comprising:

2. The method of claim 1, wherein determining, based on the target request, a node identification of a first distributed storage cluster in which the first backup data is backup-stored comprises:

3. The method of claim 2, wherein the first database is configured to store backup records of backup data in a distributed storage cluster of at least two area units, and backup records in a distributed storage cluster of different area units indicating the same backup data are stored using the same object.

4. The method of claim 1, wherein the object storage system of the first distributed storage cluster comprises a first object storage system and a second object storage system; wherein, the liquid crystal display device comprises a liquid crystal display device,

5. The method of claim 4, wherein, in the case where the processing operation information indicates that the first backup data is processed in the same distributed storage cluster, the performing, based on the node identification and the target request, the target processing on the first backup data in an object storage system of the first distributed storage cluster comprises:

6. The method of claim 5, wherein, in the event that the processing operation information indicates that creation of the first backup data is to be performed, the determining target configuration information based on the target request comprises:

7. The method of claim 5, wherein, in the case where the processing operation information indicates that a target backup operation is performed, the determining target configuration information based on the target request comprises:

8. The method of claim 5, wherein the scheduling the node to identify the first interface of the corresponding node, and performing target processing on the first backup data in the object storage system corresponding to the target configuration information, comprises:

9. The method of claim 1, wherein, in the case where the processing operation information indicates that the restoration of the first backup data in the distributed storage cluster of the cross-regional unit is performed, the performing, based on the node identification and the target request, the target processing on the first backup data in the object storage system of the first distributed storage cluster includes:

10. The method according to claim 8 or 9, characterized in that the method further comprises:

11. The method of claim 9, wherein prior to downloading the first backup data from the object storage system in which the source volume identification indicates the source volume in the target request to the target volume indicated by the target volume identification in the target request, the method further comprises:

acquiring metadata of the first backup data and metadata of the source volume;

12. The method according to claim 9, wherein the method further comprises:

13. A backup processing apparatus, the apparatus comprising:

14. A backup processing system, the system comprising: front-end equipment, control equipment and node equipment; wherein, the liquid crystal display device comprises a liquid crystal display device,

15. An electronic device comprising a processor, a memory, a computer program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the backup processing method as claimed in any one of claims 1 to 12.

16. A readable storage medium, characterized in that the readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the backup processing method according to any of claims 1 to 12.