CN118152185A - Method, system, device, computer equipment and medium for recovering backup of fragmented clusters - Google Patents

Method, system, device, computer equipment and medium for recovering backup of fragmented clusters Download PDF

Info

Publication number
CN118152185A
CN118152185A CN202410351521.5A CN202410351521A CN118152185A CN 118152185 A CN118152185 A CN 118152185A CN 202410351521 A CN202410351521 A CN 202410351521A CN 118152185 A CN118152185 A CN 118152185A
Authority
CN
China
Prior art keywords
backup
cluster
master node
backup file
target
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410351521.5A
Other languages
Chinese (zh)
Inventor
赵庆亚
杨维敏
余剑
马立珂
王子骏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Dingjia Computer Technology Co ltd
Original Assignee
Anhui Dingjia Computer Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Dingjia Computer Technology Co ltd filed Critical Anhui Dingjia Computer Technology Co ltd
Priority to CN202410351521.5A priority Critical patent/CN118152185A/en
Publication of CN118152185A publication Critical patent/CN118152185A/en
Pending legal-status Critical Current

Links

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application relates to a method, a system, a device, computer equipment and a storage medium for backup and recovery of a fragment cluster. The method comprises the following steps: the backup server responds to the backup request of the fragmented clusters, and acquires backup files respectively corresponding to target slave nodes contained in the source clusters to be backed up; obtaining backup file paths corresponding to the backup files respectively, and first master node identifiers corresponding to the target slave nodes respectively, and constructing a first corresponding relation; responding to a fragment cluster recovery request aiming at a source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing a second corresponding relation; the first corresponding relation and the second corresponding relation are used for acquiring each backup file from the backup file path corresponding to each second master node identifier and storing the backup file to each master node corresponding to the second master node identifier. By adopting the method, the complexity of backup and recovery of the fragmented clusters can be reduced.

Description

Method, system, device, computer equipment and medium for recovering backup of fragmented clusters
Technical Field
The present application relates to the field of data processing technologies, and in particular, to a method, a system, an apparatus, a computer device, a storage medium, and a computer program product for backup and recovery of a sharded cluster.
Background
With the development of data processing technology, a technology of managing data by using a sliced cluster is presented, where the sliced cluster is generally composed of a plurality of slices and a configuration service, and each slice and configuration service further includes a master-slave relationship, when the data is backed up, it is generally required to backup data from each slave node, because the number of slices is multiple, a plurality of backup files are generated, and when the data is restored, the corresponding backup data files need to be restored to the corresponding master nodes.
In the conventional technology, for data backup and restoration of a sharded cluster, because corresponding backup data files need to be restored to corresponding master nodes, users are usually required to manually memorize the attribute of each backup file and copy the attributes into the corresponding nodes one by one.
However, if the number of the fragments in the fragment cluster is large, the number of the backup files is correspondingly increased, and the backup recovery method greatly increases the operation complexity of the user and is easy to make errors due to the fact that the user is required to memorize the attribute of each backup file, so that the complexity of the existing fragment cluster backup recovery method is high.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a method, a system, an apparatus, a computer device, a storage medium, and a computer program product for a partitioned cluster backup and restore method, which can reduce the complexity of the partitioned cluster backup and restore method.
In a first aspect, the present application provides a backup and restore method for a partitioned cluster, which is applied to a backup server, and includes:
Responding to a fragment cluster backup request, and acquiring backup files respectively corresponding to target slave nodes contained in a source cluster to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively;
the backup file path corresponding to each backup file and the first master node identifier corresponding to each target slave node are obtained, and a first corresponding relation between each first master node identifier and each backup file path is constructed;
Responding to a fragment cluster recovery request aiming at the source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing a second corresponding relation between each first master node identifier and each second master node identifier; the first correspondence and the second correspondence are configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
In one embodiment, the source cluster includes a plurality of first slices and a first configuration server, where each of the first slices and the first configuration server includes a master node and a plurality of slave nodes; a first proxy is arranged in a main node contained in each first fragment and each first configuration server; the responding to the backup request of the fragmented cluster, obtaining backup files respectively corresponding to each target slave node contained in the source cluster to be backed up, comprises the following steps: responding to the backup request of the fragment cluster, and acquiring the target slave node from the slave nodes respectively corresponding to the first fragments in the source cluster and the slave nodes contained in the first configuration server; generating a cluster backup task according to the target slave node, and sending the cluster backup task to each first fragment and a first proxy set by a first configuration server; the cluster backup task is used for each first agent to acquire a target slave node from a plurality of slave nodes corresponding to each first agent according to the cluster backup task, and acquire backup files from the target slave nodes respectively and send the backup files to the backup server.
In one embodiment, each first agent is further configured to obtain a first master node identifier of a master node corresponding to each first agent, and obtain a backup file path of a backup file corresponding to each first agent in the backup server; the obtaining the backup file paths corresponding to the backup files respectively and the first master node identifiers corresponding to the target slave nodes respectively includes: and receiving backup file paths which are sent by the first agents and respectively correspond to the first agents, and first main node identifiers respectively corresponding to the first agents.
In one embodiment, each of the first correspondence relationships is stored by a database table; after the first correspondence between each first primary node identifier and each backup file path is constructed, the method further includes: writing each first corresponding relation into a database table, and generating a database table identifier corresponding to the database table; the responding to the fragment cluster recovery request for the source cluster, obtaining second master node identifiers of all master nodes in the backup file target cluster for recovering the source cluster, and constructing second corresponding relations between the first master node identifiers and the second master node identifiers, including: responding to the fragment cluster recovery request, and acquiring a library table identifier matched with the fragment cluster recovery request; acquiring a database table matched with the database table identifier, and acquiring each first master node identifier from the matched database table; and obtaining second master node identifiers of all master nodes in the target cluster, and constructing a second corresponding relation between each first master node identifier and each second master node identifier.
In one embodiment, the target cluster includes a plurality of second fragments and second configuration servers, where each second fragment and configuration server includes a master node and a plurality of slave nodes; a second agent is arranged in each second fragment and a main node contained in the second configuration server; after the second correspondence between each first master node identifier and each second master node identifier is constructed, the method further includes: constructing a cluster recovery task according to each second corresponding relation; the cluster recovery task is sent to each second fragment and a second proxy set by a second configuration server; the cluster recovery task is used for each second agent to acquire a first master node identifier corresponding to each second master node identifier according to the cluster recovery task; inquiring the database table based on each first main node identifier to obtain a backup file path corresponding to each second main node identifier; and acquiring each backup file from the backup file path corresponding to each second master node identifier, and storing the backup file to each master node corresponding to the second master node identifier.
In a second aspect, the present application further provides a backup and restore system for a partitioned cluster, including: the system comprises a backup server, a first proxy and a second proxy, wherein the first proxy and the second proxy are in communication connection with the backup server, the first proxy is arranged on different main nodes in a source cluster to be backed up, and the second proxy is arranged on different main nodes in a target cluster for recovering backup files of the source cluster; wherein:
The backup server is used for responding to the partitioned cluster backup request, receiving backup files sent by each first agent, receiving backup file paths corresponding to the backup files respectively, and first main node identifiers corresponding to the first agents respectively; constructing a first corresponding relation between each first main node identifier and each backup file path;
The backup server is further configured to obtain second master node identifiers of all the master nodes in the target cluster in response to a sharded cluster recovery request for the source cluster, and construct second correspondence between each first master node identifier and each second master node identifier;
And each second agent is configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
In one embodiment, the source cluster includes a plurality of first slices and a first configuration server, where each of the first slices and the first configuration server includes a master node and a plurality of slave nodes; each first agent is further configured to obtain a target slave node from a plurality of slave nodes corresponding to each first agent, and obtain backup files from the target slave nodes respectively; sending each backup file to a first agent corresponding to the first configuration server; and the first agent corresponding to the first configuration server is used for sending each backup file to the backup server.
In a third aspect, the present application further provides a backup and restore device for a partitioned cluster, which is applied to a backup server, and includes:
the backup file acquisition module is used for responding to the backup request of the fragmented clusters and acquiring backup files respectively corresponding to each target slave node contained in the source cluster to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively;
The first relation construction module is used for acquiring backup file paths corresponding to the backup files respectively, first master node identifiers corresponding to the target slave nodes respectively, and constructing a first corresponding relation between the first master node identifiers and the backup file paths;
The second relation construction module is used for responding to a fragment cluster recovery request aiming at the source cluster, acquiring second main node identifiers of all main nodes in a target cluster of a backup file for recovering the source cluster, and constructing a second corresponding relation between each first main node identifier and each second main node identifier; the first correspondence and the second correspondence are configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
In a fourth aspect, the present application also provides a computer device comprising a memory storing a computer program and a processor implementing the steps of the method according to any one of the embodiments of the first aspect when the computer program is executed by the processor.
In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of the embodiments of the first aspect.
In a fifth aspect, the application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method according to any of the embodiments of the first aspect.
The method, the system, the device, the computer equipment, the storage medium and the computer program product for restoring the backup of the fragmented clusters are used for responding to the backup request of the fragmented clusters through the backup server to acquire backup files respectively corresponding to target slave nodes contained in the source clusters to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively; the method comprises the steps of obtaining backup file paths corresponding to backup files respectively, first master node identifiers corresponding to target slave nodes respectively, and constructing first corresponding relations between the first master node identifiers and the backup file paths; responding to a fragment cluster recovery request aiming at a source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing second corresponding relations between all the first master node identifiers and all the second master node identifiers; the first corresponding relation and the second corresponding relation are used for acquiring each backup file from the backup file path corresponding to each second master node identifier according to the second master node identifier corresponding to each master node in the target cluster and the first corresponding relation and the second corresponding relation, and storing the backup files to each master node corresponding to the second master node identifier. According to the application, when the backup server performs the sharded cluster backup, the backup files corresponding to each target slave node of different master nodes are obtained, and the backup file paths and the first master node identifiers of the master nodes corresponding to the corresponding target slave nodes can be utilized to construct the first corresponding relation between the target slave nodes and the first master node identifiers.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for those skilled in the art.
FIG. 1 is an application environment diagram of a sharded cluster backup recovery method in one embodiment;
FIG. 2 is a flowchart of a method for backup and restore of a partitioned cluster in one embodiment;
FIG. 3 is a flow diagram of a process for obtaining a backup file in one embodiment;
FIG. 4 is a flow chart of constructing a second correspondence in one embodiment;
FIG. 5 is a schematic diagram illustrating a structure of a partitioned cluster backup and restore system according to an embodiment;
FIG. 6 is an application scenario diagram of a MongoDB cluster backup restoration method in one embodiment;
FIG. 7 is a schematic diagram of a backup workflow in one embodiment;
FIG. 8 is a schematic workflow diagram of a backup service in one embodiment;
FIG. 9 is a schematic diagram of a recovery workflow in one embodiment;
FIG. 10 is a block diagram illustrating a structure of a partitioned cluster backup and restore apparatus in one embodiment;
FIG. 11 is an internal block diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The method for restoring the backup of the fragmented clusters, which is provided by the embodiment of the application, can be applied to an application environment shown in figure 1. The backup server may be communicatively connected to servers in the source cluster and the target cluster, where each of the source cluster and the target cluster may include a plurality of slices, respectively, slice 1, slice 2, …, and slice n, and each of the slices includes a plurality of node servers, which may be a master node server and a plurality of slave node servers. Specifically, when the backup server receives a sharded cluster backup request for the source cluster, the backup server may respond to the request, thereby obtaining backup files of target slave nodes corresponding to different master nodes of the source cluster, and backup file paths and first master node identifiers corresponding to each backup file, thereby constructing a first corresponding relationship between the first master node identifiers and the backup file paths. And if the fragment cluster recovery request is received, constructing a second corresponding relation with each first master node identifier by using the second master node identifier of each master node in the target cluster. And finally, obtaining a backup file path corresponding to each second master node identifier through the first corresponding relation and the second corresponding relation, so that after obtaining a corresponding backup file through the backup file path, storing the corresponding backup file in the master node in the target cluster corresponding to each second master node identifier. The backup server and each node server can be implemented by independent servers.
In one embodiment, as shown in fig. 2, a method for recovering backup of a partitioned cluster is provided, and the method is applied to the backup server in fig. 1 for illustration, and includes the following steps:
Step S201, responding to a fragment cluster backup request, and acquiring backup files respectively corresponding to target slave nodes contained in a source cluster to be backed up; each target slave node corresponds to a different master node in the source cluster.
The backup server is a server for providing a backup service of a fragmented cluster, the source cluster refers to a cluster needing to be backed up, and the fragmented cluster backup request is a request for requesting the backup server to execute the backup of the fragmented cluster aiming at the source cluster, and the request can be triggered by a user and carries an identification of the source cluster needing to be backed up. The backup files are files to be backed up in the source cluster, and the backup files can be provided by each target slave node corresponding to different master nodes in the source cluster.
For example, for the source cluster including the segment 1, the segment 2, the segment … and the segment n, since each segment includes one master node and a plurality of slave nodes, corresponding backup files can be obtained from the target slave nodes of each segment, and the backup files as different segments are the backup file 1, the backup file 2, the backup file … and the backup file n.
Step S202, backup file paths corresponding to the backup files respectively and first master node identifiers corresponding to the target slave nodes respectively are obtained, and first corresponding relations between the first master node identifiers and the backup file paths are constructed.
The backup file path is a file storage path for storing the different backup files, and the first master node identifier is a master node corresponding to a slave node for identifying the backup file, for example, the backup file 1 is provided by a slave node of the partition 1 in the source cluster, the corresponding first master node identifier may be a master node identifier of the partition 1, and for the backup file 2 is provided by a slave node of the partition 2 in the source cluster, the corresponding first master node identifier may be a master node identifier of the partition 2. Then, a first corresponding relation between each backup file path and the first main node identifier can be established respectively.
Step S203, responding to a fragment cluster recovery request for a source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing second corresponding relations between all the first master node identifiers and all the second master node identifiers; the first corresponding relation and the second corresponding relation are used for acquiring each backup file from the backup file path corresponding to each second master node identifier according to the second master node identifier corresponding to each master node in the target cluster and the first corresponding relation and the second corresponding relation, and storing the backup files to each master node corresponding to the second master node identifier.
The fragment cluster recovery request is a request for recovering the backup file of the source cluster, and the request can also be triggered by a user and carries an identifier of the target cluster for recovering the backup file. The second master node identifier is an identifier of each master node in the target cluster, similar to the source cluster, the target cluster may also include a plurality of slices, and each slice also includes one master node and a plurality of slave nodes, and the second master node identifier may identify a different master node of the target cluster. After receiving the request for restoring the fragmented clusters of the source cluster, the backup server can determine the target cluster for restoring the backup file and the second master node identifiers of all the master nodes in the target cluster, and then can establish the corresponding relations between the different first master node identifiers and the second master node identifiers respectively.
For example, the first master node identifier may include a master node identifier of the partition 1 in the source cluster and a master node identifier of the partition 2, the second master node identifier may include a master node identifier of the partition 1 in the target cluster and a master node identifier of the partition 2, and the backup server may respectively construct a corresponding relationship between the master node identifier of the partition 1 in the source cluster and the master node identifier of the partition 1 in the target cluster, and a corresponding relationship between the master node identifier of the partition 2 in the source cluster and the master node identifier of the partition 2 in the target cluster as the second corresponding relationship.
After the first correspondence and the second correspondence are constructed, the corresponding backup file path can be queried according to the second master node identifier of each master node in the target cluster, for example, the backup file path corresponding to the master node identifier of the fragment 1 in the target cluster is the path 1, and if the backup file 1 is stored in the path, the backup file 1 can be stored in the master node of the fragment 1 in the target cluster. And similarly, the backup file path corresponding to the main node identifier of the fragment 2 in the target cluster is the path 2, and if the backup file 2 is stored in the path, the backup file 2 can be stored in the main node of the fragment 2 in the target cluster.
In the method for restoring the backup of the fragmented clusters, the backup server responds to the backup request of the fragmented clusters to acquire backup files respectively corresponding to target slave nodes contained in the source clusters to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively; the method comprises the steps of obtaining backup file paths corresponding to backup files respectively, first master node identifiers corresponding to target slave nodes respectively, and constructing first corresponding relations between the first master node identifiers and the backup file paths; responding to a fragment cluster recovery request aiming at a source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing second corresponding relations between all the first master node identifiers and all the second master node identifiers; the first corresponding relation and the second corresponding relation are used for acquiring each backup file from the backup file path corresponding to each second master node identifier according to the second master node identifier corresponding to each master node in the target cluster and the first corresponding relation and the second corresponding relation, and storing the backup files to each master node corresponding to the second master node identifier. According to the application, when the sharded cluster is backed up, the backup files corresponding to all target slave nodes of different master nodes are obtained through the backup server, and the backup file paths and the first master node identifiers of the master nodes corresponding to the corresponding target slave nodes can be utilized to construct the first corresponding relation between the target slave nodes and the first master node identifiers.
In one embodiment, the source cluster includes a plurality of first slices and a first configuration server, where each of the first slices and the first configuration server includes a master node and a plurality of slave nodes; a first proxy is arranged in a main node contained in each first fragment and each first configuration server; as shown in fig. 3, step S201 may further include:
step S301, in response to the backup request of the sharded cluster, obtains a target slave node from the slave nodes corresponding to each first shard in the source cluster and the slave nodes included in the first configuration server.
In this embodiment, the source cluster may include multiple slices, i.e., multiple first slices, and one configuration server, i.e., a first configuration server, where each of the first slices and the first configuration server may include a master node server and multiple slave node servers, and in each master node server, a proxy service, i.e., a first proxy, may be further provided, and the proxy service may be used to perform a backup task.
Specifically, when the backup server receives the backup request of the sharded cluster, one of the first shards in the source cluster and the slave nodes contained in the first server needs to be determined as the target slave node.
For example, the source cluster includes a slice 1, a slice 2 and a configuration server, and each of the slice 1, the slice 2 and the configuration server includes a master node and two slave nodes, and the backup server can determine 1 from the two slave nodes of the slice 1, the slice 2 and the configuration server as target slave nodes of the slice 1, the slice 2 and the configuration server, respectively.
Step S302, generating a cluster backup task according to a target slave node, and sending the cluster backup task to each first fragment and a first agent set by a first configuration server; and the cluster backup task is used for each first agent to acquire target slave nodes from a plurality of slave nodes corresponding to each first agent according to the cluster backup task, and acquire backup files from the target slave nodes respectively and send the backup files to the backup server.
The cluster backup task is a task for enabling each first agent in the source cluster to execute file backup, and after the backup server completes the determination of the target slave node, the backup server can also generate a corresponding cluster backup task based on the target slave node and send the corresponding cluster backup task to each first partition and the first agent set by the first configuration server. And then each first proxy can respectively extract backup files from the target slave nodes according to the target slave nodes indicated in the cluster backup task and send the backup files to the backup server. For example, for the first agent of the segment 1, the target slave node is the slave node 1, then the first agent may obtain the backup file from the slave node 1 of the segment 1 and send the backup file to the backup server, and for the first agent of the segment 2, the target slave node is the slave node 1, then the first agent may obtain the backup file from the slave node 1 of the segment 2 and send the backup file to the backup server.
In this embodiment, the backup server may further determine each first partition in the source cluster and the target slave node corresponding to the first configuration server, and generate a cluster backup task, so as to obtain, by using each corresponding first agent, a backup file from the target slave node to send the backup file.
Further, each first agent is further configured to obtain a first master node identifier of a master node corresponding to each first agent, and obtain a backup file path of a backup file corresponding to each first agent in a backup server; step S202 may further include: and receiving backup file paths respectively corresponding to the first agents and first main node identifiers respectively corresponding to the first agents.
In this embodiment, each first agent may send the corresponding backup file to the backup server for storage, and may also obtain a corresponding backup path of the backup file when the backup file is stored in a backup manner, and send the backup path to the backup server, so that the backup server may receive the backup file path of the backup file in the backup server. Meanwhile, the first master node identification of each master node can be sent to the backup server by the corresponding first agent, and the time for sending the first master node identification can be sent when the source cluster is backed up or before backup, for example, when the first agent is configured for each master node, and can be sent after the first agent starts to operate. The backup server can obtain the backup file paths respectively corresponding to the backup files and the first master node identifications respectively corresponding to the target slave nodes by receiving the backup file paths and the first master node identifications respectively sent by the first agents.
In this embodiment, the backup file path and the first master node identifier may also be actively sent to the backup server by the first proxy, so that the working pressure of the backup server may be further reduced.
In one embodiment, each first correspondence is stored by a database table; after step S202, the method may further include: and writing each first corresponding relation into a database table, and generating a database table identifier corresponding to the database table.
In this embodiment, the first correspondence may be stored through a database table, where the database table identifier is an identifier for identifying a different database table, and after the backup server completes construction of the first correspondence, the backup server may write the first correspondence into the database table, manage each first correspondence through the database table, and then may generate a database table identifier corresponding to the database table.
As shown in fig. 4, step S203 may further include:
Step S401, responding to a fragment cluster recovery request, and acquiring a library table identifier matched with the fragment cluster recovery request;
Step S402, a database table matched with the database table identification is obtained, and each first main node identification is obtained from the matched database table.
In this embodiment, after receiving the fragment cluster recovery request, the backup server may respond to the fragment cluster recovery request and determine a source cluster corresponding to the recovery request, thereby determining a corresponding library table identifier, and then may query a corresponding database table by using the library table identifier, thereby extracting a first master node identifier of each master node included in the source cluster from the database table.
Step S403, obtaining second master node identifiers of all the master nodes in the target cluster, and constructing second corresponding relations between all the first master node identifiers and all the second master node identifiers.
The backup server may obtain the second master node identifier of each master node in the target cluster, where the second master node identifier may be obtained and stored in advance by the backup server, or may be obtained from the target cluster when the backup file recovery task is performed, and then the second correspondence between each first master node identifier and each second master node identifier may be respectively constructed.
In this embodiment, the backup server may further store the first corresponding relationship through the database table, and when performing backup recovery, query each first master node identifier included in the first corresponding relationship through the database table identifier of the database table, so as to implement construction of the second corresponding relationship with the second master node identifier.
Further, the target cluster comprises a plurality of second fragments and a second configuration server, and each second fragment and configuration server comprises a master node and a plurality of slave nodes; a second agent is arranged in each second fragment and a main node contained in the second configuration server; after step S403, the method may further include: constructing a cluster recovery task according to each second corresponding relation; the cluster recovery task is sent to each second fragment and a second proxy set by a second configuration server; the cluster recovery task is used for each second agent to acquire a first master node identification corresponding to each second master node identification according to the cluster recovery task; inquiring a database table based on each first main node identifier to obtain a backup file path corresponding to each second main node identifier; and acquiring each backup file from the backup file path corresponding to each second master node identifier, and storing the backup file to each master node corresponding to the second master node identifier.
Similar to the structure of the source cluster, the target cluster may also be composed of a plurality of slices, namely, a second slice, and a configuration server, namely, a second configuration server, and each of the second slice and the second configuration server may include a master node server and a plurality of slave node servers, and in each master node server, a proxy service, namely, a second proxy, may be further provided, and the proxy service may be used to perform a recovery task.
Specifically, after the backup server completes the construction of the second corresponding relationship, the backup server may further construct a corresponding cluster recovery task by using the second corresponding relationship, and send the cluster recovery task to each second segment and the second agent set by the second configuration server, where the cluster recovery task may be used to enable each second agent in the target cluster to execute file recovery. And then, each second agent can obtain the first main node identification corresponding to each second agent according to the second corresponding relation contained in the cluster recovery task and the second main node identification of the main node corresponding to the second agent, and further, the database table is queried through the first main node identification to obtain the backup file path corresponding to each second main node identification, so that the backup file is obtained from the corresponding backup file path to store.
For example, for the second agent of the target cluster fragment 1, the corresponding relationship between the target cluster fragment 1 and the source cluster fragment 1 is recorded in the second corresponding relationship, and then the second agent may query the database table by using the identifier of the source cluster fragment 1, so as to obtain the backup file path 1, so that the second agent may obtain the backup file 1 from the backup file path 1 and store the backup file 1. Similarly, for the second agent of the shard 2, the corresponding relationship between the target cluster shard 2 and the source cluster shard 2 is recorded in the second corresponding relationship, and then the second agent can query the database table by using the identifier of the source cluster shard 2, so as to obtain the backup file path 2, and therefore, the second agent can obtain the backup file 2 from the backup file path 2 and store the backup file 2.
In this embodiment, the backup server may further obtain, by using the corresponding second agents, a first master node identifier of the corresponding source cluster master node based on the second correspondence by using the corresponding second agents, and query the database table based on the first master node identifier, to obtain a backup file path to achieve acquisition of the backup file. By the method, the working pressure in the file recovery process of the backup server can be reduced, synchronous recovery of different fragments can be realized, and recovery efficiency is improved.
In one embodiment, as shown in FIG. 5, a tiled cluster backup-restore system is provided, which may include: the backup server 501, and a first agent 502 and a second agent 503 which are in communication connection with the backup server 501, wherein the first agent 502 is arranged at different main nodes in a source cluster to be backed up, and the second agent 503 is arranged at different main nodes in a target cluster for recovering backup files of the source cluster; wherein:
The backup server 501 may be configured to, when receiving a sharded cluster backup request, respond to the sharded cluster backup request, thereby receiving backup files sent by each first agent 502, backup file paths of each backup file, and first master node identifiers of corresponding master nodes sent by each first agent 502 from first agents 502 set in different master nodes in a source cluster to be backed up. And then, constructing the corresponding relation between each first main node identifier and the backup file path as a first corresponding relation.
The backup server 501 may be further configured to determine, when receiving a sharded cluster restoration request for the source cluster, a primary node identifier of each primary node included in the target cluster for restoring the backup file of the source cluster, that is, a second primary node identifier, so that a correspondence relationship with the first primary node identifier is constructed by using the second primary node identifier, and the second correspondence relationship is used as a second correspondence relationship.
And each second agent 503 may obtain a backup file path matched with each second agent 503 according to the second master node identifier of each corresponding master node and the first corresponding relationship and the second corresponding relationship constructed by the backup server 501, so as to obtain a corresponding backup file from the corresponding backup file path, and store the corresponding backup file to the corresponding master node respectively.
The system for backup and recovery of the fragmented clusters comprises: the backup server 501, and a first agent 502 and a second agent 503 which are communicatively connected with the backup server 501, wherein the first agent 502 is arranged at different main nodes in a source cluster to be backed up, and the second agent 503 is arranged at different main nodes in a target cluster for recovering backup files of the source cluster; wherein: the backup server 501 is configured to receive backup files sent by each first agent 502 in response to a sharded cluster backup request, and receive backup file paths corresponding to each backup file respectively, and first primary node identifiers corresponding to each first agent 502 respectively; constructing a first corresponding relation between each first main node identifier and each backup file path; the backup server 501 is further configured to obtain second master node identifiers of all the master nodes in the target cluster in response to a sharded cluster recovery request for the source cluster, and construct a second correspondence between each first master node identifier and each second master node identifier; each second agent 503 is configured to obtain each backup file from the backup file paths corresponding to each second master node identifier according to the second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier. According to the embodiment, the backup and recovery system for the fragmented clusters can realize that backup data corresponding to a plurality of nodes in the source clusters are respectively backed up in corresponding main nodes in the target clusters, the user does not need to memorize the attribute of each backup file, and the user can realize backup by only focusing on the source clusters and the target clusters.
In one embodiment, the source cluster includes a plurality of first slices and a first configuration server, where each of the first slices and the first configuration server includes a master node and a plurality of slave nodes; each first agent 502 is further configured to obtain a target slave node from a plurality of slave nodes corresponding to each first agent 502, and obtain backup files from the target slave nodes respectively; sending each backup file to a first proxy corresponding to a first configuration server; and a first proxy corresponding to the first configuration server, configured to send each backup file to the backup server 501.
In this embodiment, the source cluster may be composed of a plurality of slices, i.e. first slices, and one configuration server, i.e. first configuration server, and each of the first slices and the first configuration server may include a master node and a plurality of slave nodes at the same time. When each first agent 502 sends a backup file to the backup server 501, because the backup time of each backup file is uncertain, each first agent 502 needs to determine a target slave node from a plurality of slave nodes corresponding to the first agent 502, obtain the backup file from the target slave node, send the backup file to the first configuration server respectively, and then the first configuration server uniformly sends each configuration file to the backup server 501.
For example, after the first agent 502 set in the source cluster partition 1 obtains the backup file from the target slave node of the partition 1, that is, the backup file 1 needs to be sent to the first configuration server, and similarly, after the first agent 502 set in the source cluster partition 2 obtains the backup file from the target slave node of the partition 2, that is, the backup file 2 needs to be sent to the first agent corresponding to the first configuration server, and then the first agent corresponding to the first configuration server integrates and sends the backup file 1 and the backup file 2 to the backup server 501 for storage.
In this embodiment, after each first agent 502 obtains the backup file from the target slave node, the backup file may be sent to the first agent corresponding to the first configuration server, and then the first agent corresponding to the first configuration server is sent to the backup server 501 for storage.
In one embodiment, an automatic backup and recovery method for a MongoDB sharded cluster is further provided, where the MongoDB sharded cluster is generally composed of a plurality of shards and a configuration service, each shard and the configuration service have a master-slave relationship, and when the backup is performed, data is generally backed up from each slave node, so that a plurality of backup files are generated, and when the backup is performed, the corresponding backup data files need to be recovered to the corresponding master nodes.
Therefore, when the partitioned cluster is backed up, each partitioned sends own main node identification information and master-slave attributes, after the data of each partitioned is backed up, the backup sets record the data of the corresponding slave node of which main node the data belongs to, and marks the identification of each main node of the source cluster on the corresponding data. When recovering, firstly providing the attribute information of the backup source cluster, obtaining the corresponding relation between the main node identification of the source cluster and the main node identification of the target cluster through selection and calculation, and respectively transmitting the corresponding relation to the corresponding fragments of the target cluster. And inquiring the path of the data file through the identification of the master node of the source cluster on the sharding of the target cluster, further obtaining file data corresponding to the sharding master node, and completing the recovery process.
The process can be applied to an application scenario as shown in fig. 6, where the source cluster has three slices and one configuration server, which in turn contains three host nodes, 1 master and 2 slaves, respectively. The target cluster is similarly configured, and the software system needs to install agents on each host node and needs to be interconnected with the backup service.
And when in backup, backup data are acquired from a plurality of slave nodes of the source cluster, the acquired backup files are uploaded to be stored, information recording is carried out in a backup set, and the backup set is uniquely indexed by the id of the backup set. When recovering, only the information of the backup set is needed to be obtained through the id index, the backup set is used for inquiring the files of each backup, and the files are respectively recovered to the target clusters.
Therefore, the user only needs to care about the source cluster and the target cluster and which backup cluster data to restore, and does not need to care about the internal configuration of each cluster and the corresponding relationship of each fragment of the source cluster and the target cluster.
The method can be realized by the following steps:
1. Backup process
After each agent starts to operate, the unique identifier and the master-slave attribute of the node are sent to the backup service, when the backup is initiated, the backup service randomly selects one slave node from each partition and sends a backup task to backup, when the slave node receives the backup task, the agent acquires backup data from the slave node and sends the backup data to the storage service, meanwhile, the path of the backup file stored on the slave node and the master node identifier corresponding to the slave node are firstly sent to the agent of the configuration service to be summarized, after the data backup of each agent is completed, the configuration service uniformly sends the data to the backup service, the backup set is generated by the backup service, and the backup set is indexed by the unique id so as to be used when the backup set is recovered, and the work flow of the backup set can be shown in figure 7.
2. Design of backup set
The backup set is generated by the backup service, and is indexed by a unique id, and the format design of the index to obtain the current backup set is shown in table 1:
Table 1 format table of backup set
The backup set organization is designed as a database table, the paths of all files generated in the backup process and the main node identification information of each fragment of the corresponding source cluster are recorded in the database table, and the backup file paths can be queried through the main node identification.
3. Working process of backup service in recovery
Before the recovery task is initiated, each host on the target cluster runs the agent service, and after the agent runs, the node identifier and the master-slave attribute are actively sent to the backup service. When the data needs to be restored to the target cluster, the detail content of the backup set is queried through the backup set id, as shown in table 1. Acquiring source data to be recovered through the content of the table 1, and establishing a one-to-one correspondence between source cluster master node identifiers and target cluster master node identifiers through master node information of target clusters, as shown in the table 2:
TABLE 2 Master node correspondence table for Source Cluster and target Cluster
And then, the backup service respectively sends a recovery task to the main node agent of the target cluster, wherein the recovery task comprises the main node identifier of the source cluster and the main node identifier of the target cluster so as to represent that the data corresponding to a certain main node of the source cluster is recovered to the main node of the target node, and the operation flow of the backup service is shown in fig. 8.
4. Recovery procedure
After the target cluster master node receives the recovery task, the task includes the information of the source cluster master node identifier, so that a specific path of the backup file can be obtained through the source cluster master node identifier lookup table 1, and further corresponding data is obtained and recovered to the target cluster master node, and the working flow of the target cluster master node can be shown in fig. 9.
The embodiment overcomes the defect that a plurality of backup files are generated after the user needs to manage the backup, organizes the plurality of backup files into one backup set, provides convenience for the generation, selection and deletion of cluster backup sets, simplifies the backup and recovery operation of MongoDB fragmented clusters, and only needs to care about the backup from a certain source cluster when the user backs up and recovers, and only needs to care about the recovery of certain backup set data to a certain target cluster when the user recovers. The corresponding relation between the source cluster data and the target cluster nodes is automatically processed. In addition, the universal interface of MongoDB is adopted to acquire the cluster information, so that the method is applicable to backup and recovery of MongoDB clusters of various versions.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a shard cluster backup and recovery device for realizing the shard cluster backup and recovery method. The implementation scheme of the solution to the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitation in the embodiments of the one or more slicing cluster backup and restore devices provided below may refer to the limitation of the slicing cluster backup and restore method hereinabove, and will not be repeated herein.
In one embodiment, as shown in fig. 10, there is provided a partitioned cluster backup recovery device, applied to a backup server, including: a backup file acquisition module 1001, a first relationship construction module 1002, and a second relationship construction module 1003, wherein:
The backup file obtaining module 1001 is configured to obtain, in response to a sharded cluster backup request, backup files corresponding to target slave nodes included in a source cluster to be backed up respectively; each target slave node corresponds to different master nodes in the source cluster respectively;
A first relationship construction module 1002, configured to obtain backup file paths corresponding to each backup file, and first master node identifiers corresponding to each target slave node, and construct a first correspondence between each first master node identifier and each backup file path;
A second relationship construction module 1003, configured to obtain, in response to a sharded cluster restoration request for the source cluster, second master node identifiers of respective master nodes in a target cluster for restoring a backup file of the source cluster, and construct a second correspondence between each first master node identifier and each second master node identifier; the first corresponding relation and the second corresponding relation are used for acquiring each backup file from the backup file path corresponding to each second master node identifier according to the second master node identifier corresponding to each master node in the target cluster and the first corresponding relation and the second corresponding relation, and storing the backup files to each master node corresponding to the second master node identifier.
In one embodiment, the source cluster includes a plurality of first slices and a first configuration server, where each of the first slices and the first configuration server includes a master node and a plurality of slave nodes; a first proxy is arranged in a main node contained in each first fragment and each first configuration server; the backup file obtaining module 1001 is further configured to obtain, in response to a backup request of the sharded cluster, a target slave node from slave nodes corresponding to each first shard in the source cluster and slave nodes included in the first configuration server; generating a cluster backup task according to a target slave node, and sending the cluster backup task to each first fragment and a first agent set by a first configuration server; and the cluster backup task is used for each first agent to acquire target slave nodes from a plurality of slave nodes corresponding to each first agent according to the cluster backup task, and acquire backup files from the target slave nodes respectively and send the backup files to the backup server.
In one embodiment, each first agent is further configured to obtain a first master node identifier of a master node corresponding to each first agent, and obtain a backup file path of a backup file corresponding to each first agent in a backup server; the first relationship construction module 1002 is further configured to receive backup file paths sent by each first agent and corresponding to each first agent, and first primary node identifiers corresponding to each first agent.
In one embodiment, each first correspondence is stored by a database table; the first relationship construction module 1002 is further configured to write each first corresponding relationship into a database table, and generate a library table identifier corresponding to the database table; the second relationship construction module 1003 is further configured to obtain, in response to the shard cluster recovery request, a library table identifier that matches the shard cluster recovery request; acquiring a database table matched with the database table identifier, and acquiring each first main node identifier from the matched database table; and obtaining second master node identifiers of all the master nodes in the target cluster, and constructing a second corresponding relation between each first master node identifier and each second master node identifier.
In one embodiment, the target cluster comprises a plurality of second fragments and a second configuration server, wherein each second fragment and configuration server comprises a master node and a plurality of slave nodes; a second agent is arranged in each second fragment and a main node contained in the second configuration server; the second relationship construction module 1003 is further configured to construct a cluster recovery task according to each second correspondence; the cluster recovery task is sent to each second fragment and a second proxy set by a second configuration server; the cluster recovery task is used for each second agent to acquire a first master node identification corresponding to each second master node identification according to the cluster recovery task; inquiring a database table based on each first main node identifier to obtain a backup file path corresponding to each second main node identifier; and acquiring each backup file from the backup file path corresponding to each second master node identifier, and storing the backup file to each master node corresponding to the second master node identifier.
The modules in the partitioned cluster backup and recovery device can be realized in whole or in part by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one exemplary embodiment, a computer device is provided, which may be a server, and the internal structure thereof may be as shown in fig. 11. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used to store backup file data. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a sharded cluster backup restoration method.
It will be appreciated by those skilled in the art that the structure shown in FIG. 11 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magneto-resistive random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.

Claims (10)

1. A method for backup and restoration of a partitioned cluster, which is applied to a backup server, the method comprising:
Responding to a fragment cluster backup request, and acquiring backup files respectively corresponding to target slave nodes contained in a source cluster to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively;
the backup file path corresponding to each backup file and the first master node identifier corresponding to each target slave node are obtained, and a first corresponding relation between each first master node identifier and each backup file path is constructed;
Responding to a fragment cluster recovery request aiming at the source cluster, acquiring second master node identifiers of all master nodes in a target cluster of a backup file for recovering the source cluster, and constructing a second corresponding relation between each first master node identifier and each second master node identifier; the first correspondence and the second correspondence are configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
2. The method of claim 1, wherein the source cluster comprises a plurality of first slices and a first configuration server, each of the first slices and the first configuration server comprising a master node and a plurality of slave nodes; a first proxy is arranged in a main node contained in each first fragment and each first configuration server;
the responding to the backup request of the fragmented cluster, obtaining backup files respectively corresponding to each target slave node contained in the source cluster to be backed up, comprises the following steps:
Responding to the backup request of the fragment cluster, and acquiring the target slave node from the slave nodes respectively corresponding to the first fragments in the source cluster and the slave nodes contained in the first configuration server;
Generating a cluster backup task according to the target slave node, and sending the cluster backup task to each first fragment and a first proxy set by a first configuration server; the cluster backup task is used for each first agent to acquire a target slave node from a plurality of slave nodes corresponding to each first agent according to the cluster backup task, and acquire backup files from the target slave nodes respectively and send the backup files to the backup server.
3. The method of claim 2, wherein each first agent is further configured to obtain a first master node identifier of a master node corresponding to each first agent, and obtain a backup file path of a backup file corresponding to each first agent in the backup server;
The obtaining the backup file paths corresponding to the backup files respectively and the first master node identifiers corresponding to the target slave nodes respectively includes:
and receiving backup file paths which are sent by the first agents and respectively correspond to the first agents, and first main node identifiers respectively corresponding to the first agents.
4. The method of claim 1, wherein each of the first correspondence relationships is stored by a database table;
after the first correspondence between each first primary node identifier and each backup file path is constructed, the method further includes:
writing each first corresponding relation into a database table, and generating a database table identifier corresponding to the database table;
The responding to the fragment cluster recovery request for the source cluster, obtaining second master node identifiers of all master nodes in the backup file target cluster for recovering the source cluster, and constructing second corresponding relations between the first master node identifiers and the second master node identifiers, including:
responding to the fragment cluster recovery request, and acquiring a library table identifier matched with the fragment cluster recovery request;
acquiring a database table matched with the database table identifier, and acquiring each first master node identifier from the matched database table;
And obtaining second master node identifiers of all master nodes in the target cluster, and constructing a second corresponding relation between each first master node identifier and each second master node identifier.
5. The method of claim 4, wherein the target cluster comprises a plurality of second fragments and second configuration servers, each second fragment and configuration server comprising a master node and a plurality of slave nodes; a second agent is arranged in each second fragment and a main node contained in the second configuration server;
After the second correspondence between each first master node identifier and each second master node identifier is constructed, the method further includes:
constructing a cluster recovery task according to each second corresponding relation;
The cluster recovery task is sent to each second fragment and a second proxy set by a second configuration server; the cluster recovery task is used for each second agent to acquire a first master node identifier corresponding to each second master node identifier according to the cluster recovery task; inquiring the database table based on each first main node identifier to obtain a backup file path corresponding to each second main node identifier; and acquiring each backup file from the backup file path corresponding to each second master node identifier, and storing the backup file to each master node corresponding to the second master node identifier.
6. A sharded cluster backup recovery system, the system comprising: the system comprises a backup server, a first proxy and a second proxy, wherein the first proxy and the second proxy are in communication connection with the backup server, the first proxy is arranged on different main nodes in a source cluster to be backed up, and the second proxy is arranged on different main nodes in a target cluster for recovering backup files of the source cluster; wherein:
The backup server is used for responding to the partitioned cluster backup request, receiving backup files sent by each first agent, receiving backup file paths corresponding to the backup files respectively, and first main node identifiers corresponding to the first agents respectively; constructing a first corresponding relation between each first main node identifier and each backup file path;
The backup server is further configured to obtain second master node identifiers of all the master nodes in the target cluster in response to a sharded cluster recovery request for the source cluster, and construct second correspondence between each first master node identifier and each second master node identifier;
And each second agent is configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
7. The system of claim 6, wherein the source cluster comprises a plurality of first slices and a first configuration server, each of the first slices and the first configuration server comprising a master node and a plurality of slave nodes;
Each first agent is further configured to obtain a target slave node from a plurality of slave nodes corresponding to each first agent, and obtain backup files from the target slave nodes respectively; sending each backup file to a first agent corresponding to the first configuration server;
And the first agent corresponding to the first configuration server is used for sending each backup file to the backup server.
8. A sharded cluster backup and restore device, applied to a backup server, comprising:
the backup file acquisition module is used for responding to the backup request of the fragmented clusters and acquiring backup files respectively corresponding to each target slave node contained in the source cluster to be backed up; each target slave node corresponds to different master nodes in the source cluster respectively;
The first relation construction module is used for acquiring backup file paths corresponding to the backup files respectively, first master node identifiers corresponding to the target slave nodes respectively, and constructing a first corresponding relation between the first master node identifiers and the backup file paths;
The second relation construction module is used for responding to a fragment cluster recovery request aiming at the source cluster, acquiring second main node identifiers of all main nodes in a target cluster of a backup file for recovering the source cluster, and constructing a second corresponding relation between each first main node identifier and each second main node identifier; the first correspondence and the second correspondence are configured to obtain each backup file from a backup file path corresponding to each second master node identifier according to a second master node identifier corresponding to each master node in the target cluster and the first correspondence and the second correspondence, and store the backup file to each master node corresponding to the second master node identifier.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 5 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 5.
CN202410351521.5A 2024-03-26 2024-03-26 Method, system, device, computer equipment and medium for recovering backup of fragmented clusters Pending CN118152185A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410351521.5A CN118152185A (en) 2024-03-26 2024-03-26 Method, system, device, computer equipment and medium for recovering backup of fragmented clusters

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410351521.5A CN118152185A (en) 2024-03-26 2024-03-26 Method, system, device, computer equipment and medium for recovering backup of fragmented clusters

Publications (1)

Publication Number Publication Date
CN118152185A true CN118152185A (en) 2024-06-07

Family

ID=91294670

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410351521.5A Pending CN118152185A (en) 2024-03-26 2024-03-26 Method, system, device, computer equipment and medium for recovering backup of fragmented clusters

Country Status (1)

Country Link
CN (1) CN118152185A (en)

Similar Documents

Publication Publication Date Title
CN107528872B (en) Data recovery method and device and cloud storage system
US20190196728A1 (en) Distributed storage system-based data processing method and storage device
CN106776130B (en) Log recovery method, storage device and storage node
US9690823B2 (en) Synchronizing copies of an extent in an append-only storage system
US20150254320A1 (en) Using colocation hints to facilitate accessing a distributed data storage system
CN109582213B (en) Data reconstruction method and device and data storage system
CN105824846B (en) Data migration method and device
CN110046062B (en) Distributed data processing method and system
CN110532123B (en) Fault transfer method and device of HBase system
WO2019057081A1 (en) Data storage method, data query method, computer device, and storage medium
CN109407975B (en) Data writing method, computing node and distributed storage system
CN112925759A (en) Data file processing method and device, storage medium and electronic device
CN116303789A (en) Parallel synchronization method and device for multi-fragment multi-copy database and readable medium
CN111400248A (en) Method for writing data and recovering data and file system
DE112016004457T5 (en) Duplicating data in data storage devices of a linked volume
CN118152185A (en) Method, system, device, computer equipment and medium for recovering backup of fragmented clusters
CN116049306A (en) Data synchronization method, device, electronic equipment and readable storage medium
CN111522883B (en) Backup method, device, equipment and storage medium of object data
JP2017208113A (en) Data storage method, data storage apparatus, and storage device
CN117370078B (en) Database backup management method, device, computer equipment and storage medium
CN117539690B (en) Method, device, equipment, medium and product for merging and recovering multi-disk data
CN114666401B (en) Device information processing method, device, computer device and storage medium
CN115730016B (en) Data synchronization method, system, device, computer equipment and storage medium
CN114442962B (en) Data reading method, device, equipment and storage medium
CN117931769A (en) Data migration method, device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination