CN115268801B - Backup system and method for block device - Google Patents
Backup system and method for block device Download PDFInfo
- Publication number
- CN115268801B CN115268801B CN202211205460.9A CN202211205460A CN115268801B CN 115268801 B CN115268801 B CN 115268801B CN 202211205460 A CN202211205460 A CN 202211205460A CN 115268801 B CN115268801 B CN 115268801B
- Authority
- CN
- China
- Prior art keywords
- backup
- backed
- carrier
- block device
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0638—Organizing or formatting or addressing of data
- G06F3/064—Management of blocks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1456—Hardware arrangements for backup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3037—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a memory, e.g. virtual memory, cache
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/065—Replication mechanisms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Human Computer Interaction (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a backup system and a backup method of block equipment, which comprise the following steps: the automatic scanning module is used for scanning the block equipment to be backed up in the distributed storage according to a preset backup period; the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier; the calculation module is used for determining a target backup carrier of the block equipment to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block equipment to be backed up, and sending data in the block equipment to be backed up to a working module corresponding to the target backup carrier; the working module is used for backing up the received data to the corresponding backup carrier based on a preset backup strategy. The method for incrementally backing up the block equipment to be backed up to at least one backup carrier at regular time realizes full-automatic backup of the block equipment, and the mass file system is used as the backup carrier, so that the flexibility and maintainability of the backup of the block equipment are increased, and the backup cost of the block equipment is reduced.
Description
Technical Field
The present invention relates to the field of data storage technologies, and in particular, to a backup system and method for a block device.
Background
The ceph is a distributed storage system, has the characteristics of high performance, high reliability, expandability and the like, and supports seamless data copying and concurrent reading and writing. Rbd (RADOS block device, block storage) in Ceph has become the storage cornerstone of most cloud scene cloud platforms.
In the prior art, a method of backing up multiple clusters from one ceph cluster to another ceph cluster is generally adopted for the backup of rbd, and the backup scheme requires that both a source cluster and a destination cluster serving as a backup medium are necessarily ceph clusters, and has the disadvantages of complex configuration, poor maintainability and high cost.
Disclosure of Invention
In view of this, the present invention aims to provide a backup system and method for a block device, which implement full-automatic backup of the block device by a method of incrementally backing up the block device to be backed up to at least one backup carrier at regular time, and use a mass file system as the backup carrier, thereby increasing the flexibility and maintainability of backup of the block device, and further reducing the backup cost of the block device.
In a first aspect, an embodiment of the present invention provides a backup system for a block device, where the system includes: the system comprises an automatic scanning module, a computing module, a monitoring module, a plurality of working modules and a plurality of backup carriers; the monitoring module is respectively in communication connection with the automatic scanning module and the computing module; each working module is respectively in communication connection with the computing module and the corresponding backup carrier; the automatic scanning module is used for scanning the to-be-backed-up block equipment in the distributed storage according to a preset backup period; the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier; the computing module is used for determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier; and the working module is used for receiving the data of the block device to be backed up and backing up the received data to the corresponding backup carrier based on a preset backup strategy.
Further, the monitoring module is further configured to notify the automatic scanning module to scan the device to be backed up in the distributed storage to trigger a new round of backup process when it is monitored that a newly added backup carrier exists, or a backup carrier is deleted, or a backup carrier is disconnected.
Further, the step of scanning the to-be-backed-up block devices in the distributed storage according to the preset backup period includes: performing full-automatic directional pool scanning on the distributed storage according to a preset backup period to obtain a device list of blocks to be backed up; the oriented pool is a set of all block devices including the block devices to be backed up in the distributed storage system.
Further, the step of determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes, and the hash value of the block device to be backed up includes: calculating the hash value of each block device to be backed up in the block device list to be backed up through a static hash function based on the block device list to be backed up; and determining a target backup carrier of the block equipment to be backed up through pseudo-random operation according to the number of the carrier nodes of the available backup carrier, the weight of the carrier nodes and the hash value of the block equipment to be backed up.
Further, the backup carrier contains one or more folders.
Further, the step of receiving data of the block device to be backed up and backing up the received data to the corresponding backup carrier based on a preset backup strategy includes: receiving data of the block equipment to be backed up, and judging whether historical data of the block equipment to be backed up is stored in a backup carrier corresponding to the working module; if the historical data of the block equipment to be backed up is not stored, backing up the data of the block equipment to be backed up to a corresponding backup carrier; and if the historical data of the block equipment to be backed up is stored, determining the received difference data between the data of the block equipment to be backed up and the historical data, and backing up the difference data to the corresponding backup carrier.
Further, the operation module is further configured to: judging whether the retention period of the data of the block device to be backed up exceeds a preset retention data window period or not; if so, merging the data of the block devices to be backed up.
In a second aspect, an embodiment of the present invention provides a backup system applied to any one of the above block devices, where the method includes: scanning the to-be-backed-up block equipment in the distributed storage according to a preset backup period through an automatic scanning module; monitoring the online state of the backup carrier in real time through a monitoring module to determine an available backup carrier; determining a target backup carrier of the block device to be backed up from the available backup carriers through a computing module according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to a working module corresponding to the target backup carrier; and receiving the data of the block device to be backed up through the working module, and backing up the received data to a corresponding backup carrier based on a preset backup strategy.
In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program operable on the processor, and the processor implements the method described above when executing the computer program.
In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, where the program code causes the processor to execute the method described above.
The embodiment of the invention provides a backup system and a backup method of block equipment, wherein the system comprises the following steps: the system comprises an automatic scanning module, a computing module, a monitoring module, a plurality of working modules and a plurality of backup carriers; the monitoring module is respectively in communication connection with the automatic scanning module and the computing module; each working module is respectively in communication connection with the computing module and the corresponding backup carrier; the automatic scanning module is used for scanning the to-be-backed-up block equipment in the distributed storage according to a preset backup period; the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier; the computing module is used for determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier; and the working module is used for receiving the data of the block device to be backed up and backing up the received data to the corresponding backup carrier based on a preset backup strategy. In the method, at least one backup carrier corresponding to the block device to be backed up is determined by timing the block device to be backed up and monitoring the online state of the backup carrier in real time, so as to realize full-automatic backup of the block device.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic diagram of a backup system of a block device according to an embodiment of the present invention;
fig. 2 is a schematic diagram of another block device backup system according to a second embodiment of the present invention;
fig. 3 is a flowchart of a block device backup method according to a second embodiment of the present invention.
Icon: 1-an automatic scanning module; 2-a calculation module; 3-a monitoring module; 4-a working module; 5-backup carrier.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The storage of the rbd blocks provided by ceph storage becomes a storage base stone of cloud platforms of most cloud scenes, and the importance of cloud data is self-evident, so that the security needs to be ensured. The invention aims to add an additional safety backup function on the basis of the original ceph block storage, export the block devices in the ceph storage into one or more folders, and can re-import the previously backed-up data when the original data is damaged or lost, thereby reducing the disaster loss to the minimum. Further increasing the security of the data.
For the understanding of the present embodiment, the following detailed description will be given of the embodiment of the present invention.
The first embodiment is as follows:
fig. 1 is a schematic diagram of a backup system of a block device according to an embodiment of the present invention.
Referring to fig. 1, the block device backup system includes an automatic scanning module 1, a computing module 2, a monitoring module 3, a plurality of working modules 4, and a plurality of backup carriers 5; the monitoring module 3 is respectively in communication connection with the automatic scanning module 1 and the computing module 2; each working module 4 is in communication connection with the computing module 2 and the corresponding backup carrier 5.
And the automatic scanning module 1 is used for scanning the to-be-backed-up block devices in the distributed storage according to a preset backup period.
Here, the preset backup period may be freely customized, and the user may be set as the preset backup period (how often the backup is performed) in terms of hours, days, weeks, and months. The user can automatically scan the device of the block to be backed up according to the preset backup period. And obtaining a device list of the blocks to be backed up by adopting a full-automatic directional pool scanning method.
And the monitoring module 3 is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier 5.
Here, the backup carrier is a common folder, and supports multiple File formats such as linux xfs File System, ext4 (Fourth-generation extended File System), ext3 (Third-generation extended File System), ext2 (Second-generation extended File System), and the like, as long as the folder can be read and written, so that the limitation on the hardware storage device is greatly reduced, and the storage device may be a local storage device such as a common hard disk, a disk array, and the like, or a Network File System or a shared File System such as NFS (Network File System ), glusterfs, cephFS File System, and the like.
And the calculating module 2 is used for determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module 4 corresponding to the target backup carrier.
Here, the calculation module pulls the list of available backup carriers in the monitoring module, and obtains the number of carrier nodes and the weight of the carrier nodes of the available backup carriers from the list of available backup carriers. The bearer node weights are determined for the bearer capacities.
Specifically, based on the device list of the blocks to be backed up, calculating the hash value of each device of the blocks to be backed up in the device list of the blocks to be backed up through a static hash function; and determining a target backup carrier of the block equipment to be backed up through pseudo-random operation according to the number of the carrier nodes of the available backup carrier, the weight of the carrier nodes and the hash value of the block equipment to be backed up.
And the working module 4 is used for receiving the data of the block device to be backed up and backing up the received data to a corresponding backup carrier based on a preset backup strategy.
Here, the backup carrier contains one or more folders. Considering that the writing performance and space size of a single backup carrier (folder) are likely to be limited, it may cause a backup failure (insufficient storage space) or a backup not completed within one backup cycle (too slow disk writing speed). The backup system can also perform distributed backup, namely: a plurality of folders are designated, incremental backup processing is carried out on data, and a backup system can automatically carry out pseudo-random algorithm calculation on the distribution condition of the data to be backed up according to the capacity of a carrier (the larger the storage space is, the more the data which are responsible for backup are). Therefore, the data backup can be simultaneously carried out by the multiple carriers, the data of the data to be backed up are not repeated and not omitted, and the backup speed and the flexibility of the backup carriers are greatly improved.
The incremental backup is adopted as a backup mode, if the data in the block device is exported according to the full quantity, various problems such as overlarge data quantity, overlong backup time consumption, insufficient backup space and the like can be caused, so that the backup system utilizes the incremental read-write characteristic of ceph rbd and realizes the long-term backup requirement by a mode of one-time full quantity multiple increment, the backup space is saved, and the backup time is also saved.
In one embodiment, the operational module is further configured to: judging whether the retention period of the data of the block device to be backed up exceeds a preset retention data window period or not; if so, merging the data of the block devices to be backed up.
Here, the preset reserved data window period can be freely customized according to the actual needs of the user, and the user can configure the reserved data window period according to days, weeks, months and years. For example, if the user selects to backup once a day and the retention period selects one month, the backup system will automatically backup once a day and retain up to 30 days of data, and data exceeding the backup period will be automatically merged by the system.
In an embodiment, the monitoring module is further configured to notify the automatic scanning module to scan the to-be-backed-up block device in the distributed storage to trigger a new round of backup process when it is monitored that a newly added backup carrier exists, or a backup carrier is deleted, or a backup carrier is disconnected.
Here, if the backup carrier is already full, the backup system will automatically look for other carriers to attempt backup. If the folder cannot be accessed due to the fact that the storage carrier device is damaged by hardware, the backup system also supports replacement of the backup carrier, after a new carrier is replaced, the backup system can recalculate data distribution of the data to be backed up, and automatically perform data migration and data compensation.
The embodiment of the invention provides a backup system of block equipment, which comprises: the system comprises an automatic scanning module, a computing module, a monitoring module, a plurality of working modules and a plurality of backup carriers; the monitoring module is respectively in communication connection with the automatic scanning module and the computing module; each working module is respectively in communication connection with the computing module and the corresponding backup carrier; the automatic scanning module is used for scanning the block equipment to be backed up in the distributed storage according to a preset backup period; the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier; the computing module is used for determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier; and the working module is used for receiving the data of the block device to be backed up and backing up the received data to a corresponding backup carrier based on a preset backup strategy. In the method, at least one backup carrier corresponding to the block device to be backed up is determined by timing the block device to be backed up and monitoring the online state of the backup carrier in real time, so as to realize full-automatic backup of the block device.
Example two:
fig. 2 is a schematic diagram of another block device backup system according to a second embodiment of the present invention.
Fig. 3 is a flowchart of a block device backup method according to a second embodiment of the present invention.
Referring to fig. 2 and 3, the steps of the block device backup method include:
and step S101, scanning the to-be-backed-up block equipment in the distributed storage according to a preset backup period through an automatic scanning module.
After the backup service is started, the automatic scanning module performs full-automatic directional pool scanning on the distributed storage according to a preset backup period, selects the device to be backed up to generate a device list of the block to be backed up, and simultaneously sends the device list of the block to be backed up to the computing module. The directed pool is poolA, and is a set of all block devices including block devices to be backed up in the distributed storage system, and the block devices to be backed up are rbd1, rbd2, rbd3 and the like.
And S102, monitoring the online state of the backup carrier in real time through the monitoring module, and determining an available backup carrier.
Here, the backup service may be started according to a preset backup period, or may be actively triggered by the monitoring module. When monitoring that a newly added backup carrier exists, or a backup carrier is deleted, or a backup carrier is disconnected, informing an automatic scanning module to scan the to-be-backed block device in the distributed storage to trigger a new round of backup process.
Step S103, determining a target backup carrier of the block device to be backed up from the available backup carriers through the calculating module according to the number of the carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier.
Here, the computing module pulls the list of available backup carriers in the monitoring module according to the list of the block devices to be backed up in the automatic scanning module. And calculating the final backup carrier position of each block device to be backed up through pseudo-random operation, then sorting and packaging the block device information to be backed up corresponding to each backup carrier, and respectively sending the information to the working modules corresponding to the backup carriers.
And step S104, receiving the data of the block device to be backed up through the working module, and backing up the received data to a corresponding backup carrier based on a preset backup strategy.
Here, the working module receives data of the block device to be backed up and judges whether historical data of the block device to be backed up is stored in a backup carrier corresponding to the working module; if the historical data of the block equipment to be backed up is not stored, backing up the data of the block equipment to be backed up to a corresponding backup carrier; and if the historical data of the block equipment to be backed up is stored, determining the received difference data between the data of the block equipment to be backed up and the historical data, and backing up the difference data to the corresponding backup carrier. The backup carrier contains one or more folders.
Specifically, after receiving data of a block device to be backed up, the working module self-detects whether historical data owned by a current backup carrier is consistent with the received data of the block device to be backed up, and after the detected historical data is different from the data of the block device to be backed up, full-quantity, deletion and incremental backup are respectively carried out on newly added, deleted and original backup data to the backup carrier according to a strategy.
The embodiment of the invention provides a backup method of block equipment, and the system comprises the following steps: the system comprises an automatic scanning module, a computing module, a monitoring module, a plurality of working modules and a plurality of backup carriers; the monitoring module is respectively in communication connection with the automatic scanning module and the computing module; each working module is respectively in communication connection with the computing module and the corresponding backup carrier; the automatic scanning module is used for scanning the to-be-backed-up block equipment in the distributed storage according to a preset backup period; the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier; the computing module is used for determining a target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier; and the working module is used for receiving the data of the block device to be backed up and backing up the received data to the corresponding backup carrier based on a preset backup strategy. In the method, at least one backup carrier corresponding to the block device to be backed up is determined by timing the block device to be backed up and monitoring the online state of the backup carrier in real time, so as to realize full-automatic backup of the block device.
The embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when the processor executes the computer program, the steps of the backup method for the block device provided in the foregoing embodiment are implemented.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to perform the steps of the backup method for the block device in the foregoing embodiment.
The computer program product provided in the embodiment of the present invention includes a computer-readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the method embodiment, which is not described herein again.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplification of description, but do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.
Claims (10)
1. A backup system for a block device, the system comprising: the system comprises an automatic scanning module, a computing module, a monitoring module, a plurality of working modules and a plurality of backup carriers; the monitoring module is respectively in communication connection with the automatic scanning module and the computing module; each working module is respectively in communication connection with the computing module and the corresponding backup carrier;
the automatic scanning module is used for scanning the block equipment to be backed up in the distributed storage according to a preset backup period;
the monitoring module is used for monitoring the online state of the backup carrier in real time and determining an available backup carrier;
the computing module is used for determining a target backup carrier of the block device to be backed up from the available backup carrier according to the number of the carrier nodes of the available backup carrier, the weight of the carrier nodes and the hash value of the block device to be backed up, and sending data in the block device to be backed up to the working module corresponding to the target backup carrier;
and the working module is used for receiving the data of the block device to be backed up and backing up the received data to a corresponding backup carrier based on a preset backup strategy.
2. The backup system of block devices according to claim 1, wherein the monitoring module is further configured to notify the automatic scanning module to scan the block devices to be backed up in the distributed storage to trigger a new round of backup process when it is monitored that a newly added backup carrier exists, or a backup carrier is deleted, or a backup carrier is disconnected.
3. The backup system for block devices according to claim 1, wherein the step of scanning the block devices to be backed up in the distributed storage according to the preset backup cycle comprises:
performing full-automatic directional pool scanning on the distributed storage according to a preset backup period to obtain a device list of blocks to be backed up;
the directed pool is a set of all block devices in the distributed storage system, wherein the block devices to be backed up are included in the set.
4. The backup system of a block device according to claim 3, wherein the step of determining the target backup carrier of the block device to be backed up from the available backup carriers according to the number of carrier nodes of the available backup carriers, the weight of the carrier nodes, and the hash value of the block device to be backed up comprises:
calculating the hash value of each block device to be backed up in the block device list to be backed up through a static hash function based on the block device list to be backed up;
and determining a target backup carrier of the block equipment to be backed up through pseudo-random operation according to the carrier node number of the available backup carrier, the carrier node weight and the hash value of the block equipment to be backed up.
5. The backup system of a block device according to any of claims 1 to 4, wherein said backup carrier contains one or more folders.
6. The backup system of the block device according to claim 1, wherein the step of receiving the data of the block device to be backed up and backing up the received data to the corresponding backup carrier based on a preset backup policy comprises:
receiving data of the block device to be backed up, and judging whether historical data of the block device to be backed up is stored in a backup carrier corresponding to the working module;
if the historical data of the block equipment to be backed up is not stored, backing up the data of the block equipment to be backed up to a corresponding backup carrier;
if the historical data of the block device to be backed up is stored, determining the received difference data between the data of the block device to be backed up and the historical data, and backing up the difference data to a corresponding backup carrier.
7. The backup system of the block device of claim 1, wherein the work module is further configured to:
judging whether the retention period of the data of the block device to be backed up exceeds a preset retention data window period or not;
if so, merging the data of the block devices to be backed up.
8. A backup method of a block device, applied to the backup system of a block device according to any one of claims 1 to 7, the method comprising:
scanning the block equipment to be backed up in the distributed storage according to a preset backup period through an automatic scanning module;
monitoring the online state of the backup carrier in real time through a monitoring module, and determining an available backup carrier;
determining a target backup carrier of the block device to be backed up from the available backup carrier according to the number of the carrier nodes of the available backup carrier, the weight of the carrier nodes and the hash value of the block device to be backed up by a computing module, and sending data in the block device to be backed up to a working module corresponding to the target backup carrier;
and receiving the data of the block device to be backed up through a working module, and backing up the received data to a corresponding backup carrier based on a preset backup strategy.
9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program operable on the processor, wherein the processor implements the method of claim 8 when executing the computer program.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program executes the steps of the method according to claim 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211205460.9A CN115268801B (en) | 2022-09-30 | 2022-09-30 | Backup system and method for block device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211205460.9A CN115268801B (en) | 2022-09-30 | 2022-09-30 | Backup system and method for block device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115268801A CN115268801A (en) | 2022-11-01 |
| CN115268801B true CN115268801B (en) | 2023-01-10 |
Family
ID=83758167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211205460.9A Active CN115268801B (en) | 2022-09-30 | 2022-09-30 | Backup system and method for block device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115268801B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104902009A (en) * | 2015-04-27 | 2015-09-09 | 浙江大学 | Erasable encoding and chained type backup-based distributed storage system |
| CN109144406A (en) * | 2017-06-28 | 2019-01-04 | 华为技术有限公司 | Metadata storing method, system and storage medium in distributed memory system |
| CN111930556A (en) * | 2020-10-14 | 2020-11-13 | 广东睿江云计算股份有限公司 | Backup method and device for distributed storage data |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8595188B2 (en) * | 2009-11-06 | 2013-11-26 | International Business Machines Corporation | Operating system and file system independent incremental data backup |
| US10909005B2 (en) * | 2019-02-25 | 2021-02-02 | Datto, Inc. | Object-level metadata-preserving cross heterogeneous operating systems backup and restore apparatuses, methods and systems |
-
2022
- 2022-09-30 CN CN202211205460.9A patent/CN115268801B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104902009A (en) * | 2015-04-27 | 2015-09-09 | 浙江大学 | Erasable encoding and chained type backup-based distributed storage system |
| CN109144406A (en) * | 2017-06-28 | 2019-01-04 | 华为技术有限公司 | Metadata storing method, system and storage medium in distributed memory system |
| CN111930556A (en) * | 2020-10-14 | 2020-11-13 | 广东睿江云计算股份有限公司 | Backup method and device for distributed storage data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115268801A (en) | 2022-11-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407040B (en) | A kind of duplicating remote data method and system | |
| US9075838B2 (en) | Method and apparatus for an improved file repository | |
| EP2414928B1 (en) | Data redistribution in data replication systems | |
| CN109597567B (en) | Data processing method and device | |
| CN103535014B (en) | A kind of network store system, data processing method and client | |
| CN101137984A (en) | Systems, methods, and software for distributed loading of databases | |
| CN111857592A (en) | Data storage method and device based on object storage system and electronic equipment | |
| CN103986779A (en) | Application data transmission method, device and terminal equipment | |
| CN111399760B (en) | NAS cluster metadata processing method and device, NAS gateway and medium | |
| CN115756955A (en) | Data backup and data recovery method and device and computer equipment | |
| CN109189326B (en) | Management method and device of distributed cluster | |
| CN115268801B (en) | Backup system and method for block device | |
| CN113051428B (en) | Method and device for back-up storage at front end of camera | |
| CN112925672A (en) | Data recovery method, device, equipment and storage medium | |
| CN112596959A (en) | Distributed storage cluster data backup method and device | |
| CN116737466A (en) | Backup processing method, device, system, electronic equipment and readable storage medium | |
| CN113296685A (en) | Data processing method and device and computer readable storage medium | |
| CN103685359B (en) | Data processing method and device | |
| CN111176886A (en) | Database mode switching method and device and electronic equipment | |
| CN115470041A (en) | Data disaster recovery management method and device | |
| CN107025150A (en) | A kind of system and method for realizing the control of data backup real-time recovery | |
| CN113821176B (en) | Data migration processing method, device and storage medium | |
| CN117742602A (en) | Cluster data migration method and device under private cloud and electronic equipment | |
| CN107864096B (en) | Path adjusting method and device | |
| CN113031861B (en) | Data storage method, device, equipment and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |