CN110716829A - Block storage backup method, terminal and storage medium based on Ceph technology - Google Patents
Block storage backup method, terminal and storage medium based on Ceph technology Download PDFInfo
- Publication number
- CN110716829A CN110716829A CN201910959183.2A CN201910959183A CN110716829A CN 110716829 A CN110716829 A CN 110716829A CN 201910959183 A CN201910959183 A CN 201910959183A CN 110716829 A CN110716829 A CN 110716829A
- Authority
- CN
- China
- Prior art keywords
- ceph
- image
- technology
- backup
- rbd
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a block storage backup method, a terminal and a storage medium based on a Ceph technology, belongs to data storage, and aims to solve the technical problem of how to realize incremental backup of data and distributed storage of backup based on the Ceph. The method provides block storage service through a render project of OpenStack, integrates Ceph into the render project of OpenStack, and backs up data to a Ceph cluster through RBD snapshot technology of Ceph and in an incremental back-up mode. The terminal comprises a processor and a memory for storing a computer program, the processor being configured to invoke program instructions to perform the above-described method. The storage medium stores a computer program comprising program instructions which, when executed by a processor, the processor performs the above-described method.
Description
Technical Field
The invention relates to the field of data storage, in particular to a block storage backup method, a terminal and a storage medium based on a Ceph technology.
Background
With the arrival of the informatization wave, the informatization data well-spraying type of all walks of life increases. According to the report of data era 2025 issued by IDC, data generated worldwide every year will increase from 33ZB in 2018 to 175ZB, which is equivalent to 491EB generated daily. Data security is more and more concerned by people, so a complete disaster recovery backup system is more and more important.
The shader project of the OpenStack (open source cloud computing management platform) is mainly used for providing block storage services, and through the shader project, a user can integrate different storage back ends and provide storage resource services to the outside through a unified OpenStack API interface. Ceph is a unified, distributed storage system with excellent performance, high reliability and high scalability. By "unified" is meant that a set of storage systems simultaneously provides the user with three functions of object storage, block storage, and file storage. The distributed storage in the Ceph system means the real system expandability without a central structure and a theoretical upper limit, and the problem of performance bottleneck in the traditional Scale-up type storage in the process of capacity expansion is really solved. While the COW (Copy-on-Write) feature provided by Ceph provides us with a backup strategy based on snapshot technology of rbd (rados Block devices).
Based on the above analysis, how to implement incremental backup of data and distributed storage of backup based on Ceph is a technical problem to be solved.
Disclosure of Invention
The technical task of the invention is to provide a block storage backup method, a terminal and a storage medium based on a Ceph technology to solve the problem of how to realize incremental backup of data and distributed storage of backup based on the Ceph.
In a first aspect, the invention provides a block storage backup method based on a Ceph technology, which provides a block storage service through a circle project of an OpenStack, integrates the Ceph into the circle project of the OpenStack, and backs up data into a Ceph cluster through an RBD snapshot technology of the Ceph in an incremental backup mode.
In the above real-time method, incremental backup is performed by using a Ceph rbd (rados Block devices) snapshot technology, and the backed-up data is stored in a Ceph cluster. Because the mode of incremental backup is adopted, the data backup speed is greatly increased, the storage capacity occupied by backup data is greatly reduced, and the cost is reduced. The Ceph cluster adopts a multi-copy mechanism, and an excellent CRUSH (controlled Replication UnderScalable hashing) algorithm is added to perform distributed storage on data, so that the security of the data is ensured.
Preferably, Ceph is integrated into the folder project of OpenStack, serves as the storage back-end of OpenStack, and is configured as the back-end driver of the folder-backup service.
Preferably, the method for backing up data into the Ceph cluster by adopting an incremental backup method through the RBD snapshot technology of the Ceph comprises the following steps:
creating a block device mirror image with the name of image and writing basic data;
all data in the block device mirror image are derived through an RBD command of Ceph and are recorded as image _ backup;
writing new data into the block device image, and creating a snapshot rbd/image @ snap1 by using a snapshot technology of Ceph;
incremental data between image _ backup and rbd/image @ snap1 is derived through a Ceph command, is recorded as image _ to _ snap1, and metadata is backed up.
Preferably, after creating the snapshot rbd/image @ snap1 by the snapshot technology of Ceph, calling a Ceph protection command to protect the snapshot rbd/image @ snap1 and calling a Ceph clone command to clone the snapshot rbd/image @ snap 1.
Preferably, when the backup is restored, the restoration is performed in sequence according to the sequence of the backup files.
Preferably, a block device image named image is created in the RBD pool.
In a second aspect, the present invention provides a terminal, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, and the memory is used for storing a computer program, where the computer program includes program instructions, and the processor is configured to call the program instructions to execute the block storage backup method based on the Ceph technology according to any one of the first aspects.
In a third aspect, the present invention provides a storage medium which is a computer-readable storage medium storing a computer program comprising program instructions which, when executed by a processor, the processor performs the block storage backup method based on the Ceph technique according to any one of the first aspect.
The block storage backup method, the terminal and the storage medium based on the Ceph technology have the following advantages that:
1. the backed-up data is stored in the Ceph cluster, and compared with the data stored in other media, the data security and reliability are improved;
2. when the backed-up data reaches the upper limit of the capacity of the storage medium, the Ceph cluster can quickly and conveniently expand the capacity;
3. compared with the mode of adopting full backup or differential backup, the mode of adopting incremental backup has higher efficiency of incremental backup and smaller occupied storage space.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a block storage backup method based on the Ceph technology in embodiment 1.
Detailed Description
The present invention is further described in the following with reference to the drawings and the specific embodiments so that those skilled in the art can better understand the present invention and can implement the present invention, but the embodiments are not to be construed as limiting the present invention, and the embodiments and the technical features of the embodiments can be combined with each other without conflict.
The embodiment of the invention provides a block storage backup method, a terminal and a storage medium based on a Ceph technology, which are used for solving the technical problem of how to realize incremental backup of data and distributed storage of backup based on the Ceph.
Example 1:
according to the block storage backup method based on the Ceph technology, the block storage service is provided through the Binder project of the OpenStack, the Ceph is integrated into the Binder project of the OpenStack, and data are backed up into a Ceph cluster through the RBD snapshot technology of the Ceph in an incremental backup mode.
As shown in fig. 1, the method comprises the following steps:
s100, integrating the Ceph into a render project of the OpenStack, and taking the Ceph as a storage back end of the OpenStack;
s200, configuring Ceph as a rear-end drive of a render-backup service;
s300, creating a Volume (translated into a Volume) with the name of image in the RBD pool, and writing basic data;
s400, exporting all data in the mirror image of the block device through an RBD command of Ceph, namely full backup data, and recording as image _ backup;
s500, writing new data into the block device mirror image, and creating a snapshot rbd/image @ snap1 through a snapshot technology of Ceph;
s600, incremental data between image _ backup and rbd/image @ snap1 is derived through a Ceph command and recorded as image _ to _ snap1, and metadata is backed up.
If the snapshot is to be read and written, the corresponding snapshot needs to be cloned and the cloned snapshot needs to be protected first. In step S500, after creating the snapshot rbd/image @ snap1 by the snapshot technology of Ceph, the Ceph protection command protection snapshot rbd/image @ snap1 is called, and the Ceph clone command clone snapshot rbd/image @ snap1 is called.
On the basis of the above embodiment, when the backup is restored, the restoration is performed in sequence according to the sequence of the backup files.
The bottom layer of the Block storage backup method based on the Ceph technology is realized based on the Ceph technology, the Block storage service is provided by using a render project of OpenStack, incremental backup is carried out by using a snapshot technology of Ceph RBD (RADOS Block Devices), and the backup data is stored in a Ceph cluster. Because the mode of incremental backup is adopted, the data backup speed is greatly increased, the storage capacity occupied by backup data is greatly reduced, and the cost is reduced; the Ceph cluster adopts a multi-copy mechanism, and an excellent CRUSH (controlled Replication UnderScalable hashing) algorithm is added to perform distributed storage on data, so that the security of the data is ensured.
Example 2:
the terminal comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected with each other, the memory is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the block storage backup method based on the Ceph technology disclosed in embodiment 1. The terminal may be an electronic computer.
Example 3:
a storage medium of the present invention is a computer-readable storage medium, in which a computer program is stored, the computer program including program instructions, when executed by a processor, the processor executing the block storage backup method based on the Ceph technology disclosed in embodiment 1. The storage medium includes a mechanical hard disk, a solid state hard disk, and the like.
The above-mentioned embodiments are merely preferred embodiments for fully illustrating the present invention, and the scope of the present invention is not limited thereto. The equivalent substitution or change made by the technical personnel in the technical field on the basis of the invention is all within the protection scope of the invention. The protection scope of the invention is subject to the claims.
Claims (8)
1. The block storage backup method based on the Ceph technology is characterized in that block storage service is provided through a circle project of OpenStack, Ceph is integrated into the circle project of OpenStack, and data is backed up into a Ceph cluster through the RBD snapshot technology of Ceph in an incremental backup mode.
2. The block storage backup method based on Ceph technology of claim 1, wherein Ceph is integrated into a render project of OpenStack, Ceph is used as a storage back-end of OpenStack, and Ceph is configured as a back-end driver of a render-backup service.
3. The block storage backup method based on the Ceph technology of claim 1, wherein the data is backed up to the Ceph cluster by an incremental backup method through the RBD snapshot technology of the Ceph, and the method comprises the following steps:
creating a block device mirror image with the name of image and writing basic data;
all data in the block device mirror image are derived through an RBD command of Ceph and are recorded as image _ backup;
writing new data into the block device image, and creating a snapshot rbd/image @ snap1 by using a snapshot technology of Ceph;
incremental data between image _ backup and rbd/image @ snap1 is derived through a Ceph command, is recorded as image _ to _ snap1, and metadata is backed up.
4. The block storage backup method based on Ceph technology of claim 3, wherein after creating the snapshot rbd/image @ snap1 by the snapshot technology of Ceph, calling Ceph protection command to protect the snapshot rbd/image @ snap1 and calling Ceph clone command to clone the snapshot rbd/image @ snap 1.
5. The block storage backup method based on the Ceph technology as claimed in claim 3 or 4, wherein the backup is restored sequentially according to the sequence of the backup files.
6. The method for block storage backup based on Ceph technology as claimed in claim 3 or 4, wherein a block device image named image is created in RBD pool.
7. Terminal, characterized in that it comprises a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, the memory being intended to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to execute a block storage backup method based on Ceph technology according to any of claims 1 to 6.
8. Storage medium, characterized in that it is a computer-readable storage medium, which stores a computer program comprising program instructions which, when executed by a processor, the processor performs the block storage backup method based on the Ceph technique according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910959183.2A CN110716829A (en) | 2019-10-10 | 2019-10-10 | Block storage backup method, terminal and storage medium based on Ceph technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910959183.2A CN110716829A (en) | 2019-10-10 | 2019-10-10 | Block storage backup method, terminal and storage medium based on Ceph technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110716829A true CN110716829A (en) | 2020-01-21 |
Family
ID=69211372
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910959183.2A Pending CN110716829A (en) | 2019-10-10 | 2019-10-10 | Block storage backup method, terminal and storage medium based on Ceph technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110716829A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111399774A (en) * | 2020-03-09 | 2020-07-10 | 星辰天合（北京）数据科技有限公司 | Data processing method and device based on snapshot under distributed storage system |
| CN111597078A (en) * | 2020-05-15 | 2020-08-28 | 山东汇贸电子口岸有限公司 | Timed backup method and system for copying ceph block storage data to object storage |
| CN111654410A (en) * | 2020-04-28 | 2020-09-11 | 长沙证通云计算有限公司 | Gateway request monitoring method, device, equipment and medium |
| CN111737048A (en) * | 2020-05-29 | 2020-10-02 | 苏州浪潮智能科技有限公司 | Openstack docking storage virtual machine volume backup processing method and device |
| CN111984469A (en) * | 2020-08-07 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Incremental snapshot capacity expansion method and device |
| CN113806145A (en) * | 2021-09-10 | 2021-12-17 | 杭州信核数据科技股份有限公司 | Backup recovery method and device of OpenStack virtualization platform based on Ceph storage |
| CN114003439A (en) * | 2021-12-30 | 2022-02-01 | 深圳市科力锐科技有限公司 | Data backup method, device, equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107197022A (en) * | 2017-06-02 | 2017-09-22 | 华南理工大学 | OpenStack storage optimization methods and system |
-
2019
- 2019-10-10 CN CN201910959183.2A patent/CN110716829A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107197022A (en) * | 2017-06-02 | 2017-09-22 | 华南理工大学 | OpenStack storage optimization methods and system |
Non-Patent Citations (3)
| Title |
|---|
| OSC_7DHD4AD7: "ceph中rbd的增量备份和恢复", 《开源博客，HTTPS://MY.OSCHINA.NET/U/4277648/BLOG/3246222》 * |
| SOLAR: "openstack-- cinder-backup 利用ceph实现增量备份", 《HTTPS://ZHANGCHENCHEN.GITHUB.IO/2017/05/09/OPENSTACK-CINDER-INCREMENTAL-BACKUP-WITH-CEPH/》 * |
| 杨皓森等: ""面向OpenStack/Ceph的虚拟机备份系统研究"", 《计算机系统应用》 * |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111399774A (en) * | 2020-03-09 | 2020-07-10 | 星辰天合（北京）数据科技有限公司 | Data processing method and device based on snapshot under distributed storage system |
| CN111399774B (en) * | 2020-03-09 | 2021-09-14 | 星辰天合（北京）数据科技有限公司 | Data processing method and device based on snapshot under distributed storage system |
| CN111654410A (en) * | 2020-04-28 | 2020-09-11 | 长沙证通云计算有限公司 | Gateway request monitoring method, device, equipment and medium |
| CN111597078A (en) * | 2020-05-15 | 2020-08-28 | 山东汇贸电子口岸有限公司 | Timed backup method and system for copying ceph block storage data to object storage |
| CN111737048A (en) * | 2020-05-29 | 2020-10-02 | 苏州浪潮智能科技有限公司 | Openstack docking storage virtual machine volume backup processing method and device |
| CN111737048B (en) * | 2020-05-29 | 2022-06-10 | 苏州浪潮智能科技有限公司 | Openstack docking storage virtual machine volume backup processing method and device |
| CN111984469A (en) * | 2020-08-07 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Incremental snapshot capacity expansion method and device |
| CN111984469B (en) * | 2020-08-07 | 2022-08-19 | 苏州浪潮智能科技有限公司 | Incremental snapshot capacity expansion method and device |
| CN113806145A (en) * | 2021-09-10 | 2021-12-17 | 杭州信核数据科技股份有限公司 | Backup recovery method and device of OpenStack virtualization platform based on Ceph storage |
| CN114003439A (en) * | 2021-12-30 | 2022-02-01 | 深圳市科力锐科技有限公司 | Data backup method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110716829A (en) | Block storage backup method, terminal and storage medium based on Ceph technology | |
| JP6495568B2 (en) | Method, computer readable storage medium and system for performing incremental SQL server database backup | |
| US20200329101A1 (en) | Enhanced network attached storage (nas) services interfacing to cloud storage | |
| US8856080B2 (en) | Backup using metadata virtual hard drive and differential virtual hard drive | |
| US7979649B1 (en) | Method and apparatus for implementing a storage lifecycle policy of a snapshot image | |
| US8015157B2 (en) | File sharing system, file server, and method for managing files | |
| US20200293571A1 (en) | Targeted search of backup data using facial recognition | |
| US12056384B2 (en) | Skipping data backed up in prior backup operations | |
| CN101398775B (en) | Techniques for virtual archiving | |
| US12019665B2 (en) | Targeted search of backup data using calendar event data | |
| JP2006209636A (en) | Method for maintaining snapshot | |
| CN106407040A (en) | Remote data copy method and system | |
| CN102685194B (en) | Memory device migration and the method and system be redirected | |
| US10802928B2 (en) | Backup and restoration of file system | |
| JP2016051294A (en) | Backup method, restoration method, storage system, and program | |
| CN104714858A (en) | Data backup method, data recovery method and device | |
| CN108351821A (en) | Data reconstruction method and storage device | |
| JP5868986B2 (en) | Recovery by item | |
| CN104216801A (en) | Data copying method oriented to virtual environment and data copying system oriented to virtual environment | |
| US20220179985A1 (en) | User entitlement management system | |
| CN112800019A (en) | Data backup method and system based on Hadoop distributed file system | |
| CN106528338A (en) | Remote data replication method, storage equipment and storage system | |
| US9251020B1 (en) | Systems and methods for file-level replication | |
| CN114328005B (en) | Method and system for incremental backup of container data | |
| CN101937378A (en) | Method for carrying out back-up protection on data of storage equipment and computer system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200121 |
|
| RJ01 | Rejection of invention patent application after publication |