CN112181734B - Multi-back-end storage method based on circular encoder - Google Patents
Multi-back-end storage method based on circular encoder Download PDFInfo
- Publication number
- CN112181734B CN112181734B CN202011236892.7A CN202011236892A CN112181734B CN 112181734 B CN112181734 B CN 112181734B CN 202011236892 A CN202011236892 A CN 202011236892A CN 112181734 B CN112181734 B CN 112181734B
- Authority
- CN
- China
- Prior art keywords
- backup
- storage
- type
- service
- name
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1456—Hardware arrangements for backup
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1469—Backup restoration techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the technical field of cloud computing, and particularly provides a cinder-based multi-back-end storage method, which comprises the following steps of S1, creating a backup type: binding with a storage strategy of a storage back end, and selecting the storage strategy used by the storage back end to create a backup by specifying a backup type when the backup is created; s2, registering information contained in the backup type into a service table; s3, scheduling the backup to a storage medium; s4, adding a backing type in the cinder database; s5, modifying the host value of the cis-backup into host @ backup nd by the service table; s6, creating a backup, wherein backing type and az need to be specified; s7, the circulating is dispatched to the circulating-backing according to the backing _ name associated with the az and the backing type by the circulating; and S8, selecting a storage strategy by the circular-backup according to the storage _ policy associated with the backup type. Compared with the prior art, the method can realize different storage strategies from backup storage to backup storage.
Description
Technical Field
The invention relates to the technical field of cloud computing, and particularly provides a multiple-back-end storage method based on a shader.
Background
At present, a cloud computing technology provides various cloud computing resources for a cloud computing user through resource cloud, and a cloud hard disk technology belongs to an integral part of the cloud computing technology and provides elastic storage resources for the user through a cloud platform technology. The cloud hard disk backup technology creates backup for the cloud hard disk, can realize that the cloud hard disk is rolled back to the state of creating the backup moment, ensures the safety of user service data to the maximum extent, and ensures the safety of the cloud hard disk of a user.
Currently, a method for realizing cloud hard disk data backup by a shader mainly comprises two modes, namely a snapshot technology and a backup technology. The snapshot technology is supported by a circder volume, and a snapshot created by a cloud hard disk depends on the cloud hard disk. If the cloud hard disk needs to be deleted, all snapshots created by the cloud hard disk are deleted. This greatly limits the business requirements of the cloud hard disk lifecycle management and users by the shaders.
The other technology is a backup technology which is realized through a component chopper backup, and compared with a snapshot technology, the technology releases the dependency relationship between backup and a cloud hard disk. The user can establish the backup for the cloud hard disk, does not need to care about the state of the cloud hard disk, and can restore the cloud hard disk to the moment of establishing the backup according to the business requirement without caring about the state of the cloud hard disk. Currently, the shader has implemented the storage of the backup of the cloud hard disk into different available domains, and a user can store the service data stored in the cloud hard disk into different available domains according to the service requirement of the user.
The access performance of the backup is affected by the storage medium. If a backup is stored on an SSD type of media, its access speed will be greater than the access speed stored on a SATA type of media. In order to realize the performance that a user can autonomously select a backup, the access path of the backup needs to be scheduled, and the backup is stored in different media. The method is characterized by changing the service scene of a user and the diversity of the cloud hard disk storage media. The user needs to store the backup in different media to distinguish the performance of the backup, including the speed of creation and recovery of the backup, etc. However, current support of a cloud hard disk by a shader is limited to storing backups in different available areas, and cannot meet business requirements of users.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a strong-practicability multi-backend storage method based on the shader.
The technical scheme adopted by the invention for solving the technical problems is as follows:
a multi-back-end storage method based on a shader comprises the following steps:
s1, creating a backup upType: binding with a storage strategy of a storage back end, and selecting the storage strategy used by the storage back end to create a backup by specifying a backup type when the backup is created;
s2, registering information contained in the backupType into a service table;
s3, scheduling the backup to a storage medium;
s4, adding a backup type into the cider database;
s5, modifying the host value of the cis-backup into host @ backup nd by the service table;
s6, creating a backup, wherein backing type and az need to be specified;
s7, the circulating is dispatched to the circulating-backing according to the backing _ name associated with the az and the backing type by the circulating;
and S8, selecting a storage strategy by the circular-backup according to the storage _ policy associated with the backup type.
Further, in step S1, the backup type mainly includes backup _ name and storage _ policy. In the process of creating backup, the circle selects a circle-backup through the backup _ name, and the circle-backup selects a storage strategy through the storage _ policy.
Further, if the backup type does not specify backup _ name, randomly selecting an available circular-backup under az;
if backing UpType does not specify storage _ policy, then the default storage policy is selected.
Further, in step S2, initializing a binary, a host, and a topic, obtaining a service according to the binary and the host, and if not, creating the service;
if yes, judging whether the service is a circular-backup, and if yes, establishing/starting an rcpserver;
if not, initializing a host = host @ background # backup _ type of the service, inquiring a load record of the service in a load table, and if not, creating the load record; if the service exists, initializing the maximum load and the standard load of the service, and creating/starting the rcpserver.
Further, in step S3, a request is created, whether the value of Backup _ type is normal high, and if not, an abnormal exit is thrown;
if yes, selecting service with the same host as the volume to create backup, and if yes, selecting all services with the host, az and backup _ type meeting the requirements;
if not, selecting all services of which az and backup _ type meet the requirements, judging whether available services exist or not, and if not, throwing out an abnormal exit;
if yes, the service selects the storage medium according to the backup _ type to create backup.
Further, in step S4, a backup type is added to the finder database, including id, name, backup _ name and storage _ policy fields.
Further, the id is uuid of backup type; name is the name of backupType; the backup _ name is a storage back end name corresponding to the service, and the value of the backup _ name is swift or posix; storage _ policy is a storage policy of the storage backend.
Further, if the backup _ name is swift, the storage _ policy = standby _ IA or default-place, and if the backup _ name is posix, the value of the storage _ policy is used as a subdirectory, and backup is stored under the root directory/storage _ policy.
Compared with the prior art, the multi-back-end storage method based on the cider has the following outstanding beneficial effects:
according to the method, the resource backup type is introduced, so that backup storage can be carried out to different backings for storage, and different storage strategies can be realized. The user can store the cloud hard disk backup in different storage media in the same available domain according to the service scene of the user, and the performance difference of the backup is realized.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a frame diagram of a cinder-based multi-backend storage method;
FIG. 2 is a schematic flow chart of service registration in a plural back-end storage method based on a shader;
FIG. 3 is a schematic flow chart of multi-backend scheduling in a plural-backend storage method based on a shader;
FIG. 4 is a prior art frame diagram of a cider-based multi-back-end storage method.
Detailed Description
The present invention will be described in further detail with reference to specific embodiments in order to better understand the technical solutions of the present invention. It should be apparent that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A preferred embodiment is given below:
as shown in fig. 4, the current cinder supports cloud hard disk backup. The user can store the cloud hard disk backup into different available domains, different storage back ends including ceph, SAN and the like can be connected under different az (available domains), and the cloud hard disk backup can be stored into different storage media. However, it cannot implement storage of different cloud hard disk backups in different storage media in the same available domain.
As shown in fig. 1, in order to implement the storage of the backup into different storage backend and storage medium under the same available area, a circular-based multi-backend storage method is proposed herein.
And the backup type is a backup type and is bound with a storage strategy of the storage back end, and the backup is created by selecting the storage strategy used by the storage back end through specifying the backup type when the backup is created.
The backupType mainly contains backup _ name and storage _ policy. In the process of creating backup, the circle selects a circle-backup through the backup _ name, and the circle-backup selects a storage strategy through the storage _ policy. The render-backup may support multiple storage strategies or only one storage strategy. If the backup type does not specify the backup _ name, randomly selecting the available circular-backup under az; if backing UpType does not specify storage _ policy, then a default storage policy is selected (swift selects default-placement, posix selects root directory).
As shown in fig. 2, it is shown how to register the information contained in the backup type into the service table when the sender-backup is started, so that the last started service has a tag of the back-end information, so that subsequent back-end scheduling can implement scheduling according to the tag in the service.
The method comprises the following specific steps:
initializing a binary, a host and a topic, acquiring a service according to the binary and the host, and if the binary, the host and the host do not exist, establishing the service;
if yes, judging whether the service is a circular-backup, and if yes, establishing/starting an rcpserver;
if not, initializing a host = host @ background # backup _ type of the service, inquiring a load record of the service in the load table, and if not, creating the load record; if the service exists, initializing the maximum load and the standard load of the service, and creating/starting the rcpserver.
As shown in fig. 3, it adds the scheduling of the storage back-end and the storage policy on the basis of the scheduling of the original available domain, and implements the scheduling of backup to the storage medium.
The method comprises the following specific steps:
creating a request, judging whether the value of Backup _ type is normal high or not, and if not, throwing an abnormal exit;
if yes, selecting a service with the same host as the volume to create a backup, and if yes, selecting all services with the host, az and backup _ type meeting the requirements;
if not, selecting all services of which az and backup _ type meet the requirements, judging whether available services exist or not, and if not, throwing out an abnormal exit;
if yes, the service selects the storage medium according to the backup _ type to create backup.
Add backupType (Table 1) to the circular database, containing id, name, backup _ name and storage _ policy fields. id is uuid of the backupType; the name is the name of the backup type, and the name is globally unique; the backup _ name is a storage back end name corresponding to the service, and the value of the backup _ name is swift or posix; the storage _ policy is a storage policy of a storage back end, if the backup _ name is swift, the storage _ policy = STANDARD _ IA or default-place, if the backup _ name is positx, the value of the storage _ policy is used as a subdirectory, and backup is stored under the root directory/storage _ policy.
| Field | Type | Null | Default | Extra |
| id | int(11) | NO | NULL | auto_increment |
| name | varchar(255) | YES | NULL | |
| backend_name | varchar(255) | YES | NULL | |
| storage_policy | varchar(255) | YES | NULL |
Table 1
In the circumferentially database, the services table (Table 2) modifies the host value of circumferentially-backup to host @ backup. And updating the services table when Service is started/created.
| Field | Type | Null | key | Default | Extra |
| created_at | datetime | YES | NULL | ||
| updated_at | datetime | YES | NULL | ||
| deleted_at | datetime | YES | NULL | ||
| deleted | tinyint(1) | YES | NULL | ||
| id | int(11) | NO | PRI | NULL | auto_increment |
| host | varchar(255) | YES | NULL | ||
| binary | varchar(255) | YES | NULL | ||
| topic | varchar(255) | YES | NULL | ||
| report_count | int(11) | NO | NULL | ||
| disabled | tinyint(1) | YES | NULL | ||
| availability_zone | varchar(255) | YES | NULL | ||
| disabled_reason | varchar(255) | YES | NULL | ||
| modified_at | datetime | YES | NULL | ||
| rpc_current_version | varchar(36) | YES | NULL | ||
| object_current_version | varchar(36) | YES | NULL | ||
| replication_status | varchar(36) | YES | NULL | ||
| frozen | tinyint(1) | YES | NULL | ||
| cluster_name | varchar(255) | YES | NULL | ||
| uuid | varchar(36) | YES | UNI | NULL |
Table 2
A backup is created, requiring specification of backupType and az.
And the circular is dispatched to the circular-backup according to the backup _ name associated with the az and the backup type by the circular.
The circular-backup selects a storage strategy according to storage _ policy associated with the backup type.
The above embodiments are only specific cases of the present invention, and the protection scope of the present invention includes but is not limited to the above embodiments, and any suitable changes or substitutions according to the claims of the present invention for a cider-based multi-backend storage method and by any person of ordinary skill in the art should fall within the protection scope of the present invention.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (5)
1. A multi-back-end storage method based on a shader is characterized by comprising the following steps:
s1, creating Backup _ type: binding with a storage strategy of a storage back end, and selecting the storage strategy used by the storage back end to create a Backup by specifying a Backup _ type when the Backup is created;
s2, registering information contained in the Backup _ type into a service table;
s3, scheduling the Backup to a storage medium, and selecting the storage medium by the service according to the Backup _ type to create the Backup;
s4, adding Backup _ type into the shader database;
s5, modifying the host value of the cis-backup into host @ backup nd by the service table;
s6, creating Backup, wherein Backup _ type and an available field az need to be specified;
s7, dispatching the folder to the folder-folder by the folder according to the folder _ name associated with the az and the folder type;
s8, selecting a storage strategy by the cinder-Backup according to storage _ policy associated with the Backup _ type;
adding a Backup _ type in a shader database, wherein the Backup _ type comprises id fields, name fields, backup _ name fields and storage _ policy fields, and the id is uuid of the Backup _ type; name is the name of Backup _ type; the backup _ name is a storage back end name corresponding to the service, and the backup _ name value is swift or posix; the storage _ policy is a storage strategy of a storage back end;
if the backup _ name is swift, then storage _ policy = STANDARD _ IA or default-place, if the backup _ name is posix, then the value of storage _ policy is used as a subdirectory, and backup will be stored under the root directory/storage _ policy.
2. The method as claimed in claim 1, wherein in the step S1, in the process of creating backup, the circle selects circle-backup through backup _ name, and the circle-backup selects storage policy through storage _ policy.
3. The cinder-based multi-backend storage method according to claim 2, wherein if Backup _ type does not specify Backup _ name, then randomly selecting a cinder-Backup available under az;
if Backup _ type does not specify storage _ policy, then a default storage policy is selected.
4. The cinder-based multi-backend storage method according to claim 1, wherein in step S2, binary, host and topic are initialized, service is obtained according to binary and host, and if not, service is created;
if yes, judging whether the service is a circular-backup, and if yes, establishing or starting an rcpserver;
if not, initializing a host = host @ background # Backup _ type of the service, inquiring a load record of the service in the load table, and if not, creating the load record; if the service exists, initializing the maximum load and the standard load of the service, and creating or starting the rcpserver.
5. The cinder-based multi-backend storage method according to claim 1, wherein in step S3, a request is created whether the value of Backup _ type is normal or high, and if not, an abnormal exit is thrown; if yes, selecting a service with the same host name as the volume to create a backup;
if the service with the same host name as the volume exists, selecting all services with host, az and Backup _ type meeting the requirements; if not, selecting all services with az and Backup _ type meeting the requirements;
finally, judging whether available service exists or not, and if not, throwing out an abnormal exit; and if so, the service selects the storage medium according to the Backup _ type to create Backup.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011236892.7A CN112181734B (en) | 2020-11-09 | 2020-11-09 | Multi-back-end storage method based on circular encoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011236892.7A CN112181734B (en) | 2020-11-09 | 2020-11-09 | Multi-back-end storage method based on circular encoder |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112181734A CN112181734A (en) | 2021-01-05 |
| CN112181734B true CN112181734B (en) | 2023-03-31 |
Family
ID=73917209
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011236892.7A Active CN112181734B (en) | 2020-11-09 | 2020-11-09 | Multi-back-end storage method based on circular encoder |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112181734B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113741907A (en) * | 2021-08-23 | 2021-12-03 | 中国人寿保险股份有限公司上海数据中心 | Multi-type back-end storage hybrid deployment system and method based on Openstack cloud platform |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708430A (en) * | 2016-11-30 | 2017-05-24 | 浪潮软件集团有限公司 | Cloud hard disk implementation method under cloud computing architecture |
| CN110399250A (en) * | 2019-06-26 | 2019-11-01 | 苏州浪潮智能科技有限公司 | A kind of OpenStack cloud hard disk automatic backup method and system based on customized strategy |
| CN111722957A (en) * | 2020-02-19 | 2020-09-29 | 王春宝 | Timed backup method for copying block data to os |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7136883B2 (en) * | 2001-09-08 | 2006-11-14 | Siemens Medial Solutions Health Services Corporation | System for managing object storage and retrieval in partitioned storage media |
| US7974950B2 (en) * | 2007-06-05 | 2011-07-05 | International Business Machines Corporation | Applying a policy criteria to files in a backup image |
| US8812436B2 (en) * | 2010-05-04 | 2014-08-19 | Symantec Corporation | Schedule based data lifecycle management |
| CN111367475B (en) * | 2020-03-10 | 2023-05-09 | 山东省电子口岸有限公司 | Automatic configuration method for butt joint G2 storage under prism deployment based on palm |
| CN111597078A (en) * | 2020-05-15 | 2020-08-28 | 山东汇贸电子口岸有限公司 | Timed backup method and system for copying ceph block storage data to object storage |
-
2020
- 2020-11-09 CN CN202011236892.7A patent/CN112181734B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106708430A (en) * | 2016-11-30 | 2017-05-24 | 浪潮软件集团有限公司 | Cloud hard disk implementation method under cloud computing architecture |
| CN110399250A (en) * | 2019-06-26 | 2019-11-01 | 苏州浪潮智能科技有限公司 | A kind of OpenStack cloud hard disk automatic backup method and system based on customized strategy |
| CN111722957A (en) * | 2020-02-19 | 2020-09-29 | 王春宝 | Timed backup method for copying block data to os |
Non-Patent Citations (2)
| Title |
|---|
| 关于Ceph优化企业云后端存储方案的研究;解辰辉等;《铁路计算机应用》;20190425(第04期);全文 * |
| 张继鹏等.基于Openstack的块存储与对象存储服务平台设计与搭建.《电脑编程技巧与维护》.2020,(第09期), * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112181734A (en) | 2021-01-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220188003A1 (en) | Distributed Storage Method and Device | |
| CN101253484B (en) | Method for storing data from client and the client | |
| CN101520805B (en) | Distributed file system and file processing method thereof | |
| US9223789B1 (en) | Range retrievals from archived data objects according to a predefined hash tree schema | |
| US20200344322A1 (en) | Resource scheduling method, apparatus, device and system | |
| US7024529B2 (en) | Data back up method and its programs | |
| US9552242B1 (en) | Log-structured distributed storage using a single log sequence number space | |
| TW202101216A (en) | Change-based restore from a cloud-based data protection service | |
| CN101499073B (en) | Continuous storage data storing and managing method and system based on access frequency | |
| US7774313B1 (en) | Policy enforcement in continuous data protection backup systems | |
| US20070294320A1 (en) | Automated priority restores | |
| US20080243878A1 (en) | Removal | |
| CN104793988A (en) | Cross-database distributed transaction implementation method and device | |
| US8650158B2 (en) | File cloning across different filesets | |
| US10102389B2 (en) | Access permissions management system and method | |
| CN102591982A (en) | Method and system of performing incremental sql server database backups | |
| EP2983104A1 (en) | System and method for secure multi-tenancy in datadomain operating system (ddos), a purpose built backup appliance (pbba) operating system | |
| EP2622456B1 (en) | Optimized recovery | |
| EP2488949A1 (en) | De-duplication storage system with multiple indices for efficient file storage | |
| CN103294167B (en) | A kind of low energy consumption cluster-based storage reproducing unit based on data behavior and method | |
| CN103186554A (en) | Distributed data mirroring method and data storage node | |
| EP2404250A1 (en) | Merging records from different databases | |
| CN112181734B (en) | Multi-back-end storage method based on circular encoder | |
| CN102045399B (en) | Cloud computing mode file system and file reading method | |
| US20160139996A1 (en) | Methods for providing unified storage for backup and disaster recovery and devices thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |