CN110109781A - A kind of data back up method for s3 cloud service - Google Patents
A kind of data back up method for s3 cloud service Download PDFInfo
- Publication number
- CN110109781A CN110109781A CN201910589056.8A CN201910589056A CN110109781A CN 110109781 A CN110109781 A CN 110109781A CN 201910589056 A CN201910589056 A CN 201910589056A CN 110109781 A CN110109781 A CN 110109781A
- Authority
- CN
- China
- Prior art keywords
- data
- cloud service
- local
- backup
- memory space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Computer And Data Communications (AREA)
- Hardware Redundancy (AREA)
Abstract
The invention belongs to field of computer technology, disclose a kind of data back up method for s3 cloud service, the present invention passes through network mapping, the S3 memory space serviced is led directly into local, it is used by local server, the map storage of S3 is being converted to local equipment by FUSE the relevant technologies, is then combined by living document system and S3 cloud service and is completed the data backup based on s3 cloud service.The invention enables the Backup and Restore system of S3 cloud service and traditional server backup it is equally fully functional, use is more convenient；The user of S3 cloud service, which no longer needs to be grasped additional operative knowledge system, can be used S3 cloud service；Snapshot and the filing of S3 service etc. function may be implemented in client；And client can quickly carry out data recovery, also can choose the time point being restored to.
Description
Technical field
The invention belongs to field of computer technology, in particular to a kind of data backup and resume side for s3 cloud service
Method.
Background technique
S3(simplestorage service) it is a kind of simple storage service, show as the one big storage on network
Disk, it may be convenient to store the mass data of enterprise or individual.For the use of S3 cloud service, the traditional method of industry is general
It is the interface using S3 cloud service itself, directly carries out the upload down operation of data, such way can satisfy general
Backup and Restore scene.But if client has further need, data is such as needed to be restored to specified time point or handle
Data are filed, and traditional S3 Backup and Restore system cannot achieve more data manipulations due to lacking related art method
Function.
Summary of the invention
Goal of the invention: the present invention in view of the problems of the existing technology, when providing one kind can be restored to specified data
Between put or the data back up method for s3 cloud service that data are filed.
Technical solution: to achieve the above object, the present invention provides a kind of data back up method for s3 cloud service, packets
Include following steps:
Step 1: locally adding S3 memory space, the S3 memory space serviced is mapped to by local by s3backer software；
And the memory space that will be mapped to local S3 service is converted into device block；
Step 2: being instructed using the zpool in living document system, creation includes the pond zfs of s3 local device；
Step 3: creation storage logical volume on the pond zfs comprising s3 local device created in step；
Step 4: data are backed up；
Step 5: after storage location is written in data, snapshot processing being done to currently stored logical volume.
Wherein, further include in step 1 memory space of S3 service is divided into it is multiple be mapped to local, form more parts and reflect
It penetrates.After possessing more parts of mappings, it is referred to the principle of disk array, combination is mirrored into, raid5, and raid6 etc. conveniently has battle array
The scene of column demand uses.
It further, is that memory space is mapped to by client by network protocol to the method for data backup in the step 5
End, the read-write operation of data is directly carried out in client.
It further, is by storing by third party's network backup software to the method for data backup in the step 5
The backup operation of data is carried out between end and client.
Working principle: the present invention leads directly to local by network mapping, by the S3 memory space serviced, by local service
Device uses, and is converting the map storage of S3 to local equipment by FUSE the relevant technologies, is then passing through living document
System and S3 cloud service, which combine, completes the data backup based on s3 cloud service.
The utility model has the advantages that compared with prior art, the invention enables the Backup and Restore systems and traditional server of S3 cloud service
Backup is equally fully functional, and use is more convenient；The user of S3 cloud service no longer needs to be grasped additional operative knowledge system i.e.
S3 cloud service can be used；Snapshot and the filing of S3 service etc. function may be implemented in client；And client can be counted quickly
According to recovery, the time point being restored to also can choose；Meanwhile the powerful of S3 cloud service function is relied on, but also local dynamic
File system has more storage sources.Storage administrator can provide data storage using traditional disk array
Space also can be used same mode and go to provide data space using cloud service.Without additional deployment means and
In the case where knowledge, it is no longer limited by local memory capacity limitation.
Detailed description of the invention
Fig. 1 is that S3 Stored Procedure figure is added in the present invention；
Fig. 2 is the work flow diagram of data backup in the present invention；
Fig. 3 is the work flow diagram that data are restored in the present invention.
Specific embodiment
Further explanation is done to the present invention with reference to the accompanying drawing.
The present embodiment is based on mainstream Linux system, such as the redhat/ of debian/ubuntu or the rpm series of deb series
Centos etc. has used part third party's open source software to construct or reduction procedure framework.Wherein it is used for using s3backer
The offer in cloud storage space, it realizes the function of converting S3 memory space to disk, using s3backer, simplifies scheme
The implementation process of interior local device cloud service uses scheduling of the fuse to kernel without additional attention.As shown in Figure 1, this reality
Addition S3 Stored Procedure in example is applied to mainly comprise the steps that
Step 11: the S3 memory space serviced being mapped to local, then device block is translated by s3backer software, directly
It connects and is used as local device.In general, the memory space of each S3 service is referred to as bucket in server-side, we are referred to as
The data bucket of cloud service, a bucket can be converted into a local infrastructure device.One bucket can also be converted into local more
A infrastructure device does more parts of mappings.Either one or multiple infrastructure devices, towards local living document system when
Time be it is similar, they are eventually switched to a unified storage pool by we.S3 memory space by conversion mapping is this
The experience and knowledge of the infrastructure device on ground, all local file systems can be reused, and hard link is also avoided, and extension belongs to
Property, POSIX locking etc. etc. may lead the mistake of hair.
Step 12: whether judgment step 11 succeeds, and thens follow the steps 13 if succeeded, if it fails, if unsuccessfully move back
Out；When living document system creation pond, if the disk failure after mapping failure or conversion, is that can not successfully create into
Storage pool.
Step 13: being instructed using the zpool in living document system, creation includes the zfs(living document of s3 local device
System) pond；
Step 14: whether judgment step 13 creates success, if it succeeds, end operation, if it fails, then unsuccessfully exiting.
In more specifically usage scenario, it is also necessary to the attribute in the pond zfs be set and configure the clothes of s3backer and zfs
Business, which is thought highly of, opens recovery.The attribute in the pond zfs includes capacity limit, enabling compression etc..
After completing S3 Stored Procedure, start carry out data backup, specific backup method as shown in Fig. 2, include with
Lower step:
Step 21: the zfs(living document system comprising s3 local device created in step 13) creation storage logic on pond
Volume.
Step 22: data are backed up.Wherein, the backup mode of data it is general there are two types of, one is pass through network assist
ISCSI/IB/FC is discussed, memory space is mapped to client, the read-write operation of data is directly carried out in client；Another kind is
By rsync etc. third party's network backup software, the backup operation of data is carried out between storage and client.
Step 23: after storage location is written in data, i.e., after data backup success, currently stored logical volume being done at snapshot
Reason, storage snapshot can be used for the recovery reduction of data.Since the underlying device of zfs is formed by s3 reverse proxy, therefore data can be same
It walks to the cloud S3.
Step 24: after completing step 23, cancelling storage mapping.In the process once backed up, correlative link has not only been configured
Border will also carry out necessary recovery processing after data end of transmission, in order to avoid system environments is influenced, so needing to cancel storage
Mapping.
As shown in figure 3, using method provided in this embodiment backup data restore when the following steps are included:
Step 31: clone's processing being complied fastly to time point of specified recovery corresponding storage logical volume, that thus bears new patrols
It collects volume and is cloned into new logical volume without influencing original logical volume, data；
Step 32: being restored data to given client end using rsync etc. network backup software；Or use nfs(network text
Part system) etc. network protocols carry memory space to client；So directly using storing data without being restored.It is this
The mode of quick pull-up business is suitable for database and promptly replaces use etc. business scenario.
Step 33: data then cancel storage mapping and destroy clone volume after being successfully recovered.
When being restored using method provided in this embodiment to oracle database, can quick pull-up occasional service, be not necessarily to
It is time-consuming in the recovery process of mass data to wait.
The above is only a preferred embodiment of the present invention, it should be pointed out that: for the ordinary skill people of the art
For member, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications are also answered
It is considered as protection scope of the present invention.
Claims (4)
1. a kind of data back up method for s3 cloud service, it is characterised in that: the following steps are included:
Step 1: locally adding S3 memory space, the S3 memory space serviced is mapped to by local by s3backer software；
And the memory space that will be mapped to local S3 service is converted into device block；
Step 2: being instructed using the zpool in living document system, creation includes the pond zfs of s3 local device；
Step 3: creation storage logical volume on the pond zfs comprising s3 local device created in step；
Step 4: data are backed up；
Step 5: after storage location is written in data, snapshot processing being done to currently stored logical volume.
2. the data back up method according to claim 1 for s3 cloud service, it is characterised in that: further include in step 1
By the S3 memory space serviced be divided into it is multiple be mapped to local, form more parts of mappings.
3. the data back up method according to claim 1 for s3 cloud service, it is characterised in that: right in the step 4
The method of data backup is that memory space is mapped to client, the reading of data is directly carried out in client by network protocol
Write operation.
4. the data back up method according to claim 1 for s3 cloud service, it is characterised in that: right in the step 4
The method of data backup is to carry out the backup behaviour of data between storage end and client by third party's network backup software
Make.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910589056.8A CN110109781A (en) | 2019-07-02 | 2019-07-02 | A kind of data back up method for s3 cloud service |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910589056.8A CN110109781A (en) | 2019-07-02 | 2019-07-02 | A kind of data back up method for s3 cloud service |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110109781A true CN110109781A (en) | 2019-08-09 |
Family
ID=67495910
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910589056.8A Pending CN110109781A (en) | 2019-07-02 | 2019-07-02 | A kind of data back up method for s3 cloud service |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110109781A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112241339A (en) * | 2020-10-23 | 2021-01-19 | 浪潮云信息技术股份公司 | Network-based Redis persistence method |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484130A (en) * | 2014-12-04 | 2015-04-01 | 北京同有飞骥科技股份有限公司 | Construction method of horizontal expansion storage system |
| CN105138417A (en) * | 2015-07-10 | 2015-12-09 | 上海爱数软件有限公司 | Asynchronous replication method based on zfs send |
| US20160299908A1 (en) * | 2015-04-10 | 2016-10-13 | Commvault Systems, Inc. | Using a unix-based file system to manage and serve clones to windows-based computing clients |
-
2019
- 2019-07-02 CN CN201910589056.8A patent/CN110109781A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104484130A (en) * | 2014-12-04 | 2015-04-01 | 北京同有飞骥科技股份有限公司 | Construction method of horizontal expansion storage system |
| US20160299908A1 (en) * | 2015-04-10 | 2016-10-13 | Commvault Systems, Inc. | Using a unix-based file system to manage and serve clones to windows-based computing clients |
| CN105138417A (en) * | 2015-07-10 | 2015-12-09 | 上海爱数软件有限公司 | Asynchronous replication method based on zfs send |
Non-Patent Citations (2)
| Title |
|---|
| 佚名: "Linux远程备份—ftp方式、NFS方式", 《HTTPS://BLOG.CSDN.NET/WEIXIN_33734785/ARTICLE/DETAILS/94439598》 * |
| 佚名: "在Linux上运行ZFS", 《HTTPS://WWW.IBM.COM/DEVELOPERWORKS/CN/LINUX/L-ZFS/INDEX.HTML#ARTRELATEDTOPICS》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112241339A (en) * | 2020-10-23 | 2021-01-19 | 浪潮云信息技术股份公司 | Network-based Redis persistence method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11079966B2 (en) | Enhanced soft fence of devices | |
| US10209910B2 (en) | Copy-redirect on write | |
| US9305009B1 (en) | Synchronous replication of virtualized storage processors | |
| US9875063B2 (en) | Method for writing data to a virtual disk using a controller virtual machine and different storage and communication protocols | |
| US10067722B2 (en) | Storage system for provisioning and storing data to a virtual disk | |
| US9424151B2 (en) | Disk failure recovery for virtual disk with policies | |
| US9483205B2 (en) | Writing to a storage platform including a plurality of storage clusters | |
| US9798489B2 (en) | Cloning a virtual disk in a storage platform | |
| CN114127693A (en) | Data recovery in a virtual storage system | |
| US9864530B2 (en) | Method for writing data to virtual disk using a controller virtual machine and different storage and communication protocols on a single storage platform | |
| US11099953B2 (en) | Automatic data healing using a storage controller | |
| US20160004451A1 (en) | Storage system with virtual disks | |
| DE112020003423T5 (en) | ARCHITECTURE OF VIRTUAL STORAGE SYSTEM | |
| CN109906597B (en) | Storing and retrieving restricted and unrestricted data sets to and from a cloud network | |
| US8966382B1 (en) | Managing production and replica copies dynamically | |
| US20210055885A1 (en) | Enhanced data access using composite data views | |
| US8078585B2 (en) | Reactive file recovery based on file naming and access information | |
| US11615147B2 (en) | Mobile storage manager control application for managing a storage manager of an information management system | |
| US10817325B2 (en) | Moving storage volumes for improved performance | |
| US11144638B1 (en) | Method for storage system detection and alerting on potential malicious action | |
| US20190215363A1 (en) | Dynamic pool-based tiering for synchronization storage | |
| US10579285B2 (en) | Automatic data healing by I/O | |
| CN110109780A (en) | A kind of file-level data backup and restoration methods for NAS service | |
| CN110109781A (en) | A kind of data back up method for s3 cloud service | |
| US11656947B2 (en) | Data set recovery from a point-in-time logical corruption protection copy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190809 |
|
| RJ01 | Rejection of invention patent application after publication |