CN102281321A - Data cloud storage partitioning and backup method and device - Google Patents
Data cloud storage partitioning and backup method and device Download PDFInfo
- Publication number
- CN102281321A CN102281321A CN2011101034092A CN201110103409A CN102281321A CN 102281321 A CN102281321 A CN 102281321A CN 2011101034092 A CN2011101034092 A CN 2011101034092A CN 201110103409 A CN201110103409 A CN 201110103409A CN 102281321 A CN102281321 A CN 102281321A
- Authority
- CN
- China
- Prior art keywords
- data
- copies
- fragment
- verification
- slot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a data cloud storage partitioning and backup method and device. The design thought of the method and the device is that: the traditional method of integrally storing data on same physical storage equipment is changed, the data are partitioned into a plurality of data segments, verification information of each segment is stored, each data segment is copied into a plurality of transcripts, and the transcripts of the data segments are respectively stored on different physical storage equipment. That is to say, integral data are partitioned and backed up in a limited and controllable way, the number of the segments of the data partitioning pieces and the number of the transcripts of each segment are determined according to the level of requirements on data confidentiality, integrality and availability. When the integral data are merged, the transcripts of the data segments which pass the verification are remerged.
Description
Technical field
The present invention relates to the cloud field of storage, method and the device with Backup Data cut apart in the cloud storage of the promptly based on network guarantee data security of being made up of some servers and some network storage equipments.
Background technology
The cloud storage is in cloud computing (cloud computing) conceptive extension and develops a new notion of coming out, be meant by functions such as cluster application, grid or distributed file systems, a large amount of various dissimilar memory devices in the network are gathered collaborative work by application software, a system of storage and Operational Visit function externally is provided jointly.When the core of cloud computing system computing and processing is the storage of mass data and management, just need a large amount of memory device of configuration in the cloud computing system, cloud computing system just is transformed into a cloud storage system so, is the cloud computing system of core so the cloud storage is one with storage and management.
From the notion of cloud storage and should be used at present, its essence is to provide a kind of transparent network storage service to the user.The user is as long as connect and to enter the Internet, can the enjoy network stores service--manage the data that it is stored anywhere or anytime, just as these storage in this locality; The user can conveniently share data and other users who is stored in the cloud storage simultaneously.Therefore how to be implemented in conveniently reading of data of arbitrary network access point, to guarantee that simultaneously confidentiality, integrality, the availability of data become the key of cloud memory technology.
Domestic existing about rarely having solution or system to relate to raising data security, integrality, availability in the cloud storage patented technology, wherein:
Publication No. is: CN 101605148A (framework method of the parallel system of cloud storage) is though be kept at file fragmentation preservation, each file sheet respectively on a certain the storage server, avoided although it is so divulging a secret because of the whole file that a certain storage server causes because of assault or system vulnerability cause data to leak, heightened the confidentiality of file, but any data storage server of preserving the file sheet breaks down and causes data not read or the artificial malicious modification file sheet of quilt, and then the integrality of file and availability are just suffered thorough destruction.
Publication No. is: the back-up storage that CN 101977243A (cloud data storage allocation (DDCS)) carries out indifference, incalculability restriction for the integrality that improves data and availability to the data that are kept in the storage server, this method will cause the storage server space to be consumed the whole leakage of data that (the big more data redundancy of cloud storage size is serious more) and any one storage server cause data to leak and all will cause because of assault or system vulnerability by the mass data redundancy.And do not have verification scheme in solution/system, some backs up by malicious modification when data, just can't find to judge that those backups of these data are to be modified those backups not to be modified.
It is the system architecture and the implementation method of inventing in order comprehensively to overcome the above problems that cloud is stored method and the device cut apart with Backup Data, and it need rely on following technology:
1, broadband network development.
2, WEB 2.0 technology.
3, application memory development.
4, Clustering, grid and distributed file system.
5, CDN content distribution, P2P technology, data compression technique.
6, storage virtualization technology, storage networking administrative skill.
Summary of the invention
The present invention is that a kind of cloud is stored method and the device of cutting apart with Backup Data.The system that uses described method and device to make up will strengthen confidentiality, integrality, the availability of storage data.
It is to invent to achieve these goals that cloud is stored method and the device cut apart with Backup Data.Its thinking is by changing in the past with the way on the same physical storage device of being kept at of data integrity, and employing is divided into data the check information of every section of plurality of data section preservation, again each data segment is duplicated some copies again, these data segment copies are kept at respectively on the different physical storage devices.Promptly partial data is carried out limited, controlled cutting apart and backup, the data cutting plate be how many fragments, each fragment have how many copies all foundation to deciding in requirement height, the end of data confidentiality, integrality, availability.When merging partial data, will reconsolidate by the data slot copy of verification and get final product.
In this invention, data slot is kept on the different physical storage devices, has solved that some memory device hinders for some reason or the artificial origin causes in the cloud storage data slot can not read or malicious modification causes data integrity and the available problem of being destroyed; A partial data is divided into each data slot of plurality of data fragment to be duplicated some copies again and they is preserved respectively, even some memory device leaks because of the data that system vulnerability or artificial origin cause in the cloud storage, what reveal is a certain fragment of partial data only, has improved the confidentiality of partial data.
Be primarily characterized in that:
1, a partial data is split into the plurality of data fragment, and writes down the check information of every fragment;
2, each data slot all is replicated some copies;
3, all affiliated data slot copies of partial data all are kept on the different physical storage devices as much as possible;
4, before data slot being merged into a partial data, all data slots of forming this partial data are carried out verification;
5, the copy of same data slot carries out verification successively, has only the merging that just can be used for partial data by the data trnascription of verification.
The present invention adopts modular construction, each module polymerization some function elements, combination or realize some function separately.
The invention will be further described below in conjunction with the drawings and specific embodiments.
Description of drawings
Following accompanying drawing is used to provide further understanding of the present invention, and constitutes the part of specification, is used from explanation the present invention with embodiments of the invention one, is not construed as limiting the invention.In the accompanying drawings:
Fig. 1 is functional structure chart among the embodiment of system of the present invention;
Fig. 2 is the basic flow sheet of partition data among the embodiment of the inventive method;
Fig. 3 is the basic flow sheet of merging data among the embodiment of the inventive method.
Embodiment
Below in conjunction with accompanying drawing the present invention is done further and to elaborate.
Referring to Fig. 1, be that functional structure chart among the embodiment with the method for Backup Data and device is cut apart in cloud of the present invention storage.Wherein:
Controller 101 is responsible for specific data cutting plate hop count and every fragment number of copies, and control dispenser 102 and fusion device 106 are cut apart preservation or merged extraction data;
Dispenser 102 is divided into the data slice hop count according to the cutting plate hop count of controller 101 appointments with data, gives backup device 103 with the information such as number of copies of data slot and every fragment;
Backup device 103 according to every fragment number of copies copy data fragment copy, and is saved in respectively on the memory devices different in the memory 105, and with writing position registers 104 such as the stored position information of segments, number of copies and every fragment, check informations;
Position recorder 104, the stored position information of preservation data slice hop count, number of copies and every fragment, check information etc.;
Memory 105 is preserved the data slot copy, comprises many different physical storage devices;
Fusion device 106 receives controller 101 merging data requirements, issues the instruction of checking data fragment and receives data slot by verification to checker 107, these data slots is reconsolidated be partial data;
Checker 107, receive the merging data requirement of fusion device 106, comply with all data slot position and copy informations of obtaining these data in the position recorder 104, read and the checking data fragment, will submit fusion device 106 by the data slot of verification from memory 105.
Referring to Fig. 2, be the basic flow sheet that partition data among the embodiment with the method for Backup Data and device is cut apart in cloud of the present invention storage.Wherein:
Step 201, controller sends the partition data instruction to dispenser, and the number of copies of the divided fragment of data and every fragment.
Step 202, dispenser is divided into data slot according to the cutting plate hop count with data, and gives the backup device with information such as data slot, cutting plate hop count and every fragment number of copies.
Step 203, the backup device calculates the check information of each fragment, and it is saved in the register of memory location.
Step 204, backup device be according to every fragment number of copies copy data fragment copy, and with the data slot copy as much as possible distributed and saved in the memory in the different physical storage device.
Step 205, the backup device is given position recorder with the preservation positional information of the cutting plate hop count of data, every fragment number of copies and each data slot copy and is preserved.
Step 206, partition data is finished.
Referring to Fig. 3, be the basic flow sheet that merging data among the embodiment with the method for Backup Data and device is cut apart in cloud storage.Wherein:
Step 301, controller sends the merging data instruction to fusion device, and fusion device is delivered checker with instruction.
Step 302, checker obtain the check information of the cutting plate hop count of partial data, every fragment number of copies, each fragment and the preservation position of each data slot copy etc. and cut apart information from position recorder.
Step 303 reads arbitrary copy of all data slots.
Step 304 is carried out verification according to the check information verification of each fragment to the data slot copy that reads; If all data slots are by verification, then jump procedure 307; If data slot is arranged not by verification, then jump procedure 305.
Step 305, whether inspection does not have the copy that does not read to be kept in the memory by the data slot of verification, and then jump procedure 306 is arranged, and does not have then jump procedure 309.
Step 306 reads not the arbitrary copy that does not read by the data slot of verification, jump procedure 304 from memory.
Step 307, checker will be given fusion device by the data slot of verification, and fusion device is merged into partial data with these data slots.
Step 308, merging data is finished.
Step 309, the merging data failure.
The above only is the flow process of specific implementation method of the present invention and some basic embodiment, and its content is not to be used to limit the present invention just for the ease of understanding basic principle of the present invention and flow process.Flow chart has only represented the main logical order of the embodiment of the invention, but not the logical order that all may exist.Any personnel that certain related-art technology knowledge is arranged under the prerequisite that does not break away from the principle of the invention, can also make some modifications and variations on implementation detail and flow process, these modifications and variations also should be considered as protection scope of the present invention.In addition, the present invention sets forth each function element and step can realize with the general-purpose computations device, and they can concentrate on the single calculation element, also can be distributed in the network of a plurality of calculating dress compositions; They can be carried out executable program code by calculation element and realize, also independent the or a plurality of function elements in them or step can be made as integrated circuit and realize.The present invention does not limit any combination thereof.
Claims (6)
1. cloud is stored method and the device of cutting apart with Backup Data, its core is by changing in the past with the way on the same physical storage device of being kept at of data integrity, and employing is divided into data the check information of every section of plurality of data section preservation, again each data segment is duplicated some copies again, these data segment copies are kept at respectively on the different physical storage devices.Promptly partial data is carried out limited, controlled cutting apart and backup, the data cutting plate be how many fragments, each fragment have how many copies all foundation to deciding in requirement height, the end of data confidentiality, integrality, availability.When merging partial data, will reconsolidate by the data slot copy of verification and get final product.
2. system according to claim 1 is characterized in that, a described partial data is split into the plurality of data fragment, and writes down the check information of every fragment.
3. system according to claim 1 is characterized in that, described each data slot all is replicated some copies.
4. system according to claim 1 is characterized in that, all the data slot copies under the described partial data all are kept on the different physical storage devices as much as possible.
5. system according to claim 1 is characterized in that, and is described before data slot being merged into a partial data, and all data slots of forming this partial data are carried out verification.
6. system according to claim 1 is characterized in that, the copy of described same data slot carries out verification successively, has only the merging that just can be used for partial data by the data trnascription of verification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101034092A CN102281321A (en) | 2011-04-25 | 2011-04-25 | Data cloud storage partitioning and backup method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011101034092A CN102281321A (en) | 2011-04-25 | 2011-04-25 | Data cloud storage partitioning and backup method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102281321A true CN102281321A (en) | 2011-12-14 |
Family
ID=45106492
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011101034092A Pending CN102281321A (en) | 2011-04-25 | 2011-04-25 | Data cloud storage partitioning and backup method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102281321A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102685199A (en) * | 2012-01-18 | 2012-09-19 | 吴昊 | File safety storage method based on multiple network nodes |
| CN102880658A (en) * | 2012-08-31 | 2013-01-16 | 电子科技大学 | Distributed file management system based on seismic data processing |
| CN103514198A (en) * | 2012-06-25 | 2014-01-15 | 北京智慧风云科技有限公司 | File management method, cloud storage system and cloud terminal server in cloud storage system |
| CN103607418A (en) * | 2013-07-25 | 2014-02-26 | 上海和辰信息技术有限公司 | Large-scale data partitioning system and partitioning method based on cloud service data characteristics |
| CN103970852A (en) * | 2014-05-06 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Data de-duplication method of backup server |
| CN104184740A (en) * | 2014-09-04 | 2014-12-03 | 中电长城网际系统应用有限公司 | Credible transmission method, credible third party and credible transmission system |
| CN104618153A (en) * | 2015-01-20 | 2015-05-13 | 北京大学 | Dynamic fault-tolerant method and dynamic fault-tolerant system based on P2P in distributed parallel graph processing |
| CN105573922A (en) * | 2014-11-07 | 2016-05-11 | 中兴通讯股份有限公司 | Method and device for realizing data format conversion |
| CN106027653A (en) * | 2016-05-23 | 2016-10-12 | 华中科技大学 | Multi-cloud storage system expansion method based on RAID4 (Redundant Array of Independent Disks) |
| TWI588670B (en) * | 2016-05-25 | 2017-06-21 | 精品科技股份有限公司 | System and method for segment backup |
| WO2017113088A1 (en) * | 2015-12-29 | 2017-07-06 | 深圳大学 | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device |
| CN106934302A (en) * | 2016-12-23 | 2017-07-07 | 航天星图科技（北京）有限公司 | A kind of data validation method |
| CN107436823A (en) * | 2016-05-25 | 2017-12-05 | 精品科技股份有限公司 | System and method for automatic scheduling backup |
| CN107861843A (en) * | 2017-11-30 | 2018-03-30 | 紫光股份有限公司 | A kind of file backup system based on LAN computer |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN110493192A (en) * | 2019-07-17 | 2019-11-22 | 北京泰立鑫科技有限公司 | A kind of data safe transmission system and method based on data gateway |
| CN112035058A (en) * | 2020-07-30 | 2020-12-04 | 深圳市中盛瑞达科技有限公司 | Cloud platform storage system |
| CN112685232A (en) * | 2021-01-11 | 2021-04-20 | 河南大学 | Computer backup data monitoring method and system |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630282A (en) * | 2009-07-29 | 2010-01-20 | 国网电力科学研究院 | Data backup method based on Erasure coding and copying technology |
-
2011
- 2011-04-25 CN CN2011101034092A patent/CN102281321A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101630282A (en) * | 2009-07-29 | 2010-01-20 | 国网电力科学研究院 | Data backup method based on Erasure coding and copying technology |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102685199A (en) * | 2012-01-18 | 2012-09-19 | 吴昊 | File safety storage method based on multiple network nodes |
| CN102685199B (en) * | 2012-01-18 | 2015-09-02 | 南京交通职业技术学院 | A kind of file safety storage method of multinode Network Based |
| CN103514198A (en) * | 2012-06-25 | 2014-01-15 | 北京智慧风云科技有限公司 | File management method, cloud storage system and cloud terminal server in cloud storage system |
| CN102880658A (en) * | 2012-08-31 | 2013-01-16 | 电子科技大学 | Distributed file management system based on seismic data processing |
| CN102880658B (en) * | 2012-08-31 | 2016-11-16 | 电子科技大学 | Distributed file management system based on seismic data process |
| CN103607418A (en) * | 2013-07-25 | 2014-02-26 | 上海和辰信息技术有限公司 | Large-scale data partitioning system and partitioning method based on cloud service data characteristics |
| CN103970852A (en) * | 2014-05-06 | 2014-08-06 | 浪潮电子信息产业股份有限公司 | Data de-duplication method of backup server |
| CN104184740B (en) * | 2014-09-04 | 2019-02-05 | 中电长城网际系统应用有限公司 | Trusted transmission method, trusted third party and credible delivery system |
| CN104184740A (en) * | 2014-09-04 | 2014-12-03 | 中电长城网际系统应用有限公司 | Credible transmission method, credible third party and credible transmission system |
| WO2016070668A1 (en) * | 2014-11-07 | 2016-05-12 | 中兴通讯股份有限公司 | Method, device, and computer storage medium for implementing data format conversion |
| CN105573922B (en) * | 2014-11-07 | 2020-07-10 | 中兴通讯股份有限公司 | Method and device for realizing data format conversion |
| CN105573922A (en) * | 2014-11-07 | 2016-05-11 | 中兴通讯股份有限公司 | Method and device for realizing data format conversion |
| CN104618153B (en) * | 2015-01-20 | 2018-08-03 | 北京大学 | Dynamic fault-tolerant method and system based on P2P in the processing of distributed parallel figure |
| CN104618153A (en) * | 2015-01-20 | 2015-05-13 | 北京大学 | Dynamic fault-tolerant method and dynamic fault-tolerant system based on P2P in distributed parallel graph processing |
| WO2017113088A1 (en) * | 2015-12-29 | 2017-07-06 | 深圳大学 | Cloud service-based data storage method, integrity detection method and apparatus, and terminal device |
| CN106027653A (en) * | 2016-05-23 | 2016-10-12 | 华中科技大学 | Multi-cloud storage system expansion method based on RAID4 (Redundant Array of Independent Disks) |
| CN106027653B (en) * | 2016-05-23 | 2019-04-12 | 华中科技大学 | A kind of cloudy storage system extended method based on RAID4 |
| CN107436824A (en) * | 2016-05-25 | 2017-12-05 | 精品科技股份有限公司 | System and method for sectional backup |
| CN107436823A (en) * | 2016-05-25 | 2017-12-05 | 精品科技股份有限公司 | System and method for automatic scheduling backup |
| TWI588670B (en) * | 2016-05-25 | 2017-06-21 | 精品科技股份有限公司 | System and method for segment backup |
| CN106934302A (en) * | 2016-12-23 | 2017-07-07 | 航天星图科技（北京）有限公司 | A kind of data validation method |
| CN106934302B (en) * | 2016-12-23 | 2021-07-13 | 中科星图股份有限公司 | Data validity checking method |
| CN107861843A (en) * | 2017-11-30 | 2018-03-30 | 紫光股份有限公司 | A kind of file backup system based on LAN computer |
| CN109783275A (en) * | 2018-12-17 | 2019-05-21 | 北京春鸿科技有限公司 | The method and terminal of file backup are carried out to multiple wifi storage equipment |
| CN110493192A (en) * | 2019-07-17 | 2019-11-22 | 北京泰立鑫科技有限公司 | A kind of data safe transmission system and method based on data gateway |
| CN112035058A (en) * | 2020-07-30 | 2020-12-04 | 深圳市中盛瑞达科技有限公司 | Cloud platform storage system |
| CN112035058B (en) * | 2020-07-30 | 2021-04-30 | 深圳市中盛瑞达科技有限公司 | Cloud platform storage system |
| CN112685232A (en) * | 2021-01-11 | 2021-04-20 | 河南大学 | Computer backup data monitoring method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102281321A (en) | Data cloud storage partitioning and backup method and device | |
| CN107220148B (en) | Reconstruction method and device for redundant array of independent disks | |
| CN107844268B (en) | Data distribution method, data storage method, related device and system | |
| AU2015211305B2 (en) | Prioritizing data reconstruction in distributed storage systems | |
| US9672106B2 (en) | Architecture for implementing erasure coding | |
| US9195392B2 (en) | Distributed storage method, apparatus, and system | |
| US10140174B2 (en) | Separating storage transaction logs | |
| US10353787B2 (en) | Data stripping, allocation and reconstruction | |
| WO2018190809A1 (en) | Blockchain partial ledgers | |
| US20140181042A1 (en) | Information processor, distributed database system, and backup method | |
| CN103942112A (en) | Magnetic disk fault-tolerance method, device and system | |
| US9009431B2 (en) | Virtual snapshot system and method | |
| CN104714756A (en) | Local locking in bi-directional synchronous mirroring environment and computer system | |
| CN109690494A (en) | Layering in system memory device is fault-tolerant | |
| US11520512B2 (en) | Method for storage management, electronic device and computer program product | |
| CN111949210A (en) | Metadata storage method, system and storage medium in distributed storage system | |
| CN105404565B (en) | A kind of dual-active data guard method and device | |
| US9262289B2 (en) | Storage apparatus and failover method | |
| CN107562377A (en) | A kind of storage system hard disk dilatation implementation method | |
| US20150212847A1 (en) | Apparatus and method for managing cache of virtual machine image file | |
| US10372556B1 (en) | Recovery of nodes | |
| CN104618191A (en) | Method and device for detecting communication fault between hosts and naked storage blocks | |
| CN106155563B (en) | A kind of disk access control method and device | |
| CN116009761A (en) | Data writing method and related equipment | |
| CN113424262B (en) | Storage verification method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20111214 |