CN107154945A - A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes - Google Patents
A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes Download PDFInfo
- Publication number
- CN107154945A CN107154945A CN201710401812.0A CN201710401812A CN107154945A CN 107154945 A CN107154945 A CN 107154945A CN 201710401812 A CN201710401812 A CN 201710401812A CN 107154945 A CN107154945 A CN 107154945A
- Authority
- CN
- China
- Prior art keywords
- data
- cloud
- blocks
- files
- correcting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/06—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols the encryption apparatus using shift registers or memories for block-wise or stream coding, e.g. DES systems or RC4; Hash functions; Pseudorandom sequence generators
- H04L9/0643—Hash functions, e.g. MD5, SHA, HMAC or f9 MAC
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
- H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
- H03M13/373—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L63/00—Network architectures or network communication protocols for network security
- H04L63/04—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks
- H04L63/0428—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload
- H04L63/0435—Network architectures or network communication protocols for network security for providing a confidential data exchange among entities communicating through data packet networks wherein the data content is protected, e.g. by encrypting or encapsulating the payload wherein the sending and receiving network entities apply symmetric encryption, i.e. same key used for encryption and decryption
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- General Physics & Mathematics (AREA)
- Power Engineering (AREA)
- Probability & Statistics with Applications (AREA)
- Computing Systems (AREA)
- Storage Device Security (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes and system, the method of the data reading step of data storing steps and many cloud data centers including many cloud data centers, and including fragmentation of data layer, the system of data dispatch layer and cloud service layer, the technical effects of the invention are that, after the encrypted algorithm of data and correcting and eleting codes algorithm process, it is stored in multiple cloud data centers, both data center's hardware construction cost had been saved, the data caused by single cloud data center are avoided to use integrity problem again, blocks of files selects corresponding cloud data center according to the response time, blocks of files distribution efficiency can be improved, the service quality interacted between lifting user and cloud storage system.Blocks of files is scattered to be stored in multiple cloud data centers, and metadata is stored on meta data server, is avoided that user data is stolen in cloud data center monopolization, while avoiding system developer from obtaining user data, the storage security of lifting data center data.
Description
Technical field
The present invention relates to cloud storage technical field, more particularly to a kind of cloudy fragmentation safety storage side based on correcting and eleting codes
Method and system.
Background technology
As emerging information technology is merged with crossing for modern economy society, the data volume that every profession and trade is produced just quickly increases
Long, the positive rapid aggregation of big data is huge, species is various, data complicated and changeable, and it is collected, store, handled
With analysis application, rule, the service industry situation extracted value, created the wealth are found from mass data.Due to being concealed in data
Huge value wealth so that data are increasingly becoming a national basic strategic resource, at the same big data infrastructure, point
Analysis and the development of service technology enable the enterprise to be transformed into data driven type mechanism.Due to the importance of big data, country, enterprise
Industry, organization are required for building big data ability keeping competitiveness.China supports big data development and application energetically in recent years,
National Data Centre's platform construction is promoted, the field such as credit, traffic, medical treatment, employment government data resource appropriateness is promoted to society
Exploitation, to eliminate information island, reaches construction data power purpose.
Data center has a features such as data scale big, species is more, various structures, and require high reliability, high security,
High scalability, new demand is proposed to data storage security.Cloud storage can support vast resources data storage and effectively management hundred
Hundred million grades of files, reliable solution is provided for Constructing data center.Available data stores safe practice and to there is problem as follows：
1. data encryption：There are problems that availability of data, such as corrupted data, loss.2. Replication technology：Storage overhead is excessive；③
RAID and correcting and eleting codes technology：Data safety sex chromosome mosaicism, such as transmitting procedure are stolen.Moreover, it is assumed that storing data in single cloud
Data center, there are problems that availability of data, such as cloud data center failure, insincere.Therefore, data safety
Storage is always the key problem of data center.
Domestic patent：A kind of method for secure storing and system (CN103118089A) based on multiple cloud storage systems, are carried
Supplied include data write operation, the method for secure storing of read operation, data write operation step include deblocking, data encryption,
Redundancy encoding, digital signature, share processing and data storage word step；Data reading operation step includes decrypted metadata burst text
Part, digital signature inspection, redundancy encoding and decryption ciphertext block data sub-step.Safe storage system, including encryption and decryption volume simultaneously
Code module, redundancy encoding module, secret sharing module and Digital Signature module.But there is no guarantee that the time of phase data memory disappears
Consumption and service quality.
Foreign literature：Implementation of Cloud-RAID:A Secure and Reliable Storage
Above the Clouds (International Conference on Grid and Pervasive Computing,
Springer Berlin Heidelberg, 2013) multiple cloud storage suppliers are provided to avoid lockout issue, utilize RAID skills
Art ensures availability of data, confidentiality and integrity, but when multiple cloud storage suppliers break down, it is impossible to ensure that data can
With property, and user and system interaction service quality can not be ensured.
Therefore, for problem above, in the urgent need to a kind of cloud storage system at new applicable data center, not only meet
Information Security, reliability, confidentiality can be used, and can greatly save Constructing data center cost, at the same ensure user with
The service quality of interaction between cloud storage system.
The content of the invention
The technical problems to be solved by the invention are to provide a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes
And system, the problems such as data safety is stored is solved for data center, while lifting what is interacted between user and cloud storage system
Service quality.
In order to realize above-mentioned technical purpose, the technical scheme is that：
A kind of cloudy fragmentation method for secure storing based on correcting and eleting codes, includes the data storing steps of many cloud data centers
With the data reading step of many cloud data centers,
The data storing steps of described many cloud data centers, including fragmentation of data step, data distribution step sum
According to band storing step：
1), fragmentation of data：
Data are uploaded to user to be encrypted by key, encryption data are generated, after then encryption data is encoded
Multiple blocks of files are generated, and generate the meta data file of file block message of the record including key；
2), data distribution：
According to the response time of each cloud data center according to being ranked up from small to large, by step 1) the middle file generated
Block is distributed to each cloud data center after sequence successively；
3), data strip is stored：
Each cloud data center provides continuous address space in logic, and same encryption data is stored in different cloud data
One slice-group of supercentral blocks of files formation；
The data reading step of described many cloud data centers, including meta data file read step, fragmentation of dataization are read
Take step and data reduction step；
1., meta data file is read：
Meta data file is read from meta data server, to obtain n file block message of data；
2., fragmentation of dataization is read：
The cloud data center of corresponding document block is stored according to blocks of files information searching, and downloads blocks of files；
3., data convert：
By secret key decryption data, initial data is obtained.
A kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, the tool of described fragmentation of data step
Body step includes：
First, data Data is encrypted for Key symmetric encipherment algorithm using key, generation encryption data Data '；
Secondly, using correcting and eleting codes algorithm coding encryption data Data ', n blocks of files is generated, including k according to block
With m encoding block, and n=k+m；
Meanwhile, portion Data ' meta data file is produced, meta data file is stored in meta data server.Metadata
File includes data Data titles, size, data block number k, encoding block number m, correcting and eleting codes algorithm title, key Key.
In a kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, described fragmentation of data step,
Also include uploading user data Data progress cryptographic Hash computings using hashing algorithm, regard obtained cryptographic Hash as symmetric cryptography
The step of algorithm secret key Key.
A kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, described data distribution step it is specific
Step includes：
The quantity of cloud data center is c, and c takes natural number, and according to response time size, cloud data center is ordered as
{C1,C2,...,CcSet C, by n blocks of files distribution storage to S cloud data center, wherein S ∈ C；
If n ＞ c, the order sorted according to cloud data center distributes storage file block successively；
If 1≤n≤c, n cloud data center distribution storage file block before choosing.
In a kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, described data distribution step, ring
θ=T between seasonablee-Tb, wherein TeStart to receive the time of first data byte, T for corresponding cloud data centerbOpened for system
Begin to send request time to corresponding cloud data center.
A kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, described fragmentation of data read step
In, when downloading blocks of files, in addition to when the blocks of files failed download in some cloud data center, then according to from other cloud numbers
The survival blocks of files downloaded according to center carries out decoding calculating, so as to recover the data.
In a kind of described cloudy fragmentation method for secure storing based on correcting and eleting codes, described data convert step, also
The step of including using hashing algorithm to data progress cryptographic Hash computing after decryption, then being compared with original cryptographic Hash, if breathing out
Uncommon value is equal, then the data read are initial data, is otherwise judged to reading data failure.
A kind of cloudy fragmentation safe storage system based on correcting and eleting codes, including fragmentation of data layer, data dispatch layer and cloud
Service layer：
The fragmentation of data layer includes user's application service module, data encryption/decryption service module, data encoding/solution
Code service module and metadata service module；
(1), user's application service module is used to provide the interface service interacted with user, and rower is entered to user identity
Know registration, and be the different access of different type user setting and function privilege；
(2), data encryption/decryption service module is used to pass through key respectively in data storage and reading data for user
Service is encrypted and decrypted；
(3), data encoding service module is used to generate blocks of files, including it is many numbers that the data after encryption are carried out into cutting
Calculated according to block and by correcting and eleting codes coding and produce multiple encoding blocks；Data decoding service module is used to download from each cloud data center
Corresponding blocks of files, is calculated by correcting and eleting codes decoding and recovers encryption data；
(4), metadata service module is used for the meta data file for generating file block message of the record including key；
The data dispatch layer includes data dispatch service module and data distribution service module；
Ith, data dispatch service module is used for the response time according to system and each cloud data center, and cloud data center is pressed
It is ranked up according to excellent → difference, the blocks of files that is generated after schedule code is come with this；
II, Data distributing module be used for by the RESTful API API design of internet application (theoretical) with
Each cloud data center is attached, and blocks of files is distributed into each cloud data center；
The cloud service layer is made up of multiple cloud data centers, is used for into data storage of racking.
A kind of described cloudy fragmentation safe storage system based on correcting and eleting codes, described data encryption services module exists
During user storage data, the cryptographic Hash of data is first calculated using hashing algorithm, using cryptographic Hash as key Key, then using key
For Key symmetric encipherment algorithm encryption data；
Described data deciphering service module obtains key Key values when user reads data from meta data file, then
Key is used for Key symmetric encipherment algorithm ciphertext data.
A kind of described cloudy fragmentation safe storage system based on correcting and eleting codes, described data encoding service module will
Data after encryption use the average cutting of (n, k) correcting and eleting codes algorithm to be that k calculates m coding of generation according to block, then by encoding
Block, wherein k are equal according to the size of block and m encoding block, and symbiosis is into n=k+m blocks of files；
Record has data name, size, data block in the meta data file that described metadata service module is generated
Number k, encoding block number m, correcting and eleting codes algorithm title and key Key.
Described data decoding service module first obtains each blocks of files when user reads data from meta data file
The cloud data center information of storage, then corresponding blocks of files is downloaded from each cloud data center, if partial document block is unavailable or loses
Effect, then calculate according to the decoding of (n, k) correcting and eleting codes algorithm and recover data.
The technical effects of the invention are that：
(1) after the encrypted algorithm of data and correcting and eleting codes algorithm process, multiple cloud data centers is stored in, were both saved in data
Hard-hearted part construction cost, avoids the data caused by single cloud data center from using integrity problem, can maximize in cloud data again
Heart diversity, at utmost tolerance cloud data center are unavailable, realize fault-tolerant maximization and improve data access efficiency.
(2) blocks of files selects corresponding cloud data center according to the response time, can improve blocks of files distribution efficiency, lifting
The service quality interacted between user and cloud storage system.
(3) blocks of files is scattered is stored in multiple cloud data centers, and metadata is stored on meta data server, is avoided that cloud
User data is stolen in data center's monopolization, while avoiding system developer from obtaining user data, lifts depositing for data center's data
Store up security.
Brief description of the drawings
Fig. 1 is the cloudy fragmentation safe storage system schematic diagram based on correcting and eleting codes.
Fig. 2 is the data storage schematic flow sheet of many cloud data centers of the invention.
Fig. 3 is the digital independent schematic flow sheet of many cloud data centers of the invention.
Fig. 4 is date storage method flow chart of the present invention.
Fig. 5 is method for reading data flow chart of the present invention.
Fig. 6 is present system configuration diagram.
Embodiment
The invention will be further described with reference to the accompanying drawings and examples.
The present invention is based on RAID thoughts and multiple cloud data centers, as shown in figure 1, each user by cloud storage system by number
According to multiple cloud data centers are stored in, cloud data center is made up of some large-scale cloud storage suppliers, such as Amazon S3,
Microsoft Azure, A Liyun, Tengxun's cloud and Baidu's cloud etc..
A kind of cloudy fragmentation method for secure storing based on correcting and eleting codes provided by the present invention, including many cloud data centers
Data storage and many cloud data centers digital independent.
The data storage of many cloud data centers, is mainly made up of fragmentation of data, data distribution and data strip storage.
(1) fragmentation of data：
First, data Data is uploaded to user using hashing algorithm and carries out cryptographic Hash computing, using obtained cryptographic Hash as
Symmetric encipherment algorithm key Key, reuses the symmetric encipherment algorithm that key is Key and data Data is encrypted, generation encryption
Data Data '；
Secondly, using correcting and eleting codes algorithm coding encryption data Data ', n blocks of files is generated, including k according to block
With m encoding block, and n=k+m.
Meanwhile, portion Data ' meta data file is produced, meta data file is stored in meta data server.Metadata
File includes data Data titles, size, data block number k, encoding block number m, correcting and eleting codes algorithm title, key Key.
(2) data distribution
If system is made up of c different cloud data centers, C={ 1,2 ..., c } is designated as.
Define response time θ=Te-Tb, wherein TeStart first data byte of reception for corresponding cloud data center
Time, TbStart to send request time to corresponding cloud data center for system.
According to response time size, cloud data center is ordered as { C1,C2,...,Cc}.By n blocks of files distribution storage
To S cloud data center, wherein S ∈ C.
If n ＞ c, the order sorted according to cloud data center distributes storage file block successively；
If 1≤n≤c, n cloud data center distribution storage file block before choosing.
(3) data strip is stored
Each cloud data center provide that logical address is continuous but physics storage address may and discontinuous data storage is empty
Between, data Data ' is stored in the blocks of files referred to as slice-group in different cloud data centers.
The digital independent of many cloud data centers, mainly read by meta data file, fragmentation of data reading, cryptographic Hash inspection
Composition.
(1) meta data file is read
The meta data file stored after fragmentation of data is read from meta data server, n is obtained from meta data file
Individual file block message.
(2) fragmentation of dataization is read
The cloud data center of corresponding document block is stored according to blocks of files information searching, blocks of files is obtained from cloud data center
Patch information, starts to download blocks of files.If the blocks of files failed download in certain cloud data center, according to other survival blocks of files
Decoding, which is calculated, recovers the data.
Again by the symmetric encipherment algorithm ciphertext data that key is Key, initial data is obtained.
(3) data convert
Cryptographic Hash computing is carried out to data after decryption using hashing algorithm, then is compared with original cryptographic Hash, if equal
The data then read are initial data, if unequal, read data failure.
A kind of cloudy fragmentation safe storage system based on correcting and eleting codes provided by the present invention, including fragmentation of data layer,
Data dispatch layer and cloud service layer：
Fragmentation of data layer by user's application service, data encryption/decryption service, Metadata Service and data encoding/
Decoding service is constituted.
1. user's application service.The interface service interacted with user is provided, registration is identified to user identity, no
Same type user has different access rights and can use different functions.
2. data encryption/decryption service.
Data encryption services.When user storage data, the cryptographic Hash of data is first calculated using hashing algorithm, by cryptographic Hash
As key Key, then key is used for Key symmetric encipherment algorithm encryption data.
Data deciphering is serviced.When user reads data, key Key values are obtained from meta data file, then using key
For Key symmetric encipherment algorithm ciphertext data.
3. data encoding/decoding service.
Data encoding is serviced.Data are first averaged cutting for k evidence by the data after encryption using (n, k) correcting and eleting codes algorithm
Block, then produce m encoding block by encoding to calculate, wherein k is equal according to block and m coded block size, and symbiosis is into n=k+
M blocks of files.1 part of meta data file is generated after coding, wherein including data name, size, data block number k, encoding block
Number m, correcting and eleting codes algorithm title, key Key.
Data decoding service.When user reads data, each file block message is first obtained from meta data file, according to
The cloud data center of blocks of files information searching storage, then corresponding blocks of files is downloaded from each cloud data center, if partial document block
Unavailable or failure (failure number is less than m), then calculate according to (n, k) correcting and eleting codes algorithm coding and recover data.
4. Metadata Service
Metadata Service.Meta data file is generated after data encoding, data name, size, data block number k, volume is included
Code block number m, correcting and eleting codes algorithm title, key Key.
The data dispatch layer is made up of data dispatch service, Data distributing.
1. data dispatch service.
According to the response time of system and each cloud data center, cloud data center is ranked up according to excellent → difference, with this
Carry out the blocks of files generated after schedule code.
2. Data distributing.
System is attached by RESTful API with each cloud data center, and blocks of files is distributed in each cloud data
The heart.
The cloud service layer, is mainly made up of, cloud data center is that well-known Internet enterprises cloud is deposited some cloud data centers
Store up product, such as Amazon S3, Microsoft Azure, A Liyun, Tengxun's cloud and Baidu's cloud.
Embodiments of the invention choose 5 cloud data centers, respectively Amazon S3, Microsoft Azure, Ali
Cloud, Tengxun's cloud, Baidu's cloud, each back end position are respectively Tokyo, Eastern China data center, Hangzhou, Guangzhou, Beijing.
The cloudy fragmentation safe storage system based on correcting and eleting codes (is write a Chinese character in simplified form：RClouds), RClouds system schematics
As shown in figure 1, the data reading step of data storage and many cloud data centers including many cloud data centers；
The data storage of many cloud data centers, as shown in Fig. 2 including following three process：
(1.1) fragmentation of data
Assuming that user needs the data Data sizes stored to be 129643295B, cryptographic Hash is calculated using MD5 hash algorithms
For 6392103b85a39d790f979f7fcb21c320, then Key is used for the aes algorithm encryption data of the cryptographic Hash, generate
Encryption data Data '.
Encryption data cutting, using the processing of (6,4) RS correcting and eleting codes algorithm coding, is first that 4 sizes are by encryption data
32440320 data block Di, i ∈ [1,5], it is 32440320 encoding block E that re-encoding process, which generates 2 sizes,j, j ∈ [1,
2], a total of 6 blocks of files.After the completion of coded treatment, 1 part of meta data file is generated, meta data file includes data Data
Title, size, data block number k, encoding block number m, correcting and eleting codes algorithm title, key Key.Wherein meta data file is stored in member
In data server.
(1.2) data distribution
5 cloud data center transmission performances are assessed, are attached by RESTful API with each cloud data center, are defined
Response time θ=Te-Tb, wherein TeStart to receive the time of first data byte, T for corresponding cloud data centerbFor system
Start to send to corresponding cloud data center and send request time.θ values are ranked up from small to large, correspondence cloud data center is
Tengxun's cloud, A Liyun, Microsoft Azure, Baidu's cloud, Amazon S3.Equal-sized 6 are generated after encryption data coding
Individual blocks of files, wherein D1It is distributed to Tengxun's cloud, D2It is distributed to Ali's cloud, D3It is distributed to Microsoft Azure, D4It is distributed to hundred
Spend cloud, E1It is distributed to Amazon S3, E2It is distributed to Tengxun's cloud.
(1.3) data strip is stored
6 blocks of files are sequentially stored in different cloud data centers, and this 6 blocks of files are referred to as slice-group.
Fig. 4 is the date storage method flow chart of many cloud data centers.
The digital independent of many cloud data centers, as shown in figure 3, including following three process：
(2.1) meta data file is read
When user needs to read data, the metadata text stored after fragmentation of data is first read from meta data server
Part, obtains n file block message from meta data file, and the cloud data center of storage is searched according to fileinfo.
(2.2) fragmentation of dataization is read
Blocks of files patch information is obtained from cloud data center, starts to download blocks of files.If the file in certain cloud data center
Block failed download, then calculate according to other survival blocks of files codings and recover the data.Calculated again by key for Key symmetric cryptography
Method ciphertext data, obtains initial data.
(2.3) data convert
Cryptographic Hash computing is carried out to data after decryption using MD5 hashing algorithm, then is compared with original cryptographic Hash, if phase
It is initial data Deng the data then read, if unequal, reads data failure.
Fig. 5 is the digital independent flow chart of many cloud data centers.
RClouds system architectures are as shown in fig. 6, RClouds systems are main by fragmentation of data layer, data dispatch layer and cloud clothes
Business layer composition.
(1) fragmentation of data layer, mainly includes：1. user's application service.2. data encryption/decryption service.3. metadata takes
Business.4. data encoding/decoding service.
(2) data dispatch layer, mainly includes：1. data dispatch service.2. Data distributing.
(3) cloud service layer, mainly includes：Amazon S3, Microsoft Azure, A Liyun, Tengxun's cloud, Baidu's cloud etc.
Cloud data center.
Claims (10)
1. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes, it is characterised in that include the number of many cloud data centers
According to storing step and the data reading step of many cloud data centers,
The data storing steps of described many cloud data centers, including fragmentation of data step, data distribution step and data strip
Band storing step：
1), fragmentation of data：
Data are uploaded to user to be encrypted by key, encryption data is generated, and are generated after then encryption data is encoded
Multiple blocks of files, and generate the meta data file of file block message of the record including key；
2), data distribution：
According to the response time of each cloud data center according to being ranked up from small to large, by step 1) in generation blocks of files according to
Secondary each cloud data center being distributed to after sequence；
3), data strip is stored：
Each cloud data center provides continuous address space in logic, and same encryption data is stored in different cloud data centers
On blocks of files formation one slice-group；
The data reading step of described many cloud data centers, including meta data file read step, fragmentation of dataization read step
Rapid and data reduction step；
1., meta data file is read：
Meta data file is read from meta data server, to obtain n file block message of data；
2., fragmentation of dataization is read：
The cloud data center of corresponding document block is stored according to blocks of files information searching, and downloads blocks of files；
3., data convert：
By secret key decryption data, initial data is obtained.
2. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 1, it is characterised in that institute
The specific steps for the fragmentation of data step stated include：
First, data Data is encrypted for Key symmetric encipherment algorithm using key, generation encryption data Data '；
Secondly, using correcting and eleting codes algorithm coding encryption data Data ', n blocks of files is generated, including k according to block and m
Encoding block, and n=k+m；
Meanwhile, portion Data ' meta data file is produced, meta data file is stored in meta data server.Meta data file
Including data Data titles, size, data block number k, encoding block number m, correcting and eleting codes algorithm title, key Key.
3. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 1, it is characterised in that institute
In the fragmentation of data step stated, in addition to using hashing algorithm to user's upload data Data progress cryptographic Hash computings, will
The step of cryptographic Hash arrived is as symmetric encipherment algorithm key Key.
4. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 1, it is characterised in that institute
The specific steps for the data distribution step stated include：
The quantity of cloud data center is c, and c takes natural number, and according to response time size, cloud data center is ordered as into { C1,
C2,...,CcSet C, by n blocks of files distribution storage to S cloud data center, wherein S ∈ C；
If n ＞ c, the order sorted according to cloud data center distributes storage file block successively；
If 1≤n≤c, n cloud data center distribution storage file block before choosing.
5. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 1, it is characterised in that institute
In the data distribution step stated, response time θ=Te-Tb, wherein TeStart to receive first data for corresponding cloud data center
The time of byte, TbStart to send request time to corresponding cloud data center for system.
6. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 1, it is characterised in that institute
In the fragmentation of data read step stated, when downloading blocks of files, in addition to when the blocks of files in some cloud data center is downloaded
During failure, then decoding calculating is carried out according to the survival blocks of files downloaded from other cloud data centers, so as to recover the data.
7. a kind of cloudy fragmentation method for secure storing based on correcting and eleting codes according to claim 3, it is characterised in that institute
In the data convert step stated, in addition to using hashing algorithm to after decryption data carry out cryptographic Hash computing, then with original Hash
The step of value is compared, if cryptographic Hash is equal, the data read are initial data, are otherwise judged to reading data failure.
8. a kind of cloudy fragmentation safe storage system based on correcting and eleting codes, it is characterised in that adjusted including fragmentation of data layer, data
Spend layer and cloud service layer：
The fragmentation of data layer includes user's application service module, data encryption/decryption service module, data encoding/decoding clothes
Module of being engaged in and metadata service module；
(1), user's application service module is used to provide the interface service interacted with user, and user identity is identified and stepped on
Note, and be the different access of different type user setting and function privilege；
(2), data encryption/decryption service module is used to carry out by key respectively in data storage and reading data for user
Encryption and decryption service；
(3), data encoding service module is used to generate blocks of files, including it is multiple data blocks that the data after encryption are carried out into cutting
And the multiple encoding blocks of generation are calculated by correcting and eleting codes coding；Data decoding service module is used to download accordingly from each cloud data center
Blocks of files, pass through correcting and eleting codes decoding calculate recover encryption data；
(4), metadata service module is used for the meta data file for generating file block message of the record including key；
The data dispatch layer includes data dispatch service module and data distribution service module；
Ith, data dispatch service module is used for the response time according to system and each cloud data center, by cloud data center according to excellent
→ difference is ranked up, and the blocks of files that is generated after schedule code is carried out with this；
II, Data distributing module are used for by RESTful API (API design of internet application is theoretical) and each cloud
Data center is attached, and blocks of files is distributed into each cloud data center；
The cloud service layer is made up of multiple cloud data centers, is used for into data storage of racking.
9. a kind of cloudy fragmentation safe storage system based on correcting and eleting codes according to claim 8, it is characterised in that institute
The data encryption services module stated first calculates the cryptographic Hash of data, by cryptographic Hash in user storage data using hashing algorithm
As key Key, then key is used for Key symmetric encipherment algorithm encryption data；
Described data deciphering service module obtains key Key values, then use when user reads data from meta data file
Key is Key symmetric encipherment algorithm ciphertext data.
10. a kind of cloudy fragmentation safe storage system based on correcting and eleting codes according to claim 8, it is characterised in that
Described data encoding service module by the data after encryption using the average cutting of (n, k) correcting and eleting codes algorithms be k according to block, then
M encoding block is produced by encoding to calculate, wherein k is equal according to the size of block and m encoding block, symbiosis is into n=k+m
Blocks of files；
Being recorded in the meta data file that described metadata service module is generated has data name, size, data block number k,
Encoding block number m, correcting and eleting codes algorithm title and key Key.
Described data decoding service module first obtains the storage of each blocks of files when user reads data from meta data file
Cloud data center information, then download corresponding blocks of files from each cloud data center, if partial document block is unavailable or failure,
Calculated according to the decoding of (n, k) correcting and eleting codes algorithm and recover data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710401812.0A CN107154945A (en) | 2017-05-31 | 2017-05-31 | A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710401812.0A CN107154945A (en) | 2017-05-31 | 2017-05-31 | A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107154945A true CN107154945A (en) | 2017-09-12 |
Family
ID=59795985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710401812.0A Pending CN107154945A (en) | 2017-05-31 | 2017-05-31 | A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107154945A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107645558A (en) * | 2017-09-30 | 2018-01-30 | 华侨大学 | A kind of three-level secure cloud storage method calculated based on mist |
| CN108197484A (en) * | 2018-01-23 | 2018-06-22 | 中南民族大学 | A kind of method that node data safety is realized under distributed storage environment |
| CN109033367A (en) * | 2018-07-26 | 2018-12-18 | 电子科技大学 | A kind of attribute towards two-dimentional list file is perpendicular to cut cloudy storage method |
| CN109787986A (en) * | 2019-01-29 | 2019-05-21 | 黄策 | File fragmentation public network safe transmission method |
| CN109814812A (en) * | 2019-02-12 | 2019-05-28 | 中天宽带技术有限公司 | The rapid data transfer method placed based on content fragmentation |
| CN109815707A (en) * | 2019-01-31 | 2019-05-28 | 清华大学 | Date storage method and system |
| CN110381061A (en) * | 2019-07-19 | 2019-10-25 | 广东省新一代通信与网络创新研究院 | Cloudy storage method, method for down loading, device and the storage medium of file |
| CN110493323A (en) * | 2019-07-29 | 2019-11-22 | 华南理工大学 | Fairness document distribution method, system and storage medium based on block chain |
| CN110636141A (en) * | 2019-10-17 | 2019-12-31 | 中国人民解放军陆军工程大学 | Multi-cloud storage system based on cloud and mist cooperation and management method thereof |
| CN110659296A (en) * | 2019-09-26 | 2020-01-07 | 广州华多网络科技有限公司 | Storage method, device, equipment and computer readable medium |
| CN111310245A (en) * | 2020-03-05 | 2020-06-19 | 之江实验室 | Data encryption storage method for mimicry defense system |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
| CN112486941A (en) * | 2020-11-30 | 2021-03-12 | 佛山赛思禅科技有限公司 | Mimicry object storage system based on multiple erasure codes |
| CN112714031A (en) * | 2021-03-29 | 2021-04-27 | 中南大学 | Fault node rapid repairing method based on bandwidth sensing |
| CN113194330A (en) * | 2021-03-25 | 2021-07-30 | 电子科技大学 | Fragmented multi-cloud video resource management method and system |
| CN115514470A (en) * | 2022-11-22 | 2022-12-23 | 中网道科技集团股份有限公司 | Storage method and system for community correction data security |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103118089A (en) * | 2013-01-22 | 2013-05-22 | 华中科技大学 | Safe storage method based on a plurality of cloud storage systems and system thereof |
| CN103729470A (en) * | 2014-01-20 | 2014-04-16 | 刘强 | Secure storage method based on different cloud storage ends |
| CN104468664A (en) * | 2013-09-18 | 2015-03-25 | 中兴通讯股份有限公司 | Method and device for uploading files to cloud storage system, and method and device for downloading files from cloud storage system |
| CN104539733A (en) * | 2015-01-20 | 2015-04-22 | 电子科技大学 | Private fragmented file cloud synchronization method |
| CN104754055A (en) * | 2015-04-03 | 2015-07-01 | 易云捷讯科技（北京）有限公司 | Safety cloud storage method for use in multi-cloud environment |
| CN106603673A (en) * | 2016-12-19 | 2017-04-26 | 上海交通大学 | Fine-grained cloud storage scheduling method based on erasure codes |
| CN106657266A (en) * | 2016-11-10 | 2017-05-10 | 南京云创大数据科技股份有限公司 | Architecture and method of secure and reliable public cloud storage system based on public clouds |
-
2017
- 2017-05-31 CN CN201710401812.0A patent/CN107154945A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103118089A (en) * | 2013-01-22 | 2013-05-22 | 华中科技大学 | Safe storage method based on a plurality of cloud storage systems and system thereof |
| CN104468664A (en) * | 2013-09-18 | 2015-03-25 | 中兴通讯股份有限公司 | Method and device for uploading files to cloud storage system, and method and device for downloading files from cloud storage system |
| CN103729470A (en) * | 2014-01-20 | 2014-04-16 | 刘强 | Secure storage method based on different cloud storage ends |
| CN104539733A (en) * | 2015-01-20 | 2015-04-22 | 电子科技大学 | Private fragmented file cloud synchronization method |
| CN104754055A (en) * | 2015-04-03 | 2015-07-01 | 易云捷讯科技（北京）有限公司 | Safety cloud storage method for use in multi-cloud environment |
| CN106657266A (en) * | 2016-11-10 | 2017-05-10 | 南京云创大数据科技股份有限公司 | Architecture and method of secure and reliable public cloud storage system based on public clouds |
| CN106603673A (en) * | 2016-12-19 | 2017-04-26 | 上海交通大学 | Fine-grained cloud storage scheduling method based on erasure codes |
Non-Patent Citations (4)
| Title |
|---|
| 卿昱: "《云计算安全技术》", 31 December 2016 * |
| 李泽锋: "《数据档案馆建设与管理》", 31 December 2005 * |
| 王健宗: "云存储服务质量的若干关键问题研究", 《中国博士学位论文全文数据库信息科技辑》 * |
| 顾瑜: "云计算环境下数据保护关键技术研究", 《中国博士学位论文全文数据库信息科技辑》 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107645558A (en) * | 2017-09-30 | 2018-01-30 | 华侨大学 | A kind of three-level secure cloud storage method calculated based on mist |
| CN108197484A (en) * | 2018-01-23 | 2018-06-22 | 中南民族大学 | A kind of method that node data safety is realized under distributed storage environment |
| CN108197484B (en) * | 2018-01-23 | 2020-03-10 | 中南民族大学 | Method for realizing node data security in distributed storage environment |
| CN109033367A (en) * | 2018-07-26 | 2018-12-18 | 电子科技大学 | A kind of attribute towards two-dimentional list file is perpendicular to cut cloudy storage method |
| CN109787986A (en) * | 2019-01-29 | 2019-05-21 | 黄策 | File fragmentation public network safe transmission method |
| CN109815707A (en) * | 2019-01-31 | 2019-05-28 | 清华大学 | Date storage method and system |
| CN109814812B (en) * | 2019-02-12 | 2022-03-01 | 中天宽带技术有限公司 | Rapid data transfer method based on content fragmentation placement |
| CN109814812A (en) * | 2019-02-12 | 2019-05-28 | 中天宽带技术有限公司 | The rapid data transfer method placed based on content fragmentation |
| CN110381061A (en) * | 2019-07-19 | 2019-10-25 | 广东省新一代通信与网络创新研究院 | Cloudy storage method, method for down loading, device and the storage medium of file |
| CN110493323A (en) * | 2019-07-29 | 2019-11-22 | 华南理工大学 | Fairness document distribution method, system and storage medium based on block chain |
| CN110659296A (en) * | 2019-09-26 | 2020-01-07 | 广州华多网络科技有限公司 | Storage method, device, equipment and computer readable medium |
| CN110659296B (en) * | 2019-09-26 | 2021-02-12 | 广州方硅信息技术有限公司 | Storage method, device, equipment and computer readable medium |
| CN110636141A (en) * | 2019-10-17 | 2019-12-31 | 中国人民解放军陆军工程大学 | Multi-cloud storage system based on cloud and mist cooperation and management method thereof |
| CN111310245A (en) * | 2020-03-05 | 2020-06-19 | 之江实验室 | Data encryption storage method for mimicry defense system |
| CN112052141A (en) * | 2020-09-02 | 2020-12-08 | 平安科技（深圳）有限公司 | Data fragment verification method and device, computer equipment and readable storage medium |
| CN112486941A (en) * | 2020-11-30 | 2021-03-12 | 佛山赛思禅科技有限公司 | Mimicry object storage system based on multiple erasure codes |
| CN113194330A (en) * | 2021-03-25 | 2021-07-30 | 电子科技大学 | Fragmented multi-cloud video resource management method and system |
| CN112714031A (en) * | 2021-03-29 | 2021-04-27 | 中南大学 | Fault node rapid repairing method based on bandwidth sensing |
| CN112714031B (en) * | 2021-03-29 | 2021-06-22 | 中南大学 | Fault node rapid repairing method based on bandwidth sensing |
| CN115514470A (en) * | 2022-11-22 | 2022-12-23 | 中网道科技集团股份有限公司 | Storage method and system for community correction data security |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107154945A (en) | A kind of cloudy fragmentation method for secure storing and system based on correcting and eleting codes | |
| TWI720918B (en) | Consenus of shared blockchain data storage based on error correction code | |
| TWI740575B (en) | Method, system and device for prioritizing shared blockchain data storage | |
| KR102412024B1 (en) | Indexing and recovery of encoded blockchain data | |
| CN111902817A (en) | Block chain data storage based on shared nodes and error correction coding | |
| AU2015259417A1 (en) | Distributed secure data storage and transmission of streaming media content | |
| AU2019320956B2 (en) | Data security of shared blockchain data storage based on error correction code | |
| CN103118089A (en) | Safe storage method based on a plurality of cloud storage systems and system thereof | |
| CN103530201A (en) | Safety data repetition removing method and system applicable to backup system | |
| CN114466015B (en) | Data storage system and method based on multi-cloud architecture | |
| CN108182367B (en) | A kind of encrypted data chunk client De-weight method for supporting data to update | |
| CN111095210A (en) | Storing shared blockchain data based on error correction coding | |
| CN109451004A (en) | Cloud storage system and method based on cloud bridge | |
| CN108183973A (en) | A kind of Streaming Media quantum flow block data structure and encipher-decipher method | |
| CN104967660B (en) | A kind of network performance method for improving towards more cloud frameworks | |
| KR20210058746A (en) | Dynamic blockchain data storage based on error correction codes | |
| CN106991332A (en) | The method and device that a kind of mass data is stored safely | |
| CN101945125A (en) | Method and device for transmitting documents | |
| CN109491591A (en) | A kind of information diffusion method suitable for cloudy storage system | |
| CN103248713A (en) | Synchronous disc data security protection method | |
| CN111565144A (en) | Data layered storage management method for instant communication tool | |
| CN115499244A (en) | Streaming data safe transmission and storage method based on data lake | |
| CN110648140B (en) | Multi-chain matching method and device based on block chain | |
| CN110958285B (en) | Data storage system based on block chain | |
| RU2656836C2 (en) | System and method of interaction of users with cloud target data storage |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170912 |