CN106708653A - Mixed tax administration data security protecting method based on erasure code and multi-copy - Google Patents
Mixed tax administration data security protecting method based on erasure code and multi-copy Download PDFInfo
- Publication number
- CN106708653A CN106708653A CN201611252092.8A CN201611252092A CN106708653A CN 106708653 A CN106708653 A CN 106708653A CN 201611252092 A CN201611252092 A CN 201611252092A CN 106708653 A CN106708653 A CN 106708653A
- Authority
- CN
- China
- Prior art keywords
- data
- correcting
- eleting codes
- tax
- many
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000003860 storage Methods 0.000 claims abstract description 75
- 238000012545 processing Methods 0.000 claims abstract description 13
- 238000007726 management method Methods 0.000 claims description 37
- 238000012360 testing method Methods 0.000 claims description 18
- 230000008569 process Effects 0.000 claims description 9
- 230000009467 reduction Effects 0.000 claims description 7
- 238000013523 data management Methods 0.000 claims description 5
- 238000013500 data storage Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 239000011248 coating agent Substances 0.000 claims description 3
- 238000000576 coating method Methods 0.000 claims description 3
- 230000014759 maintenance of location Effects 0.000 claims description 2
- 238000011084 recovery Methods 0.000 abstract 1
- 238000013459 approach Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000001568 sexual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a mixed tax administration data security protecting method based on erasure code and multi-copy. The method comprises the following steps: while the tax administration data of a tax administration data distributed storage system is normal, starting the multi-copy and erasure code storage mode storage flow of the tax administration data; and while the tax administration data of the tax administration data distributed storage system is failure, starting the tax administration data fault-tolerance processing flow. The method is capable of executing the sub-mode storage by using the tax administration data features of different times, distributing the erasure coding task to different nodes, using the mode of copy first and erasure code second, comprehensively improving the security of the whole tax administration data and the data recovery performance, improving the coding performance of the whole system, and guaranteeing the data security before finishing the erasure coding.
Description
Technical field
The present invention relates to computer data management technical field, and in particular to a kind of mixing based on correcting and eleting codes with many copies
Tax big data method for security protection.
Background technology
With the development that deepens continuously of economic globalization and China's economy, China's taxpayer's quantity rapidly increases, the tax category are got over
Hair is abundant, and in face of increasingly huger tax data, distributed storage is a storage scheme for main flow, with sexual valence very high
Than and autgmentability.For tax data, its problem of data safety in distributed storage environment is the pass for being worth research
Key point.Distributed memory system includes great deal of nodes, and node failure or outside invasion are likely to cause data imperfect.For
Loss of data is avoided, generally using the fault-tolerance approach based on redundant data, redundancy fault-tolerant mainly there are two kinds：One kind is many copies
It is fault-tolerant, by replicate redundant data carry out it is fault-tolerant；Another kind is fault-tolerant correcting and eleting codes, is held by encoding generation redundant data
It is wrong.
The fault-tolerance approach being widely used at present is fault-tolerant based on many copies for replicating：By former data duplication into c copy,
When c data trnascription then being distributed into c different memory node, so any c-1 node failure, each data is at least
Also 1 copy is present.Many copies are fault-tolerant have the advantages that it is simple easily realize, computing cost is few, data access performance is good.But
Many copies are fault-tolerant also to be had the shortcomings that to protrude very much：Storage overhead is very big.It is this for tax data very huge in itself, and always
Keep for the data of rapid growth, many copies based on duplication are fault-tolerant and do not apply to.
With the growth of data explosion formula, correcting and eleting codes are fault-tolerant because it can be identical even with much lower storage overhead offer
Data reliability higher, also begins to turn into study hotspot in recent years.The fault-tolerant strategy of correcting and eleting codes is：One data is divided into c
Individual data block, is then encoded into c data block n (n ＞ c) individual encoding block and is distributed in n different disk, so when node loses
During effect, as long as the data also have c encoding block to exist, it becomes possible to decode former data to come.With three copies being widely used
Fault-tolerant networks are compared, and memory space consumption can both be reduced by 53% by RS correcting and eleting codes, also fault-tolerant ability can be improved into one simultaneously
Times.Degraded performance when but the defect of correcting and eleting codes is data reconstruction, especially in distributed storage, because data reconstruction is needed
Want multiple nodes to cooperate, inevitably bring substantial amounts of network resource consumption and computing resource to consume.For tax number
For this distributed data, this will turn into the critical bottleneck of whole system performance.
The content of the invention
In view of this, in order to solve above mentioned problem of the prior art, the present invention proposes that one kind is secondary based on correcting and eleting codes and more
This mixing tax big data method for security protection.
The present invention is solved the above problems by following technological means：
A kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies, when tax data distribution is deposited
When the tax data of storage system is normal, start many copies and correcting and eleting codes storage mode Stored Procedure of tax data；
When the tax data of tax data distributed memory system fails, start tax data fault-tolerant processing flow；
Many copies comprise the following steps with correcting and eleting codes storage mode Stored Procedure：
Step S11, tax data is temporally divided into historical data and Recent data, and the Recent data includes multiple
Different Recent data bags；
Step S12, the Recent data is stored in many copy memory modules according to many copy storage modes, is gone through described
History data are stored in correcting and eleting codes memory module according to correcting and eleting codes storage mode；
Step S13, when Recent data coating is labeled as completion status, then dumps to correcting and eleting codes by the Recent data bag
Memory module is so as to history of forming data；
The tax data fault-tolerant processing flow comprises the following steps：
Step S21, according to many copy memory module data management nodes, judges that failure tax data storage is deposited in many copies
In storage module or in correcting and eleting codes memory module；
Step S22, if the tax data of failure is stored in many copy memory modules, according to the note of many replica management nodes
Record and send test packet to related many copy memory nodes, it is corresponding with failure tax data according to the selection of test packet feedback delay
Copy, and copy is reverted into effective tax data；
Step S23, if the tax data of failure is stored in correcting and eleting codes memory module, need to further search for correcting and eleting codes pipe
The record of node is managed, test packet is sent to related correcting and eleting codes memory node, then selected successively according to test packet feedback delay
Corresponding encoded block is selected, after obtaining sufficient amount encoding block, you can reduction recovers tax data；
The tax data distributed memory system, for providing storage and fault tolerant service for tax data；
The data that the tax data is input into for the client of tax data distributed memory system；
The historical data is the data before tax data distributed memory system time division points, is stored in correcting and eleting codes
Memory module；
The Recent data, is the data after tax data distributed memory system time division points, and storage is more secondary
This memory module；
Many copy memory modules, for storing and processing Recent data, including copy memory module data more than
Management node and copy memory node more than at least one；
Many replica management nodes, duplication, distribution and storage for managing data in many copy memory modules, and it is right
Data message is recorded；
Many copy memory nodes, for storing Recent data；
The correcting and eleting codes memory module, for storing and processing historical data, including a correcting and eleting codes management node with extremely
A few correcting and eleting codes memory node；
The correcting and eleting codes management node is for managing the coding of data in correcting and eleting codes memory module, distribution and storing and right
Data message is recorded；
The correcting and eleting codes memory node, for store historical data；
Many copy storage modes, for reading, storing, record by tax data distribution formula storage system and extensive
Multiple Recent data；
The correcting and eleting codes storage mode, for by tax data distribution formula storage system come unloading, reading, record and it is extensive
Multiple historical data；
The encoding block is the encoding block formed after the Recent data of unloading is by subpackage and coding, and storage is deposited in correcting and eleting codes
Storage node, tax data is reverted to for the reduction during tax data fault-tolerant processing.
Further, the correcting and eleting codes storage mode described in step S12 is comprised the following steps：
Step S1221, judges whether the outside visiting frequency to correcting and eleting codes memory module is less than by correcting and eleting codes management node
Visiting frequency threshold value, so as to judge that whether current correcting and eleting codes memory module, in idle condition, is if it is activated all to entangle and deleted
Code memory node；
Step S1222, makes the following judgment to the correcting and eleting codes memory node that each is activated：The correcting and eleting codes memory node
Storage load whether exceed storage loading thresholds, and whether the offered load of the correcting and eleting codes memory node fully loaded more than network
Threshold value, if be no more than, unloading data is treated to the request of many replica management nodes；
Step S1223, by after after unloading data encoding, distributing and be stored in correcting and eleting codes memory node, and will distribute information
Record is in correcting and eleting codes management node；
Step S1224, after confirming data conversion storage success, by the tax data and its pair of unloading in many copy memory modules
This whole is deleted；
It is described to treat that unloading data are some copy data that unloading data are applied in many replica management nodes records, should
The selection principle of copy data need to meet load balancing, and the copy data is used to distributing and being stored in correcting and eleting codes storage section after encoding
Point；
The distribution information is the record information that multiple encoding blocks are distributed to multiple correcting and eleting codes memory nodes, many for guiding
Individual encoding block reduction reverts to tax data.
Further, in many copy storage modes described in step S12, the handling process for writing Recent data is included such as
Lower step：
Step S1231, when client sends Recent data request write-in, many replica management nodes are responded；
Step S1232, the tax data to writing carries out duplication and forms copy, and the tax data and its copy that will be write
It is dividedly stored in different many copy memory nodes；
Step S1233, the storage information of the tax data that will be write is recorded in many replica management nodes.
Further, in many copy storage modes described in step S12, the handling process for reading Recent data is included such as
Lower step：
Step S1241, when client sends Recent data read requests, many replica management nodes respond and basis
Record and send test packet to related many copy memory nodes and ask computational load；
Step S1242, the time delay fed back by test packet and the computational load of related many copy memory nodes are come comprehensive
Select corresponding many copy memory nodes；
Step S1243, the tax data that the distribution according to many replica management nodes allows in corresponding many copy memory nodes
It is sent directly in client；
The client is the client of distributed memory system, for writing and reading tax data.
Further, in step S1232, the tax data of write-in and its location mode of copy are by same tax data
Different copies be physically separated, select different rack or computer room storage.
Further, the process that the encoding block is formed is comprised the following steps：
Step 61, treats that the Recent data of unloading is packetized into C data block；
Step 62, N number of encoding block is encoded into by C data block, and the number of the N is more than C；
Step 63, N number of encoding block is distributed to N number of different correcting and eleting codes memory module；
Step 64, information record is distributed in the correcting and eleting codes management node where encoding block by N number of encoding block.
The present invention carries out merotype storage using the tax data feature of different time, comprehensively improves whole tax data
Security and data repairing performance.The characteristics of there is phasic Chang due to the visiting frequency of tax data, recent data
Visiting frequency is highest, and the visiting frequency of historical data is then relatively low.Correcting and eleting codes are used in the low data of visiting frequency,
Memory space utilization rate can be improved, many copies are used in visiting frequency data high, improve data repairing performance.
Secondly, be distributed in erasure code task on different nodes by the present invention, and takes into full account node during selection node
Loading condition, computational load and network transport load are shared in multiple nodes, improve the overall coding efficiency of system.
Again, the present invention ensure that the data before erasure code completion using the pattern of correcting and eleting codes after first copy
Security, compensate for the situation of loss of data in the coding being likely to encounter when fault-tolerant using correcting and eleting codes merely.
Brief description of the drawings
Technical scheme in order to illustrate more clearly the embodiments of the present invention, below will be to wanting needed for embodiment description
The accompanying drawing for using is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present invention, right
For those of ordinary skill in the art, on the premise of not paying creative work, it can also be obtained according to these accompanying drawings
His accompanying drawing.
Fig. 1 is a kind of workflow based on correcting and eleting codes with the mixing tax big data method for security protection of many copies of the present invention
Cheng Tu；
Fig. 2 is the structural representation of many copy memory modules of the invention；
Fig. 3 is the structural representation of correcting and eleting codes memory module of the invention.
Specific embodiment
To enable the above objects, features and advantages of the present invention more obvious understandable, below in conjunction with accompanying drawing and specifically
Embodiment technical scheme is described in detail.It is pointed out that described embodiment is only this hair
Bright a part of embodiment, rather than whole embodiments, based on the embodiment in the present invention, those of ordinary skill in the art are not having
There is the every other embodiment made and being obtained under the premise of creative work, belong to the scope of protection of the invention.
As shown in figure 1, a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies, when tax number
According to the tax data of distributed memory system it is normal when, many copies and the correcting and eleting codes storage mode for starting tax data store stream
Journey；
When the tax data of tax data distributed memory system fails, start tax data fault-tolerant processing flow；
Many copies comprise the following steps with correcting and eleting codes storage mode Stored Procedure：
Step S11, tax data is temporally divided into historical data and Recent data, and the Recent data includes multiple
Different Recent data bags；
Step S12, the Recent data is stored in many copy memory modules according to many copy storage modes, is gone through described
History data are stored in correcting and eleting codes memory module according to correcting and eleting codes storage mode；
Step S13, the Recent data bag when Recent data coating is labeled as completion status, then dumped to entangle and deleted
Code memory module is so as to history of forming data；
The tax data fault-tolerant processing flow comprises the following steps：
Step S21, according to many copy memory module data management nodes, judges that failure tax data storage is deposited in many copies
In storage module or in correcting and eleting codes memory module；
Step S22, if the tax data of failure is stored in many copy memory modules, according to the note of many replica management nodes
Record and send test packet to related many copy memory nodes, it is corresponding with failure tax data according to the selection of test packet feedback delay
Copy, and copy is reverted into effective tax data；
Step S23, if the tax data of failure is stored in correcting and eleting codes memory module, need to further search for correcting and eleting codes management
The record of node, test packet is sent to related correcting and eleting codes memory node, is then selected successively according to test packet feedback delay
Corresponding encoded block, after obtaining sufficient amount encoding block, you can reduction recovers tax data；
The tax data distributed memory system, for providing storage and fault tolerant service for tax data；
The data that the tax data is input into for the client of tax data distributed memory system；
The historical data is the data before tax data distributed memory system time division points, is stored in correcting and eleting codes
Memory module；
The Recent data, is the data after tax data distributed memory system time division points, and storage is more secondary
This memory module.
As shown in Fig. 2 many copy memory modules, for storing and processing Recent data, including copy is deposited more than one
Storage module data management node and copy memory node more than at least one；
Many replica management nodes, duplication, distribution and storage for managing data in many copy memory modules, and it is right
Data message is recorded；
Many copy memory nodes, for storing Recent data.
As shown in figure 3, the correcting and eleting codes memory module, for storing and processing historical data, including a correcting and eleting codes pipe
Reason node and at least one correcting and eleting codes memory node；
The correcting and eleting codes management node is for managing the coding of data in correcting and eleting codes memory module, distribution and storing and right
Data message is recorded；
The correcting and eleting codes memory node, for store historical data.
Many copy storage modes, for reading, storing, record by tax data distribution formula storage system and extensive
Multiple Recent data；
The correcting and eleting codes storage mode, for by tax data distribution formula storage system come unloading, reading, record and it is extensive
Multiple historical data；
The encoding block is the encoding block formed after the Recent data of unloading is by subpackage and coding, and storage is deposited in correcting and eleting codes
Storage node, tax data is reverted to for the reduction during tax data fault-tolerant processing.
Correcting and eleting codes storage mode described in step S12 is comprised the following steps：
Step S1221, judges whether the outside visiting frequency to correcting and eleting codes memory module is less than by correcting and eleting codes management node
Visiting frequency threshold value, so as to judge that whether current correcting and eleting codes memory module, in idle condition, is if it is activated all to entangle and deleted
Code memory node；
Step S1222, makes the following judgment to the correcting and eleting codes memory node that each is activated：The correcting and eleting codes memory node
Storage load whether exceed storage loading thresholds, and whether the offered load of the correcting and eleting codes memory node fully loaded more than network
Threshold value, if be no more than, unloading data is treated to the request of many replica management nodes；
Step S1223, by after after unloading data encoding, distributing and be stored in correcting and eleting codes memory node, and will distribute information
Record is in correcting and eleting codes management node；
Step S1224, after confirming data conversion storage success, by the tax data and its pair of unloading in many copy memory modules
This whole is deleted；
It is described to treat that unloading data are some copy data that unloading data are applied in many replica management nodes records, should
The selection principle of copy data need to meet load balancing, and the copy data is used to distributing and being stored in correcting and eleting codes storage section after encoding
Point；
The distribution information is the record information that multiple encoding blocks are distributed to multiple correcting and eleting codes memory nodes, many for guiding
Individual encoding block reduction reverts to tax data.
In many copy storage modes described in step S12, the handling process for writing Recent data comprises the following steps：
Step S1231, when client sends Recent data request write-in, many replica management nodes are responded；
Step S1232, the tax data to writing carries out duplication and forms copy, and the tax data and its copy that will be write
It is dividedly stored in different many copy memory nodes；
Step S1233, the storage information of the tax data that will be write is recorded in many replica management nodes.
In many copy storage modes described in step S12, the handling process for reading Recent data comprises the following steps：
Step S1241, when client sends Recent data read requests, many replica management nodes respond and basis
Record and send test packet to related many copy memory nodes and ask computational load；
Step S1242, the time delay fed back by test packet and the computational load of related many copy memory nodes are come comprehensive
Select corresponding many copy memory nodes；
Step S1243, the tax data that the distribution according to many replica management nodes allows in corresponding many copy memory nodes
It is sent directly in client；
The client is the client of distributed memory system, for writing and reading tax data.
In step S1232, the tax data of write-in and its location mode of copy are that the difference of same tax data is secondary
Originally it is physically separated, selects different rack or computer room storages.
The process that the encoding block is formed is comprised the following steps：
Step 61, treats that the Recent data of unloading is packetized into C data block；
Step 62, N number of encoding block is encoded into by C data block, and the number of the N is more than C；
Step 63, N number of encoding block is distributed to N number of different correcting and eleting codes memory module；
Step 64, information record is distributed in the correcting and eleting codes management node where encoding block by N number of encoding block.
The present invention carries out merotype storage using the tax data feature of different time, comprehensively improves whole tax data
Security and data repairing performance.The characteristics of there is phasic Chang due to the visiting frequency of tax data, recent data
Visiting frequency is highest, and the visiting frequency of historical data is then relatively low.Correcting and eleting codes are used in the low data of visiting frequency,
Memory space utilization rate can be improved, many copies are used in visiting frequency data high, improve data repairing performance.
Secondly, be distributed in erasure code task on different nodes by the present invention, and takes into full account node during selection node
Loading condition, computational load and network transport load are shared in multiple nodes, improve the overall coding efficiency of system.
Again, the present invention ensure that the data before erasure code completion using the pattern of correcting and eleting codes after first copy
Security, compensate for the situation of loss of data in the coding being likely to encounter when fault-tolerant using correcting and eleting codes merely.
Embodiment described above only expresses several embodiments of the invention, and its description is more specific and detailed, but simultaneously
Therefore the limitation to the scope of the claims of the present invention can not be interpreted as.It should be pointed out that for one of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to guarantor of the invention
Shield scope.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (6)
1. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies, it is characterised in that when tax number
According to the tax data of distributed memory system it is normal when, many copies and the correcting and eleting codes storage mode for starting tax data store stream
Journey；
When the tax data of tax data distributed memory system fails, start tax data fault-tolerant processing flow；
Many copies comprise the following steps with correcting and eleting codes storage mode Stored Procedure：
Step S11, tax data is temporally divided into historical data and Recent data, and the Recent data includes multiple different
Recent data bag；
Step S12, the Recent data is stored in many copy memory modules according to many copy storage modes, by the history number
Stored in correcting and eleting codes memory module according to according to correcting and eleting codes storage mode；
Step S13, when Recent data coating is labeled as completion status, then dumps to correcting and eleting codes storage by the Recent data bag
Module is so as to history of forming data；
The tax data fault-tolerant processing flow comprises the following steps：
Step S21, according to many copy memory module data management nodes, judges that failure tax data storage stores mould in many copies
In block or in correcting and eleting codes memory module；
Step S22, if the tax data storage of failure is in many copy memory modules, according to many replica management nodes record to
Related many copy memory nodes send test packet, according to test packet feedback delay selection pair corresponding with failure tax data
This, and copy is reverted into effective tax data；
Step S23, if the tax data of failure is stored in correcting and eleting codes memory module, need to further search for correcting and eleting codes management node
Record, to related correcting and eleting codes memory node send test packet, then selected successively accordingly according to test packet feedback delay
Encoding block, after obtaining sufficient amount encoding block, you can reduction recovers tax data.
2. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies as claimed in claim 1, its
It is characterised by, the correcting and eleting codes storage mode described in step S12 is comprised the following steps：
Whether step S1221, judge the outside visiting frequency to correcting and eleting codes memory module less than access by correcting and eleting codes management node
Frequency threshold, so as to judge that whether current correcting and eleting codes memory module, in idle condition, if it is activates whole correcting and eleting codes and deposits
Storage node；
Step S1222, makes the following judgment to the correcting and eleting codes memory node that each is activated：The correcting and eleting codes memory node is deposited
Whether storage load exceedes storage loading thresholds, and whether the offered load of the correcting and eleting codes memory node is fully loaded with threshold more than network
Value, if be no more than, unloading data is treated to the request of many replica management nodes；
Step S1223, by after after unloading data encoding, distributing and be stored in correcting and eleting codes memory node, and will distribute information record
In correcting and eleting codes management node；
Step S1224 is complete by the tax data and its copy of unloading in many copy memory modules after confirming data conversion storage success
Delete in portion.
3. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies as claimed in claim 1, its
It is characterised by, in many copy storage modes described in step S12, the handling process for writing Recent data comprises the following steps：
Step S1231, when client sends Recent data request write-in, many replica management nodes are responded；
Step S1232, the tax data to writing carries out duplication and forms copy, and the tax data and its copy of write-in are separated
It is stored in different many copy memory nodes；
Step S1233, the storage information of the tax data that will be write is recorded in many replica management nodes.
4. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies as claimed in claim 1, its
It is characterised by, in many copy storage modes described in step S12, the handling process for reading Recent data comprises the following steps：
Step S1241, when client sends Recent data read requests, many replica management nodes are responded and according to record
Test packet is sent to related many copy memory nodes and ask computational load；
Step S1242, the time delay and the computational load of related many copy memory nodes fed back by test packet are come comprehensive selection
Corresponding many copy memory nodes；
Step S1243, the tax data that the distribution according to many replica management nodes allows in corresponding many copy memory nodes is direct
It is sent in client.
5. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies as claimed in claim 3, its
It is characterised by, in step S1232, the tax data of write-in and its location mode of copy are that the difference of same tax data is secondary
Originally it is physically separated, selects different rack or computer room storages.
6. a kind of mixing tax big data method for security protection based on correcting and eleting codes with many copies as claimed in claim 1, its
It is characterised by, the process that the encoding block is formed is comprised the following steps：
Step 61, treats that the Recent data of unloading is packetized into C data block；
Step 62, N number of encoding block is encoded into by C data block, and the number of the N is more than C；
Step 63, N number of encoding block is distributed to N number of different correcting and eleting codes memory module；
Step 64, information record is distributed in the correcting and eleting codes management node where encoding block by N number of encoding block.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611252092.8A CN106708653B (en) | 2016-12-29 | 2016-12-29 | Mixed tax big data security protection method based on erasure code and multiple copies |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611252092.8A CN106708653B (en) | 2016-12-29 | 2016-12-29 | Mixed tax big data security protection method based on erasure code and multiple copies |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106708653A true CN106708653A (en) | 2017-05-24 |
| CN106708653B CN106708653B (en) | 2020-06-30 |
Family
ID=58904096
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611252092.8A Active CN106708653B (en) | 2016-12-29 | 2016-12-29 | Mixed tax big data security protection method based on erasure code and multiple copies |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106708653B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108196978A (en) * | 2017-12-22 | 2018-06-22 | 新华三技术有限公司 | Date storage method, device, data-storage system and readable storage medium storing program for executing |
| CN108255432A (en) * | 2018-01-12 | 2018-07-06 | 郑州云海信息技术有限公司 | Write operation control method, system, device and storage medium based on bedding storage |
| CN110196682A (en) * | 2018-06-15 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Data managing method, calculates equipment and storage medium at device |
| CN110209670A (en) * | 2019-05-09 | 2019-09-06 | 北京猫盘技术有限公司 | Data processing method and device based on network storage equipment cluster |
| CN111008181A (en) * | 2019-10-31 | 2020-04-14 | 苏州浪潮智能科技有限公司 | Method, system, terminal and storage medium for switching storage strategies of distributed file system |
| CN111381767A (en) * | 2018-12-28 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN111782582A (en) * | 2019-06-14 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Data conversion method, system and name node |
| CN112965660A (en) * | 2021-02-09 | 2021-06-15 | 山东英信计算机技术有限公司 | Method, system, device and medium for feeding back information of double storage pools |
| CN114398006A (en) * | 2021-12-24 | 2022-04-26 | 中国电信股份有限公司 | Distributed storage mode control method, device, equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103118133A (en) * | 2013-02-28 | 2013-05-22 | 浙江大学 | Mixed cloud storage method based on file access frequency |
| CN105472047A (en) * | 2016-02-03 | 2016-04-06 | 天津书生云科技有限公司 | Storage system |
-
2016
- 2016-12-29 CN CN201611252092.8A patent/CN106708653B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103118133A (en) * | 2013-02-28 | 2013-05-22 | 浙江大学 | Mixed cloud storage method based on file access frequency |
| CN105472047A (en) * | 2016-02-03 | 2016-04-06 | 天津书生云科技有限公司 | Storage system |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108196978B (en) * | 2017-12-22 | 2021-03-09 | 新华三技术有限公司 | Data storage method, device, data storage system and readable storage medium |
| CN108196978A (en) * | 2017-12-22 | 2018-06-22 | 新华三技术有限公司 | Date storage method, device, data-storage system and readable storage medium storing program for executing |
| CN108255432A (en) * | 2018-01-12 | 2018-07-06 | 郑州云海信息技术有限公司 | Write operation control method, system, device and storage medium based on bedding storage |
| CN110196682A (en) * | 2018-06-15 | 2019-09-03 | 腾讯科技（深圳）有限公司 | Data managing method, calculates equipment and storage medium at device |
| CN111381767B (en) * | 2018-12-28 | 2024-03-26 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN111381767A (en) * | 2018-12-28 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Data processing method and device |
| CN110209670B (en) * | 2019-05-09 | 2022-03-25 | 北京猫盘技术有限公司 | Data processing method and device based on network storage device cluster |
| CN110209670A (en) * | 2019-05-09 | 2019-09-06 | 北京猫盘技术有限公司 | Data processing method and device based on network storage equipment cluster |
| CN111782582A (en) * | 2019-06-14 | 2020-10-16 | 北京京东尚科信息技术有限公司 | Data conversion method, system and name node |
| CN111008181A (en) * | 2019-10-31 | 2020-04-14 | 苏州浪潮智能科技有限公司 | Method, system, terminal and storage medium for switching storage strategies of distributed file system |
| CN112965660A (en) * | 2021-02-09 | 2021-06-15 | 山东英信计算机技术有限公司 | Method, system, device and medium for feeding back information of double storage pools |
| CN112965660B (en) * | 2021-02-09 | 2023-08-08 | 山东英信计算机技术有限公司 | Method, system, equipment and medium for double storage pool information feedback |
| CN114398006A (en) * | 2021-12-24 | 2022-04-26 | 中国电信股份有限公司 | Distributed storage mode control method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106708653B (en) | 2020-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106708653A (en) | Mixed tax administration data security protecting method based on erasure code and multi-copy | |
| US9823980B2 (en) | Prioritizing data reconstruction in distributed storage systems | |
| CN103942112B (en) | Disk tolerance method, apparatus and system | |
| CN106527993B (en) | Mass file storage method and device in a kind of distributed system | |
| CN102411637B (en) | Metadata management method of distributed file system | |
| CN102142006A (en) | File processing method and device of distributed file system | |
| CN105120003B (en) | A kind of method for realizing data backup under cloud environment | |
| CN102024016A (en) | Rapid data restoration method for distributed file system (DFS) | |
| CN108196978A (en) | Date storage method, device, data-storage system and readable storage medium storing program for executing | |
| CN105045917B (en) | A kind of the distributed data restoration methods and device of Case-based Reasoning | |
| US10346066B2 (en) | Efficient erasure coding of large data objects | |
| CN107291889A (en) | A kind of date storage method and system | |
| CN101515296A (en) | Data updating method and device | |
| CN105635252B (en) | Hadoop distributed file system HDFS erasure code redundancy backup method | |
| CN105159603A (en) | Repair method for distributed data storage system | |
| CN106776795A (en) | Method for writing data and device based on Hbase databases | |
| CN102142032A (en) | Method and system for reading and writing data of distributed file system | |
| US20140380091A1 (en) | Information processing apparatus, computer-readable recording medium having stored program for controlling information processing apparatus, and method for controlling information processing apparatus | |
| CN104965835A (en) | Method and apparatus for reading and writing files of a distributed file system | |
| CN104715044A (en) | Distributed system and data manipulation method thereof | |
| KR101254179B1 (en) | Method for effective data recovery in distributed file system | |
| CN103399943A (en) | Communication method and communication device for parallel query of clustered databases | |
| CN110268397B (en) | Efficient optimized data layout method applied to data warehouse system | |
| US8312237B2 (en) | Automated relocation of in-use multi-site protected data storage | |
| CN111670560A (en) | Electronic device, system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |