CN110532804A - A kind of secure storage control method based on big data - Google Patents
A kind of secure storage control method based on big data Download PDFInfo
- Publication number
- CN110532804A CN110532804A CN201910834991.6A CN201910834991A CN110532804A CN 110532804 A CN110532804 A CN 110532804A CN 201910834991 A CN201910834991 A CN 201910834991A CN 110532804 A CN110532804 A CN 110532804A
- Authority
- CN
- China
- Prior art keywords
- node
- block
- module
- value
- big data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Storage Device Security (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A kind of secure storage control method and system based on big data determines the size of big data to be stored this method comprises: receiving big data to be stored, and confirms its completeness and efficiency；Assess local storage space；The type for judging large data objects selects its corresponding strategy；According to corresponding strategy, coding and piecemeal are carried out to big data；Assess capacity and the safety of node space；According to piecemeal and safety and capacity, big data is stored.This method and system can be directed to produced by user or big data to be stored and processed, attribute based on each memory node, by big data safely, reliably, reasonably, part and then also original place storage can not be intercepted by hacker to the node, so that the big data does not influence under fire, it is not influenced by node failure or physical change, to realize the secure storage of big data.
Description
Technical field
The present invention relates to electric data processing fields, and more specifically, are related to a kind of secure storage based on big data
Control method and system.
Background technique
With the development of computer technology and the progress of network technology, huge change is had occurred in the work and life of the mankind
Change, present people such as carry out social interaction by mobile terminal, obtain daily in the information of contact, reading, storage, processing
News, inquiry knowledge element, shopping, amusement etc. are taken, this increases the data volume created exponentially.The magnanimity number being consequently formed
According to referred to as big data.Big data also brings some security risks while bringing advantage to the user.Such as, illegal point
Son will appreciate that the purchase of user to the privacy information that can obtain user after user's progress big data processing and analysis, such as electric business website
Object habit, social software provider will appreciate that the social networks of user and good friend get in touch with situation, and search engine provider will appreciate that
The retrieval habit and focus of user；In addition, when the accessing terminal to network of user accesses, network information data by
The calculated attack and destruction of unprincipled fellow and hacker, user's important information can be compromised, this will bring to computer user
Interests loss.And the development of big data technology and cloud is closely related, by being potentiated and enhanced network letter using cloud
The safety for ceasing data storage, as the focus got more and more attention.Therefore using cloud come to big data carry out backup and
Storage is desirable means.
However, being directed to cloud storage in the prior art, the equilibrium of load is focused mainly on, i.e., how load is reasonably distributed
To each node；Or further, by encrypting and distributed storage to data, that is, the reliability of data itself is improved
To enhance safety.However rarer prior art concern is directed to produced by user or big data to be stored and processed, and
The characteristics of attribute of each memory node, such as the storage medium characteristic (magnetism, semiconductor, phase transformation) and object of different memory nodes
Manage operating parameter (such as erasable number, such as the failure odds of damage etc) and security feature (such as attacked or
The probability of destruction), to the secure storage bring hidden danger of big data；Simultaneously in the prior art, local by carrying out big data
With cloud distributed storage, however hacker or criminal are by stealing the partial data of any of local or cloud, so that it may
To obtain big data information by decryption (the user's big data having is accordingly, direct even without encryption link), so as to cause
The leakage of privacy of user or personal information.How the attribute reasonably based on each memory node, safely, reliably by big data
Ground, reasonably, part and then also original place storage can not be intercepted by hacker to the node so that the big data does not influence under fire,
It is not influenced by node failure or physical change, this is in this field without disclosure but problem to be solved.
Summary of the invention
An object of the present invention is to provide a kind of secure storage control method and system based on big data, being capable of needle
To produced by user or big data to be stored and processed, based on the attribute of each memory node, safely, reliably by big data
Ground, reasonably, part and then also original place storage can not be intercepted by hacker to the node so that the big data does not influence under fire,
It is not influenced by node failure or physical change, to realize the secure storage of big data.
A kind of technical solution that the present invention takes to solve above-mentioned technical problem are as follows: secure storage control based on big data
Method processed, comprising: step S1 receives big data to be stored, and determines the size of big data to be stored, and confirms its integrality
And validity；Step S2 assesses local storage space；Step S3 judges the type of large data objects, selects its corresponding plan
Slightly；Step S4 carries out coding and piecemeal to big data according to corresponding strategy；Step S5 assesses the capacity and peace of node space
Quan Xing；Step S6 stores big data according to piecemeal and safety and capacity.
In one embodiment, in step sl, big data to be stored is received, determines the big of big data to be stored
It is small, and confirm that its completeness and efficiency further comprises: the big data with storage is received via data link, and determines it
Data length and data block information, based on the data size information for including in data block, by being compared with physical length information
Completeness and efficiency relatively to determine big data；In step s 2, assessment local storage space further comprises: by originally
Ground memory manager sends space querying request, by local storage manager inquire mapping table and space idle state it
Afterwards, its space free message and its storage initial address are sent.
In one embodiment, in step s3, the type for judging large data objects selects its corresponding strategy further
Include: to analyze data to be stored, confirms that it belongs to text, program or image information；If belonging to text and program, select
Select partial data coding, piecemeal storage；If it is image information, then information is selected to big data provider's display pattern, is choosing
Compression Strategies are selected still without Compression Strategies；Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, and R is 2
Each unit of positive integer pwoer, sub-block is represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；
Such as down conversion is carried out for each sub-block therein:Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；As p=q=0, the value of T (p, q) is the first numerical value,
Remaining T (p, q) is second value；First numerical value of each sub-block is successively sorted, and successively by the second value of each sub-block
Arrangement forms the first sequence of values and second value sequence.
In one embodiment, in step s 5, the capacity and safety for assessing node space further comprise: step
S51 determines available cloud node, and obtains the storage media types of node of the cloud for storing big data；Step S52,
Based on different node storage media types, the upper limit value TT of its read-write erasing access times is obtained by searching for table and can be used
Memory space, and obtain the erasing times of the read-write ERW and safety coefficient SE of the different blocks of the storage medium of each node
With failure exception probability FA, wherein individual node includes positive integer block, and different blocks have due to historical operation
Different read-write erasing times, different blocks are with different safety since rogue program is attacked or infected to history
Number, different blocks generate failure and with different failure exception probability due to abnormal；Wherein safety coefficient is the area in history
The number of rogue program and the ratio by operation amount are attacked or infected to block, failure exception probability be in history the block due to
The abnormal number for generating failure and the ratio by operation amount；Step S53 assesses the security performance of each node:Wherein k indicates the serial number of node；J indicates to be directed to k-th of node
The number of blocks included, its value is positive integer；J indicates the serial number for the block that node includes；wtkIndicate that node k is used
Storage medium weighted value, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium be higher than semiconductor
The weighted value of storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；Step S54, is based on
The numerical value of the security performance of the node k of calculating, is ranked up according to descending order；Step S55, by the node of sequence is corresponding can
It is sent to secure storage controller with memory space, the operation for step S6；Step S56 is used when this method finally determines
When block j in node k, the erasing times of the read-write ERW of block j is increased by 1.
In one embodiment, in step s 5, the capacity and safety for assessing node space further comprise: step
S51 determines available cloud node, and obtains the storage media types of node of the cloud for storing big data；Step S52,
Based on different node storage media types, the upper limit value TT of its read-write erasing access times is obtained by searching for table and can be used
Memory space, and obtain the erasing times of the read-write ERW and safety coefficient SE of the different blocks of the storage medium of each node
With failure exception probability FA, wherein individual node includes positive integer block, and different blocks have due to historical operation
Different read-write erasing times, different blocks are with different safety since rogue program is attacked or infected to history
Number, different blocks generate failure and with different failure exception probability due to abnormal；The safety coefficient is the area in history
The number of rogue program and the ratio by operation amount are attacked or infected to block, failure exception probability be in history the block due to
The abnormal number for generating failure and the ratio by operation amount；Step S53 assesses each of each node k being likely to be used
The security performance of block j:Wherein k indicates the serial number of node；J is indicated
The serial number for the block that node includes；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between
Numerical value, wherein the weighted value of magnetic storage medium be higher than semiconductor storage medium weighted value, the weight of semiconductor storage medium
Value is higher than the weighted value of phase change memory medium；Step S54, the security performance of each block j of each node k based on calculating
Numerical value is ranked up according to descending order；Step S55 sends the corresponding free memory of the block of the node of sequence to
Secure storage controller, the operation for step S6；Step S56, when this method, which finally determines, uses the block j in node k,
The erasing times of the read-write ERW of block j is increased by 1.
In one embodiment, in step s 4, according to corresponding strategy, coding is carried out to big data and piecemeal is further
It include: step S41, based on the strategy selected in previous steps, partial data or image determining or to text and program
Uncompressed data carry out coding and piecemeal, or by Compression Strategies to second value sequence carry out coding and piecemeal；Step
S42 sets coding parameter and nuisance parameter for determining partial data or second value sequence；Step S43, based on volume
Code parameter and nuisance parameter, are divided into k part for partial data or second value sequence average；Step S44 creates finite field
The matrix of upper sequency spectrum；Step S45 creates encoder matrix table according to the matrix；Step S46, for k part, by the matrix
It is multiplied with encoder matrix, the matrix of consequence encoded；Step S47, by the matrix of consequence of coding according to being provided in configuration information
Size piecemeal arrangement.
In one embodiment, in step s 6, according to piecemeal and safety and capacity, to big data stored into
One step includes: step S61: determining that the object in step S3 is uncompressed partial data or compressed first numerical value sequence
Column and second value sequence, then follow the steps S62 if it is the former, no to then follow the steps S63；Step S62: if in step S3
It is uncompressed partial data, then arranges the data for arranging piecemeal in step S4 to store to cloud, and execute step S64；
Step S63: it if in step S3 being compressed first sequence of values and second value sequence, arranges the first sequence of values
It is stored in local, and the storage of second value sequence beyond the clouds and is executed into step S64；Step S64: according to obtained in step S5
Containing available space, ranked node or block, the data in step S62 or S63 wait store to cloud are successively deposited
The node or block of the sequence are stored up, and secure storage controller is recorded in its memory node and its storage address information
In；Step S65: when the data wait store are completed, the response of secure storage is returned to local user.
According to an exemplary embodiment of the invention, a kind of secure storage control system based on big data is also claimed.
The secure storage control system based on big data includes: the first module, for receiving big data to be stored, is determined wait store
Big data size, and confirm its completeness and efficiency；Second module, for assessing local storage space；Third module,
For judging the type of large data objects, its corresponding strategy is selected；4th module, for being counted to big according to corresponding strategy
According to carry out coding and piecemeal；5th module, for assessing capacity and the safety of node space；6th module, for basis point
Block and safety and capacity, store big data.
In one embodiment, the first module is further used for: the big data with storage is received via data link, and
Determine its data length and data block information, based on the data size information for including in data block, by with physical length information
It is compared to determine the completeness and efficiency of big data；Second module is further used for: by local storage management
Device sends space querying request and sends its space after inquiring mapping table and space idle state by local storage manager
Free message and its storage initial address.
In one embodiment, third module is further used for: analyzing data to be stored, confirms that it belongs to text, journey
Sequence or image information；If belonging to text and program, partial data coding, piecemeal storage are selected；If it is image information,
Information then is selected to big data provider's display pattern, is to select Compression Strategies or without Compression Strategies；Compression Strategies include: by
Big data is divided into the sub-block comprising R × R unit, the positive integer pwoer that R is 2, and each unit of sub-block is represented as (x, y)；Its
Part of the middle marginal portion less than R binary zero value polishing；Such as down conversion is carried out for each sub-block therein:Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；As p=q=0, the value of T (p, q) is the first numerical value,
Remaining T (p, q) is second value；First numerical value of each sub-block is successively sorted, and successively by the second value of each sub-block
Arrangement forms the first sequence of values and second value sequence.
In one embodiment, the 5th module further comprises: May Day module, for determining available cloud node,
And obtain the storage media types of node of the cloud for storing big data；Five or two module, for being deposited based on different nodes
Storage media type, the upper limit value TT and free memory of its read-write erasing access times are obtained by searching for table, and are obtained
The erasing times of the read-write ERW and safety coefficient SE and failure exception probability of the different blocks of the storage medium of each node
FA, wherein individual node includes positive integer block, and different blocks are wiped due to historical operation with different read-writes
Except number, different blocks have different safety coefficients since rogue program is attacked or infected to history, different blocks due to
It is abnormal to generate failure and there is different failure exception probability；Wherein safety coefficient is that evil is attacked or infected to the block in history
The number for program of anticipating and the ratio by operation amount, failure exception probability are the block in history due to abnormal time for generating failure
It counts and by the ratio of operation amount；Five or three module, for assessing the security performance of each node:Wherein k indicates the serial number of node；J indicates to be directed to k-th of node
The number of blocks included, its value is positive integer；J indicates the serial number for the block that node includes；wtkIndicate that node k is used
Storage medium weighted value, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium be higher than semiconductor
The weighted value of storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；The May 4th module is used
In the numerical value of the security performance of the node k based on calculating, it is ranked up according to descending order；Five or five module, for that will sort
The corresponding free memory of node be sent to secure storage controller, the operation for the 6th module；Five or six module is used
In when this method finally determines and uses the block j in node k, the erasing times of the read-write ERW of block j is increased by 1.
In one embodiment, the 5th module further comprises: May Day module, for determining available cloud node,
And obtain the storage media types of node of the cloud for storing big data；Five or two module, for being deposited based on different nodes
Storage media type, the upper limit value TT and free memory of its read-write erasing access times are obtained by searching for table, and are obtained
The erasing times of the read-write ERW and safety coefficient SE and failure exception probability of the different blocks of the storage medium of each node
FA, wherein individual node includes positive integer block, and different blocks are wiped due to historical operation with different read-writes
Except number, different blocks have different safety coefficients since rogue program is attacked or infected to history, different blocks due to
It is abnormal to generate failure and there is different failure exception probability；The safety coefficient is that evil is attacked or infected to the block in history
The number for program of anticipating and the ratio by operation amount, failure exception probability are the block in history due to abnormal time for generating failure
It counts and by the ratio of operation amount；Five or three module, the peace of each block j for assessing each node k being likely to be used
Full performance:Wherein k indicates the serial number of node；J indicates that node includes
Block serial number；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between numerical value,
Wherein the weighted value of magnetic storage medium is higher than the weighted value of semiconductor storage medium, and the weighted value of semiconductor storage medium is higher than phase
Become the weighted value of storage medium；The May 4th module, the security performance of each block j for each node k based on calculating
Numerical value is ranked up according to descending order；Five or five module, the corresponding free memory of block of the node for that will sort
It is sent to secure storage controller, the operation for the 6th module；Five or six module, for finally determining when this method using section
When block j in point k, the erasing times of the read-write ERW of block j is increased by 1.
In one embodiment, the 4th module further comprises: the 4th 1 module, for based in previous block select
Strategy determines or carries out coding and piecemeal, Huo Zhetong to the partial data of text and program or the uncompressed data of image
Overcompression strategy carries out coding and piecemeal to second value sequence；Four or two module, for for determining partial data or
Second value sequence sets coding parameter and nuisance parameter；Four or three module will for being based on coding parameter and nuisance parameter
Partial data or second value sequence average are divided into k part；Four or four module, for creating the square of sequency spectrum in finite field
Battle array；Four or five module, for creating encoder matrix table according to the matrix；Four or six module, for being directed to k part, by the square
Battle array is multiplied with encoder matrix, the matrix of consequence encoded；Four or seven module, the matrix of consequence for that will encode is according to confidence
The arrangement of size piecemeal specified in breath.
In one embodiment, the 6th module further comprises: the 6th 1 module: for determining the object in third module
It is uncompressed partial data or compressed first sequence of values and second value sequence, is then proceeded to if it is the former
Six or two module, otherwise to the six or three module；Six or two module: if in third module being uncompressed partial data,
It arranges the data for arranging piecemeal in the 4th module to store to cloud, and proceeds to the six or four module；Six or three module: if the
It is compressed first sequence of values and second value sequence in three modules, then arranges the first sequence of values being stored in local,
And the storage of second value sequence beyond the clouds and is proceeded into the six or four module；Six or four module: for being obtained according in the 5th module
Arrive containing available space, ranked node or block, by the six or two module or the six or three module wait store cloud
The data at end successively store the node or block to the sequence, and peace is recorded in its memory node and its storage address information
In full storage control；Six or five module: for returning to secure storage to local user when the data wait store are completed
Response.
Detailed description of the invention
In the accompanying drawings by way of example rather than the embodiment of the present invention is shown by way of limitation, wherein phase
Same appended drawing reference indicates identical element, in which:
According to an exemplary embodiment of the invention, Fig. 1 illustrates a kind of stream of secure storage control method based on big data
Cheng Tu.
Specific embodiment
Before carrying out following specific embodiments, certain words and phrase used in the patent document are illustrated
Definition may be advantageous: term " includes " and "comprising" and its derivative mean to include without limiting；Term "or" is
Include, it is meant that and/or；Phrase " with ... it is associated ", " associated with it " and its derivative might mean that including quilt
Be included in ... it is interior, with ... interconnection, include be comprised in ... it is interior, be connected to ... or with ... connect, be coupled to ... or
With ... couple, can be with ... communicate, with ... cooperation interweaves, and side by side, approaches ..., be bound to ... or with ... binding, tool
Have, attribute having ..., etc.；And term " controller " mean to control any equipment of at least one operation, system or its
Component, such equipment may be realized with some combinations of hardware, firmware or software or wherein at least two.It should be noted that
: functionality associated with any specific controller may be centralization or distributed, either local or remote
Journey.The definition for being used for certain words and phrase is provided through patent document, it should be understood by those skilled in the art that: if not
In most cases, in many cases, such definition is suitable for word and phrase existing and define in this way not
To use.
In the following description, several specific embodiments with reference to attached drawing and are diagrammatically shown.It will be appreciated that
It is contemplated that and other embodiments can be made without departing from the scope of the present disclosure or spirit.Therefore, described in detail below should not be by
Think in a limiting sense.
According to an exemplary embodiment of the invention, Fig. 1 illustrates a kind of stream of secure storage control method based on big data
Cheng Tu.Wherein the secure storage control method based on big data includes:
Step S1 receives big data to be stored, and determines the size of big data to be stored, and confirms its integrality and have
Effect property；
Step S2 assesses local storage space；
Step S3 judges the type of large data objects, selects its corresponding strategy；
Step S4 carries out coding and piecemeal to big data according to corresponding strategy；
Step S5 assesses capacity and the safety of node space；
Step S6 stores big data according to piecemeal and safety and capacity.
Preferably, in step sl, big data to be stored is received, determines the size of big data to be stored, and is confirmed
Its completeness and efficiency further comprises: receiving the big data with storage via data link, and determines its data length
And data block information, based on the data size information for including in data block, by being compared to determine with physical length information
The completeness and efficiency of big data.
By the step, it can effectively guarantee the completeness and efficiency of original big data.
Preferably, in step s 2, assessment local storage space further comprises: by sending out to local storage manager
It send space querying to request, after inquiring mapping table and space idle state by local storage manager, sends its space free time
Information and its storage initial address.
By the step, the ability of being locally stored can be effectively obtained, not secure storage control is prepared.
Preferably, in step s3, the type for judging large data objects, selects its corresponding strategy to further comprise: point
Data to be stored are analysed, confirm that it belongs to text, program or image information；If belonging to text and program, selection is complete
Data encoding, piecemeal storage；If it is image information, then information is selected to big data provider's display pattern, is selection compression
Strategy is still without Compression Strategies.
Preferably, the image information of no Compression Strategies is the partial data of image.
It is further preferred that Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, R be 2 it is just whole
Number power, each unit of sub-block are represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For
Each sub-block therein carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q.
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value.By each sub-block
First numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence
Column.
Preferably, corresponding decompression strategy are as follows:
Preferably, the first numerical value and second value can be respectively stored in local and cloud in the next steps.
By the step, accordingly even when hacker or criminal intercept the piece of the first sequence of values and second value sequence
Section, can not also restore the image of original big data, to control the secure storage of big data.
Preferably, in step s 4, according to corresponding strategy, coding is carried out to big data and piecemeal further comprises:
Step S41, based on the strategy selected in previous steps, partial data or figure determining or to text and program
The uncompressed data of picture carries out coding and piecemeal, or carries out coding and piecemeal to second value sequence by Compression Strategies；
Step S42 sets coding parameter and nuisance parameter for determining partial data or second value sequence；
Step S43 is based on coding parameter and nuisance parameter, and partial data or second value sequence average are divided into k
Part；
Step S44 creates the matrix of sequency spectrum in finite field；
Step S45 creates encoder matrix table according to the matrix；
The matrix is multiplied by step S46 for k part with encoder matrix, the matrix of consequence encoded；
Step S47 arranges the matrix of consequence of coding according to size piecemeal specified in configuration information.
Preferably, in step s 5, the capacity and safety for assessing node space further comprise:
Step S51 determines available cloud node, and obtains the storage medium class of node of the cloud for storing big data
Type；
Step S52 obtains its read-write erasing access times based on different node storage media types by searching for table
Upper limit value TT and free memory, and obtain the read-write erasing times of the different blocks of the storage medium of each node
ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks by
There are the different erasing times of read-write in historical operation, different blocks attack due to history or are infected rogue program and had
There is different safety coefficients, different blocks generate failure and with different failure exception probability due to abnormal；Wherein safety system
Number is the number and the ratio by operation amount that rogue program was attacked or infected to the block in history, and failure exception probability is to go through
The block is due to the abnormal number for generating failure and by the ratio of operation amount in history；
Step S53 assesses the security performance of each node:
Wherein k indicates the serial number of node；J is indicated for the
Included by k node, its value be positive integer number of blocks；J indicates the serial number for the block that node includes；wtkIndicate node k
The weighted value of used storage medium, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium is higher than
The weighted value of semiconductor storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；
Step S54, the numerical value of the security performance of the node k based on calculating, is ranked up according to descending order；
The corresponding free memory of the node of sequence is sent secure storage controller by step S55, is used for step S6
Operation；
Step S56 wipes the read-write of block j secondary when this method, which finally determines, uses the block j in node k
Number ERW increases by 1.
Preferably, aforementioned including but not limited to be read by operation, write-in, erasing, refresh, be pre-charged.
Alternatively, in step s 5, the capacity and safety for assessing node space further comprise:
Step S51 determines available cloud node, and obtains the storage medium class of node of the cloud for storing big data
Type；
Step S52 obtains its read-write erasing access times based on different node storage media types by searching for table
Upper limit value TT and free memory, and obtain the read-write erasing times of the different blocks of the storage medium of each node
ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks by
There are the different erasing times of read-write in historical operation, different blocks attack due to history or are infected rogue program and had
There is different safety coefficients, different blocks generate failure and with different failure exception probability due to abnormal；The safety system
Number is the number and the ratio by operation amount that rogue program was attacked or infected to the block in history, and failure exception probability is to go through
The block is due to the abnormal number for generating failure and by the ratio of operation amount in history；
Step S53 assesses the security performance of each block j for each node k being likely to be used:
Wherein k indicates the serial number of node；J indicates that node includes
Block serial number；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between numerical value,
Wherein the weighted value of magnetic storage medium is higher than the weighted value of semiconductor storage medium, and the weighted value of semiconductor storage medium is higher than phase
Become the weighted value of storage medium；
Step S54, the numerical value of the security performance of each block j of each node k based on calculating, according to descending order into
Row sequence；
Step S55 sends secure storage controller for the corresponding free memory of the block of the node of sequence, is used for
The operation of step S6；
Step S56 wipes the read-write of block j secondary when this method, which finally determines, uses the block j in node k
Number ERW increases by 1.
By the step, it can comprehensively consider the difference and its history secure data of storage medium, can guarantee
Big data to be stored is stored on node and may be decreased its destroyed probability with maximum.
Preferably, in step s 6, according to piecemeal and safety and capacity, big data is carried out to store further packet
It includes:
Step S61: determine that the object in step S3 is uncompressed partial data or compressed first numerical value sequence
Column and second value sequence, then follow the steps S62 if it is the former, no to then follow the steps S63；
Step S62: if in step S3 being uncompressed partial data, the number for arranging piecemeal in step S4 is arranged
Cloud is arrived according to storage, and executes step S64；
Step S63: it if in step S3 being compressed first sequence of values and second value sequence, arranges first
Sequence of values is stored in local, and the storage of second value sequence beyond the clouds and is executed step S64；
Step S64: according to obtained in step S5 containing available space, ranked node or block, by step
Data in S62 or S63 wait store to cloud successively store node or block to the sequence, and by its memory node and its
Storage address information is recorded in secure storage controller；
Step S65: when the data wait store are completed, the response of secure storage is returned to local user.
Pass through the step, it is ensured that the secure storage and control of data, and for image data may also be ensured that it is black
Visitor or criminal can not restore the image of original big data, to ensure privacy of user and information security.
Accordingly, the application further relates to a kind of secure storage control system based on big data comprising:
First module determines the size of big data to be stored, and confirm that it is complete for receiving big data to be stored
Property and validity；
Second module, for assessing local storage space；
Third module selects its corresponding strategy for judging the type of large data objects；
4th module, for carrying out coding and piecemeal to big data according to corresponding strategy；
5th module, for assessing capacity and the safety of node space；
6th module, for being stored to big data according to piecemeal and safety and capacity.
Preferably, the first module is further used for: receiving the big data with storage via data link, and determines its number
According to length and data block information, based on the data size information for including in data block, by being compared with physical length information
To determine the completeness and efficiency of big data.
Preferably, the second module is further used for: by sending space querying request to local storage manager, by this
Ground memory manager is inquired after mapping table and space idle state, its space free message and its storage starting point are sent
Location.
Preferably, third module is further used for: analyzing data to be stored, confirms that it belongs to text, program is still schemed
As information；If belonging to text and program, partial data coding, piecemeal storage are selected；If it is image information, then to big number
Information is selected according to provider's display pattern, is to select Compression Strategies or without Compression Strategies.
Preferably, the image information of no Compression Strategies is the partial data of image.
It is further preferred that Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, R be 2 it is just whole
Number power, each unit of sub-block are represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For
Each sub-block therein carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q.
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value.By each sub-block
First numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence
Column.
Preferably, corresponding decompression strategy are as follows:
Preferably, the first numerical value and second value can be respectively stored in local and cloud in the operation of subsequent module
End.
Preferably, the 4th module further comprises:
4th 1 module, for based on the strategy selected in previous block, complete number determining or to text and program
According to the uncompressed data of perhaps image carry out coding and piecemeal or by Compression Strategies to second value sequence carry out coding and
Piecemeal；
Four or two module, for setting coding parameter and redundancy for determining partial data or second value sequence
Parameter；
Four or three module, for being based on coding parameter and nuisance parameter, by partial data or second value sequence average point
It is cut into k part；
Four or four module, for creating the matrix of sequency spectrum in finite field；
Four or five module, for creating encoder matrix table according to the matrix；
Four or six module, for for k part, which to be multiplied with encoder matrix, the matrix of consequence encoded；
Four or seven module, the matrix of consequence for that will encode are arranged according to size piecemeal specified in configuration information.
Preferably, the 5th module further comprises:
May Day module for determining available cloud node, and obtains depositing for node of the cloud for storing big data
Storage media type；
Five or two module, for based on different node storage media types, obtaining its read-write erasing by searching for table to make
With the upper limit value TT and free memory of number, and the read-write for obtaining the different blocks of the storage medium of each node is wiped
Except number ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different
Block has the different erasing times of read-write due to historical operation, and malice journey is attacked or infected to different blocks due to history
Sequence and there is different safety coefficients, different blocks are generated failure and are had different failure exception probability due to abnormal；Wherein
Safety coefficient is that the number of rogue program and the ratio by operation amount are attacked or infected to the block in history, and failure exception is general
Rate is the block in history due to the abnormal number for generating failure and by the ratio of operation amount；
Five or three module, for assessing the security performance of each node:
Wherein k indicates the serial number of node；J is indicated for the
Included by k node, its value be positive integer number of blocks；J indicates the serial number for the block that node includes；wtkIndicate node k
The weighted value of used storage medium, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium is higher than
The weighted value of semiconductor storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；
The May 4th module, the numerical value of the security performance for the node k based on calculating, is ranked up according to descending order；
Five or five module is used for sending secure storage controller for the corresponding free memory of the node of sequence
In the operation of the 6th module；
Five or six module, for when this method is finally determined using block j in node k, by the read-write of block j
Erasing times ERW increases by 1.
Preferably, aforementioned including but not limited to be read by operation, write-in, erasing, refresh, be pre-charged.
Alternatively, the 5th module further comprises:
May Day module for determining available cloud node, and obtains depositing for node of the cloud for storing big data
Storage media type；
Five or two module, for based on different node storage media types, obtaining its read-write erasing by searching for table to make
With the upper limit value TT and free memory of number, and the read-write for obtaining the different blocks of the storage medium of each node is wiped
Except number ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different
Block has the different erasing times of read-write due to historical operation, and malice journey is attacked or infected to different blocks due to history
Sequence and there is different safety coefficients, different blocks are generated failure and are had different failure exception probability due to abnormal；It is described
Safety coefficient is that the number of rogue program and the ratio by operation amount are attacked or infected to the block in history, and failure exception is general
Rate is the block in history due to the abnormal number for generating failure and by the ratio of operation amount；
Five or three module, the security performance of each block j for assessing each node k being likely to be used:
Wherein k indicates the serial number of node；J indicates that node includes
Block serial number；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between numerical value,
Wherein the weighted value of magnetic storage medium is higher than the weighted value of semiconductor storage medium, and the weighted value of semiconductor storage medium is higher than phase
Become the weighted value of storage medium；
The May 4th module, the numerical value of the security performance of each block j for each node k based on calculating, according to passing
Subtract order to be ranked up；
Five or five module, for sending secure storage control for the corresponding free memory of the block of the node of sequence
Device, the operation for the 6th module；
Five or six module, for when this method is finally determined using block j in node k, by the read-write of block j
Erasing times ERW increases by 1.
Preferably, the 6th module further comprises:
6th 1 module: for determining that the object in third module is uncompressed partial data or compressed
One sequence of values and second value sequence, the six or two module is then proceeded to if it is the former, otherwise to the six or three module；
Six or two module: it if in third module being uncompressed partial data, arranges piecemeal in the 4th module
The data of arrangement are stored to cloud, and proceed to the six or four module；
Six or three module: it if being compressed first sequence of values and second value sequence in third module, arranges
First sequence of values is stored in local, and the storage of second value sequence beyond the clouds and is proceeded into the six or four module；
Six or four module: for according to containing available space, ranked node or area obtained in the 5th module
Data in six or two module or the six or three module wait store to cloud are successively stored node or area to the sequence by block
Block, and its memory node and its storage address information are recorded in secure storage controller；
Six or five module: for returning to the response of secure storage to local user when the data wait store are completed.
Above-mentioned each technical term is the routine techniques term with common meaning in this field, in order not to obscure this
The emphasis of invention, is not further explained it herein.
To sum up, in the inventive solutions, the secure storage control method by using a kind of based on big data
And system, it can be directed to produced by user or big data to be stored and processed will be counted greatly based on the attribute of each memory node
According to safely, reliably, reasonably, can not by hacker intercept part so that also original place storage to the node so that the big data
It does not influence under fire, is not influenced by node failure or physical change, to realize the secure storage of big data.
It will be appreciated that example and reality of the invention can be realized in the form of the combination of hardware, software or hardware and software
Apply example.As described above, any main body for executing this method can be stored, in the form of volatility or non-volatile holographic storage, such as
Equipment is stored, as ROM, whether no matter can erasing or is rewritable, or in the form of a memory, such as RAM, storage core
Piece, equipment or integrated circuit or on the readable medium of light or magnetic, such as CD, DVD, disk or tape.It will be appreciated that
Storage equipment and storage medium are suitable for storing the example of the machine readable storage of one or more programs, upon being performed,
One or more of programs realize example of the invention.Via any medium, such as it is loaded with by wired or wireless coupling
Signal of communication can electronically transmit example of the invention, and example suitably includes identical content.
It is to be noted that because the present invention is solved for produced by user or big data to be stored and processed, base
In the attribute of each memory node, by big data safely, reliably, reasonably, can not by hacker intercept part so that also original place
The node is stored, so that the big data does not influence under fire, is not influenced by node failure or physical change, to realize
The technical issues of secure storage of big data, uses technician in the art after reading this description according to it
Technological means to understand is instructed, and obtains advantageous effects, so claimed side in the following claims
Case belongs to the technical solution on patent law purposes.In addition, because the claimed technical solution of appended claims can be in work
It is made or used in industry, therefore the program has practicability.
The above, preferable specific embodiment only of the invention, but protection scope of the present invention is not limited to
This, anyone skilled in the art in the technical scope disclosed by the present invention, the variation that can readily occur in or replaces
It changes, should all forgive within protection scope of the present invention.Unless be otherwise expressly recited, otherwise disclosed each feature is only
It is equivalent or similar characteristics a example for general series.Therefore, protection scope of the present invention should be with claims
Subject to protection scope.
Claims (10)
1. a kind of secure storage control method based on big data, comprising:
Step S1, receives big data to be stored, and determines the size of big data to be stored, and confirms its integrality and effectively
Property；
Step S2 assesses local storage space；
Step S3 judges the type of large data objects, selects its corresponding strategy；
Step S4 carries out coding and piecemeal to big data according to corresponding strategy；
Step S5 assesses capacity and the safety of node space；
Step S6 stores big data according to piecemeal and safety and capacity.
2. the secure storage control method according to claim 1 based on big data, in which:
In step sl, big data to be stored is received, determines the size of big data to be stored, and confirms its integrality and has
Effect property further comprises: the big data with storage received via data link, and determines its data length and data block information,
Based on the data size information for including in data block, the integrality of big data is determined by being compared to physical length information
And validity；
In step s 2, assessment local storage space further comprises: by sending space querying to local storage manager
Request sends its space free message and its deposits after inquiring mapping table and space idle state by local storage manager
Store up initial address.
3. the secure storage control method according to claim 2 based on big data, in which:
In step s3, the type for judging large data objects, selects its corresponding strategy to further comprise: analyzing number to be stored
According to confirming that it belongs to text, program or image information；If belonging to text and program, partial data coding, piecemeal are selected
Storage；If it is image information, then information is selected to big data provider's display pattern, is to select Compression Strategies or without compression
Strategy；
Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, the positive integer pwoer that R is 2, each list of sub-block
Member is represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For each son therein
Block carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value；By the first of each sub-block
Numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence；With
And
In step s 5, the capacity and safety for assessing node space further comprise:
Step S51 determines available cloud node, and obtains the storage media types of node of the cloud for storing big data；
Step S52 obtains the upper limit of its read-write erasing access times by searching for table based on different node storage media types
Value TT and free memory, and obtain the storage medium of each node different blocks the erasing times of read-write ERW with
And safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks are due to going through
History operates and has the different erasing times of read-write, and different blocks have not since rogue program is attacked or infected to history
Same safety coefficient, different blocks generate failure and with different failure exception probability due to abnormal；Wherein safety coefficient is
The number and the ratio by operation amount, failure exception probability that rogue program was attacked or infected to the block in history are in history
The block is due to the abnormal number for generating failure and by the ratio of operation amount；
Step S53 assesses the security performance of each node:
Wherein k indicates the serial number of node；J indicates to be directed to k-th
Included by node, its value be positive integer number of blocks；J indicates the serial number for the block that node includes；wtkIndicate node k institute
The weighted value of the storage medium of use, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium is higher than half
The weighted value of conductor storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；
Step S54, the numerical value of the security performance of the node k based on calculating, is ranked up according to descending order；
Step S55 sends secure storage controller for the corresponding free memory of the node of sequence, the behaviour for step S6
Make；
Step S56, when this method, which finally determines, uses the block j in node k, by the erasing times of the read-write ERW of block j
Increase by 1.
4. the secure storage control method according to claim 2 based on big data, in which:
In step s3, the type for judging large data objects, selects its corresponding strategy to further comprise: analyzing number to be stored
According to confirming that it belongs to text, program or image information；If belonging to text and program, partial data coding, piecemeal are selected
Storage；If it is image information, then information is selected to big data provider's display pattern, is to select Compression Strategies or without compression
Strategy；
Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, the positive integer pwoer that R is 2, each list of sub-block
Member is represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For each son therein
Block carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value；By the first of each sub-block
Numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence；With
And
In step s 5, the capacity and safety for assessing node space further comprise:
Step S51 determines available cloud node, and obtains the storage media types of node of the cloud for storing big data；
Step S52 obtains the upper limit of its read-write erasing access times by searching for table based on different node storage media types
Value TT and free memory, and obtain the storage medium of each node different blocks the erasing times of read-write ERW with
And safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks are due to going through
History operates and has the different erasing times of read-write, and different blocks have not since rogue program is attacked or infected to history
Same safety coefficient, different blocks generate failure and with different failure exception probability due to abnormal；The safety coefficient is
The number and the ratio by operation amount, failure exception probability that rogue program was attacked or infected to the block in history are in history
The block is due to the abnormal number for generating failure and by the ratio of operation amount；
Step S53 assesses the security performance of each block j for each node k being likely to be used:
Wherein k indicates the serial number of node；J indicates the area that node includes
The serial number of block；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between numerical value, wherein
The weighted value of magnetic storage medium is higher than the weighted value of semiconductor storage medium, and the weighted value of semiconductor storage medium is deposited higher than phase transformation
The weighted value of storage media；
Step S54, the numerical value of the security performance of each block j of each node k based on calculating, is arranged according to descending order
Sequence；
The corresponding free memory of the block of the node of sequence is sent secure storage controller by step S55, is used for step
The operation of S6；
Step S56, when this method, which finally determines, uses the block j in node k, by the erasing times of the read-write ERW of block j
Increase by 1.
5. the secure storage control method according to claim 3 or 4 based on big data, in which:
In step s 4, according to corresponding strategy, coding is carried out to big data and piecemeal further comprises:
Step S41 is determined or based on the strategy selected in previous steps to the partial data of text and program or image
Uncompressed data carries out coding and piecemeal, or carries out coding and piecemeal to second value sequence by Compression Strategies；
Step S42 sets coding parameter and nuisance parameter for determining partial data or second value sequence；
Step S43 is based on coding parameter and nuisance parameter, partial data or second value sequence average is divided into k part；
Step S44 creates the matrix of sequency spectrum in finite field；
Step S45 creates encoder matrix table according to the matrix；
The matrix is multiplied by step S46 for k part with encoder matrix, the matrix of consequence encoded；
Step S47 arranges the matrix of consequence of coding according to size piecemeal specified in configuration information；And
In step s 6, according to piecemeal and safety and capacity, carrying out storage to big data further comprises:
Step S61: determine the object in step S3 be uncompressed partial data or compressed first sequence of values and
Second value sequence thens follow the steps S62 if it is the former, no to then follow the steps S63；
Step S62: if in step S3 being uncompressed partial data, the data for arranging piecemeal in step S4 is arranged to deposit
It stores up to cloud, and executes step S64；
Step S63: it if in step S3 being compressed first sequence of values and second value sequence, arranges the first numerical value
Sequence is stored in local, and the storage of second value sequence beyond the clouds and is executed step S64；
Step S64: according to obtained in step S5 containing available space, ranked node or block, by step S62 or
Data in S63 wait store to cloud successively store node or block to the sequence, and by its memory node and its storage
Address information recording is into secure storage controller；
Step S65: when the data wait store are completed, the response of secure storage is returned to local user.
6. a kind of secure storage control system based on big data, comprising:
First module determines the size of big data to be stored for receiving big data to be stored, and confirm its integrality and
Validity；
Second module, for assessing local storage space；
Third module selects its corresponding strategy for judging the type of large data objects；
4th module, for carrying out coding and piecemeal to big data according to corresponding strategy；
5th module, for assessing capacity and the safety of node space；
6th module, for being stored to big data according to piecemeal and safety and capacity.
7. the secure storage control system according to claim 6 based on big data, in which:
First module is further used for: receiving the big data with storage via data link, and determines its data length sum number
According to block message, based on the data size information for including in data block, big number is determined by being compared to physical length information
According to completeness and efficiency；
Second module is further used for: by sending space querying request to local storage manager, by local storage pipe
After managing device inquiry mapping table and space idle state, its space free message and its storage initial address are sent.
8. the secure storage control system according to claim 7 based on big data, in which:
Third module is further used for: analyzing data to be stored, confirms that it belongs to text, program or image information；If
Belong to text and program, then selects partial data coding, piecemeal storage；It is if it is image information, then aobvious to big data provider
Show mode selecting information, is to select Compression Strategies or without Compression Strategies；
Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, the positive integer pwoer that R is 2, each list of sub-block
Member is represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For each son therein
Block carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value；By the first of each sub-block
Numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence；With
And
5th module further comprises:
May Day module, for determining available cloud node, and the storage for obtaining node of the cloud for storing big data is situated between
Matter type；
Five or two module, for obtaining its read-write by searching for table and wiping using secondary based on different node storage media types
Several upper limit value TT and free memory, and obtain the erasing time of read-write of the different blocks of the storage medium of each node
Number ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks
There are different read-write erasing times due to historical operation, different blocks attacked or infected rogue program due to history and
With different safety coefficients, different blocks generate failure and with different failure exception probability due to abnormal；Wherein safety
Coefficient is that the number of rogue program is attacked or infected to the block in history and the ratio by operation amount, failure exception probability are
The block is due to the abnormal number for generating failure and by the ratio of operation amount in history；
Five or three module, for assessing the security performance of each node:
Wherein k indicates the serial number of node；J indicates to be directed to k-th
Included by node, its value be positive integer number of blocks；J indicates the serial number for the block that node includes；wtkIndicate node k institute
The weighted value of the storage medium of use, the weighted value be (0,1] between numerical value, wherein the weighted value of magnetic storage medium is higher than half
The weighted value of conductor storage medium, the weighted value of semiconductor storage medium are higher than the weighted value of phase change memory medium；
The May 4th module, the numerical value of the security performance for the node k based on calculating, is ranked up according to descending order；
Five or five module, for sending secure storage controller for the corresponding free memory of the node of sequence, for the
The operation of six modules；
Five or six module, for when this method finally determines and uses the block j in node k, the read-write of block j to be wiped
Number ERW increases by 1.
9. the secure storage control system according to claim 7 based on big data, in which:
Third module is further used for: analyzing data to be stored, confirms that it belongs to text, program or image information；If
Belong to text and program, then selects partial data coding, piecemeal storage；It is if it is image information, then aobvious to big data provider
Show mode selecting information, is to select Compression Strategies or without Compression Strategies；
Compression Strategies include: that big data is divided into the sub-block comprising R × R unit, the positive integer pwoer that R is 2, each list of sub-block
Member is represented as (x, y)；Wherein part of the marginal portion less than R binary zero value polishing；For each son therein
Block carries out such as down conversion:
Wherein as r=0,And when r ≠ 0, C (r)=1, wherein r is p or q；
As p=q=0, the value of T (p, q) is the first numerical value, remaining T (p, q) is second value；By the first of each sub-block
Numerical value successively sorts, and the second value of each sub-block is arranged successively, and forms the first sequence of values and second value sequence；With
And
5th module further comprises:
May Day module, for determining available cloud node, and the storage for obtaining node of the cloud for storing big data is situated between
Matter type；
Five or two module, for obtaining its read-write by searching for table and wiping using secondary based on different node storage media types
Several upper limit value TT and free memory, and obtain the erasing time of read-write of the different blocks of the storage medium of each node
Number ERW and safety coefficient SE and failure exception probability FA, wherein individual node includes positive integer block, and different blocks
There are different read-write erasing times due to historical operation, different blocks attacked or infected rogue program due to history and
With different safety coefficients, different blocks generate failure and with different failure exception probability due to abnormal；The safety
Coefficient is that the number of rogue program is attacked or infected to the block in history and the ratio by operation amount, failure exception probability are
The block is due to the abnormal number for generating failure and by the ratio of operation amount in history；
Five or three module, the security performance of each block j for assessing each node k being likely to be used:
Wherein k indicates the serial number of node；J indicates the area that node includes
The serial number of block；wtkIndicate node k used by storage medium weighted value, the weighted value be (0,1] between numerical value, wherein
The weighted value of magnetic storage medium is higher than the weighted value of semiconductor storage medium, and the weighted value of semiconductor storage medium is deposited higher than phase transformation
The weighted value of storage media；
The May 4th module, the numerical value of the security performance of each block j for each node k based on calculating are secondary according to successively decreasing
Sequence is ranked up；
Five or five module, for sending secure storage controller for the corresponding free memory of the block of the node of sequence,
Operation for the 6th module；
Five or six module, for when this method finally determines and uses the block j in node k, the read-write of block j to be wiped
Number ERW increases by 1.
10. the secure storage control system based on big data according to claim 8 or claim 9, in which:
4th module further comprises:
4th 1 module, for based on the strategy selected in previous block, determine or to the partial data of text and program or
The uncompressed data of person's image encodes and piecemeal, or second value sequence is encoded and divided by Compression Strategies
Block；
Four or two module, for setting coding parameter and nuisance parameter for determining partial data or second value sequence；
Partial data or second value sequence average are divided into k for being based on coding parameter and nuisance parameter by the four or three module
A part；
Four or four module, for creating the matrix of sequency spectrum in finite field；
Four or five module, for creating encoder matrix table according to the matrix；
Four or six module, for for k part, which to be multiplied with encoder matrix, the matrix of consequence encoded；
Four or seven module, the matrix of consequence for that will encode are arranged according to size piecemeal specified in configuration information；And
6th module further comprises:
6th 1 module: for determining that the object in third module is uncompressed partial data or compressed first number
Value sequence and second value sequence, the six or two module is then proceeded to if it is the former, otherwise to the six or three module；
Six or two module: if in third module being uncompressed partial data, piecemeal in the 4th module is arranged in arrangement
Data store to cloud, and proceed to the six or four module；
Six or three module: it if in third module being compressed first sequence of values and second value sequence, arranges the
One sequence of values is stored in local, and the storage of second value sequence beyond the clouds and is proceeded to the six or four module；
Six or four module: for according to obtained in the 5th module contain available space, ranked node or block, will
Data in six or two module or the six or three module wait store to cloud successively store node or block to the sequence, and will
Its memory node and its storage address information are recorded in secure storage controller；
Six or five module: for returning to the response of secure storage to local user when the data wait store are completed.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910834991.6A CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
| CN202010210196.2A CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910834991.6A CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210196.2A Division CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110532804A true CN110532804A (en) | 2019-12-03 |
| CN110532804B CN110532804B (en) | 2020-06-05 |
Family
ID=68667015
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210196.2A Expired - Fee Related CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
| CN201910834991.6A Active CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210196.2A Expired - Fee Related CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN111428266B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177765A (en) * | 2020-01-06 | 2020-05-19 | 广州知弘科技有限公司 | Financial big data processing method, storage medium and system |
| CN115174591A (en) * | 2022-07-01 | 2022-10-11 | 深圳市玄羽科技有限公司 | Intelligent manufacturing system and data security management method thereof |
| CN116627361A (en) * | 2023-07-25 | 2023-08-22 | 江苏华存电子科技有限公司 | Storage comparison management method and system for redundant information |
| CN117235013A (en) * | 2023-11-10 | 2023-12-15 | 中科数创(临沂)数字科技有限公司 | Intelligent archive management method based on artificial intelligence |
| CN118069658A (en) * | 2024-04-19 | 2024-05-24 | 北京威斯盾网络科技有限公司 | Control method of database system and database system |
| CN118170325A (en) * | 2024-05-13 | 2024-06-11 | 邯郸鉴晨网络科技有限公司 | Big data storage and processing control method |
| CN118535652A (en) * | 2024-07-25 | 2024-08-23 | 卓世智星(青田)元宇宙科技有限公司 | Big data storage method and system |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114675789B (en) * | 2022-04-20 | 2023-06-16 | 深圳有方信息技术有限公司 | Big data analysis and storage system and method based on computer system |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187948A (en) * | 2007-12-20 | 2008-05-28 | 中国科学院计算技术研究所 | A continuous data protection system and its realization method |
| CN104636673A (en) * | 2015-03-10 | 2015-05-20 | 四川中科腾信科技有限公司 | Safe data storage method under big data background |
| US20160255117A1 (en) * | 2011-03-18 | 2016-09-01 | Zscaler, Inc. | Mobile device security, device management, and policy enforcement in a cloud based system |
| CN106203164A (en) * | 2016-07-01 | 2016-12-07 | 何钟柱 | The big Data Resources Management System of information security based on trust computing and cloud computing |
| US10097572B1 (en) * | 2016-06-07 | 2018-10-09 | EMC IP Holding Company LLC | Security for network computing environment based on power consumption of network devices |
| CN109032499A (en) * | 2018-06-09 | 2018-12-18 | 西安电子科技大学 | A kind of data access method of Distributed Storage, information data processing terminal |
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN109960459A (en) * | 2017-12-23 | 2019-07-02 | 王彬 | A kind of data-storage system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10079731B2 (en) * | 2015-11-23 | 2018-09-18 | International Business Machines Corporation | Client-space network monitoring |
| CN106230982B (en) * | 2016-09-08 | 2019-07-16 | 哈尔滨工程大学 | A kind of dynamic self-adapting secure cloud storage method considering node reliability |
| CN109933501B (en) * | 2017-12-15 | 2022-08-23 | 中国移动通信集团浙江有限公司 | Capacity evaluation method and device of application system |
| CN110062199B (en) * | 2018-01-19 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Load balancing method and device and computer readable storage medium |
| CN109446020B (en) * | 2018-09-21 | 2022-05-13 | 曙光云计算集团有限公司 | Dynamic evaluation method and device of cloud storage system |
| CN110162270B (en) * | 2019-04-29 | 2020-08-25 | 平安国际智慧城市科技股份有限公司 | Data storage method, storage node and medium based on distributed storage system |
-
2019
- 2019-09-05 CN CN202010210196.2A patent/CN111428266B/en not_active Expired - Fee Related
- 2019-09-05 CN CN201910834991.6A patent/CN110532804B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101187948A (en) * | 2007-12-20 | 2008-05-28 | 中国科学院计算技术研究所 | A continuous data protection system and its realization method |
| US20160255117A1 (en) * | 2011-03-18 | 2016-09-01 | Zscaler, Inc. | Mobile device security, device management, and policy enforcement in a cloud based system |
| CN104636673A (en) * | 2015-03-10 | 2015-05-20 | 四川中科腾信科技有限公司 | Safe data storage method under big data background |
| US10097572B1 (en) * | 2016-06-07 | 2018-10-09 | EMC IP Holding Company LLC | Security for network computing environment based on power consumption of network devices |
| CN106203164A (en) * | 2016-07-01 | 2016-12-07 | 何钟柱 | The big Data Resources Management System of information security based on trust computing and cloud computing |
| CN109960459A (en) * | 2017-12-23 | 2019-07-02 | 王彬 | A kind of data-storage system |
| CN109032499A (en) * | 2018-06-09 | 2018-12-18 | 西安电子科技大学 | A kind of data access method of Distributed Storage, information data processing terminal |
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
Non-Patent Citations (4)
| Title |
|---|
| DUYGU SINANC TERZI 等: "A survey on security and privacy issues in big data", 《2015 10TH INTERNATIONAL CONFERENCE FOR INTERNET TECHNOLOGY AND SECURED TRANSACTIONS (ICITST)》 * |
| SEUNGMIN KANG 等: "A Security-Aware Data Placement Mechanism for Big Data Cloud Storage Systems", 《2016 IEEE 2ND INTERNATIONAL CONFERENCE ON BIG DATA SECURITY ON CLOUD (BIGDATASECURITY), IEEE INTERNATIONAL CONFERENCE ON HIGH PERFORMANCE AND SMART COMPUTING (HPSC), AND IEEE INTERNATIONAL CONFERENCE ON INTELLIGENT DATA AND SECURITY (IDS)》 * |
| 吴晓玲 等: "基于云存储架构的分布式大数据安全容错存储算法", 《中国电子科学研究院学报》 * |
| 吴章玲 等: "基于PCM的大数据存储与管理研究综述", 《计算机研究与发展》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177765A (en) * | 2020-01-06 | 2020-05-19 | 广州知弘科技有限公司 | Financial big data processing method, storage medium and system |
| CN115174591A (en) * | 2022-07-01 | 2022-10-11 | 深圳市玄羽科技有限公司 | Intelligent manufacturing system and data security management method thereof |
| CN115174591B (en) * | 2022-07-01 | 2023-10-13 | 深圳市玄羽科技有限公司 | Intelligent manufacturing system and data security management method thereof |
| CN116627361A (en) * | 2023-07-25 | 2023-08-22 | 江苏华存电子科技有限公司 | Storage comparison management method and system for redundant information |
| CN116627361B (en) * | 2023-07-25 | 2023-11-17 | 江苏华存电子科技有限公司 | Storage comparison management method and system for redundant information |
| CN117235013A (en) * | 2023-11-10 | 2023-12-15 | 中科数创(临沂)数字科技有限公司 | Intelligent archive management method based on artificial intelligence |
| CN117235013B (en) * | 2023-11-10 | 2024-01-30 | 中科数创(临沂)数字科技有限公司 | Intelligent archive management method based on artificial intelligence |
| CN118069658A (en) * | 2024-04-19 | 2024-05-24 | 北京威斯盾网络科技有限公司 | Control method of database system and database system |
| CN118069658B (en) * | 2024-04-19 | 2024-06-25 | 北京威斯盾网络科技有限公司 | Control method of database system and database system |
| CN118170325A (en) * | 2024-05-13 | 2024-06-11 | 邯郸鉴晨网络科技有限公司 | Big data storage and processing control method |
| CN118170325B (en) * | 2024-05-13 | 2024-08-16 | 北京励栋科技发展有限公司 | Big data storage and processing control method |
| CN118535652A (en) * | 2024-07-25 | 2024-08-23 | 卓世智星(青田)元宇宙科技有限公司 | Big data storage method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110532804B (en) | 2020-06-05 |
| CN111428266A (en) | 2020-07-17 |
| CN111428266B (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532804A (en) | A kind of secure storage control method based on big data | |
| US20170300396A1 (en) | Allocating data based on hardware faults | |
| US20030126434A1 (en) | File security system using a security class and method for managing an encryption key | |
| JP6393375B2 (en) | Electronic device network, electronic device and inspection process thereof | |
| CN1282322C (en) | Portable information storage medium and its authetification method | |
| CN105122202A (en) | File system operation on multi-tiered volume | |
| US10083194B2 (en) | Process for obtaining candidate data from a remote storage server for comparison to a data to be identified | |
| CN107707660B (en) | Cloud storage method and system with selectable algorithm and based on identity verification technology | |
| CN110046494A (en) | Big data processing method and system based on terminal | |
| US20110071991A1 (en) | Systems and methods for geometric data compression and encryption | |
| CN115408350A (en) | Log compression method, log recovery method, log compression device, log recovery device, computer equipment and storage medium | |
| CN107395587A (en) | A kind of data managing method and system based on multipoint cooperative mechanism | |
| CN101568929B (en) | Information converting technique | |
| Alshede et al. | Ensemble Voting-Based Anomaly Detection for a Smart Grid Communication Infrastructure. | |
| CN115834257A (en) | Cloud electric power data safety protection method and protection system | |
| KR102093764B1 (en) | Managment server for managing the server and storage | |
| CN108090364A (en) | A kind of localization method and system in data leak source | |
| Leung et al. | Flexible compression of big data | |
| CN105024818B (en) | A method of building data encryption storage and exchange system | |
| CN104881253A (en) | Array erasure coding method free of error correction parameter limitation | |
| CN117009683A (en) | Codeword data processing method and device, electronic equipment and storage medium | |
| CN118018326B (en) | Data security encryption method and system based on distributed storage | |
| US11816242B2 (en) | Log compression and obfuscation using embeddings | |
| CN116488947B (en) | Security element treatment method | |
| CN115664799B (en) | Data exchange method and system applied to information technology security |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200511 Address after: 201702 Shanghai city Qingpu District No. 599, building 1, 2 Gaojing Road, room 208 Applicant after: SHANGHAI V&G INFORMATION TECHNOLOGY Co.,Ltd. Address before: 510000 1011, building H5, Luogang Aoyuan Plaza, 1904 Chuang Kai Avenue, Huangpu District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU ZHIHONG TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |