CN110532804B - Safe storage control method based on big data - Google Patents
Safe storage control method based on big data Download PDFInfo
- Publication number
- CN110532804B CN110532804B CN201910834991.6A CN201910834991A CN110532804B CN 110532804 B CN110532804 B CN 110532804B CN 201910834991 A CN201910834991 A CN 201910834991A CN 110532804 B CN110532804 B CN 110532804B
- Authority
- CN
- China
- Prior art keywords
- module
- node
- block
- data
- big data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
Abstract
A big data based secure storage control method and system, the method includes: receiving big data to be stored, determining the size of the big data to be stored, and confirming the integrity and the validity of the big data to be stored; evaluating a local storage space; judging the type of the big data object, and selecting a corresponding strategy; coding and partitioning the big data according to a corresponding strategy; evaluating the capacity and the safety of a node space; the big data is stored according to the block and the safety and the capacity. The method and the system can safely, reliably and reasonably store the big data which is generated by a user or is to be stored and processed to the node in place on the basis of the attribute of each storage node, wherein the big data cannot be intercepted by a hacker and then is not stored to the node in place, so that the big data is not influenced by attack and is not influenced by node faults or physical changes, and the safe storage of the big data is realized.
Description
Technical Field
The present invention relates to the field of electrical data processing, and more particularly, to a big data based secure storage control method and system.
Background
With the development of computer technology and the advancement of network technology, the work and life of human beings have changed greatly, and people now touch, read, store and process information such as social interaction through mobile terminals, acquisition of news, inquiry of knowledge elements, shopping, entertainment and the like every day, which has resulted in the increase of the amount of created data by times. The massive data thus formed is referred to as big data. Big data brings convenience to users and brings some safety hazards. For example, after a lawbreaker processes and analyzes the big data of the user, the lawbreaker can obtain the privacy information of the user, for example, the e-commerce website can know the shopping habit of the user, the social software provider can know the social relationship and the friend-friendly contact condition of the user, and the search engine provider can know the retrieval habit and the focus of the user; in addition, when the user terminal accesses the network for access, the network information data is deliberately attacked and destroyed by lawless persons and hackers, and the user important information is leaked, which brings benefit loss to the computer user. The big data technology is closely related to the development of the cloud technology, and the security of network information data storage is enhanced and improved by using the cloud technology as a focus of increasing importance. It is therefore desirable to utilize cloud technology for backing up and storing large data.
However, in the prior art, for cloud storage, the load balancing is mainly focused, that is, how to reasonably distribute the load to each node; or further, the data is stored in a distributed mode by encrypting the data, namely, the reliability of the data is improved, so that the safety is enhanced. However, there is less prior art concerning the characteristics of the big data generated or to be stored and processed for the user, as well as the properties of each storage node, such as the storage medium characteristics (magnetism, semiconductor, phase change) and physical operating parameters (e.g. the number of erasures, the probability of occurrence of a failure such as a crash) and the security characteristics (e.g. the probability of being attacked or destroyed) of the different storage nodes, which brings about a hidden danger for the secure storage of big data; meanwhile, in the prior art, by performing local and cloud distributed storage on big data, however, a hacker or a lawless person can acquire big data information by decryption (some big data of a user or even no encryption link, correspondingly, is directly obtained) by stealing part of data in either local or cloud, thereby causing disclosure of user privacy or personal information. How to reasonably store big data to a storage node safely, reliably, reasonably and irretrievably by a hacker and further in situ based on the attribute of each storage node, so that the big data is not affected by an attack and not affected by a node failure or physical change is a problem which is not disclosed in the field but is yet to be solved.
Disclosure of Invention
One of the objectives of the present invention is to provide a method and a system for controlling secure storage based on big data, which can safely, reliably, reasonably, and further store the big data in place to a node, based on the attribute of each storage node, aiming at the big data generated by a user or to be stored and processed, so that the big data is not affected by an attack and is not affected by a node failure or a physical change, thereby implementing secure storage of the big data.
The technical scheme adopted by the invention to solve the technical problems is as follows: a big data based secure storage control method comprises the following steps: step S1, receiving big data to be stored, determining the size of the big data to be stored, and confirming the integrity and the validity of the big data; step S2, evaluating the local storage space; step S3, judging the type of big data object, selecting the corresponding strategy; step S4, according to the corresponding strategy, the big data is coded and blocked; step S5, evaluating the capacity and safety of the node space; step S6, storing the big data according to the block and the security and capacity.
In one embodiment, in step S1, receiving the big data to be stored, determining the size of the big data to be stored, and confirming the integrity and validity further includes: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block; in step S2, evaluating the local storage space further includes: after the local memory manager queries the mapping table and the space free state by sending a space query request to the local memory manager, the space free information and the storage starting address of the space free information are sent.
In one embodiment, in step S3, determining the type of big data object, and selecting its corresponding policy further comprises: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, displaying mode selection information to a big data provider, and selecting a compression strategy or a non-compression strategy; the compression strategy comprises the following steps: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q; when p is 0, the value of T (p, q) is a first value and the remaining T (p, q) is a second value; and sequentially sequencing the first numerical values of the sub-blocks and sequentially sequencing the second numerical values of the sub-blocks to form a first numerical sequence and a second numerical sequence.
In one embodiment, in step S5, the evaluating the capacity and the security of the node space further comprises: step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data; step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically; step S53, evaluating the security performance of each node:
wherein k represents the serial number of the node; j represents the number of blocks included for the kth node whose value is a positive integer; j represents the sequence number of the block included in the node; wt. ofkRepresents a weight value of a storage medium employed by the node k, the weight value being (0, 1)]Wherein the magnetic storage medium has a higher weight value than the semiconductor storage medium, the semiconductor storage mediumIs higher than the weight value of the phase change storage medium; step S54, based on the calculated safety performance value of the node k, sorting according to descending order; step S55, sending the available storage space corresponding to the sorted nodes to the safe storage controller for the operation of step S6; in step S56, when the method finally determines to use the block j in the node k, the read-write-erase count ERW of the block j is increased by 1.
In one embodiment, in step S5, the evaluating the capacity and the security of the node space further comprises: step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data; step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically; step S53, evaluating the security performance of each block j of each node k that may be used:
wherein k represents the serial number of the node; j represents the sequence number of the block included in the node; wt. ofkRepresents a weight value of a storage medium employed by the node k, the weight value being (0, 1)]Wherein the weight value of the magnetic storage medium is higher than the weight value of the semiconductor storage medium, and the weight value of the semiconductor storage medium is higher than the weight value of the phase change storage medium; step S54, based on the calculated numerical value of the safety performance of each block j of each node k, sorting according to descending order; step S55, sending the available storage space corresponding to the block of the sorted node to the secure storage controller for the operation of step S6; in step S56, when the method finally determines to use the block j in the node k, the read-write-erase count ERW of the block j is increased by 1.
In one embodiment, in step S4, encoding and blocking the big data according to the corresponding policy further comprises: step S41, determining, based on the strategy selected in the previous step, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy; step S42, setting coding parameters and redundancy parameters aiming at the determined complete data or the second numerical sequence; step S43, equally dividing the complete data or the second numerical sequence into k parts based on the encoding parameters and the redundancy parameters; step S44, creating a matrix with full rank on the upper column of the finite field; step S45, creating a coding matrix table according to the matrix; step S46, aiming at k parts, multiplying the matrix by a coding matrix to obtain a coding result matrix; in step S47, the result matrix of the encoding is arranged in blocks according to the size specified in the configuration information.
In one embodiment, storing the big data according to the blocking and the security and capacity in step S6 further comprises: step S61: determining whether the object in step S3 is uncompressed complete data or compressed first and second numerical sequences, if the object is the uncompressed complete data, performing step S62, otherwise performing step S63; step S62: if the step S3 is uncompressed complete data, arranging for the data arranged in blocks in the step S4 to be stored in the cloud, and performing the step S64; step S63: if the first numerical value sequence and the second numerical value sequence are compressed in step S3, arranging for the first numerical value sequence to be stored locally and the second numerical value sequence to be stored in the cloud and performing step S64; step S64: sequentially storing the data to be stored to the cloud end in the step S62 or S63 to the sorted nodes or blocks according to the sorted nodes or blocks containing the available space obtained in the step S5, and recording the storage nodes and the storage address information thereof to the secure storage controller; step S65: and when the data to be stored is finished, returning a response of the safe storage to the local user.
According to an exemplary embodiment of the present invention, a big data based secure storage control system is also claimed. The big data based secure storage control system comprises: the device comprises a first module, a second module and a third module, wherein the first module is used for receiving big data to be stored, determining the size of the big data to be stored and confirming the integrity and the validity of the big data to be stored; a second module for evaluating a local storage space; the third module is used for judging the type of the big data object and selecting a corresponding strategy; a fourth module for encoding and blocking the big data according to the corresponding strategy; a fifth module for evaluating capacity and security of a node space; and the sixth module is used for storing the big data according to the block and the safety and the capacity.
In one embodiment, the first module is further to: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block; the second module is further to: after the local memory manager queries the mapping table and the space free state by sending a space query request to the local memory manager, the space free information and the storage starting address of the space free information are sent.
In one embodiment, the third module is further to: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, displaying mode selection information to a big data provider, and selecting a compression strategy or a non-compression strategy; the compression strategy comprises the following steps: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q; when p is 0, the value of T (p, q) is a first value and the remaining T (p, q) is a second value; and sequentially sequencing the first numerical values of the sub-blocks and sequentially sequencing the second numerical values of the sub-blocks to form a first numerical sequence and a second numerical sequence.
In one embodiment, the fifth module further comprises: the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data; a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically; a fifth module, configured to evaluate security performance of each node:
wherein k represents the serial number of the node; j represents the number of blocks included for the kth node whose value is a positive integer; j represents the sequence number of the block included in the node; wt. ofkRepresents a weight value of a storage medium employed by the node k, the weight value being (0, 1)]Wherein the weight value of the magnetic storage medium is higher than the weight value of the semiconductor storage medium, and the weight value of the semiconductor storage medium is higher than the weight value of the phase change storage medium; a fifth module for countingSequencing the calculated numerical values of the safety performance of the nodes k according to a descending order; a fifth module, configured to send an available storage space corresponding to the sorted node to the secure storage controller, for operation of the sixth module; a fifth module for increasing the number of times ERW of read-write-erase of the block j by 1 when the method finally determines to use the block j in the node k.
In one embodiment, the fifth module further comprises: the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data; a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically; a fifth module for evaluating the security performance of each block j of each node k that may be used:
wherein k represents the serial number of the node; j represents the sequence number of the block included in the node; wt. ofkRepresents a weight value of a storage medium employed by the node k, the weight value being (0, 1)]Wherein the weight value of the magnetic storage medium is higher than the weight value of the semiconductor storage medium, and the weight value of the semiconductor storage medium is higher than the weight value of the phase change storage medium; a fifth module, configured to sort in descending order based on the calculated numerical values of the security performance of each block j of each node k; a fifth module for sorting the regions of nodesSending the available storage space corresponding to the block to a secure storage controller for operation of a sixth module; a fifth module for increasing the number of times ERW of read-write-erase of the block j by 1 when the method finally determines to use the block j in the node k.
In one embodiment, the fourth module further comprises: a fourth module for determining, based on the strategy selected in the previous module, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy; a fourth second module, configured to set coding parameters and redundancy parameters for the determined complete data or the second numerical sequence; a fourth third module for equally dividing the complete data or the second numerical sequence into k parts based on the coding parameters and the redundancy parameters; a fourth module, configured to create a matrix with full rank on the upper column of the finite field; a fifth module for creating a coding matrix table according to the matrix; a fourth sixth module, configured to multiply the matrix with the coding matrix for k parts to obtain a coding result matrix; and a fourth seventh module, configured to arrange the encoded result matrix in blocks according to the size specified in the configuration information.
In one embodiment, the sixth module further comprises: a sixth module: for determining whether an object in the third module is uncompressed complete data or compressed first and second numerical sequences, if the former then proceeding to the sixth second module, otherwise to the sixth third module; a sixth second module: if the third module is uncompressed complete data, arranging to store the data arranged in blocks in the fourth module to the cloud end, and continuing to the sixth fourth module; a sixth third module: if the third module is the compressed first numerical sequence and the second numerical sequence, arranging for the first numerical sequence to be stored locally and the second numerical sequence to be stored in the cloud and continuing to the sixth fourth module; a sixth fourth module: the system comprises a sixth module, a sixth storage module, a secure storage controller and a third module, wherein the sixth module is used for sequentially storing data to be stored to a cloud end in the sixth module or the sixth module to the sorted nodes or blocks according to the sorted nodes or blocks which contain available space and are obtained in the fifth module, and storing storage nodes and storage address information of the storage nodes into the secure storage controller; a sixth module: and the response for returning the safe storage to the local user when the data to be stored is finished.
Drawings
Embodiments of the invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
fig. 1 illustrates a flowchart of a big data based secure storage control method according to an exemplary embodiment of the present invention.
Detailed Description
Before proceeding with the following detailed description, it may be advantageous to set forth definitions of certain words and phrases used throughout this patent document: the terms "include" and "comprise," as well as derivatives thereof, mean inclusion without limitation; the term "or" is inclusive, meaning and/or; the phrases "associated with," "associated with," and derivatives thereof may mean to include, be included within, with, interconnect with, contain, be included within, be connected to, or be connected with, be coupled to, or be coupled with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to, or be bound with, have properties of, etc.; while the term "controller" means any device, system or component thereof that controls at least one operation, such a device may be implemented in hardware, firmware or software, or some combination of at least two of the same. It should be noted that: the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, as those skilled in the art will understand: in many, if not most instances, such definitions apply to prior as well as future uses of such defined words and phrases.
In the following description, reference is made to the accompanying drawings that show, by way of illustration, several specific embodiments. It will be understood that: other embodiments are contemplated and may be made without departing from the scope or spirit of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense.
Fig. 1 illustrates a flowchart of a big data based secure storage control method according to an exemplary embodiment of the present invention. The big data-based secure storage control method comprises the following steps:
step S1, receiving big data to be stored, determining the size of the big data to be stored, and confirming the integrity and the validity of the big data;
step S2, evaluating the local storage space;
step S3, judging the type of big data object, selecting the corresponding strategy;
step S4, according to the corresponding strategy, the big data is coded and blocked;
step S5, evaluating the capacity and safety of the node space;
step S6, storing the big data according to the block and the security and capacity.
Preferably, in step S1, the receiving the big data to be stored, determining the size of the big data to be stored, and confirming the integrity and validity further includes: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block.
Through the steps, the integrity and the effectiveness of the original big data can be effectively ensured.
Preferably, in step S2, the evaluating the local storage space further comprises: after the local memory manager queries the mapping table and the space free state by sending a space query request to the local memory manager, the space free information and the storage starting address of the space free information are sent.
By this step, the local storage capability can be effectively acquired, and no provision is made for secure storage control.
Preferably, in step S3, determining the type of the big data object, and selecting the corresponding policy further includes: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, the mode selection information is displayed to the big data provider, and whether the compression strategy or the non-compression strategy is selected.
Preferably, the image information of the non-compression strategy is complete data of the image.
Further preferably, the compression strategy comprises: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q.
When p is 0, T (p, q) has a first value, and the rest of T (p, q) has a second value. And sequentially sequencing the first numerical values of the sub-blocks and sequentially sequencing the second numerical values of the sub-blocks to form a first numerical sequence and a second numerical sequence.
Preferably, the corresponding decompression policy is:
preferably, the first and second values may be stored locally and in the cloud, respectively, in a subsequent step.
Through this step, even if a hacker or a lawless person intercepts the segments of the first numerical sequence and the second numerical sequence, it cannot restore the image of the original big data, thereby controlling the safe storage of the big data.
Preferably, in step S4, encoding and blocking the big data according to the corresponding policy further includes:
step S41, determining, based on the strategy selected in the previous step, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy;
step S42, setting coding parameters and redundancy parameters aiming at the determined complete data or the second numerical sequence;
step S43, equally dividing the complete data or the second numerical sequence into k parts based on the encoding parameters and the redundancy parameters;
step S44, creating a matrix with full rank on the upper column of the finite field;
step S45, creating a coding matrix table according to the matrix;
step S46, aiming at k parts, multiplying the matrix by a coding matrix to obtain a coding result matrix;
in step S47, the result matrix of the encoding is arranged in blocks according to the size specified in the configuration information.
Preferably, in step S5, the evaluating the capacity and the security of the node space further comprises:
step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data;
step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
step S53, evaluating the security performance of each node:
step S54, based on the calculated safety performance value of the node k, sorting according to descending order;
step S55, sending the available storage space corresponding to the sorted nodes to the safe storage controller for the operation of step S6;
in step S56, when the method finally determines to use the block j in the node k, the read-write-erase count ERW of the block j is increased by 1.
Preferably, the aforementioned operations include, but are not limited to, reading, writing, erasing, refreshing, precharging.
Alternatively, in step S5, the evaluating the capacity and the security of the node space further comprises:
step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data;
step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
step S53, evaluating the security performance of each block j of each node k that may be used:
step S54, based on the calculated numerical value of the safety performance of each block j of each node k, sorting according to descending order;
step S55, sending the available storage space corresponding to the block of the sorted node to the safe storage controller for the operation of step S6;
in step S56, when the method finally determines to use the block j in the node k, the read-write-erase count ERW of the block j is increased by 1.
By the step, the difference of the storage media and the historical safety data of the storage media can be comprehensively considered, and the large data to be stored can be guaranteed to be stored on the node, so that the probability of damage of the large data can be reduced to the maximum extent.
Preferably, in step S6, storing the big data according to the block and the security and capacity further comprises:
step S61: determining whether the object in step S3 is uncompressed complete data or compressed first and second numerical sequences, if the object is the uncompressed complete data, performing step S62, otherwise performing step S63;
step S62: if the step S3 is uncompressed complete data, arranging for the data arranged in blocks in the step S4 to be stored in the cloud, and performing the step S64;
step S63: if the first numerical value sequence and the second numerical value sequence are compressed in step S3, arranging for the first numerical value sequence to be stored locally and the second numerical value sequence to be stored in the cloud and performing step S64;
step S64: sequentially storing the data to be stored to the cloud end in the step S62 or S63 to the sorted nodes or blocks according to the sorted nodes or blocks containing the available space obtained in the step S5, and recording the storage nodes and the storage address information thereof to the secure storage controller;
step S65: and when the data to be stored is finished, returning a response of the safe storage to the local user.
Through the steps, the safe storage and control of the data can be ensured, and hackers or lawbreakers can also be ensured to be incapable of restoring the image of the original big data aiming at the image data, so that the privacy and the information safety of users are ensured.
Correspondingly, the application also relates to a safe storage control system based on big data, which comprises:
the device comprises a first module, a second module and a third module, wherein the first module is used for receiving big data to be stored, determining the size of the big data to be stored and confirming the integrity and the validity of the big data to be stored;
a second module for evaluating a local storage space;
the third module is used for judging the type of the big data object and selecting a corresponding strategy;
a fourth module for encoding and blocking the big data according to the corresponding strategy;
a fifth module for evaluating capacity and security of a node space;
and the sixth module is used for storing the big data according to the block and the safety and the capacity.
Preferably, the first module is further configured to: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block.
Preferably, the second module is further configured to: after the local memory manager queries the mapping table and the space free state by sending a space query request to the local memory manager, the space free information and the storage starting address of the space free information are sent.
Preferably, the third module is further configured to: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, the mode selection information is displayed to the big data provider, and whether the compression strategy or the non-compression strategy is selected.
Preferably, the image information of the non-compression strategy is complete data of the image.
Further preferably, the compression strategy comprises: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q.
When p is 0, T (p, q) has a first value, and the rest of T (p, q) has a second value. And sequentially sequencing the first numerical values of the sub-blocks and sequentially sequencing the second numerical values of the sub-blocks to form a first numerical sequence and a second numerical sequence.
Preferably, the corresponding decompression policy is:
preferably, the first and second values may be stored locally and in the cloud, respectively, in operation of subsequent modules.
Preferably, the fourth module further comprises:
a fourth module for determining, based on the strategy selected in the previous module, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy;
a fourth second module, configured to set coding parameters and redundancy parameters for the determined complete data or the second numerical sequence;
a fourth third module for equally dividing the complete data or the second numerical sequence into k parts based on the coding parameters and the redundancy parameters;
a fourth module, configured to create a matrix with full rank on the upper column of the finite field;
a fifth module for creating a coding matrix table according to the matrix;
a fourth sixth module, configured to multiply the matrix with the coding matrix for k parts to obtain a coding result matrix;
and a fourth seventh module, configured to arrange the encoded result matrix in blocks according to the size specified in the configuration information.
Preferably, the fifth module further comprises:
the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data;
a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
a fifth module, configured to evaluate security performance of each node:
a fifth module for sorting in descending order based on the calculated value of the security performance of the node k;
a fifth module, configured to send an available storage space corresponding to the sorted node to the secure storage controller, for operation of the sixth module;
a fifth module for increasing the number of times ERW of read-write-erase of the block j by 1 when the method finally determines to use the block j in the node k.
Preferably, the aforementioned operations include, but are not limited to, reading, writing, erasing, refreshing, precharging.
Alternatively, the fifth module further comprises:
the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data;
a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
a fifth module for evaluating the security performance of each block j of each node k that may be used:
a fifth module, configured to sort in descending order based on the calculated numerical values of the security performance of each block j of each node k;
a fifth module, configured to send an available storage space corresponding to the block of the sorted node to the secure storage controller, for operation of the sixth module;
a fifth module for increasing the number of times ERW of read-write-erase of the block j by 1 when the method finally determines to use the block j in the node k.
Preferably, the sixth module further comprises:
a sixth module: for determining whether an object in the third module is uncompressed complete data or compressed first and second numerical sequences, if the former then proceeding to the sixth second module, otherwise to the sixth third module;
a sixth second module: if the third module is uncompressed complete data, arranging to store the data arranged in blocks in the fourth module to the cloud end, and continuing to the sixth fourth module;
a sixth third module: if the third module is the compressed first numerical sequence and the second numerical sequence, arranging for the first numerical sequence to be stored locally and the second numerical sequence to be stored in the cloud and continuing to the sixth fourth module;
a sixth fourth module: the system comprises a sixth module, a sixth storage module, a secure storage controller and a third module, wherein the sixth module is used for sequentially storing data to be stored to a cloud end in the sixth module or the sixth module to the sorted nodes or blocks according to the sorted nodes or blocks which contain available space and are obtained in the fifth module, and storing storage nodes and storage address information of the storage nodes into the secure storage controller;
a sixth module: and the response for returning the safe storage to the local user when the data to be stored is finished.
The above-mentioned technical terms are conventional technical terms having ordinary meanings in the art, and are not further explained herein in order not to obscure the point of the present invention.
In summary, in the technical solution of the present invention, by using a method and a system for controlling secure storage based on big data, it is able to safely, reliably, reasonably and not intercepted by a hacker and further store the big data to a node in place based on the attribute of each storage node for big data generated by a user or to be stored and processed, so that the big data is not affected by an attack and is not affected by a node failure or a physical change, thereby implementing secure storage of the big data.
It will be understood that: the examples and embodiments of the invention may be implemented in hardware, software, or a combination of hardware and software. As mentioned above, any body performing this method may be stored, for example, in the form of volatile or non-volatile storage, for example, a storage device, like a ROM, whether erasable or rewritable or not, or in the form of memory, such as for example a RAM, a memory chip, a device or an integrated circuit, or on an optically or magnetically readable medium, such as for example a CD, a DVD, a magnetic disk or a magnetic tape. It will be understood that: storage devices and storage media are examples of machine-readable storage suitable for storing one or more programs that, when executed, implement examples of the present invention. Examples of the present invention may be conveyed electronically via any medium, such as a communications signal carried by a wired or wireless coupling, and the examples contain the same where appropriate.
It should be noted that: because the invention solves the technical problem that the big data generated by a user or to be stored and processed can be safely, reliably and reasonably stored in the node in a way that the big data can not be intercepted by a hacker and then can be stored in the node in situ based on the attribute of each storage node, so that the big data is not influenced by attack and is not influenced by node failure or physical change, thereby realizing the safe storage of the big data, and the technical means which can be understood by technical personnel in the technical field according to the teaching after reading the specification is adopted, and the beneficial technical effect is obtained, the scheme claimed in the appended claims belongs to the technical scheme in the meaning of patent law. Furthermore, the solution claimed in the appended claims has utility since it can be manufactured or used in industry.
The above description is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (6)
1. A big data based secure storage control method comprises the following steps:
step S1, receiving big data to be stored, determining the size of the big data to be stored, and confirming the integrity and the validity of the big data;
step S2, evaluating the local storage space;
step S3, judging the type of big data object, selecting the corresponding strategy;
step S4, according to the corresponding strategy, the big data is coded and blocked;
step S5, evaluating the capacity and safety of the node space;
step S6, storing the big data according to the block, the safety and the capacity;
wherein in step S1, receiving the big data to be stored, determining the size of the big data to be stored, and confirming the integrity and validity further comprises: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block;
in step S2, evaluating the local storage space further includes: sending a space query request to a local memory manager, and sending space idle information and a storage initial address of the space idle information after the local memory manager queries a mapping table and a space idle state;
in step S5, the evaluating the capacity and the security of the node space further includes any one of scenario 1 and scenario 2:
scheme 1 includes:
step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data;
step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
step S53, evaluating the security performance of each node:
step S54, based on the calculated safety performance value of the node k, sorting according to descending order;
step S55, sending the available storage space corresponding to the sorted nodes to the safe storage controller for the operation of step S6;
step S56, when the method finally determines to use the block j in the node k, increasing the number of times ERW of read, write and erase of the block j by 1;
the scheme 2 comprises the following steps:
step S51, determining available cloud nodes, and acquiring storage medium types of the nodes used by the cloud for storing big data;
step S52, based on different node storage medium types, obtaining the upper limit value TT and the available storage space of the read-write-erase using times through a lookup table, and obtaining the read-write-erase times ERW, the safety factor SE and the fault abnormal probability FA of different blocks of the storage medium of each node, wherein a single node comprises a positive integer of blocks, different blocks have different read-write-erase times due to historical operation, different blocks have different safety factors due to historical attack or malicious program infection, and different blocks have different fault abnormal probabilities due to abnormal fault; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
step S53, evaluating the security performance of each block j of each node k that may be used:
step S54, based on the calculated numerical value of the safety performance of each block j of each node k, sorting according to descending order;
step S55, sending the available storage space corresponding to the block of the sorted node to the safe storage controller for the operation of step S6;
in step S56, when the method finally determines to use the block j in the node k, the read-write-erase count ERW of the block j is increased by 1.
2. The big-data-based secure storage control method according to claim 1, wherein the step S5 includes scenario 1:
in step S3, determining the type of the big data object, and selecting the corresponding policy further includes: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, displaying mode selection information to a big data provider, and selecting a compression strategy or a non-compression strategy;
the compression strategy comprises the following steps: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q;
when p is 0, the value of T (p, q) is a first value and the remaining T (p, q) is a second value; and sequencing the first numerical values of the sub-blocks in sequence, and sequencing the second numerical values of the sub-blocks in sequence to form a first numerical sequence and a second numerical sequence.
3. The big-data-based secure storage control method of claim 1, wherein the step S5 includes scenario 2:
in step S3, determining the type of the big data object, and selecting the corresponding policy further includes: analyzing data to be stored, and determining whether the data belongs to text, programs or image information; if the text and the program belong to, selecting complete data encoding and block storage; if the image information is the image information, displaying mode selection information to a big data provider, and selecting a compression strategy or a non-compression strategy;
the compression strategy comprises the following steps: dividing the large data into sub-blocks comprising R × R cells, R being a positive integer power of 2, each cell of a sub-block being represented as (x, y); wherein the portion of the edge portion which is less than R is filled with binary zero values; for each of the sub-blocks, performing the following transformation:
wherein when r is 0, the compound is,
and r ≠ 0, c (r) ═ 1, where r is p or q;
when p is 0, the value of T (p, q) is a first value and the remaining T (p, q) is a second value; and sequencing the first numerical values of the sub-blocks in sequence, and sequencing the second numerical values of the sub-blocks in sequence to form a first numerical sequence and a second numerical sequence.
4. The big-data-based secure storage control method according to claim 2 or 3, wherein:
in step S4, encoding and blocking the big data according to the corresponding policy further comprises:
step S41, determining, based on the strategy selected in the previous step, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy;
step S42, setting coding parameters and redundancy parameters aiming at the determined complete data or the second numerical sequence;
step S43, equally dividing the complete data or the second numerical sequence into k parts based on the encoding parameters and the redundancy parameters;
step S44, creating a matrix with full rank on the upper column of the finite field;
step S45, creating an encoding matrix according to the matrix;
step S46, multiplying the matrix of step S44 by the coding matrix for k parts to obtain a result matrix of coding;
step S47, arranging the result matrix of coding in blocks according to the size specified in the configuration information; and
in step S6, storing the big data according to the chunking and the security and capacity further comprises:
step S61: determining whether the object in step S3 is uncompressed complete data or compressed first and second numerical sequences, if the object is the uncompressed complete data, performing step S62, otherwise performing step S63;
step S62: if the step S3 is uncompressed complete data, arranging for the data arranged in blocks in the step S4 to be stored in the cloud, and performing the step S64;
step S63: if the first numerical value sequence and the second numerical value sequence are compressed in step S3, arranging for the first numerical value sequence to be stored locally and the second numerical value sequence to be stored in the cloud and performing step S64;
step S64: sequentially storing the data to be stored to the cloud end in the step S62 or S63 to the sorted nodes or blocks according to the sorted nodes or blocks containing the available space obtained in the step S5, and recording the storage nodes and the storage address information thereof to the secure storage controller;
step S65: and when the data to be stored is finished, returning a response of the safe storage to the local user.
5. A big-data based secure storage control system, comprising:
the device comprises a first module, a second module and a third module, wherein the first module is used for receiving big data to be stored, determining the size of the big data to be stored and confirming the integrity and the validity of the big data to be stored;
a second module for evaluating a local storage space;
the third module is used for judging the type of the big data object and selecting a corresponding strategy;
a fourth module for encoding and blocking the big data according to the corresponding strategy;
a fifth module for evaluating capacity and security of a node space;
the sixth module is used for storing the big data according to the blocks, the safety and the capacity;
wherein the first module is further to: receiving the big data stored by the tape via the data link and determining the data length and data block information thereof, and determining the integrity and validity of the big data by comparing with the actual length information based on the data size information contained in the data block;
the second module is further to: sending a space query request to a local memory manager, and sending space idle information and a storage initial address of the space idle information after the local memory manager queries a mapping table and a space idle state;
the fifth module further comprises any one of scheme 1 and scheme 2:
scheme 1 is as follows:
the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data;
a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by a malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
a fifth module, configured to evaluate security performance of each node:
a fifth module for sorting in descending order based on the calculated value of the security performance of the node k;
a fifth module, configured to send an available storage space corresponding to the sorted node to the secure storage controller, for operation of the sixth module;
a fifth module, configured to increase the number of times ERW of read, write, and erase of the block j by 1 when the system finally determines to use the block j in the node k;
scheme 2 is as follows:
the fifth module is used for determining available cloud nodes and acquiring the storage medium types of the nodes used by the cloud for storing the big data;
a fifth module, configured to obtain, based on different node storage medium types, an upper limit TT and an available storage space of the read-write-erase usage times of the storage medium through a lookup table, and obtain read-write-erase times ERW, a safety factor SE, and a failure anomaly probability FA of different blocks of the storage medium of each node, where a single node includes a positive integer of blocks, and different blocks have different read-write-erase times due to historical operations, different blocks have different safety factors due to historical attacks or malicious program infection, and different blocks have different failure anomaly probabilities due to anomalies; the safety factor is the ratio of the number of times that the block is attacked or infected by the malicious program to the number of times that the block is operated historically, and the fault abnormal probability is the ratio of the number of times that the block generates faults due to the abnormality to the number of times that the block is operated historically;
a fifth module for evaluating the security performance of each block j of each node k that may be used:
a fifth module, configured to sort in descending order based on the calculated numerical values of the security performance of each block j of each node k;
a fifth module, configured to send an available storage space corresponding to the block of the sorted node to the secure storage controller, for operation of the sixth module;
a fifth module for increasing the number of times ERW of read-write-erase of block j in node k by 1 when the system finally determines to use block j.
6. The big-data based secure storage control system of claim 5, wherein:
the fourth module further comprises:
a fourth module for determining, based on the strategy selected in the previous module, either to encode and block the complete data of the text and the program or the uncompressed data of the image, or to encode and block the second numerical sequence by means of a compression strategy;
a fourth second module, configured to set coding parameters and redundancy parameters for the determined complete data or the second numerical sequence;
a fourth third module for equally dividing the complete data or the second numerical sequence into k parts based on the coding parameters and the redundancy parameters;
a fourth module, configured to create a matrix with full rank on the upper column of the finite field;
a fourth and fifth module for creating an encoding matrix from the matrix;
a fourth sixth module, configured to multiply, for the k parts, the matrix created by the fourth module with a coding matrix to obtain a coded result matrix;
a seventh module, configured to arrange the encoded result matrix in blocks according to the size specified in the configuration information; and
the sixth module further comprises:
a sixth module: for determining whether an object in the third module is uncompressed complete data or compressed first and second numerical sequences, if the former then proceeding to the sixth second module, otherwise to the sixth third module;
a sixth second module: if the third module is uncompressed complete data, arranging to store the data arranged in blocks in the fourth module to the cloud end, and continuing to the sixth fourth module;
a sixth third module: if the third module is the compressed first numerical sequence and the second numerical sequence, arranging for the first numerical sequence to be stored locally and the second numerical sequence to be stored in the cloud and continuing to the sixth fourth module;
a sixth fourth module: the system comprises a sixth module, a sixth storage module, a secure storage controller and a third module, wherein the sixth module is used for sequentially storing data to be stored to a cloud end in the sixth module or the sixth module to the sorted nodes or blocks according to the sorted nodes or blocks which contain available space and are obtained in the fifth module, and storing storage nodes and storage address information of the storage nodes into the secure storage controller;
a sixth module: and the response for returning the safe storage to the local user when the data to be stored is finished.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010210196.2A CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
| CN201910834991.6A CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910834991.6A CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210196.2A Division CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110532804A CN110532804A (en) | 2019-12-03 |
| CN110532804B true CN110532804B (en) | 2020-06-05 |
Family
ID=68667015
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910834991.6A Active CN110532804B (en) | 2019-09-05 | 2019-09-05 | Safe storage control method based on big data |
| CN202010210196.2A Expired - Fee Related CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010210196.2A Expired - Fee Related CN111428266B (en) | 2019-09-05 | 2019-09-05 | Storage node space capacity and security evaluation method based on big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN110532804B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111177765A (en) * | 2020-01-06 | 2020-05-19 | 广州知弘科技有限公司 | Financial big data processing method, storage medium and system |
| CN114675789B (en) * | 2022-04-20 | 2023-06-16 | 深圳有方信息技术有限公司 | Big data analysis and storage system and method based on computer system |
| CN115174591B (en) * | 2022-07-01 | 2023-10-13 | 深圳市玄羽科技有限公司 | Intelligent manufacturing system and data security management method thereof |
| CN116627361B (en) * | 2023-07-25 | 2023-11-17 | 江苏华存电子科技有限公司 | Storage comparison management method and system for redundant information |
| CN117235013B (en) * | 2023-11-10 | 2024-01-30 | 中科数创(临沂)数字科技有限公司 | Intelligent archive management method based on artificial intelligence |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104636673A (en) * | 2015-03-10 | 2015-05-20 | 四川中科腾信科技有限公司 | Safe data storage method under big data background |
| CN109032499A (en) * | 2018-06-09 | 2018-12-18 | 西安电子科技大学 | A kind of data access method of Distributed Storage, information data processing terminal |
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
| CN109960459A (en) * | 2017-12-23 | 2019-07-02 | 王彬 | A kind of data-storage system |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN100555289C (en) * | 2007-12-20 | 2009-10-28 | 中国科学院计算技术研究所 | A kind of continuous data protection system and its implementation |
| US9369433B1 (en) * | 2011-03-18 | 2016-06-14 | Zscaler, Inc. | Cloud based social networking policy and compliance systems and methods |
| US10079731B2 (en) * | 2015-11-23 | 2018-09-18 | International Business Machines Corporation | Client-space network monitoring |
| US10097572B1 (en) * | 2016-06-07 | 2018-10-09 | EMC IP Holding Company LLC | Security for network computing environment based on power consumption of network devices |
| CN106203164B (en) * | 2016-07-01 | 2017-10-27 | 上海宽惠网络科技股份有限公司 | Information security big data resource management system based on trust computing and cloud computing |
| CN106230982B (en) * | 2016-09-08 | 2019-07-16 | 哈尔滨工程大学 | A kind of dynamic self-adapting secure cloud storage method considering node reliability |
| CN109933501B (en) * | 2017-12-15 | 2022-08-23 | 中国移动通信集团浙江有限公司 | Capacity evaluation method and device of application system |
| CN110062199B (en) * | 2018-01-19 | 2020-07-10 | 杭州海康威视系统技术有限公司 | Load balancing method and device and computer readable storage medium |
| CN109446020B (en) * | 2018-09-21 | 2022-05-13 | 曙光云计算集团有限公司 | Dynamic evaluation method and device of cloud storage system |
| CN110162270B (en) * | 2019-04-29 | 2020-08-25 | 平安国际智慧城市科技股份有限公司 | Data storage method, storage node and medium based on distributed storage system |
-
2019
- 2019-09-05 CN CN201910834991.6A patent/CN110532804B/en active Active
- 2019-09-05 CN CN202010210196.2A patent/CN111428266B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104636673A (en) * | 2015-03-10 | 2015-05-20 | 四川中科腾信科技有限公司 | Safe data storage method under big data background |
| CN109960459A (en) * | 2017-12-23 | 2019-07-02 | 王彬 | A kind of data-storage system |
| CN109032499A (en) * | 2018-06-09 | 2018-12-18 | 西安电子科技大学 | A kind of data access method of Distributed Storage, information data processing terminal |
| CN109542352A (en) * | 2018-11-22 | 2019-03-29 | 北京百度网讯科技有限公司 | Method and apparatus for storing data |
Non-Patent Citations (2)
| Title |
|---|
| A Security-Aware Data Placement Mechanism for Big Data Cloud Storage Systems;Seungmin Kang 等;《2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS)》;20160704;全文 * |
| 基于PCM的大数据存储与管理研究综述;吴章玲 等;《计算机研究与发展》;20151231;第52卷(第2期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111428266A (en) | 2020-07-17 |
| CN110532804A (en) | 2019-12-03 |
| CN111428266B (en) | 2020-11-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110532804B (en) | Safe storage control method based on big data | |
| Goodrich et al. | Invertible bloom lookup tables | |
| US9165002B1 (en) | Inexpensive deletion in a data storage system | |
| US9378075B2 (en) | Reducing interference through controlled data access | |
| CN109064031B (en) | Project affiliate credit evaluation method based on block chain, block chain and storage medium | |
| JP2021513141A (en) | Generation and identification of 2D barcodes | |
| US20090094318A1 (en) | Smart access to a dispersed data storage network | |
| CN106874348B (en) | File storage and index method and device and file reading method | |
| CN108268354A (en) | Data safety monitoring method, background server, terminal and system | |
| US20130179413A1 (en) | Compressed Distributed Storage Systems And Methods For Providing Same | |
| EP3154202B1 (en) | Encoding program, encoding method, encoding device, decoding program, decoding method, and decoding device | |
| US9619657B2 (en) | Method and apparatus for storing redeem code, and method and apparatus for verifying redeem code | |
| CN112073444B (en) | Data set processing method and device and server | |
| CN105893169A (en) | File storage method and system based on erasure codes | |
| CN108133026B (en) | Multi-data processing method, system and storage medium | |
| CN117061254A (en) | Abnormal flow detection method, device and computer equipment | |
| CN111797240A (en) | Enterprise entity unique ID creation method, storage medium, electronic device and system | |
| CN108090364A (en) | A kind of localization method and system in data leak source | |
| CN115834257A (en) | Cloud electric power data safety protection method and protection system | |
| Mungamuru et al. | Privacy, Preservation and Performance: The 3 P's of Distributed Data Management | |
| CN111444270B (en) | Method and system for controlling harmful information based on block chain | |
| Chavhan et al. | Implementation of improved inline deduplication scheme for distributed cloud storage | |
| CN113674083A (en) | Internet financial platform credit risk monitoring method, device and computer system | |
| US20240126923A1 (en) | Log compression and obfuscation using embeddings | |
| Zhao et al. | An Automatically Privacy Protection Solution for Implementing the Right to Be Forgotten in Embedded System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200511 Address after: 201702 Shanghai city Qingpu District No. 599, building 1, 2 Gaojing Road, room 208 Applicant after: SHANGHAI V&G INFORMATION TECHNOLOGY Co.,Ltd. Address before: 510000 1011, building H5, Luogang Aoyuan Plaza, 1904 Chuang Kai Avenue, Huangpu District, Guangzhou City, Guangdong Province Applicant before: GUANGZHOU ZHIHONG TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |