CN110647503A - Distributed storage method and device - Google Patents
Distributed storage method and device Download PDFInfo
- Publication number
- CN110647503A CN110647503A CN201910954998.1A CN201910954998A CN110647503A CN 110647503 A CN110647503 A CN 110647503A CN 201910954998 A CN201910954998 A CN 201910954998A CN 110647503 A CN110647503 A CN 110647503A
- Authority
- CN
- China
- Prior art keywords
- file
- block chain
- storage
- private
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/137—Hash-based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/134—Distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/178—Techniques for file synchronisation in file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
Abstract
The invention discloses a distributed storage system, which realizes the distribution autonomy of storage nodes in the distributed storage system and the data exchange and data sharing among the storage nodes based on a consensus algorithm by utilizing the characteristics of decentralization and trust removal in the construction process of the distributed storage system. Compared with the existing mainstream centralized cloud service system, the network node can establish a many-to-many contact relation instead of one-to-many contact relation, so that the data transmission efficiency can be greatly improved; the blocking encapsulation, anonymous exchange and asymmetric encryption algorithm of data are adopted in the data storage, exchange, uploading and downloading processes, so that the safety and reliability of the whole ecological chain of data cloud storage are greatly guaranteed. Compared with the existing mainstream centralized cloud service system, the cloud storage system can greatly improve the operation efficiency of the system on the basis of improving the data security and reduce the operation cost of the cloud storage service.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for distributed storage of data.
Background
Blockchain technology is gaining increasing attention as digital cryptocurrency, such as bitcoin, becomes more popular. The block chain technology is a novel decentralized protocol and can safely store digital assets such as digital currency, equity and bond rights. The block chain technology effectively solves the consensus problem in the problem of the general of Byzantin by means of data encryption, timestamps, distributed consensus, economic incentive and the like, realizes decentralized point-to-point transaction in a distributed system with nodes not needing mutual trust, thereby effectively reducing the trust cost of practical economy and redefining the property right system of the Internet era.
Although the blockchain technique significantly improves the security and reliability of data, the storage scalability of the current blockchain technique is poor. Taking bitcoin as an example, 465402 blocks are generated in total by 5, 8 and 5 months in 2017, the total capacity is 107.89GB, and 9892723 certified addresses are on the chain. Because the blockchain technique requires that each complete node in the bitcoin network holds complete blockchain information, currently, approximately 1000 ten thousand nodes contribute more than 100GB of disk space to store blockchain data. That is, the current bitcoin system uses a storage space of approximately 1000PB to store only about 100GB of data, which wastes storage space greatly. And the capacity of the bitcoin and the number of participating nodes are increased rapidly with the passage of time, and the block chain technology occupies a large amount of storage space of a large number of nodes more and more. This also greatly limits the development and application of database systems based on blockchain technology.
Meanwhile, the block chain platform EOS intelligent contract vulnerability event pushes the block chain security up to the wind gap wave tip again. An attacker may control all nodes in a blockchain network to be desired after a series of operations by issuing a "smart contract" containing malicious code.
In order to increase the storage expansibility and the storage safety of the blockchain technology, the invention provides a blockchain storage capacity extensible model, and a data copy allocation strategy is adopted to optimize the model.
Disclosure of Invention
Aiming at the technical problems, particularly in the traditional distributed data storage, the technical personnel in the field improve the distributed storage mode and the block chain technology, and the technical problems of the prior art, namely the excessive redundancy problem of the distributed file storage, need to be solved; secondly, the storage expansibility of the block chain technology is increased; and thirdly, solving the problem that the intelligent contract is illegally attacked.
The present invention is directed to at least solving the problems of the prior art. Therefore, the invention discloses a distributed storage system based on a blockchain technology, which introduces the blockchain technology in the construction process of the distributed storage system, and realizes the distribution autonomy of storage nodes in the distributed storage system and the data exchange and data sharing among the storage nodes based on a consensus algorithm by utilizing the characteristics of decentralization and distrust of the blockchain technology. Compared with a centralized cloud service system, the network nodes can establish a many-to-many contact relation instead of a one-to-many contact relation, and the data transmission efficiency can be greatly improved; the blocking encapsulation, anonymous exchange and asymmetric encryption algorithm of data are adopted in the data storage, exchange, uploading and downloading processes, so that the safety and reliability of the whole ecological chain of data cloud storage are greatly guaranteed. The cloud storage service management method and the cloud storage service management system can greatly improve the operation efficiency of the system and reduce the operation cost of the cloud storage service on the basis of improving the data security.
In order to achieve the purpose, the invention adopts the technical scheme that:
a distributed storage method is characterized in that a stored file is represented by a hash value and distributed by a BitTorrent protocol, and specifically comprises the following steps:
s1, constructing a block chain network based on an interplanetary file system (IPFS) protocol;
s2, constructing a private chain different from the block chain network;
s3, an uploader in the distributed storage network generates an uploading request and records the uploading request in a contract of the private chain;
s4, dividing the file to be stored into a plurality of files and uploading the files to the block chain network based on a specific protocol;
s5, a storage request is generated by a storage person in the distributed storage network, and the storage request is recorded in the contract;
s6, synchronously storing an uploading request and a storage request of a file by a DHT (distributed hash table) on each node of the block chain network;
and S7, storing the file fragments through a BitTorrent protocol.
Preferably, the upload request includes a split number of files, a backup number of files, a file description, a public key of the uploader, an account name of the uploader, an uploader ID, and a file ID.
Preferably, the specific protocol in step S4 is WebRTC and/or uTP.
Preferably, the private chain comprises: the core block chain, private block chain equipment and central server, private block chain equipment is including the node server, be provided with hardware in the node server and prevent hot wall, private block chain equipment passes through central server and core block chain connection, private block chain equipment has been set up again based on core block chain, a plurality of block chain nodes of this private block chain equipment configuration, this private chain is built and can establish a plurality of private chain nodes based on block chain platform, set up central server, establish private chain control management platform through central server, this private chain control management platform includes node control platform and node management.
Preferably, the required file blocks are obtained from the block chain of the interplanetary file system according to the user request, the file blocks required by the file are preferentially obtained from nodes near the user, the public key submitted by the user is utilized to decrypt the data, and the file is merged at the client to obtain the complete file.
Preferably, there is provided a distributed storage apparatus that performs the storage method according to claim 1, including:
the terminal service layer is used for generating an uploading request by an uploader, a storage request by a storer and a downloading request by a downloader, and segmenting a file to be stored into a plurality of file segments according to the uploading request; the block chain network layer is used for storing the uploading request, the downloading request and the storage information of the file, and is used for performing authority verification on a downloader who initiates the downloading request; and the distributed storage network is used for storing the file blocks, performing authority verification on a storage person when the file is uploaded, and performing authority verification on a download person when the file is downloaded.
The invention further discloses an electronic device, which is characterized by comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the distributed storage method described above via execution of the executable instructions.
The invention further discloses a computer-readable storage medium on which a computer program is stored, characterized in that the computer program realizes the above-mentioned distributed storage method when being executed by a processor.
Drawings
The invention will be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the embodiments. In the drawings, like reference numerals designate corresponding parts throughout the different views.
FIG. 1 is a flow chart of a distributed storage method of the present invention.
FIG. 2 is a block diagram of the distributed storage system of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Referring to fig. 1, a flowchart of a distributed storage method according to an embodiment of the present invention is shown.
Example one
A distributed storage method is characterized in that a stored file is represented by a hash value and distributed by a BitTorrent protocol, and specifically comprises the following steps:
s1, constructing a block chain network based on an interplanetary file system (IPFS) protocol;
s2, constructing a private chain different from the block chain network;
s3, an uploader in the distributed storage network generates an uploading request and records the uploading request in a contract of the private chain;
s4, dividing the file to be stored into a plurality of files and uploading the files to the block chain network based on a specific protocol;
s5, a storage request is generated by a storage person in the distributed storage network, and the storage request is recorded in the contract;
s6, synchronously storing an uploading request and a storage request of a file by a DHT (distributed hash table) on each node of the block chain network;
and S7, storing the file fragments through a BitTorrent protocol.
Preferably, the private chain comprises: the core block chain, private block chain equipment and central server, private block chain equipment is including the node server, be provided with hardware in the node server and prevent hot wall, private block chain equipment passes through central server and core block chain connection, private block chain equipment has been set up again based on core block chain, a plurality of block chain nodes of this private block chain equipment configuration, this private chain is built and can establish a plurality of private chain nodes based on block chain platform, set up central server, establish private chain control management platform through central server, this private chain control management platform includes node control platform and node management.
Preferably, the required file blocks are obtained from the block chain of the interplanetary file system according to the user request, the file blocks required by the file are preferentially obtained from nodes near the user, the public key submitted by the user is utilized to decrypt the data, and the file is merged at the client to obtain the complete file.
In particular, a hash value is a unique and extremely compact representation of a piece of data as a numerical value. If a piece of plaintext is hashed and even if only one letter of the piece is altered, the subsequent hash will produce a different value. It is computationally virtually impossible to find two different inputs for which the hash is the same value. Message Authentication Code (MAC) hash functions are typically used with digital signatures to sign data, while Message Detection Code (MDC) hash functions are used for data integrity.
And storing the hash value data of each user in a user chain of the user, constructing the hash value data on the user chain into a default library tree at intervals of a first preset time, calculating a root hash of the default library tree, and storing the root hash into an anchor chain block.
For steps S3-S6, the DHT (distributed hash table) on each node of the blockchain network stores data synchronously, the problem of hash authentication of distributed data in the blockchain data is solved by introducing a trusted tree, and balance is performed between data security and data redundancy amount by means of single signature multiple authentication, specifically, the step S3 generates a storage request, the step S4 segments the original data, the step S5 completes recording of the segmented data, and the step S6 implements uploading of the data. The partitioned sub-data are stored in a block form under the condition of intelligent contract management in a transaction information form, one block is divided into a block head and a block body, all transaction related information is included in the block body, the block head comprises a hash value of a previous block, a timestamp, a random number, a difficulty coefficient and a Merkle tree root hash value in the block, the timestamp is the block generation time, the random number is the solution of a mathematical problem, the difficulty coefficient can dynamically adjust the difficulty of the mathematical problem, and the Merkle tree is the only identification of transactions in the block body. The hash values of the blocks are formed by the constituent elements through a certain hash algorithm, the blocks correspond to the hash value of the previous block in the block through the parenthosh in the block, the blocks are connected end to form a chain structure, and the longest chain formed from the created block is called as a main chain. In the process of adding blocks into a block chain one by one, a plurality of new blocks are often added into a main chain simultaneously to generate branches due to network delay, and here, the difficulty coefficients need to be compared, and the branch chain with the largest accumulated workload in the whole chain is taken as the main chain to continue the block adding process. The time stamp in the block head ensures the orderliness of the block and has certain precaution effect on tampering or forging the block data. The difficulty coefficient is used for dynamically adjusting the mining difficulty degree and keeping the block-producing time to float up and down in the expected time, and meanwhile, the difficulty coefficient is a verification basis for workload certification.
The Merkle tree finally forms a Merkle tree root hash value by grouping and hashing the transactions, inserting the hash value into the Merkle tree and carrying out recursive hashing, wherein the hash value of all nodes in a path leading to the node is changed when the data of any node in the Merkle tree is changed, so that the Merkle tree root hash value is packaged into a block chain head, and the correctness of all transactions in the block can be verified only by verifying the root hash value.
In the above embodiments, the transaction tree and receipt tree are Merkle trees, and the Merkle Patricia tree is designed for account state storage to store account state trees.
When a transaction or an account is executed, the state tree needs to be queried or modified, operations of increasing, deleting and modifying are frequently carried out, the state tree needs to have the capabilities of fast querying and fast calculating the root hash, and the Merkle tree and the dictionary tree are combined, so that the lookup efficiency is improved, and the root hash value can be fast calculated. With such a data structure, the RootHash can be generated quickly after an insert or modify operation without the need to regenerate a tree. While the depth of the tree is limited, even considering that an attacker may deliberately make some transactions, it does not make the tree infinitely extensive, otherwise the attacker may perform a Denial-of-Service Attack (DoS) by manipulating the depth of the tree, so that the update becomes extremely slow. And the root hash value of the tree is only related to the data of the leaf node and is not related to the updating sequence of the data. Updating in another order does not change the hash value of the root.
In the first embodiment, the data and the requester are associated by means of signature hash authentication in the above manner, and a balance is found between node data redundancy and authentication time.
Example two
A distributed storage method is characterized in that a stored file is represented by a hash value and distributed by a BitTorrent protocol, and specifically comprises the following steps:
s1, constructing a block chain network based on an interplanetary file system (IPFS) protocol;
s2, constructing a private chain different from the block chain network;
s3, an uploader in the distributed storage network generates an uploading request and records the uploading request in a contract of the private chain;
s4, dividing the file to be stored into a plurality of files and uploading the files to the block chain network based on a specific protocol;
s5, a storage request is generated by a storage person in the distributed storage network, and the storage request is recorded in the contract;
s6, synchronously storing an uploading request and a storage request of a file by a DHT (distributed hash table) on each node of the block chain network;
and S7, storing the file fragments through a BitTorrent protocol.
Preferably, the upload request includes a split number of files, a backup number of files, a file description, a public key of the uploader, an account name of the uploader, an uploader ID, and a file ID.
Preferably, the specific protocol in step S4 is WebRTC and/or uTP.
Preferably, the private chain comprises: the core block chain, private block chain equipment and central server, private block chain equipment is including the node server, be provided with hardware in the node server and prevent hot wall, private block chain equipment passes through central server and core block chain connection, private block chain equipment has been set up again based on core block chain, a plurality of block chain nodes of this private block chain equipment configuration, this private chain is built and can establish a plurality of private chain nodes based on block chain platform, set up central server, establish private chain control management platform through central server, this private chain control management platform includes node control platform and node management.
Preferably, the required file blocks are obtained from the block chain of the interplanetary file system according to the user request, the file blocks required by the file are preferentially obtained from nodes near the user, the public key submitted by the user is utilized to decrypt the data, and the file is merged at the client to obtain the complete file.
The method specifically adopts an IPFS distribution mechanism, namely the IPFS changes the distribution mechanism of the WEB content from the original, so that decentralization is completed. Unlike the existing WEB network, the network is based on HTTP protocol, which is a centralized network (based on TCP protocol). Regardless of how the content distribution is distributed, no matter how many servers are distributed around the world. The centralized nature still exists.
The IPFS protocol is computed by a hash function and the same file is addressed uniquely, i.e. the uniqueness of the addresses will point to the same resource. Unlike the URL, the address calculated by the function is generated by being secured by an encryption algorithm.
Each web page opened by the browser is a collection of files, such as sound graphics and like web page element files. These files are uniquely placed on the IPFS, they can be opened by address, and on this basis, for example, users near the user visit the same website, users can get directly from nearby users, and physical distance is closer to open faster, as with all blockchain technologies, the IPFS is based on P2P and forms a peer-to-peer transport network. One can connect more and larger networks together. Such networks do not have a central node and resources stored in the IPFS may be scattered over non-centralized storage areas.
At S1, an IPFS network of peer-to-peer protocol is constructed, a distributed hash table is maintained in the distributed network, i.e. the hash table is stored in the network, when a new file is added to the IPFS distributed network, the hash value of the file is used as an index of the file, and is the same as that of other block chain networks, and when the content of the stored file is changed, the hash value is changed due to the change of the data of the file. When the data are uploaded to the IPFS network nodes, the node data are backed up with at least 2 other nodes, so that the data are prevented from being lost when a node server goes wrong, the stored redundancy is guaranteed not to be too high, the data are transmitted and backed up between the nodes, and the network transmission pressure of a client is reduced.
In step S2, a regional private blockchain is constructed by configuring a plurality of nodes in the regional network, where the plurality of nodes are a plurality of servers controllable by the constructor, and the constructor has the highest authority of the plurality of nodes, and high-authority management monitoring on the IPFS network is realized by recording information such as list information, logs, and exception reports of data transmission in the controllable blockchain.
EXAMPLE III
In order to facilitate better understanding of the present application, referring to a block diagram of a distributed storage system shown in fig. 2, based on the block diagram of the system, the embodiment provides a distributed storage method, where a file to be stored is represented by a hash value and distributed by a BitTorrent protocol, and the method specifically includes:
s1, constructing a block chain network based on an interplanetary file system (IPFS) protocol;
s2, constructing a private chain different from the block chain network;
s3, an uploader in the distributed storage network generates an uploading request and records the uploading request in a contract of the private chain;
s4, dividing the file to be stored into a plurality of files and uploading the files to the block chain network based on a specific protocol;
s5, a storage request is generated by a storage person in the distributed storage network, and the storage request is recorded in the contract;
s6, synchronously storing an uploading request and a storage request of a file by a DHT (distributed hash table) on each node of the block chain network;
and S7, storing the file fragments through a BitTorrent protocol.
Preferably, the upload request includes a split number of files, a backup number of files, a file description, a public key of the uploader, an account name of the uploader, an uploader ID, and a file ID.
Preferably, the specific protocol in step S4 is WebRTC and/or uTP.
Preferably, the private chain comprises: the core block chain, private block chain equipment and central server, private block chain equipment is including the node server, be provided with hardware in the node server and prevent hot wall, private block chain equipment passes through central server and core block chain connection, private block chain equipment has been set up again based on core block chain, a plurality of block chain nodes of this private block chain equipment configuration, this private chain is built and can establish a plurality of private chain nodes based on block chain platform, set up central server, establish private chain control management platform through central server, this private chain control management platform includes node control platform and node management.
Preferably, the required file blocks are obtained from the block chain of the interplanetary file system according to the user request, the file blocks required by the file are preferentially obtained from nodes near the user, the public key submitted by the user is utilized to decrypt the data, and the file is merged at the client to obtain the complete file.
Preferably, the device of the distributed storage method of the application block chain comprises a terminal service layer, a storage layer and a download layer, wherein the terminal service layer is used for an uploader to generate an upload request, a depositor to generate a storage request and a downloader to generate a download request, and segmenting a file to be stored into a plurality of file segments according to the upload request; the block chain network layer is used for storing the uploading request, the downloading request and the storage information of the file, and is used for performing authority verification on a downloader who initiates the downloading request; and the distributed storage network is used for storing the file blocks, performing authority verification on a storage person when the file is uploaded, and performing authority verification on a download person when the file is downloaded.
Preferably, in the step of the data storage process of the block chain storage capacity, the user node first calculates the number of copies to be stored in each block, each new block of the block chain is calculated on the basis of the previous block, and the security of different blocks is different, so that, in the embodiment, the security of each block is calculated according to three blocks, a data copy allocation policy of the block chain is provided, the storage process is optimized, and the data copy allocation policy is obtained by first obtaining the minimum number of copies to be stored in each block according to the total number of system nodes, then determining the number of the same copies to be stored in a certain number of continuous blocks according to the system security requirement, and finally calculating the number of the copies to be stored in each block according to the difficulty setting of the block chain.
In the third embodiment, for step S1, the building a blockchain network based on an interplanetary file system (IPFS) protocol; the interplanetary file system IPFS is a distributed file system based on the P2P network, and is designed to be expected to connect all computing devices using the same file system. IPFS is somewhat similar to the internet in a sense, but it should be more viewed as a gittorent-like BitTorrent cluster with version management functionality. IPFS provides a high throughput content addressing based block storage model. This forms a generalized Merkle directed acyclic graph MerkleDAG. The Merkel directed acyclic graph is a data structure on which a file system with a version control function, a block chain and even a Web site which can never be stopped can be established. The IPFS technology combines the technologies of a distributed hash table, block exchange with an incentive mechanism, a self-authentication namespace and the like, and the characteristics that no single point of failure exists and mutual trust is not needed between nodes can be achieved.
In an IPFS network users are content-based addressed. When a file is added to an IPFS node, it gets a new name. This name is actually a cryptographic hash that is computed from the contents of the file. It is guaranteed by encryption that the hash always represents only the content of the file. When a file is fetched to the IPFS distributed network through a hash value, a node having data can be quickly found by using a DHT data structure, so that the data is obtained, and whether the data is correct or not is verified by using the hash.
A normal block and a summary block are arranged in the IPFS network. There is a certain span between summary blocks, i.e. one summary block appears every n normal blocks. And the regular summary blocks are arranged to take the task of regularly cleaning the block information. The summary block has a function of sorting currently available transaction outputs besides a function of verifying transactions according to a normal flow, and the nodes write the sorted transaction outputs into files and store the files in the IPFS network. And after successful storage, writing the hash value for acquiring the file into the block header of the summary block. The workflow of the data writing and broadcasting summary block is consistent with that of the normal block. In this way, file correctness is verified, and the error block is discarded after an error is found.
As for the construction of a private chain distinct from the blockchain network; the method is mainly used for storing reliability evaluation of each node responsible for storage block segmentation and simultaneously recording user information, transaction important information is stored in a private chain building mode, optionally, encryption is carried out through a one-way function, a user can record file use extraction and other operations through configuring an intelligent contract in the private chain, all operation behaviors can be recorded due to the fact that the operation has the characteristic of being not capable of being tampered, a manager can conveniently determine the access condition of the file through the identity of the recorded identity, and optionally, the operation is combined with big data analysis to determine the browsing preference and the like of an accessor.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Although the invention has been described above with reference to various embodiments, it should be understood that many changes and modifications may be made without departing from the scope of the invention. It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this invention. The above examples are to be construed as merely illustrative and not limitative of the remainder of the disclosure. After reading the description of the invention, the skilled person can make various changes or modifications to the invention, and these equivalent changes and modifications also fall into the scope of the invention defined by the claims.
Claims (8)
1. A distributed storage method is characterized in that a stored file is represented by a hash value and distributed by a BitTorrent protocol, and specifically comprises the following steps:
s1, constructing a block chain network based on an interplanetary file system (IPFS) protocol;
s2, constructing a private chain different from the block chain network;
s3, an uploader in the distributed storage network generates an uploading request and records the uploading request in a contract of the private chain;
s4, dividing the file to be stored into a plurality of files, and uploading the block chain network based on a specific protocol;
s5, a storage request is generated by a storage person in the distributed storage network, and the storage request is recorded in the contract;
s6, synchronously storing an uploading request and a storage request of a file by a DHT (distributed hash table) on each node of the block chain network;
and S7, storing the file fragments through a BitTorrent protocol.
2. The distributed storage method according to claim 1, wherein the upload request includes a split number of files, a backup number of files, a file description, a public key of an uploader, an account name of the uploader, an uploader ID, and a file ID.
3. The distributed storage method according to claim 2, wherein the specific protocol in step S4 is WebRTC and/or uTP.
4. A distributed storage method according to claim 3, wherein said private chain comprises: core block chain, private block chain equipment and central server, private block chain equipment is including the node server, be provided with hardware in the node server and prevent hot wall, private block chain equipment passes through central server and core block chain connection, private block chain equipment has been set up based on core block chain, a plurality of block chain nodes of equipment configuration based on this private block chain, this private chain is built and can establish a plurality of private chain nodes based on block chain platform, set up central server, establish private chain control management platform through central server, this private chain control management platform includes node monitoring platform and node management.
5. The distributed storage method according to claim 4, wherein the required file blocks are obtained from the blockchain of the interplanetary file system according to a user request, the file blocks required by the file are preferentially obtained from nodes near the user, data decryption is performed by using a public key submitted by the user, and the files are merged at the client to obtain a complete file.
6. A distributed storage apparatus, characterized in that a distributed storage apparatus that executes the storage method according to claim 1, comprises:
the terminal service layer is used for generating an uploading request by an uploader, a storage request by a storer and a downloading request by a downloader, and segmenting a file to be stored into a plurality of file segments according to the uploading request; the block chain network layer is used for storing the uploading request, the downloading request and the storage information of the file, and is used for performing authority verification on a downloader who initiates the downloading request; and the distributed storage network is used for storing the file blocks, performing authority verification on a storage person when the file is uploaded, and performing authority verification on a download person when the file is downloaded.
7. An electronic device, comprising:
a processor; and the number of the first and second groups,
a memory for storing executable instructions of the processor;
wherein the processor is configured to perform the distributed storage method of any of claims 1-5 via execution of the executable instructions.
8. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the distributed storage method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910954998.1A CN110647503A (en) | 2019-10-09 | 2019-10-09 | Distributed storage method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910954998.1A CN110647503A (en) | 2019-10-09 | 2019-10-09 | Distributed storage method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110647503A true CN110647503A (en) | 2020-01-03 |
Family
ID=68993700
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910954998.1A Pending CN110647503A (en) | 2019-10-09 | 2019-10-09 | Distributed storage method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110647503A (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274273A (en) * | 2020-01-22 | 2020-06-12 | 北京瑞卓喜投科技发展有限公司 | Contract intelligent method and system and storage medium |
| CN111309523A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Data reading and writing method, data remote copying method and device and distributed storage system |
| CN111414426A (en) * | 2020-03-26 | 2020-07-14 | 北京云图科瑞科技有限公司 | Data processing method and system based on block chain |
| CN111526050A (en) * | 2020-04-17 | 2020-08-11 | 昆明大棒客科技有限公司 | Network deployment system, method, device and storage medium for block chaining-out block nodes |
| CN111586109A (en) * | 2020-04-17 | 2020-08-25 | 上海坤仪金科信息技术有限公司 | Block chain cloud storage user data deduplication solution |
| CN111767250A (en) * | 2020-06-10 | 2020-10-13 | 钛星投资(深圳)有限公司 | Decentralized storage method, downloading method and storage system |
| CN111949630A (en) * | 2020-08-10 | 2020-11-17 | 大连理工大学 | Distributed block chain storage method based on file review mechanism |
| CN112257048A (en) * | 2020-12-21 | 2021-01-22 | 南京韦科韬信息技术有限公司 | Information security protection method and device |
| CN112272155A (en) * | 2020-09-15 | 2021-01-26 | 山东鲁能软件技术有限公司 | E-mail receiving and sending method based on block chain |
| CN112272156A (en) * | 2020-09-15 | 2021-01-26 | 山东鲁能软件技术有限公司 | E-mail receiving and sending system based on block chain |
| CN112311764A (en) * | 2020-09-28 | 2021-02-02 | 苏州浪潮智能科技有限公司 | Server data exchange network security system |
| CN112667746A (en) * | 2020-12-30 | 2021-04-16 | 浙江甲骨文超级码科技股份有限公司 | Block chain based data storage method and equipment, electronic device and storage equipment |
| CN112733194A (en) * | 2021-01-26 | 2021-04-30 | 长威信息科技发展股份有限公司 | Trusted circulation method and system for block chain data |
| CN112835855A (en) * | 2021-02-22 | 2021-05-25 | 北京众享比特科技有限公司 | File storage method and system based on block chain |
| CN112866090A (en) * | 2021-01-20 | 2021-05-28 | 临沂呆马区块链网络科技有限公司 | Instant communication system and method fusing block chain and point-to-point communication |
| CN113094334A (en) * | 2021-03-22 | 2021-07-09 | 四块科技(天津)有限公司 | Digital service method, device, equipment and storage medium based on distributed storage |
| CN113312663A (en) * | 2021-05-31 | 2021-08-27 | 尧领有限公司 | Distributed data storage method and system, and computer readable storage medium |
| CN113987063A (en) * | 2021-09-23 | 2022-01-28 | 北京连山科技股份有限公司 | Data particle distribution system based on block chain |
| CN114760316A (en) * | 2020-12-28 | 2022-07-15 | 富泰华工业（深圳）有限公司 | Block chain construction method and related equipment |
| CN115529319A (en) * | 2022-08-04 | 2022-12-27 | 熵链科技(厦门)有限公司 | Storage service regulation and control method, system, equipment and storage medium based on IPFS network |
| CN115964754A (en) * | 2023-03-16 | 2023-04-14 | 长城数字能源(西安)科技有限公司 | Big data secure storage method and device |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206237441U (en) * | 2016-12-19 | 2017-06-09 | 杜伯仁 | Resource-sharing block chain network |
| CN107249046A (en) * | 2017-08-15 | 2017-10-13 | 李俊庄 | A kind of distributed cloud storage system construction method based on block chain |
| CN108076148A (en) * | 2017-12-15 | 2018-05-25 | 成都链网络科技有限公司 | Storage system based on block chain |
| CN108111585A (en) * | 2017-12-15 | 2018-06-01 | 成都链网络科技有限公司 | Distributed storage method based on block chain |
| CN109587253A (en) * | 2018-12-11 | 2019-04-05 | 浙江大学 | A method of distributed storage is realized to file based on block chain |
| CN110032545A (en) * | 2019-03-27 | 2019-07-19 | 远光软件股份有限公司 | File memory method, system and electronic equipment based on block chain |
| CN110109874A (en) * | 2019-05-13 | 2019-08-09 | 北京共识数信科技有限公司 | A kind of non-stop layer distributed document retrieval method based on block chain |
| CN110289060A (en) * | 2019-06-20 | 2019-09-27 | 福州数据技术研究院有限公司 | A kind of personal medical data storage method under the chain based on random Merkle tree |
-
2019
- 2019-10-09 CN CN201910954998.1A patent/CN110647503A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN206237441U (en) * | 2016-12-19 | 2017-06-09 | 杜伯仁 | Resource-sharing block chain network |
| CN107249046A (en) * | 2017-08-15 | 2017-10-13 | 李俊庄 | A kind of distributed cloud storage system construction method based on block chain |
| CN108076148A (en) * | 2017-12-15 | 2018-05-25 | 成都链网络科技有限公司 | Storage system based on block chain |
| CN108111585A (en) * | 2017-12-15 | 2018-06-01 | 成都链网络科技有限公司 | Distributed storage method based on block chain |
| CN109587253A (en) * | 2018-12-11 | 2019-04-05 | 浙江大学 | A method of distributed storage is realized to file based on block chain |
| CN110032545A (en) * | 2019-03-27 | 2019-07-19 | 远光软件股份有限公司 | File memory method, system and electronic equipment based on block chain |
| CN110109874A (en) * | 2019-05-13 | 2019-08-09 | 北京共识数信科技有限公司 | A kind of non-stop layer distributed document retrieval method based on block chain |
| CN110289060A (en) * | 2019-06-20 | 2019-09-27 | 福州数据技术研究院有限公司 | A kind of personal medical data storage method under the chain based on random Merkle tree |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111274273A (en) * | 2020-01-22 | 2020-06-12 | 北京瑞卓喜投科技发展有限公司 | Contract intelligent method and system and storage medium |
| CN111309523A (en) * | 2020-02-16 | 2020-06-19 | 西安奥卡云数据科技有限公司 | Data reading and writing method, data remote copying method and device and distributed storage system |
| CN111414426A (en) * | 2020-03-26 | 2020-07-14 | 北京云图科瑞科技有限公司 | Data processing method and system based on block chain |
| CN111526050A (en) * | 2020-04-17 | 2020-08-11 | 昆明大棒客科技有限公司 | Network deployment system, method, device and storage medium for block chaining-out block nodes |
| CN111586109A (en) * | 2020-04-17 | 2020-08-25 | 上海坤仪金科信息技术有限公司 | Block chain cloud storage user data deduplication solution |
| CN111526050B (en) * | 2020-04-17 | 2023-09-12 | 昆明大棒客科技有限公司 | Network deployment system, method, equipment and storage medium for block chain out-of-block node |
| CN111586109B (en) * | 2020-04-17 | 2023-04-28 | 上海坤仪金科信息技术有限公司 | Block chain cloud storage user data deduplication solution method |
| CN111767250A (en) * | 2020-06-10 | 2020-10-13 | 钛星投资(深圳)有限公司 | Decentralized storage method, downloading method and storage system |
| CN111949630A (en) * | 2020-08-10 | 2020-11-17 | 大连理工大学 | Distributed block chain storage method based on file review mechanism |
| CN112272155A (en) * | 2020-09-15 | 2021-01-26 | 山东鲁能软件技术有限公司 | E-mail receiving and sending method based on block chain |
| CN112272156A (en) * | 2020-09-15 | 2021-01-26 | 山东鲁能软件技术有限公司 | E-mail receiving and sending system based on block chain |
| CN112311764B (en) * | 2020-09-28 | 2022-05-20 | 苏州浪潮智能科技有限公司 | Server data exchange network security system |
| CN112311764A (en) * | 2020-09-28 | 2021-02-02 | 苏州浪潮智能科技有限公司 | Server data exchange network security system |
| CN112257048A (en) * | 2020-12-21 | 2021-01-22 | 南京韦科韬信息技术有限公司 | Information security protection method and device |
| CN114760316A (en) * | 2020-12-28 | 2022-07-15 | 富泰华工业（深圳）有限公司 | Block chain construction method and related equipment |
| CN114760316B (en) * | 2020-12-28 | 2023-09-26 | 富泰华工业（深圳）有限公司 | Block chain construction method and related equipment |
| CN112667746A (en) * | 2020-12-30 | 2021-04-16 | 浙江甲骨文超级码科技股份有限公司 | Block chain based data storage method and equipment, electronic device and storage equipment |
| CN112866090A (en) * | 2021-01-20 | 2021-05-28 | 临沂呆马区块链网络科技有限公司 | Instant communication system and method fusing block chain and point-to-point communication |
| CN112733194A (en) * | 2021-01-26 | 2021-04-30 | 长威信息科技发展股份有限公司 | Trusted circulation method and system for block chain data |
| CN112835855A (en) * | 2021-02-22 | 2021-05-25 | 北京众享比特科技有限公司 | File storage method and system based on block chain |
| CN113094334A (en) * | 2021-03-22 | 2021-07-09 | 四块科技(天津)有限公司 | Digital service method, device, equipment and storage medium based on distributed storage |
| CN113312663A (en) * | 2021-05-31 | 2021-08-27 | 尧领有限公司 | Distributed data storage method and system, and computer readable storage medium |
| CN113987063B (en) * | 2021-09-23 | 2022-06-24 | 北京连山科技股份有限公司 | Data particle distribution system based on block chain |
| CN113987063A (en) * | 2021-09-23 | 2022-01-28 | 北京连山科技股份有限公司 | Data particle distribution system based on block chain |
| CN115529319A (en) * | 2022-08-04 | 2022-12-27 | 熵链科技(厦门)有限公司 | Storage service regulation and control method, system, equipment and storage medium based on IPFS network |
| CN115964754A (en) * | 2023-03-16 | 2023-04-14 | 长城数字能源(西安)科技有限公司 | Big data secure storage method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110647503A (en) | Distributed storage method and device | |
| US11283617B2 (en) | Systems and methods for state of data management | |
| JP6865851B2 (en) | Blockchain World State Markle Patricia Tri Tree Subtree Construction | |
| Ali et al. | Blockstack: A global naming and storage system secured by blockchains | |
| CN107220559B (en) | Encryption storage method for non-tamperable file | |
| US11483622B2 (en) | Hybrid blockchains and streamchains using non-crypto hashes for securing audio-, video-, image-, and speech-based transactions and contracts | |
| TW202029694A (en) | Systems and methods for efficient and secure processing, accessing and transmission of data via a blockchain network | |
| JP2021527274A (en) | Distributed access, storage, and transfer of blockchain | |
| CN110011981B (en) | Trusted cloud storage method and system based on block chain | |
| Spasovski et al. | Proof of stake blockchain: performance and scalability for groupware communications | |
| Ali et al. | Blockstack: A new decentralized internet | |
| WO2019233614A1 (en) | A method for registration of data in a blockchain database and a method for verifying data | |
| JP2006518508A (en) | Additional hash functions in content-based addressing | |
| EP3709568A1 (en) | Deleting user data from a blockchain | |
| Hassanzadeh-Nazarabadi et al. | LightChain: Scalable DHT-based blockchain | |
| US20240080181A1 (en) | Blockchain Data Structures and Systems and Methods Therefor for Multipath Transaction Management | |
| CN111797347A (en) | Content distributed over secure channels | |
| Tang et al. | Lightweight authentication of freshness in outsourced key-value stores | |
| Vaidya et al. | Commit signatures for centralized version control systems | |
| Ali et al. | Blockstack technical whitepaper | |
| Khacef et al. | Trade-off between security and scalability in blockchain design: A dynamic sharding approach | |
| Joe et al. | Mixed mode analytics architecture for data deduplication in wireless personal cloud computing | |
| Gangadevi et al. | A survey on data integrity verification schemes using blockchain technology in Cloud Computing Environment | |
| Thakur et al. | Data integrity techniques in cloud computing: an analysis | |
| Trueman et al. | Ensuring privacy and data freshness for public auditing of shared data in cloud |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200103 |