CN112131229A - Block chain-based distributed data access method and device and storage node - Google Patents

Block chain-based distributed data access method and device and storage node Download PDF

Info

Publication number
CN112131229A
CN112131229A CN202011051905.3A CN202011051905A CN112131229A CN 112131229 A CN112131229 A CN 112131229A CN 202011051905 A CN202011051905 A CN 202011051905A CN 112131229 A CN112131229 A CN 112131229A
Authority
CN
China
Prior art keywords
data
storage node
sub
block chain
file
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011051905.3A
Other languages
Chinese (zh)
Inventor
李增兴
曾东林
李昀
邓卫兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bank of China Ltd
Original Assignee
Bank of China Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bank of China Ltd filed Critical Bank of China Ltd
Priority to CN202011051905.3A priority Critical patent/CN112131229A/en
Publication of CN112131229A publication Critical patent/CN112131229A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2282—Tablespace storage structures; Management thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
    • G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database

Abstract

The invention discloses a distributed data access method, a distributed data access device and storage nodes based on a block chain, wherein N storage nodes form a block chain network, and each storage node is distributed in different machine rooms, and the writing method comprises the following steps: the client receives the write-in request and sends the write-in request to the block chain network; the request comprises a data file and a corresponding main file identifier; when the main storage node receives the request, the data file is divided into N-1 parts of sub-data files, and sub-file identifications are distributed; determining the address of each sub data file and the corresponding check data to be stored in the storage node according to a preset parity check strategy, and sending the sub data files, the check data and the corresponding data directories thereof to a block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and storing a node address; and each storage node receives and stores the subdata file, the corresponding check data and the corresponding data directory. The invention can improve the safety of data storage and save the data storage space.

Description

Block chain-based distributed data access method and device and storage node
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a block chain-based distributed data access method, apparatus, and storage node.
Background
This section is intended to provide a background or context to the embodiments of the invention that are recited in the claims. The description herein is not admitted to be prior art by inclusion in this section.
Currently, data files are generally stored centrally in each server or repeatedly at each node of each blockchain. The centralized storage has security risk, and each node of the block chain is repeatedly stored and wastes space.
Disclosure of Invention
The embodiment of the invention provides a distributed data writing method based on a block chain, which is used for improving the safety of data storage and saving the data storage space, wherein N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different machine rooms, and the method comprises the following steps:
when receiving a data write-in request of a user, a client sends the write-in request to a block chain network; the writing request comprises a data file to be written and a corresponding main file identifier;
receiving a write request sent over a blockchain network by a selected master storage node; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
each subdata storage node receives the subdata files and the corresponding data directories which are sent through the block chain network and stores the subdata files and the corresponding data directories; and the check data storage node receives the check data and the corresponding data directory sent by the block chain network and stores the check data and the corresponding data directory.
The embodiment of the invention also provides a distributed data writing method based on the block chain, which is used for improving the safety of data storage and saving the data storage space, wherein N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different machine rooms, and the method comprises the following steps:
receiving a write request sent over a blockchain network;
splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file;
determining node address information of each sub-data file and corresponding check data to be stored in the storage node according to a preset parity check strategy;
sending each sub-data file and the corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
The embodiment of the invention also provides a distributed data reading method based on the block chain, which is used for improving the safety of data storage and saving the data storage space, wherein N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different machine rooms, and the method comprises the following steps:
when receiving a data reading request of a user, a client sends the reading request to a block chain network; the reading request comprises a main file identifier corresponding to a data file to be read;
the selected main storage node receives a reading request sent by a block chain network, searches a corresponding data directory according to a main file identifier, generates a sub-data file acquisition request according to the data directory, and sends the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
each subdata storage node receives a subdata file acquisition request sent through a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
the main storage node receives the subdata files sent by all the subdata storage nodes through the block chain network, and all the subdata files are combined to obtain a data file to be read; and feeding back the data file to be read to the client.
The embodiment of the invention also provides a distributed data reading method based on the block chain, which is used for improving the safety of data storage and saving the data storage space, wherein N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different machine rooms, and the reading method comprises the following steps:
receiving a read request sent over a blockchain network;
searching a corresponding data directory according to the main file identifier, and generating a sub-data file acquisition request according to the data directory;
sending a sub data file acquisition request to a block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
receiving subdata files sent by all storage nodes through a block chain network;
merging all the sub-data files to obtain a data file to be read;
and feeding back the data file to be read to the client.
An embodiment of the present invention further provides a block chain-based distributed data writing device, configured to improve security of data storage and save data storage space, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different machine room, where the writing device includes:
the client is used for sending a write-in request to the block chain network when receiving a data write-in request of a user; the writing request comprises a data file to be written and a corresponding main file identifier;
a selected primary storage node for receiving a write request sent over a blockchain network; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
each subdata storage node is used for receiving the subdata files and the corresponding data directories sent by the block chain network and storing the subdata files and the corresponding data directories; and the check data storage node is used for receiving the check data and the corresponding data directory sent by the block chain network and storing the check data and the corresponding data directory.
The embodiment of the present invention further provides a block chain-based distributed data write storage node, configured to improve data storage security and save data storage space, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different computer rooms, where the write storage node includes:
a first receiving unit configured to receive a write request transmitted through a blockchain network;
the data splitting unit is used for splitting the data file to be written into N-1 parts of sub-data files and distributing a sub-file identifier for each part of sub-data file;
the first determining unit is used for determining node address information of each sub-data file and the corresponding check data which should be stored in the storage node according to a preset parity check strategy;
the first sending unit is used for sending each part of the sub-data files, the corresponding data directories, the check data and the corresponding data directories to the block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
The embodiment of the present invention further provides a block chain-based distributed data reading apparatus, configured to improve security of data storage and save data storage space, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different machine room, where the reading apparatus includes:
the client is used for sending a reading request to the block chain network when receiving the data reading request of a user; the reading request comprises a main file identifier corresponding to a data file to be read;
the selected main storage node is used for receiving a reading request sent by the block chain network, searching a corresponding data directory according to the main file identifier, generating a sub-data file acquisition request according to the data directory, and sending the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
each subdata storage node is used for receiving a subdata file acquisition request sent by a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
the main storage node is used for receiving the subdata files sent by all the subdata storage nodes through the block chain network, and merging all the subdata files to obtain a data file to be read; and feeding back the data file to be read to the client.
The embodiment of the present invention further provides a block chain-based distributed data reading storage node, configured to improve data storage security and save data storage space, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in different machine rooms, where the reading storage node includes:
a second receiving unit, configured to receive a read request sent through a blockchain network; receiving subdata files sent by all storage nodes through a block chain network;
the generating unit is used for searching a corresponding data directory according to the main file identifier and generating a sub-data file acquisition request according to the data directory;
a second sending unit, configured to send the sub-data file acquisition request to the block link network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information; feeding back a data file to be read to the client;
and the merging processing unit is used for merging all the sub-data files to obtain the data file to be read.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the distributed data writing and reading method based on the block chain when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the above method for writing and reading distributed data based on a block chain is stored in the computer-readable storage medium.
In the embodiment of the invention, N storage nodes in a distributed data access scheme based on a block chain form a block chain network, N is a positive integer greater than 2, each storage node is distributed in different machine rooms, and compared with the technical scheme that in the prior art, the centralized storage has a security risk, and each node of the block chain is repeatedly stored and wastes space, the distributed data access scheme based on the block chain comprises the following steps: in the data write-in apparatus: when receiving a data write-in request of a user, a client sends the write-in request to a block chain network; the write request comprises a data file to be written and a corresponding main file identifier; receiving a write request sent over a blockchain network by a selected master storage node; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored; each subdata storage node receives the subdata files and the corresponding data directories which are sent through the block chain network and stores the subdata files and the corresponding data directories; and the check data storage node receives the check data and the corresponding data directory sent by the block chain network and stores the check data and the corresponding data directory. In the data reading time reading device: when receiving a data reading request of a user, a client sends the reading request to a block chain network; the reading request comprises a main file identifier corresponding to a data file to be read; the selected main storage node receives a reading request sent by a block chain network, searches a corresponding data directory according to a main file identifier, generates a sub-data file acquisition request according to the data directory, and sends the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information; each subdata storage node receives a subdata file acquisition request sent through a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network; the main storage node receives the subdata files sent by all the subdata storage nodes through the block chain network, and all the subdata files are combined to obtain a data file to be read; and the data file to be read is fed back to the client, so that the safety of data storage is improved and the data storage space is saved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:
FIG. 1 is a block chain based distributed data access system according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a block chain-based distributed data writing method applied to a writing apparatus according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a distributed data writing method based on a block chain applied to a write storage node according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a block chain-based distributed data reading method applied to a reading apparatus according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating a block chain-based distributed data reading method applied to read storage nodes according to an embodiment of the present invention;
FIG. 6A is a first diagram illustrating a blockchain network according to an embodiment of the present invention;
FIG. 6B is a diagram illustrating a blockchain network according to an embodiment of the present invention;
FIG. 6C is a schematic diagram illustrating a block chain based distributed data access according to an embodiment of the present invention;
FIG. 7 is a block chain-based distributed data writing apparatus or reading apparatus according to an embodiment of the present invention;
FIG. 8 is a block chain-based distributed data write storage node according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a distributed data reading storage node based on a block chain in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
The existing hard disk arrays are usually stored in the same cabinet and the same machine room, and if the cabinet fails (such as power failure), the whole hard disk fails, and the storage service cannot be provided. Therefore, there is a need for a space-saving and fault-tolerant approach.
In view of the problems in the prior art, the inventor proposes a distributed data access scheme based on a blockchain, which solves the above problems for a new blockchain distributed storage method.
The embodiment of the invention avoids repeated storage of files by establishing a block chain network and taking the hash of file data as a file name through the block chain, divides the files into a plurality of parts, disperses the files to a plurality of nodes for storage, and checks the files mutually, namely, divides the data into a plurality of nodes, distributes the nodes in different machine rooms, provides storage service across the machine rooms together, and when a certain node fails, the source data can be calculated through parity check, and the storage service is continuously provided, thus providing fault-tolerant distributed file service. Specifically, the scheme generally comprises the following steps:
1. a block chain network connecting a plurality of storage nodes;
2. data writing, data splitting and parity storage are dispersed to each storage node for storage;
3. reading data, reading data from a plurality of storage nodes, and combining bytes for returning;
4. failure of any storage node can be used to deduce the source data from the parity data of other storage nodes and provide normal data return.
Fig. 1 is a schematic structural diagram of a block chain-based distributed data access system according to an embodiment of the present invention, and as shown in fig. 1, the system includes: the writing device 10 and the reading device 20, in operation, in the data writing device: when receiving a data write-in request of a user, a client sends the write-in request to a block chain network; the write request comprises a data file to be written and a corresponding main file identifier; receiving a write request sent over a blockchain network by a selected master storage node; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored; each subdata storage node receives the subdata files and the corresponding data directories which are sent through the block chain network and stores the subdata files and the corresponding data directories; and the check data storage node receives the check data and the corresponding data directory sent by the block chain network and stores the check data and the corresponding data directory. In the data reading time reading device: when receiving a data reading request of a user, a client sends the reading request to a block chain network; the reading request comprises a main file identifier corresponding to a data file to be read; the selected main storage node receives a reading request sent by a block chain network, searches a corresponding data directory according to a main file identifier, generates a sub-data file acquisition request according to the data directory, and sends the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information; each subdata storage node receives a subdata file acquisition request sent through a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network; the main storage node receives the subdata files sent by all the subdata storage nodes through the block chain network, and all the subdata files are combined to obtain a data file to be read; and the data file to be read is fed back to the client, so that the safety of data storage is improved and the data storage space is saved.
The block chain based distributed data access system is described in detail below. First, a block chain-based distributed data writing method applied to a writing apparatus is described.
Fig. 2 is a schematic flow chart of a block chain-based distributed data writing method applied to a writing apparatus in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 2, the method includes the following steps:
step 101: when receiving a data write-in request of a user, a client sends the write-in request to a block chain network; the writing request comprises a data file to be written and a corresponding main file identifier;
step 102: receiving a write request sent over a blockchain network by a selected master storage node; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
step 103: each subdata storage node receives the subdata files and the corresponding data directories which are sent through the block chain network and stores the subdata files and the corresponding data directories; and the check data storage node receives the check data and the corresponding data directory sent by the block chain network and stores the check data and the corresponding data directory.
In a specific implementation, the main storage node may be a storage node voted according to the storage node, or may be preset.
In one embodiment, when receiving a data write request from a user, a client sends the write request to a blockchain network, which may include:
when a client receives a data writing request of a user, generating a hash value according to a data file to be written; and adding the hash value as a main file identifier into the write-in request, and sending the write-in request to the block chain network.
When the distributed data access method is specifically implemented, the hash value of the data file is used as the main file identifier, so that repeated storage is avoided, the storage space is saved, and the safety and efficiency of distributed data access are further improved.
In the step 102, each of the first N-1 storage nodes stores data to be stored in the node, and the nth storage node stores check data of other nodes (the first N-1 storage nodes). And the data directory corresponding to the verification data comprises all the subfile identifications.
The embodiment of the present invention further provides a block chain-based distributed data writing method applied to a write storage node, as described in the following embodiments. Because the principle of solving the problem of the block chain-based distributed data writing method applied to the write-in storage node is similar to that of the block chain-based distributed data writing method applied to the writing device, the implementation of the block chain-based distributed data writing method applied to the write-in storage node can refer to the implementation of the block chain-based distributed data writing method applied to the writing device, and repeated parts are not described again.
Fig. 3 is a schematic flow chart of a block chain-based distributed data writing method applied to write storage nodes in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 3, the method includes the following steps:
step 201: receiving a write request sent over a blockchain network;
step 202: splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file;
step 203: determining node address information of each sub-data file and corresponding check data to be stored in the storage node according to a preset parity check strategy;
step 204: sending each sub-data file and the corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
In one embodiment, the block chain-based distributed data writing method may further include:
receiving a subdata file and a corresponding data directory which are sent through a block chain network;
and storing the sub-data files and the corresponding data directories.
In one embodiment, the block chain-based distributed data writing method may further include:
receiving verification data and a corresponding data directory sent by a block chain network;
and storing the check data and the corresponding data directory.
In one embodiment, the master file identifier may be a hash value of the data file to be written.
The embodiment of the present invention further provides a block chain-based distributed data reading method applied to a reading apparatus, as described in the following embodiments. Because the principle of solving the problem of the block chain-based distributed data reading method applied to the reading device is similar to that of the block chain-based distributed data writing method applied to the writing device, the block chain-based distributed data reading method applied to the reading device can be implemented by referring to the implementation of the block chain-based distributed data writing method applied to the writing device, and repeated details are not repeated.
Fig. 4 is a schematic flow chart of a block chain-based distributed data reading method applied to a reading apparatus in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 4, the reading method includes the following steps:
step 301: when receiving a data reading request of a user, a client sends the reading request to a block chain network; the reading request comprises a main file identifier corresponding to a data file to be read;
step 302: the selected main storage node receives a reading request sent by a block chain network, searches a corresponding data directory according to a main file identifier, generates a sub-data file acquisition request according to the data directory, and sends the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
step 303: each subdata storage node receives a subdata file acquisition request sent through a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
step 304: the main storage node receives the subdata files sent by all the subdata storage nodes through the block chain network, and all the subdata files are combined to obtain a data file to be read; and feeding back the data file to be read to the client.
In an embodiment, the above block chain-based distributed data reading method may further include:
when each subdata storage node determines that the subdata storage node does not have a subdata file corresponding to a data file to be read according to a data directory stored by the storage node, the subdata storage node is determined to be a fault storage node, and fault storage node information is sent to a block link network;
when the main storage node receives the information of the fault storage node, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
In specific implementation, under the condition that data loss in any storage node is found, source data, namely the subdata files of the failed storage node, can be calculated through check data stored by other storage nodes, so that the following effects are achieved: under the condition of saving the data storage space, the safety of data storage is improved.
In one embodiment, the master file identifier may be a hash value of the data file to be read.
The embodiment of the present invention further provides a block chain-based distributed data reading method applied to reading a storage node, as described in the following embodiments. Because the principle of solving the problem of the block chain-based distributed data reading method applied to the read storage node is similar to that of the block chain-based distributed data writing method applied to the writing device, the implementation of the block chain-based distributed data reading method applied to the read storage node can refer to the implementation of the block chain-based distributed data writing method applied to the writing device, and repeated parts are not described again.
Fig. 5 is a schematic flow chart of a block chain-based distributed data reading method applied to reading storage nodes in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 5, the reading method includes the following steps:
step 401: receiving a read request sent over a blockchain network;
step 402: searching a corresponding data directory according to the main file identifier, and generating a sub-data file acquisition request according to the data directory;
step 403: sending a sub data file acquisition request to a block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
step 404: receiving subdata files sent by all storage nodes through a block chain network;
step 405: merging all the sub-data files to obtain a data file to be read;
step 406: and feeding back the data file to be read to the client.
In an embodiment, the above block chain-based distributed data reading method may further include:
receiving a sub data file acquisition request sent through a block chain network;
determining whether a subdata file corresponding to a data file to be read exists in the storage node according to the data directory stored in the storage node;
and when determining that the storage node has the sub-data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub-data file to the block chain network.
In an embodiment, the above block chain-based distributed data reading method may further include:
when determining that the sub data storage node does not have the sub data file corresponding to the data file to be read according to the data directory stored by the storage node, determining the sub data storage node as a fault storage node;
sending the fault storage node information to a block chain network;
when the information of the fault storage node is received, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
In order to facilitate understanding of how the present invention is implemented, an overall scheme of block chain based distributed data access in the embodiment of the present invention is described in detail below with reference to fig. 6A to 6C.
The distributed data access (writing and reading) method based on the block chain provided by the embodiment of the invention comprises the following processes:
1) a common block chain distributed storage network is built, and each storage node is connected with a block chain through a computer network, as shown in fig. 6A and 6B.
2) And when the data is written, uploading the data file and the digital signature to the block chain distributed storage network.
3) The distributed storage nodes of the block chain are used for splitting the data file, for example, each bit of each byte is stored in different nodes, parity check bits are calculated for every 4 bits and stored in other nodes, and each byte is stored in 5 nodes and issued to each block chain link point for storage. Byte splitting may also be used, checking policies such as:
and 5, node scheme: 4 bytes are respectively placed in 4 block chain nodes, and parity check is placed in the 5 th block chain node, so that 1 damaged node in the 5 block chain nodes can keep data from being lost.
Parity checking: each Bit in the 4 nodes corresponds to a group, 1 is counted, and several nodes exist, if 1 or 3 nodes exist, 1 is stored in the 5 th node, otherwise, 0 is stored.
D5 can be calculated using xor exclusive or operation (2 bits are 1 different and 0 the same):
d5 ═ D1 xor D2 xor D3 xor D4, as shown in table 1 below.
Node 1 Node 2 Node 3 Node 4 Node 5 parity bit
0 0 0 0 0
0 1 0 0 1
0 1 1 0 0
1 1 1 0 1
1 1 1 1 0
TABLE 1
4, node scheme: if 3 bytes are respectively placed in 3 block chain nodes and parity check is placed in the 4 th block chain node, any 1 damaged node in the 4 nodes can keep data from being lost.
Parity checking: the number of 1 in the 3 nodes corresponds to each Bit, and if 1 or 3, 1 is stored in the 4 th node, otherwise, 0 is stored, as shown in the following table 2.
Figure BDA0002709833440000121
Figure BDA0002709833440000131
TABLE 2
4) And when data is read, reading bit data from each node, combining the bit data into bytes and returning the bytes to the application.
5) Fault tolerance processing: if any node fails and cannot be read, the parity data of other 4 nodes can be used for calculating the source data and returning the source data to the application, so that the fault tolerance processing of the system is kept normal.
For example: assume node D3 fails to read:
the 5-node scheme can calculate each Bit of D3 with the corresponding data xor of the other 4 nodes:
D3＝D1 xor D2 xor D4 xor D5；
the 4-node scheme can calculate each Bit of D3 using the corresponding data xor of the other 3 nodes:
D3＝D1 xor D2 xor D4。
for another example, as shown in table 3 below, the data is split into 4 parts, D1, D2, D3 and D4, 1 part is placed in each node, then 4 parts are grouped into one group for each bit, the parity of each bit is calculated, D5 is generated, and D5 is stored in the 5 th node.
TABLE 3
Figure BDA0002709833440000132
Figure BDA0002709833440000141
As shown in fig. 6C, when the distributed data access system based on the block chain according to the embodiment of the present invention works:
1) the writing module (a module included by the client) receives a writing request of a user, sends the writing request to a selected or preset main storage node in a plurality of storage nodes of the block chain network through the block chain network, and the main storage node generates a hash value for file data to serve as a file name (a main file identifier), splits the file data to obtain a plurality of sub data files, generates check data and submits the check data to the block chain network.
2) The blockchain network automatically finds appropriate nodes (searching for nodes according to the determined address information to be stored in the nodes), distributively stores the split data, and registers a data directory (a file hash value, a data label and a node address).
3) And receiving the corresponding split data (subdata files) by the block chain node, and storing the split data.
4) The reading module (a module included in the client) receives a reading request of a user (the reading request includes a main file identifier corresponding to a data file to be read, namely a hash value), the user provides the file hash value, and the reading module sends the hash file reading request to the block chain.
5) The block chain network sends a sub-data file acquisition request according to a data directory (file hash value, data label number and node address), acquires the sub-data file from the storage node of each block chain, performs parity check and fault tolerance, and returns the sub-data file to the reading module.
6) And the block chain node (each storage node) reads the saved subdata files and returns the subdata files to the block chain network (main storage node), and the main storage node combines all the subdata files to obtain data to be read and provides the data to a reading module of the client.
Any block chain node (storage node) can undertake the functions of data splitting, merging, checking, forwarding and the like of the block chain network.
In summary, the block chain-based distributed data access method provided in the embodiment of the present invention realizes:
1) the Hash value of the file is used as the file name, and the same file has the same Hash value and does not need to be stored repeatedly.
2) Through the cross-node distributed storage, the machine room of a single block chain node is failed, and data can still be recovered.
Therefore, the block chain-based distributed data access scheme provided by the embodiment of the invention improves the security of data storage on the basis of saving the space of the data storage.
The embodiment of the present invention further provides a block chain-based distributed data writing apparatus, as described in the following embodiments. Because the principle of the device for solving the problem is similar to the block chain-based distributed data writing method applied to the writing storage node, the implementation of the device can refer to the implementation of the block chain-based distributed data writing method applied to the writing storage node, and repeated details are not repeated.
Fig. 7 is a schematic structural diagram of a block chain-based distributed data writing device in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 7, the writing device includes:
the client is used for sending a write-in request to the block chain network when receiving a data write-in request of a user; the writing request comprises a data file to be written and a corresponding main file identifier;
a selected primary storage node for receiving a write request sent over a blockchain network; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
each subdata storage node is used for receiving the subdata files and the corresponding data directories sent by the block chain network and storing the subdata files and the corresponding data directories; and the check data storage node is used for receiving the check data and the corresponding data directory sent by the block chain network and storing the check data and the corresponding data directory.
In one embodiment, the client may specifically be configured to: when a data writing request of a user is received, generating a hash value according to a data file to be written; and adding the hash value as a main file identifier into the write-in request, and sending the write-in request to the block chain network.
The embodiment of the present invention further provides a block chain-based distributed data writing apparatus, as described in the following embodiments. Because the principle of the device for solving the problem is similar to the block chain-based distributed data writing method applied to the writing storage node, the implementation of the device can refer to the implementation of the block chain-based distributed data writing method applied to the writing storage node, and repeated details are not repeated.
Fig. 8 is a schematic structural diagram of a block chain-based distributed data write storage node in an embodiment of the present invention, and as shown in fig. 8, N storage nodes form a block chain network, where N is a positive integer greater than 2, and each storage node is distributed in different computer rooms, where the write storage node includes:
a first receiving unit 111, configured to receive a write request sent through a blockchain network;
the data splitting unit 112 is configured to split the data file to be written into N-1 sub-data files, and allocate a sub-file identifier to each sub-data file;
a first determining unit 113, configured to determine, according to a preset parity check policy, node address information that each sub-data file and corresponding check data should be stored in a storage node;
a first sending unit 114, configured to send each sub data file and the corresponding data directory, the check data, and the corresponding data directory to the block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
In one embodiment, the first receiving unit may be further configured to: receiving a subdata file and a corresponding data directory which are sent through a block chain network;
the block chain-based distributed data write storage node may further include: and the storage processing unit is used for storing the sub-data files and the corresponding data directories.
In one embodiment, the first receiving unit may be further configured to: receiving verification data and a corresponding data directory sent by a block chain network;
the block chain-based distributed data write storage node may further include: and the storage processing unit is used for storing the verification data and the corresponding data directory.
The embodiment of the present invention further provides a distributed data reading apparatus based on a block chain, as described in the following embodiments. Because the principle of the device for solving the problem is similar to the block chain-based distributed data reading method applied to the write-in storage node, the implementation of the device can refer to the implementation of the block chain-based distributed data reading method applied to the write-in storage node, and repeated details are not repeated.
Fig. 7 is a schematic structural diagram of a distributed data reading apparatus based on a block chain in an embodiment of the present invention, and as shown in fig. 7, N storage nodes form a block chain network, where N is a positive integer greater than 2, and each storage node is distributed in a different computer room, where the distributed data reading apparatus based on a block chain includes:
the client is used for sending a reading request to the block chain network when receiving the data reading request of a user; the reading request comprises a main file identifier corresponding to a data file to be read;
the selected main storage node is used for receiving a reading request sent by the block chain network, searching a corresponding data directory according to the main file identifier, generating a sub-data file acquisition request according to the data directory, and sending the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
each subdata storage node is used for receiving a subdata file acquisition request sent by a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
the main storage node is used for receiving the subdata files sent by all the subdata storage nodes through the block chain network, and merging all the subdata files to obtain a data file to be read; and feeding back the data file to be read to the client.
In an embodiment, each storage node may be further configured to determine that the current sub-data storage node is a failed storage node when it is determined that the sub-data storage node does not have a sub-data file corresponding to a data file to be read according to a data directory stored in the current storage node, and send information of the failed storage node to the block chain network;
the master storage node is specifically configured to:
when the information of the fault storage node is received, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
Fig. 9 is a schematic structural diagram of a block chain-based distributed data reading storage node in an embodiment of the present invention, where N storage nodes form a block chain network, N is a positive integer greater than 2, and each storage node is distributed in a different computer room, as shown in fig. 9, the reading storage node includes:
a second receiving unit 211, configured to receive a read request sent through a blockchain network; receiving subdata files sent by all storage nodes through a block chain network;
a generating unit 212, configured to search a corresponding data directory according to the main file identifier, and generate a sub-data file obtaining request according to the data directory;
a second sending unit 213, configured to send the sub data file obtaining request to the blockchain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information; feeding back a data file to be read to the client;
and the merging processing unit 214 is configured to merge all the sub data files to obtain a data file to be read.
In one embodiment, the second receiving unit may be further configured to receive a sub data file acquisition request sent through a blockchain network;
the block chain based distributed data reading storage node may further include: the second determining unit is used for determining whether the storage node has a subdata file corresponding to the data file to be read according to the data directory stored by the storage node;
the second sending unit may be further configured to send the sub data file to the block chain network when it is determined that the sub data file corresponding to the data file to be read exists in the storage node according to the data directory stored in the storage node.
In one embodiment, the second determining unit may be further configured to: when determining that the sub data storage node does not have the sub data file corresponding to the data file to be read according to the data directory stored by the sub data storage node, determining the storage node as a failure storage node;
the second sending unit may be further configured to send the failed storage node information to the blockchain network;
the above read storage node may further include: the verification fault-tolerant processing unit is used for performing parity check and fault-tolerant processing according to a preset fault-tolerant processing strategy, the sub-data files of other sub-data storage nodes and the check data of the check data storage node when receiving the information of the fault storage node to obtain the sub-data file corresponding to the fault storage node;
the merge processing unit may be further to: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
The embodiment of the invention also provides computer equipment, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the distributed data writing and reading method based on the block chain when executing the computer program.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program for executing the above method for writing and reading distributed data based on a block chain is stored in the computer-readable storage medium.
In the embodiment of the invention, compared with the technical scheme that the safety risk exists in centralized storage and the space is wasted by repeated storage of each node of the block chain in the prior art, the distributed data access scheme based on the block chain improves the safety of data storage and saves the space of the data storage.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (22)

1. A distributed data writing method based on a block chain is characterized in that N storage nodes form a block chain network, N is a positive integer greater than 2, the storage nodes are distributed in different machine rooms, and the distributed data writing method based on the block chain comprises the following steps:
when receiving a data write-in request of a user, a client sends the write-in request to a block chain network; the writing request comprises a data file to be written and a corresponding main file identifier;
receiving a write request sent over a blockchain network by a selected master storage node; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
each subdata storage node receives the subdata files and the corresponding data directories which are sent through the block chain network and stores the subdata files and the corresponding data directories; and the check data storage node receives the check data and the corresponding data directory sent by the block chain network and stores the check data and the corresponding data directory.
2. The blockchain-based distributed data writing method according to claim 1, wherein the client sends a write request to the blockchain network when receiving a data write request from a user, and the method includes:
when a client receives a data writing request of a user, generating a hash value according to a data file to be written; and adding the hash value as a main file identifier into the write-in request, and sending the write-in request to the block chain network.
3. A distributed data writing method based on a block chain is characterized in that N storage nodes form a block chain network, N is a positive integer greater than 2, the storage nodes are distributed in different machine rooms, and the distributed data writing method based on the block chain comprises the following steps:
receiving a write request sent over a blockchain network;
splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file;
determining node address information of each sub-data file and corresponding check data to be stored in the storage node according to a preset parity check strategy;
sending each sub-data file and the corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
4. The block chain-based distributed data writing method according to claim 3, further comprising:
receiving a subdata file and a corresponding data directory which are sent through a block chain network;
and storing the sub-data files and the corresponding data directories.
5. The block chain-based distributed data writing method according to claim 4, further comprising:
receiving verification data and a corresponding data directory sent by a block chain network;
and storing the check data and the corresponding data directory.
6. A distributed data reading method based on a block chain is characterized in that N storage nodes form a block chain network, N is a positive integer greater than 2, the storage nodes are distributed in different machine rooms, and the distributed data reading method based on the block chain comprises the following steps:
when receiving a data reading request of a user, a client sends the reading request to a block chain network; the reading request comprises a main file identifier corresponding to a data file to be read;
the selected main storage node receives a reading request sent by a block chain network, searches a corresponding data directory according to a main file identifier, generates a sub-data file acquisition request according to the data directory, and sends the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
each subdata storage node receives a subdata file acquisition request sent through a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
the main storage node receives the subdata files sent by all the subdata storage nodes through the block chain network, and all the subdata files are combined to obtain a data file to be read; and feeding back the data file to be read to the client.
7. The blockchain-based distributed data reading method according to claim 6, further comprising:
when each subdata storage node determines that the subdata storage node does not have a subdata file corresponding to a data file to be read according to a data directory stored by the storage node, the subdata storage node is determined to be a fault storage node, and fault storage node information is sent to a block link network;
when the main storage node receives the information of the fault storage node, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
8. The block chain-based distributed data reading method according to claim 6, wherein the master file identifier is a hash value of a data file to be read.
9. A distributed data reading method based on a block chain is characterized in that N storage nodes form a block chain network, N is a positive integer greater than 2, the storage nodes are distributed in different machine rooms, and the distributed data reading method based on the block chain comprises the following steps:
receiving a read request sent over a blockchain network;
searching a corresponding data directory according to the main file identifier, and generating a sub-data file acquisition request according to the data directory;
sending a sub data file acquisition request to a block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
receiving subdata files sent by all storage nodes through a block chain network;
merging all the sub-data files to obtain a data file to be read;
and feeding back the data file to be read to the client.
10. The blockchain-based distributed data reading method according to claim 9, further comprising:
receiving a sub data file acquisition request sent through a block chain network;
determining whether a subdata file corresponding to a data file to be read exists in the storage node according to the data directory stored in the storage node;
and when determining that the storage node has the sub-data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub-data file to the block chain network.
11. The blockchain-based distributed data reading method according to claim 10, further comprising:
when determining that the sub data storage node does not have the sub data file corresponding to the data file to be read according to the data directory stored by the storage node, determining the sub data storage node as a fault storage node;
sending the fault storage node information to a block chain network;
when the information of the fault storage node is received, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
12. The utility model provides a distributed data write device based on block chain which characterized in that, N storage node constitutes the block chain network, and N is the positive integer that is greater than 2, and each storage node distributes in different computer rooms, and distributed data write device based on block chain includes:
the client is used for sending a write-in request to the block chain network when receiving a data write-in request of a user; the writing request comprises a data file to be written and a corresponding main file identifier;
a selected primary storage node for receiving a write request sent over a blockchain network; splitting a data file to be written into N-1 parts of sub-data files, and distributing a sub-file identifier for each part of sub-data file; determining node address information of each sub-data file and corresponding check data to be stored in a storage node according to a preset parity check strategy, and sending each sub-data file and corresponding data directory, the check data and the corresponding data directory to a block chain network; the data directory includes: the method comprises the steps that a main file identifier, a subfile identifier and node address information to be stored in a storage node are stored;
each subdata storage node is used for receiving the subdata files and the corresponding data directories sent by the block chain network and storing the subdata files and the corresponding data directories; and the check data storage node is used for receiving the check data and the corresponding data directory sent by the block chain network and storing the check data and the corresponding data directory.
13. The block chain-based distributed data writing apparatus according to claim 12, wherein the client is specifically configured to: when a data writing request of a user is received, generating a hash value according to a data file to be written; and adding the hash value as a main file identifier into the write-in request, and sending the write-in request to the block chain network.
14. The utility model provides a distributed data based on block chain writes into storage node, its characterized in that, N storage node constitutes the block chain network, and N is for being greater than 2 positive integer, and each storage node distributes in different computer rooms, and distributed data based on block chain writes into storage node includes:
a first receiving unit configured to receive a write request transmitted through a blockchain network;
the data splitting unit is used for splitting the data file to be written into N-1 parts of sub-data files and distributing a sub-file identifier for each part of sub-data file;
the first determining unit is used for determining node address information of each sub-data file and the corresponding check data which should be stored in the storage node according to a preset parity check strategy;
the first sending unit is used for sending each part of the sub-data files, the corresponding data directories, the check data and the corresponding data directories to the block chain network; the data directory includes: the main file identification, the subfile identification and the node address information to be stored in the storage node.
15. The blockchain-based distributed data write storage node of claim 14, the first receiving unit further to: receiving a subdata file and a corresponding data directory which are sent through a block chain network;
the distributed data write storage node based on the block chain further comprises: and the storage processing unit is used for storing the sub-data files and the corresponding data directories.
16. The utility model provides a distributed data reading device based on block chain which characterized in that, N storage node constitutes the block chain network, and N is the positive integer that is greater than 2, and each storage node distributes in different computer rooms, and distributed data reading device based on block chain includes:
the client is used for sending a reading request to the block chain network when receiving the data reading request of a user; the reading request comprises a main file identifier corresponding to a data file to be read;
the selected main storage node is used for receiving a reading request sent by the block chain network, searching a corresponding data directory according to the main file identifier, generating a sub-data file acquisition request according to the data directory, and sending the sub-data file acquisition request to the block chain network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information;
each subdata storage node is used for receiving a subdata file acquisition request sent by a block chain network; when determining that the storage node has a sub data file corresponding to the data file to be read according to the data directory stored in the storage node, sending the sub data file to a block chain network;
the main storage node is used for receiving the subdata files sent by all the subdata storage nodes through the block chain network, and merging all the subdata files to obtain a data file to be read; and feeding back the data file to be read to the client.
17. The distributed data reading apparatus based on the blockchain according to claim 16, wherein each sub data storage node is further configured to determine that the sub data storage node is a failed storage node when it is determined that the sub data file corresponding to the data file to be read does not exist in the storage node according to the data directory stored in the storage node, and send information of the failed storage node to the blockchain network;
the master storage node is specifically configured to:
when the information of the fault storage node is received, performing parity check and fault tolerance processing according to a preset fault tolerance processing strategy, the sub data files of other sub data storage nodes and check data of the check data storage node to obtain the sub data file corresponding to the fault storage node;
merging all the subdata files to obtain a data file to be read, wherein the process comprises the following steps: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
18. The utility model provides a distributed data read storage node based on block chain which characterized in that, N storage node constitutes the block chain network, and N is the positive integer that is greater than 2, and each storage node distributes in different computer rooms, and distributed data read storage node based on block chain includes:
a second receiving unit, configured to receive a read request sent through a blockchain network; receiving subdata files sent by all storage nodes through a block chain network;
the generating unit is used for searching a corresponding data directory according to the main file identifier and generating a sub-data file acquisition request according to the data directory;
a second sending unit, configured to send the sub-data file acquisition request to the block link network; the data directory includes: the method comprises the steps of identifying a main file, identifying a subfile and node address information; feeding back a data file to be read to the client;
and the merging processing unit is used for merging all the sub-data files to obtain the data file to be read.
19. The blockchain-based distributed data reading storage node of claim 18 wherein the second receiving unit is further configured to receive a sub data file acquisition request sent over a blockchain network;
the distributed data reading and storing node based on the block chain further comprises: the second determining unit is used for determining whether the storage node has a subdata file corresponding to the data file to be read according to the data directory stored by the storage node;
the second sending unit is further configured to send the sub-data file to the block chain network when determining that the sub-data file corresponding to the data file to be read exists in the storage node according to the data directory stored in the storage node.
20. The blockchain-based distributed data reading storage node of claim 19 wherein the second determination unit is further to: when determining that the sub data storage node does not have the sub data file corresponding to the data file to be read according to the data directory stored by the storage node, determining the sub data storage node as a fault storage node;
the second sending unit is also used for sending the information of the fault storage node to the block chain network;
the distributed data reading and storing node based on the block chain further comprises: the verification fault-tolerant processing unit is used for performing parity check and fault-tolerant processing according to a preset fault-tolerant processing strategy, the sub-data files of other sub-data storage nodes and the check data of the check data storage node when receiving the information of the fault storage node to obtain the sub-data file corresponding to the fault storage node;
the merging processing unit is further configured to: and merging the sub-data files corresponding to the fault storage node and the other sub-data files corresponding to the sub-data storage nodes to obtain the data file to be read.
21. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 11 when executing the computer program.
22. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program for executing the method of any one of claims 1 to 11.
CN202011051905.3A 2020-09-29 2020-09-29 Block chain-based distributed data access method and device and storage node Pending CN112131229A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011051905.3A CN112131229A (en) 2020-09-29 2020-09-29 Block chain-based distributed data access method and device and storage node

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011051905.3A CN112131229A (en) 2020-09-29 2020-09-29 Block chain-based distributed data access method and device and storage node

Publications (1)

Publication Number Publication Date
CN112131229A true CN112131229A (en) 2020-12-25

Family

ID=73844876

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011051905.3A Pending CN112131229A (en) 2020-09-29 2020-09-29 Block chain-based distributed data access method and device and storage node

Country Status (1)

Country Link
CN (1) CN112131229A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115297130A (en) * 2022-10-09 2022-11-04 四川蜀天信息技术有限公司 Distributed storage network data synchronization method based on block chain

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108769146A (en) * 2018-05-11 2018-11-06 北京奇虎科技有限公司 A kind of data transmission method, device and block catenary system based on block chain
CN110278222A (en) * 2018-03-15 2019-09-24 华为技术有限公司 The method, system and relevant device of data management in distributed file storage system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110278222A (en) * 2018-03-15 2019-09-24 华为技术有限公司 The method, system and relevant device of data management in distributed file storage system
CN108769146A (en) * 2018-05-11 2018-11-06 北京奇虎科技有限公司 A kind of data transmission method, device and block catenary system based on block chain

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115297130A (en) * 2022-10-09 2022-11-04 四川蜀天信息技术有限公司 Distributed storage network data synchronization method based on block chain
CN115297130B (en) * 2022-10-09 2022-12-09 四川蜀天信息技术有限公司 Distributed storage network data synchronization method based on block chain

Similar Documents

Publication Publication Date Title
US11797489B2 (en) System and method for using local storage to emulate centralized storage
US9411685B2 (en) Parity chunk operating method and data server apparatus for supporting the same in distributed raid system
US7685459B1 (en) Parallel backup
US8234518B2 (en) Method for voting with secret shares in a distributed system
WO2017201977A1 (en) Data writing and reading method and apparatus, and distributed object storage cluster
US7478113B1 (en) Boundaries
JP5539683B2 (en) Scalable secondary storage system and method
US10140351B2 (en) Method and apparatus for processing database data in distributed database system
US10620830B2 (en) Reconciling volumelets in volume cohorts
US7689764B1 (en) Network routing of data based on content thereof
US9436558B1 (en) System and method for fast backup and restoring using sorted hashes
US20040148306A1 (en) Hash file system and method for use in a commonality factoring system
CN103067525A (en) Cloud storage data backup method based on characteristic codes
WO2005122630B1 (en) Arrangement in a network node for secure storage and retrieval of encoded data distributed among multiple network nodes
US11656962B2 (en) Erasure coding repair availability
CN104699740A (en) Method and system for generating content addressable storage signature for a file
US20190004727A1 (en) Using a namespace to augment de-duplication
CN113051102B (en) File backup method, device, system, storage medium and computer equipment
CN106027638A (en) Hadoop data distribution method based on hybrid coding
CN112131229A (en) Block chain-based distributed data access method and device and storage node
CN112148797A (en) Block chain-based distributed data access method and device and storage node
US20180246666A1 (en) Methods for performing data deduplication on data blocks at granularity level and devices thereof
CN115563073A (en) Method and device for data processing of distributed metadata and electronic equipment
CN112947847B (en) Method, apparatus and computer program product for storing data
CA2934041C (en) Reconciling volumelets in volume cohorts

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination