CN111209262A

CN111209262A - Large-scale distributed safe storage system based on block chain

Info

Publication number: CN111209262A
Application number: CN202010024755.0A
Authority: CN
Inventors: 王凯琢; 于洁
Original assignee: Inspur Tianyuan Communication Information System Co Ltd
Current assignee: Inspur Tianyuan Communication Information System Co Ltd
Priority date: 2020-01-10
Filing date: 2020-01-10
Publication date: 2020-05-29
Anticipated expiration: 2040-01-10
Also published as: CN111209262B

Abstract

The invention provides a large-scale distributed safe storage system based on a block chain, which belongs to the technical field of computer storage and information safety. And by combining the block chain and the distributed safe storage technology, the data storage modes on the chain and under the chain are designed, so that the problem that the data is easy to be falsified is effectively solved, and the integrity of the data is greatly improved on the premise of ensuring the privacy.

Description

Large-scale distributed safe storage system based on block chain

Technical Field

The invention relates to computer storage and information security technology, in particular to a large-scale distributed security storage system based on a block chain.

Background

With the increasing demand of users for data reliability, especially for highly electronic internet enterprises, the large amount of data accumulated in the operation process is the most precious wealth and the life line of internet enterprises. Most of the traditional internet data storage schemes adopt centralized data management, the data security boundary is too obvious, and internal and external attacks are difficult to avoid, so that corresponding data security risks are brought, especially, the protection of data is lacked in the network data transmission process, the risk of illegal hacker attacks is greatly increased, and the internet data storage schemes become soft ribs in a client/server mode.

Disclosure of Invention

In order to solve the technical problems, the invention provides a large-scale distributed secure storage system based on a block chain, which is applied to the network data transmission process by using the digital signature technology and SSL secure communication guarantee protocol of the block chain to realize the secure and reliable data transmission, establishes the safety mechanism of the identities of both information sending parties and the sending content, and can also verify whether the sent information changes in the transmission process so as to guarantee the reliability, trustiness and inflexibility of the data in each link of transmission, storage and the like, thereby providing a new technical solution for the secure data storage.

The technical scheme of the invention is as follows:

a blockchain-based large-scale distributed secure storage system, comprising:

the system comprises a covering and routing module, a data organization module, a data redundancy module, a communication module, a block chain digital signature module and a data query module;

wherein the content of the first and second substances,

the overlay and routing module adopts a routing protocol;

the file information of the data organization module is stored by adopting a file in an XML format; all the file information processed by the system is managed uniformly by the module and organized according to the hierarchical levels of the root block, the directory block, the version block and the data block;

dividing a file into a plurality of blocks and placing the blocks on different nodes; each user has a root block file, the root block contains the information of the user during registration, and the information comprises the ID of a shared file directory block and the ID of a confidential file directory block;

the data redundancy module adopts a Raptor coding method of error correcting code redundancy; firstly, dividing and filling original data, converting the data into source characters with the same size, forming more than one source character into a block, wherein one block is a coding unit, then generating intermediate characters by executing coding operation, and finally generating coded characters by LT coding;

the communication module adopts SSL security protocol;

the block chain digital signature module is used for uniformly storing all public keys of the users using the distributed storage system and randomly distributing the storage positions of the public keys by the DHT;

a data query module; the function of returning all data information meeting the query condition according to the data description information input by the user is realized; the user must select a file download or deletion from the list of files returned by the query module.

Further, in the above-mentioned case,

the data organization module is divided into the following five parts

(1) Registration

The system obtains a user name and a password from the input of a user, checks whether the user name is null or not, creates a root data block and two directory blocks locally according to the user name and the password, sends a registration command 'Register', then sends the user name, judges whether the user name is registered or not, prompts the user to be registered if the user name is registered, sends the root data block and the two directory blocks to a storage server according to the ID of the root data block and the ID of the directory blocks if the user name is not registered, and finally sends a command 'End', which indicates that the data sending is finished; if the command 'Register OK' is received, the registration is successful, and if the command of successful registration is not received, the registration is failed;

(2) login to

The system obtains a user name and a password from the input of the user, calculates the ID and determines that the ID is not null; firstly, searching a root data block file of a user in a local cookie file according to a user ID, obtaining a password according to a user name, if the file is found and the taken-out password is consistent with the password input by the user, correctly logging in a system, if one condition is not met, sending a Login request command 'Login', then sending the user name, searching the root data block file of the user on each node according to the user name, downloading the root data block to the local if the root data block file is found, obtaining the password of the user according to the user name, successfully logging in if the password is correct, otherwise, failing to log in; when logging in, according to the set connection time, displaying a login progress, and if the set time is exceeded, displaying connection overtime and failing to log in;

(3) storing

The user can operate after logging in the system, file storage in the local system is selected in data storage of a main interface, files are selected by right pressing, uploading is selected in a popup menu, an uploading dialog box is popped up at the moment, whether a sharing and encryption method is selected according to the requirement of the user, and the uploading method of the data files is called after confirmation; firstly, obtaining a path of a temporary folder from a configuration file, storing all temporary files in the temporary folder, and deleting files in the temporary folder according to requirements after the task is executed; then, the ID of the shared directory and the ID of the personal file directory are obtained from the user root data block file and are used when the version block file is created; acquiring attribute information of an uploaded file, dividing the file into file blocks, storing the file block information into a data block file, creating a version file, storing the file attribute information into the file, modifying the content of a corresponding directory file according to whether the uploaded file is shared, and finally modifying the content of a root directory file; after the local operation is finished, sending a storage command to a server, and then sending two directory files, version block files and data block files;

(4) downloading

The downloading is to return a file information list through the data query module, then select a file in the file information list, right-click to pop up a menu after selection, select downloading from the menu, select the name and the storage position of the downloaded file in a downloading dialog box, and start downloading after confirmation;

the download execution process is as follows: firstly, obtaining the version ID of a file from a file information list, obtaining the data ID of the file from the version ID, then obtaining all the IDs of file blocks from the data ID, interacting with a background, downloading the data blocks corresponding to all the IDs, then combining the data blocks into a file according to the sequence of file division, downloading the file to a specified position according to a specified file name, and finishing the downloading;

(5) deleting

Deletion is also the deletion of files on the basis of queries.

The deleting process comprises the following steps: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks for the file from the storage server if the file is not local, obtains dirID and data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file; after the directory file block is obtained, deleting the corresponding file information in the file from the directory file block, and updating the directory block file; and acquiring a data file corresponding to the data ID, acquiring all file block IDs in the file, informing the storage server to delete all file block IDs, and then deleting the file. All updated files need to be updated on the network, and files to be deleted need to inform the storage server of deletion.

Four B data built locally by data inquiry module⁺The tree respectively stores file name information, file type information, file size and file creation date; when the user exits the system, the four B are used⁺The trees are encrypted respectively and written into a file, and then uploaded to a server in the point-to-point network for storage; when the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded⁺The tree is read from the file and decrypted; thus, the information of the file held by the user is recovered; when a user wants to insert or delete a file, the four B files are processed⁺The tree performs an insert or delete operation, respectively. When a user needs to inquire files, related file information needs to be input, and the system respectively obtains four B files according to the information⁺And searching in the tree, and finally solving the intersection of the search results of the four trees as a final result and returning the final result to the user.

Further, in the above-mentioned case,

the operation is as follows:

data preparation

The system provides storage services in the form of files, encodes user files into multiple blocks and places them on different nodes, and in addition, the system supports users to view previous versions of data; a data organization structure is designed, and the structure is composed of a root data block, a directory block, a version block and a data block. Each user has a root data block, and the root data block contains the file directory, the shared file directory and the signed personal information owned by the user; the directory block contains pointers pointing to the latest version block of each data file, and the version block contains more than one pointer pointing to the actual data block; the version block is also provided with a backward pointer pointing to the previous version, and more than one version shares the same data block; the pointer is the ID distributed by the DHT, and the DHT can find a block as long as the ID of the block is obtained;

wherein the content of the first and second substances,

(1) root data block

The root data block stores all personal information of the user, including information of root data block ID, user name, hash value of password, using storage space size, user file directory block ID, shared directory block ID and digital signature;

(2) directory block

The directory block contains a directory block ID, a user name, a root data block ID, the number of files, a total size of the file, a file name, a latest version block ID of the file, a digital signature, and the like. Each user has two directory blocks: a user file directory block and a shared file directory block; when a user uploads a file, if the user chooses not to share the file, the version block of the file is associated with the user file directory block, otherwise, the version block of the file is associated with the shared file directory block; the function that the user can check the shared files of the friends is realized by the shared file directory block;

(3) version block

The version block comprises a version block ID, a user name, a directory block ID, a file name, a file type, a file size, creation time, last access time, a redundancy type, the number of data blocks, IDs of all the data blocks and a previous version ID; the version blocks correspond to files, namely each file has one version block; if the user modifies the uploaded file and uploads the modified file again, the system still generates a new version block for the modified file;

(4) data block

The data block contains a data block ID, data block content and a digital signature; a file contains more than one data block, and the data blocks are the result of the file being coded; so that one version block also corresponds to more than one data block;

each block only stores one data record, each data record stores the hash value of the previous data in addition to the data itself, and the two data records are put together for hash to obtain a value which is used as the hash value of the block, so that all data are chained by the block.

Model development

Randomly selecting a node to establish a user public key management file and storing public keys of all users. The public key management file is established when the system is started, an ID value is distributed by a bottom-layer structural overlay network DHT, and the routing protocol sends the file to a corresponding storage node according to the ID value;

the public key management file is stored in the format of an XML file, and the specific format is as follows:

the username behind the Name tag represents the username, the content in the Modulus tag is the coefficient required for calculating the key, the content in the publicExponent tag is the index required for calculating the key, and the public key of the user can be calculated by the two parameter systems. When the user registers, the system finds the public key management file through the DHT, downloads the public key management file to the local, fills the contents of the three tags of the Name, the Modulus and the public Exponent, and then transmits the contents back to the server stored by the public key management file.

System verification process

The background program runs on 5 servers respectively, the servers are positioned in a local area network, 5 nodes run on each server, and 25 nodes are simulated; the foreground interface runs on the PC.

Cutting system functions to obtain a system which does not comprise the block chain digital signature and the SSL and a system which does not comprise the block chain digital signature and the SSL; respectively testing the transmission time of files with different sizes under a system with three cutting functions and a normal system, wherein the file size is from 3M to 100M; the blockchain digital signature function has the greatest impact on system performance.

The large-scale distributed safe storage system based on the block chain can interconnect storage resources scattered in regions and on a network to form a large-scale distributed mass storage resource pool, and provides high-reliability, high-availability and safe storage service for users. The system performs redundant storage on the data of the user to ensure high reliability of the data, reasonably places the data within the wide area network range to ensure high availability and access performance of the data, and can provide data sharing for different users. In addition, the system encrypts and stores the data index of the user to ensure that the data information is not leaked, uses an SSL (secure sockets layer) security protocol for transmission in the data transmission process to ensure the security of the transmission process, and performs a block chain digital signature technology on the data block to ensure the integrity of the data. The block chain digital signature technology can provide a more effective solution for the integrity problem of data, and by combining the block chain and the distributed safe storage technology, an on-chain and off-chain data storage mode is designed, so that the problem that the data is easily tampered is effectively solved, and the integrity of the data is greatly improved on the premise of guaranteeing privacy.

The invention has the advantages that

The performance change of the system after the functions of adding the block chain digital signature, the SSL and the like for improving the safety is analyzed through experimental comparison. Experimental results show that the addition of these functions affects the performance of the system, but greatly enhances the safety of the system. Compared with the traditional storage form, the method has the advantages of strong privacy, high controllability, safe data sharing and the like, and is easy to expand.

Drawings

FIG. 1 is a schematic diagram of the system topology of the present invention;

FIG. 2 is a block diagram of a large scale distributed storage system logical implementation framework;

FIG. 3 is a data organizational diagram of a large scale distributed storage system;

FIG. 4 is a schematic diagram of an organization of root data blocks and directory blocks;

FIG. 5 is a schematic diagram of version block and data block organization;

FIG. 6 is a schematic diagram of a file upload block chain digital signature process

FIG. 7 is a schematic diagram of a file download verification process

Fig. 8 is a comparison graph of file transfer times.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer and more complete, the technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention, and based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts belong to the scope of the present invention.

The system is composed of a large number of nodes distributed in the wide area network, and the nodes are interconnected in a point-to-point mode. The system topology is shown in fig. 1. All servers as providers of storage service form a distributed wide area network storage system; the user as a consumer of the storage service communicates with the whole storage system through client software to upload, download, share and the like files.

Overall system design

The system provides functions mainly as follows: data storage, data query, data sharing, and related security measures.

Data storage: the system performs redundant storage on the data of the user, and distributes the data to the wide area network range through the DHT, so that on one hand, the high reliability of the data can be ensured, and when the data on some nodes are damaged, the data can still be provided for the user through other backup recovery data; on the other hand, the quality of service is guaranteed, and data is returned to the user in a short time.

Data query: various attributes are set for user data, indexes are built on the attributes, complex query of the user on the data is supported, and substring query based on character string attributes, section query based on numerical value attributes and combined query among different attributes are supported by the system.

Data sharing: the system sets the friend function. The user can access the shared data of the friend, and meanwhile, partial data of the user can be set to be shared to be accessed by the friend.

Safety measures are as follows: user data is distributed in the wide area network range and is most likely to be tampered without being known by the user, and therefore a block chain digital signature technology is introduced to ensure the integrity of the user data; in order to avoid that the personal file information of the user is leaked due to the fact that the login account of the user is cracked, the file index of the user is encrypted so as to protect the privacy of the user; meanwhile, as the data is not safely transmitted on the wide area network, the SSL safety protocol is introduced into the system and is implanted into the point-to-point network so as to ensure the transmission safety of the data. The system logic implementation framework is shown in fig. 2.

Design of functional module

Overlay and routing module

The routing module is the basis of a large-scale distributed storage system, which maintains the association between nodes. The nodes follow a consistent routing protocol, and messages are continuously forwarded between the nodes and arrive at the corresponding nodes within a small number of hops. The system adopts a Path routing protocol.

The infrastructure adopts a route strategy of prefix matching and increasing continuously to ensure that the route is in

Finish within hop (B ═ 2)^bIs a carry system of node identities, b is a system parameter, typically 2 or 4). Each node server has a 128-bit nodeId, which is determined to be [0,2 ]¹²⁸) Usually viewed as a ring, say 2¹²⁸0. nodeId is obtained by hashing the address (IP and port) of a node as it joins the system.

The routing state of a node in a topology mainly comprises the following three parts: a routing table (routing table), a neighbor set (neighbor set), and a leaf set (leaf set). The Routing table is organized in a prefix matching manner, and stores pointers of nodes at farther positions in the address space. Stored in the Leaf set are pointers to | L | nodes whose nodeId is closest to itself in the address space. Through leaf sets, all nodes in the system are connected into a ring according to the size sequence of the nodeId, which plays a key role in ensuring the routing convergence of the Path. The neighborwood set contains some nodes that are in the network at a close distance (typically meaning with a small network delay) from the current node network. The neighborwood set is only useful in route optimization and does not contribute to ensuring the routing efficiency and route convergence of log (n) hops.

Given the destination address of a message, the paster ensures that whichever node starts from, the message will eventually be forwarded to the node in the current system whose nodeId is closest to the destination address. The processing flow after each node receives the message is as follows:

when a publish node receives a message, it first judges whether it is the routing end point according to the leaf set, if so, transfers the message to the upper application; if not, see if the route end point is in the leaf set, if so, forward the message to it, if not, i.e. pick the pointer with nodeId matching longest with the previous string of the destination address in the routing table, forward the message to the corresponding node. The method ensures that the nodeId of the node receiving the message is always closer to the target address (the former string is matched longer or the digit is closer to the target address) in the forwarding process, and can prove the convergence of the Path routing, namely the message must be finally and finallyAnd sending to the target node. Routing is at best at the point of destination because each hop makes the nodeId of the forwarding node match the destination address longer before reaching the destination node

Within a hop can be completed.

When a node joins or leaves the system, the relevant node is required to carry out state maintenance. When a node with a nodeId of X needs to join the system, first, a certain node A in the system is requested to send a message by taking X as a target address. Depending on the convergence of the routing algorithm, this message will eventually reach node Z, which is closest to node id X among all the current nodes. The node Z hands its leaf set to X to help X initialize its leaf set; while all nodes on the message forwarding path hand certain rows of their routing table to X. With this information, X initializes its routing table and leaf set. After the initialization is completed, X notifies all nodes on the message forwarding path and in the leaf set that have added the system, and the notified nodes modify their state information accordingly. The whole adding process only needs O (log)_BN) messages can be completed.

Each node explicitly probes all pointers after a period of time to see whether the corresponding node is still in the system, and if the corresponding node is not exited or failed, the corresponding node needs to be repaired when the pointer is found to be failed. If a node in a leaf set fails, the current node will repair its own leaf set through interactions with other nodes in the leaf set. If a certain node in the routing table is invalid, the current node selects another node from the routing table row where the invalid node is located, and fills the vacant position left by the invalid node with the corresponding item of the routing table. If the row in the routing table of the current node has no available node, it will select a node from the next row of the routing table to inquire the corresponding item of its routing table, and this process will continue until a pointer is obtained that can replace the failed node.

Data organization module

The file information of the module is stored by adopting a file in an XML format, so that the user data is conveniently and reasonably arranged, and the user file directory is organized according to the hierarchy. All file information processed by the system is managed by the module in a unified way, and is organized in order according to the hierarchical levels of the root block, the directory block, the version block and the data block. The system provides storage service in a file form, associates file information with user information, is convenient to store and search, and simultaneously divides files into a plurality of blocks which are placed on different nodes. Each user has a root block file that contains the user's information at the time of registration, including the ID of the shared file directory block and the ID of the secured file directory block. The implementation of the module is described in the following five parts:

(1) registration

The system obtains a user name and a password from the input of a user, checks whether the user name is null or not, creates a root data block and two directory blocks locally according to the user name and the password, sends a registration command 'Register', then sends the user name, judges whether the user name is registered or not, prompts the user to be registered if the user name is registered, sends the root data block and the two directory blocks to a storage server according to the root data block ID and the directory block ID if the user name is not registered, and finally sends a command 'End', which indicates that the data sending is finished. If the command "Register OK" is received, the registration is successful, and if the command for successful registration is not received, the registration is failed.

(2) Login to

The system obtains the user name and password from the user's input, calculates the ID and determines that it is not empty. Firstly, searching a root data block file of a user in a local cookie file according to a user ID, obtaining a password according to a user name, if the file is found and the taken-out password is consistent with the password input by the user, correctly logging in the system, if one condition is not met, sending a Login request command 'Login', then sending the user name, searching the root data block file of the user on each node according to the user name, downloading the root data block to the local, obtaining the password of the user according to the user name, successfully logging in if the verification is correct, or else, failing to log in. And displaying the login progress according to the set connection time during login, and displaying connection overtime and login failure if the set time is exceeded.

(3) Storing

The user can perform operations such as storage, access and the like after logging in the system, file storage in a local system is selected in data storage of a main interface, a right key is used for selecting a file, uploading is selected in a popup menu, an uploading dialog box is popped up at the moment, whether a sharing and encryption method is selected according to the requirement of the user, and an uploading method of the data file is called after confirmation. Firstly, obtaining the path of the temporary folder from the configuration file, storing all temporary files in the temporary folder, and deleting the files in the temporary folder according to requirements after the task is executed. Then, the ID of the shared directory and the ID of the personal file directory are obtained in the user root data block file, and are used when the version block file is created. Acquiring attribute information of an uploaded file, dividing the file into file blocks, storing the file block information into a data block file, creating a version file, storing the file attribute information into the file, modifying the content of a corresponding directory file according to whether the uploaded file is shared, and finally modifying the content of a root directory file. After the local operation is finished, a storage command is sent to the server, and then two directory files, version block files and data block files are sent.

(4) Downloading

The downloading is to return a file information list through the data query module, then select a file in the file information list, pop up a menu after selecting, select downloading from the menu, select the name and the storage position of the downloaded file in a downloading dialog box, and start downloading after confirming.

The download execution process is as follows: the method comprises the steps of firstly obtaining the version ID of a file from a file information list, obtaining the data ID of the file from the version ID, then obtaining all the IDs of file blocks from the data ID, interacting with a background, downloading the data blocks corresponding to all the IDs, then combining the data blocks into a file according to the sequence of file division, downloading the file to a specified position according to a specified file name, and finishing the downloading.

(5) Deleting

Deletion is also the deletion of files on the basis of queries. The process is as follows: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks for the file from the storage server if the file is not local, obtains the dirID and data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file. After the directory file block is obtained, deleting the corresponding file information in the file from the directory file block, and updating the directory block file; and acquiring a data file corresponding to the data ID, acquiring all file block IDs in the file, informing the storage server to delete all file block IDs, and then deleting the file. All updated files need to be updated on the network, and files to be deleted need to inform the storage server of deletion.

Data redundancy module

In a large-scale distributed storage system, the dynamic property of the nodes is high, and in order to ensure that data is still available under the condition of node failure, the data must be stored redundantly. There are two main redundancy methods at present: full copy redundancy and error correction code redundancy. Full copy redundancy refers to keeping a complete copy of multiple copies of data, so long as one copy is accessible, the data is not lost. The error correcting code is to divide the stored data into m parts and then to code and convert the data into n (n > m) parts, and any t (t is more than or equal to m) parts can be obtained when the data is recovered.

Although the complete copy redundancy is simple and intuitive, the method has the defects of large storage space consumption, poor performance when processing large files and the like. Using error correction code redundancy, storage space and maintenance bandwidth in the system can be greatly conserved, with the same reliability as copy redundancy. Therefore, the system adopts a method of Raptor coding the redundancy of the error correcting code.

One type of code suitable for use as network transmission in error correction codes is known as fountain codes. The fountain code has two characteristics: the source end can generate an infinite number of codes from the original data; the receiving end can recover the original data as long as it receives enough codes without requiring that the received codes be sequential. LT coding is the first realization of fountain codes, Raptor coding is improved from LT coding, and a pre-coding process is added before LT coding, so that higher decoding efficiency is achieved.

As shown in fig. 7, the Raptor coding adopts a multi-layer check precoding technology, the middle two layers of nodes are middle coding check units, the mapping from the input unit to the first layer of middle coding check units adopts an extended hamming code, and the mapping from the first layer of middle coding check units to the second layer of middle coding check units adopts an LDPC code.

The Raptor encoding process can be summarized as follows: the method comprises the steps of firstly segmenting and filling original data, converting the data into source characters with the same size, forming a block by a plurality of source characters, enabling one block to be a coding unit, then generating intermediate characters by executing coding operation, and finally generating coding characters by LT coding.

Raptor decoding and encoding processes are similar, firstly, Gaussian elimination is adopted to recover intermediate characters from encoded characters, and then LT encoding process is carried out on the intermediate characters to obtain original characters.

SSL secure communication module

The Secure Socket Layer (SSL) is a Security protocol based on WEB applications proposed by netscape corporation, and includes: server authentication, client authentication, data integrity over SSL link, and data confidentiality over SSL link. The SSL safety protocol mainly comprises an SSL handshake protocol and an SSL recording protocol, and also comprises an SSL modification ciphertext protocol and an SSL warning protocol, and is a protocol family consisting of four protocols. SSL is an optional layer between TCP and the application layer protocol. SSL uses public key and traditional encryption technology to realize a safe encryption tunnel on the TCP layer, and guarantees the confidentiality and integrity of information transmission.

The SSL protocol is applied to hypertext transfer for WEB clients and WEB servers, but the SSL protocol can also be applied to peer-to-peer network environments. Since in the handshake protocol the client and the server themselves can be seen as a pair of peer nodes. It can be seen that it is theoretically possible to implant SSL protocol for transmission between point-to-point single nodes and single nodes. If the SSL protocol introduced by the single node and the single node can be extended to all nodes in the network, the security of the whole point-to-point network resource interaction can be realized. However, peer-to-peer is developed on the basis of openness and privacy, and if identity authentication is required before interaction of each communication node, the original development purpose of peer-to-peer is violated.

Block chain digital signature module

Since the data storage servers are distributed in the whole internet, the user cannot effectively monitor the files uploaded by the user, and the data of the user can be illegally tampered by an owner or hacker of the server, so that the block chain digital signature scheme is provided. Data sharing in blockchain technology is a distributed accounting book, and transaction records have multiple copies, so the problem of distributed data storage is solved first. The basic unit of block chain storage is a block, the block adopts a chain structure, namely, the newly added block (similar to one row of records in a database) knows what the previous block (previous row of records) is, the root can be traced all the time, the identification of the block is the hash value of the block, and meanwhile, the chain structure reserves the track generated by the service, so that the verification can be performed according to the previous record in the newly added transaction, and the content of the block is not easy to be falsified.

The public key management problem caused by the absence of a central server in a distributed environment is solved by uniformly storing the public keys of all users using the distributed storage system and randomly distributing the storage positions of the users by the DHT. The method ensures that any server needing the public key management file can find the public key management file through the route of the DHT like downloading any common file. Meanwhile, the completeness of the data is ensured by regenerating the abstract again when the data block is downloaded and comparing the abstract with the original abstract, and the risk that the user uses the illegally tampered data is avoided.

The traditional three-tier data server protection scheme adopted by the storage system is a technical scheme for protecting important data files. The scheme divides the protection of the data server into three layers: the first layer is a remote data terminal used for backing up data files; the second layer is a camouflage transmission protocol, and transmission hiding is realized through UDP, IP and Ethernet protocol headers; and the third layer is to perform honeypot protection on the data server. Comparing the three-layer data server protection scheme with the large-scale distributed security storage system based on the block chain, the block chain digital signature scheme can be obtained to have better performance in data protection and expansibility.

Data query module

The data query module realizes the function of returning all data information meeting query conditions according to the data description information input by the user. This module is one of the core modules of the system, as it is the basis for the user to download and delete files. The user must select a file download or deletion from the list of files returned by the query module.

The module establishes four B's locally at the user⁺The tree stores file name information, file type information, file size, and file creation date, respectively. When the user exits the system, the four B are used⁺The trees are encrypted respectively and written into a file, and then uploaded to a certain server in the point-to-point network for storage. When the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded⁺The tree is read from the file and decrypted. So that the information of the file held by the user is restored. In this way, the searching operation of the user on the file is completely carried out locally, and only once network communication is carried out during login and logout, so that the bandwidth overhead of query can be effectively reduced. When a user wants to insert or delete a file, the four B files are processed⁺The tree is inserted or deleted once, the steps are the same as those of standard B⁺The tree operations are identical and will not be described in detail.

When a user needs to inquire files, related file information such as file names, file sizes and the like needs to be input, and the system respectively obtains four B files according to the information⁺A lookup is made in the tree. Finally, the search results of the four trees are used for solving the intersection and the intersection is used as the final result to be returned to the user.

Implementation procedure

Data preparation

The system provides storage services in the form of files that encode user files into blocks and place them on different nodes, and additionally, the system supports users to view previous versions of data. For this purpose, a data organization as shown in fig. 3 is provided. The structure is composed of a root data block, a directory block, a version block, and a data block. Each user has a root data block, and the root data block contains personal information such as file directories, shared file directories, signatures and the like owned by the user. The directory block contains pointers to the latest version block of each data file, and the version block contains multiple pointers to the actual data blocks. The version block also has a backward pointer pointing to the previous version, and multiple versions share the same data block. It should be noted that the pointer mentioned here is actually an ID assigned by the DHT, and the DHT can find a block as long as the ID of the block is obtained.

The following describes each block in detail:

(1) root data block

The root data block stores all personal information of the user, including information such as root data block ID, user name, hash value of password, used storage space size, user file directory block ID, shared directory block ID, and digital signature.

(2) Directory block

The directory block contains a directory block ID, a user name, a root data block ID, the number of files, a total size of the file, a file name, a latest version block ID of the file, a digital signature, and the like. Each user has two directory blocks: user file directory blocks and shared file directory blocks. When a user uploads a file, if the user chooses not to share the file, the version block of the file is associated with the user file directory block, otherwise, the version block of the file is associated with the shared file directory block. The function that the user can view the shared files of the friends is realized by the shared file directory block.

(3) Version block

The version block contains a version block ID, a user name, a directory block ID, a file name, a file type, a file size, a creation time, a last access time, a redundancy type, a data block number, an ID of all data blocks, a previous version ID, and the like. Version blocks correspond to files, that is, each file has a version block. If the user uploads the uploaded file again after modifying the uploaded file, the system still generates a new version block for the uploaded file.

(4) Data block

The data block contains a data block ID, data block content, and a digital signature. A file contains a plurality of data blocks that are the result of the file being encoded. One version block also corresponds to a plurality of data blocks.

Each block in the large-scale distributed secure storage system only stores one data record, each data record stores the hash value of the previous data besides the data, the two data records are put together for hash to obtain one value which is used as the hash value of the block, and therefore all data are chained through the blocks. Therefore, whether the data is used or not can be verified, and when a person wants to modify the information of a certain data, the fact that all blocks from the modified block to the next needs to be modified is found, which is almost impossible, and other clients cannot accept the modification even if the modification is carried out.

The privacy protection mechanism of the block chain digital signature scheme is mainly divided into two aspects of identity privacy protection and data privacy protection. Public key addresses PK are adopted to replace identities of Publisher, User and Requester on the chain, and two communication parties cannot acquire information of real companies and personnel of the opposite party. In the method, the log file is desensitized before cloud storage and chain release, information related to company confidentiality is hidden by a Publisher, and subsequent steps can be performed on desensitized data by using a User as a final confirmation party. The identity and data privacy protection mechanism can effectively prevent malicious users from calling log information to carry out targeted attack. In the sharing process, the two communication parties are anonymous so as to ensure the privacy safety of the two parties. The request is sent by a Requester, the User authorizes the request, and the Permission and the Time are used for realizing authorization operation and Time limitation on the Requester, ensuring that an authorized person cannot perform operation exceeding the authority, and canceling the Permission after exceeding the Time. The data files are released to the outside through desensitization processing, and the chain records the releasing and calling requests of each data file. And tracing the source by using a block chain mechanism and inquiring the use record under the conditions that the data file has desensitization abnormality and the data has errors.

Model development

The main purpose of the block chain digital signature scheme technology is to ensure the integrity of user data. Because in the peer-to-peer network, the nodes are not trusted with each other, the data stored on the machines of the user can be tampered at any time, so that a technology is needed for prompting the user whether the data of the user is tampered, and the digital signature technology is not suitable.

In a large-scale distributed secure storage system, the positions of all nodes are equal, and an absolute authoritative server does not exist to serve as an authentication center, so that public keys of other nodes cannot be obtained through certificates for encryption and decryption. Therefore, according to the characteristics of the distributed storage network, the adopted method is to randomly select a node to establish a user public key management file and store the public keys of all users. The public key management file is established when the system is started, an ID value is distributed by a bottom layer structural covering network DHT, and the routing protocol sends the file to a corresponding storage node according to the ID value. The essence is that the public key management file is treated as a common data file, and the working process of the system is simplified.

Taking the document delivery to the distributed storage system as an example, the block chain digital signature scheme adapted to the peer-to-peer network environment is proposed below. Suppose A is the user who uploaded the file and its public and private key pairs are (ka1, ka2) and B is the server selected by the DHT to receive the document. The system has a public key management file which stores the public keys of all users including the user A, and the private key of the user A is stored by the user A. In the scheme, in order to realize the function of finding whether the user data is tampered, the size of the signed content is considered to be indefinite, the original text content is directly signed, the operation content is large, and the time is long, so that a message digest is generated by using an SHA-1 algorithm on the original text, then the digest is signed, and finally the signature and the data are uploaded as a whole.

The process of block chain digital signature when user a uploads a file is shown in fig. 6, and the process can be described as follows:

(1) generating a summary Z1 for the data block P by using the SHA-1 algorithm;

(2) the user A uses a private key Ka1 of the user A to digitally sign the abstract Z1 by using an RSA algorithm to obtain a signature C;

(3) combining the data block P and the signature C into a new data block PC;

(4) the DHT assigns an ID to this new data block and routes it to the server where it is stored.

When a user A needs to download a file of the user A, the DHT finds all the required data blocks, a server for storing the data blocks finds a public key management file through the DHT after receiving an instruction for downloading a certain data block, the public key management file is downloaded, the public key of the user A is found in the file, the RSA signature C is decrypted by using the public key to recover a digest Z1, then, an SHA-1 algorithm is used for generating a digest Z2 for the data block P again, the digest Z2 is compared with the digest Z1 and the digest Z2, if the data block P is found to be unequal, the data block P is tampered, and the user can be informed and the downloading of the data block is stopped.

The authentication process when user a downloads the file is shown in fig. 7: the process can be described as follows:

(1) after receiving the downloading instruction, the server separates a data block P and a signature C from a data block PC;

(2) calculating a summary Z2 of the data block P by using an SHA-1 algorithm;

(3) downloading the public key to a public key management file through the DHT, and taking out the public key of the user A from the public key management file;

(4) decrypting the signature C by using the public key to obtain a digest Z1;

(5) comparing Z1 with Z2, if not equal, informing the user A that the data block P is tampered, and stopping downloading the data block P; if they are equal, the data block is passed to user A.

(6) The node A and the node B generate respective public keys and private keys before communication, the public keys can be disclosed to the outside, but the private keys cannot be leaked to a third party.

(7) Node a connects node B, exchanging public keys with each other.

(8) The node A encrypts and transmits the private key of the node A to the node B by using the public key of the node B, and similarly, the node B encrypts and transmits the private key of the node B to the node A by using the public key of the node A.

(9) At the moment, both the nodes obtain the private key of the other party, and the private key of the other party is multiplied by the private key of the nodes to generate the symmetric key K.

(10) And after the downloading is finished, verifying by using the block chain, signing a digital signature for the previous transaction and the next owner by the current owner by using the private key K, and attaching the signature to the tail to manufacture a log record. A new download is generated and broadcast to other participating nodes in the blockchain network.

(11) The propagation of all blockchain transactions, where the current owner broadcasts the transaction order to the entire network, each node will collect several unverified transaction Hash values into blocks, each of which may contain hundreds or thousands of downloads. The node that completes the fastest will propagate its own block to other nodes.

(12) And downloading amount certification, wherein each node obtains the right of creating a new block through a workload certification mechanism equivalent to solving a mathematical problem. Each node performs calculation of workload certification to decide who can verify the transaction, and the node which calculates the result fastest verifies the transaction.

(13) And (4) full node verification, wherein when one node finds a block, the node broadcasts all transactions recorded by the block with time stamps to the whole network and is checked by other nodes of the whole network, other nodes can confirm whether the download contained in the block is effective or not, and the block is accepted after the confirmation that the block is not paid repeatedly and has an effective digital signature, and the block is formally connected with a block chain at the moment and cannot be subjected to data tampering.

(14) And (4) block chain recording, checking the correctness of the block accounting by other nodes in the whole network, and competing for the next block after the legal block after the other nodes in the whole network have no error so as to form a legal accounting block. Once all nodes receive the block, the block which does not finish the node work before is invalid, each node can reestablish a block, and the next node calculation work is continued.

(15) The storage log records each time the file is uploaded and downloaded and calls for the file. And tracing the source by using a block chain mechanism and inquiring the use record under the conditions that the desensitization abnormality occurs to the storage log file and the data is wrong.

System verification process

The background program runs on 5 servers respectively, the servers are positioned in a local area network, 5 nodes run on each server, and 25 nodes are simulated; the foreground interface runs on the PC. The server and PC are configured as follows:

(1) server configuration

Hardware: langchao Yingxin server, 8G internal memory

Software: red Hat Enterprise Linux 7.2 operating System, JDK 8

(2) PC configuration

Hardware: I5-8250U CPU 1.80GHz, 16G memory, 1000M network card

Software: windows 10, JDK 8

In order to ensure the security of data in a large-scale distributed storage system, a block chain digital signature technology and an SSL secure transmission technology are introduced to enhance the security of the system, but the addition of the technologies inevitably has a certain influence on the performance of the system. In order to quantify this effect and provide a basis for future system improvement, the following performance comparison analysis was performed:

and cutting the system function to obtain a system which does not comprise the block chain digital signature and the SSL and a system which does not comprise the block chain digital signature and the SSL. And comparing and analyzing the file transmission performance of the three systems with that of a normal system.

We tested the transmission time of different sized files in three sets of clipping systems and normal systems, respectively, with file sizes from 3M to 100M. The test results are shown in fig. 8.

As can be seen from fig. 8, the performance of the system with only the blockchain digital signature function is closer to that of the normal system, while the performance of the system with only SSL is closer to that of the system without the blockchain digital signature and SSL, so it can be concluded that the impact of the blockchain digital signature function on the system performance is the greatest. It has also been found that as documents grow larger, the gap in performance grows.

Analyzing the reason, firstly, the most time consuming process of the SSL module is the handshake process, but the process is performed only once when the connection is established between the nodes, only data is encrypted during file transmission without handshake authentication again, and the SSL encrypts the data by using a DES encryption algorithm, which belongs to a symmetric key algorithm and is faster than the block chain technology used for digital signatures. With the increase of the file volume, the number of blocks into which the file is divided by the data redundancy module is increased, and the encryption work of the block chain digital signature and the SSL is increased, so that the difference between the performance of the file with the larger file volume and a system without the function is larger. Although the addition of the block chain digital signature and the SSL module has a certain impact on the data transmission performance of the system, it also provides a strong guarantee for the security of the user data during transmission and storage. It is worth a partial loss of performance in exchange for a substantial increase in safety performance.

The above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. A large-scale distributed safe storage system based on a block chain is characterized in that,

the method comprises the following steps:

wherein the content of the first and second substances,

the overlay and routing module adopts a routing protocol;

the communication module adopts SSL security protocol;

2. The storage system of claim 1,

the data organization module is divided into the following five parts

(1) Registration

(2) login to

(3) storing

(4) downloading

(5) deleting

Deletion is also the deletion of files on the basis of queries.

3. The storage system of claim 1,

the deleting process comprises the following steps: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks for the file from the storage server if the file is not local, obtains dirID and data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file; after the directory file block is obtained, deleting the corresponding file information in the file from the directory file block, and updating the directory block file; acquiring a data file corresponding to the data ID, acquiring all file block IDs in the file, informing a storage server to delete all file block IDs, and then deleting the file; all updated files need to be updated on the network, and files to be deleted need to inform the storage server of deletion.

4. The storage system of claim 1,

four B data built locally by data inquiry module⁺The tree respectively stores file name information, file type information, file size and file creation date; when the user exits the system, the four B are used⁺The trees are encrypted respectively and written into a file, and then uploaded to a server in the point-to-point network for storage; when the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded⁺The tree is read from the file and decrypted; so that the information of the file held by the user is restored.

5. The storage system of claim 4,

when a user wants to insert or delete a file, the four B files are processed⁺The tree performs an insert or delete operation, respectively.

6. The storage system of claim 5,

when a user needs to inquire files, related file information needs to be input, and the system respectively obtains four B files according to the information⁺And searching in the tree, and finally solving the intersection of the search results of the four trees as a final result and returning the final result to the user.

7. The storage system of claim 6,

the operation is as follows:

data preparation

wherein the content of the first and second substances,

(1) root data block

(2) directory block

(3) version block

(4) data block

8. The storage system of claim 7,

model development

9. The storage system of claim 8,

system verification process

The background program runs on 5 servers respectively, the servers are positioned in a local area network, 5 nodes run on each server, and 25 nodes are simulated; the foreground interface runs on the PC; cutting system functions to obtain a system which does not comprise the block chain digital signature and the SSL and a system which does not comprise the block chain digital signature and the SSL; the transmission time of files with different sizes under a system with three sets of clipping functions and a normal system is respectively tested, the file size is from 3M to 100M, and the influence of the block chain digital signature function on the system performance is the largest.