CN111209262B

CN111209262B - Large-scale distributed secure storage system based on block chain

Info

Publication number: CN111209262B
Application number: CN202010024755.0A
Authority: CN
Inventors: 王凯琢; 于洁
Original assignee: Inspur Communication Information System Co Ltd
Current assignee: Inspur Communication Information System Co Ltd
Priority date: 2020-01-10
Filing date: 2020-01-10
Publication date: 2023-06-16
Anticipated expiration: 2040-01-10
Also published as: CN111209262A

Abstract

The invention provides a large-scale distributed safe storage system based on a blockchain, which belongs to the technical field of computer storage and information security. And by combining the blockchain and the distributed secure storage technology, the data storage modes on and off the chain are designed, so that the problem that data is easy to tamper is effectively solved, and the integrity of the data is greatly improved on the premise of ensuring privacy.

Description

Large-scale distributed secure storage system based on block chain

Technical Field

The invention relates to the computer storage and information security technology, in particular to a large-scale distributed security storage system based on a blockchain.

Background

With the increasing demands of users on data reliability, especially for internet enterprises that have been highly electronic, a large amount of data materials accumulated in the running process are the most precious wealth of them, and are life lines of internet enterprise survival. The traditional internet data storage scheme mostly adopts centralized data management, the data security boundary is too obvious, and internal and external attacks are difficult to avoid, so that corresponding data security risks are brought, and particularly, the risk of illegal hacking attack is greatly increased due to lack of protection of data in the network data transmission process, and the data security risk becomes a soft rib of a client/server mode.

Disclosure of Invention

In order to solve the technical problems, the invention provides a large-scale distributed secure storage system based on a blockchain, which utilizes the digital signature technology of the blockchain and SSL secure communication guarantee protocol application to realize secure and reliable data transmission in the network data transmission process, establishes the identity of both sides of information transmission and a secure mechanism of the transmitted content, and can also verify whether the transmitted information has variation in the transmission process so as to guarantee the reliability, the credibility and the irreplaceability of all links of data transmission, storage and the like, thereby providing a new technical solution for secure data storage.

The technical scheme of the invention is as follows:

a blockchain-based large-scale distributed secure storage system, comprising:

the system comprises a coverage and routing module, a data organization module, a data redundancy module, a communication module, a blockchain digital signature module and a data query module;

wherein, the liquid crystal display device comprises a liquid crystal display device,

the coverage and routing module adopts a path routing protocol;

the file information of the data organization module is file storage in an XML format; all the file information processed by the system is subjected to unified management of the module and is organized according to the hierarchical levels of the root block, the directory block, the version block and the data block;

Dividing the file into a plurality of blocks and placing the blocks on different nodes; each user has a root block file, wherein the root block contains information of the user during registration and comprises the ID of a shared file directory block and the ID of a confidential file directory block;

the data redundancy module adopts a Raptor coding method for error correction code redundancy; firstly, dividing and filling original data, converting the data into source characters with the same size, forming a block by more than one source character, wherein the block is a coding unit, then generating intermediate characters by executing coding operation, and finally generating coding characters by LT coding;

the communication module adopts SSL safety protocol;

the block chain digital signature module is used for uniformly storing public keys of all users using the distributed storage system and randomly distributing storage positions of the public keys by the DHT;

a data query module; the function of returning all data information meeting the query condition according to the data description information input by the user is realized; the user must select a file download or delete from the list of files returned by the query module.

Further, the method comprises the steps of,

the data organization module is divided into the following five parts

(1) Registration

The system obtains a user name and a password from the input of a user, checks that the user name and the password are not empty, creates a root data block and two directory blocks locally according to the user name and the password, sends a registration command 'Register', then sends the user name, firstly judges whether the user name is registered or not, prompts the user to Register if the user name is registered, sends the root data block and the two directory blocks to a storage server according to the root data block ID and the directory block ID if the user name is not registered, and finally sends a command 'End', which indicates that data transmission is completed; if the command 'Register OK' is received, the registration is successful, and if the command of successful registration is not received, the registration failure is indicated;

(2) Login

The system obtains a user name and a password from the input of the user, calculates an ID and determines that the ID is not empty; firstly, searching a user's root data block file in a local cookie file according to a user ID, obtaining a password according to a user name, if the file is found and the obtained password is consistent with the password input by the user, correctly logging in the system, if one condition is not satisfied, sending a Login request command Login, then sending the user name, searching the user's root data block file on each node according to the user name, downloading the root data block to the local according to the user name, obtaining the user's password according to the user name, verifying the correctness, successfully logging in, otherwise, logging in fails; displaying login progress according to the set connection time when logging in, and displaying connection overtime when the login progress exceeds the set time, wherein the login fails;

(3) Storage of

The user can operate after logging in the system, select the file storage in the local system in the data storage of the main interface, right key select the file, select uploading in the popup menu, popup the uploading dialog box at this time, select whether to share and encrypt the method according to the user's need, call the uploading method of the data file after confirming; firstly, obtaining a path of a temporary folder from a configuration file, storing all temporary files in the temporary folder, and deleting the files in the temporary folder according to requirements after task execution is completed; then obtaining the ID of the shared directory and the ID of the personal file directory in the user root data block file, and using the ID when creating the version block file; acquiring attribute information of an uploaded file, dividing the file into file blocks, storing the file block information into data block files by using an ID (identity) of each file block, creating a version file, storing the file attribute information into the file, modifying the content of a corresponding directory file according to whether the uploaded file is shared or not, and finally modifying the content of a root directory file; after the local operation is finished, a Store storage command is sent to a server, and then two directory files, a version block file and a data block file are sent;

(4) Downloading

The downloading is that a file information list is returned through the data query module, then a file is selected in the file information list, a menu is popped up by a right key after the selection, the downloading is selected from the menu, the name and the storage position of the downloaded file are selected in a downloading dialog box, and the downloading is started after the confirmation;

the download execution process is as follows: firstly, obtaining a version ID of a file from a file information list, obtaining a data ID of the file from the version ID, obtaining all IDs of file blocks from the data ID, interacting with a background, downloading data blocks corresponding to all IDs, merging the data blocks into a file according to the sequence of file segmentation, downloading the file into a designated position by designated file names, and completing the downloading;

(5) Deletion of

Deletion is also the deletion of files on a query basis.

The deleting process is as follows: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks for the version block file from the storage server if the version block file is not local, obtains the dirID and the data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file; after the catalog file block is obtained, corresponding file information in the file is deleted from the catalog file block, and the catalog block file is updated; and obtaining a data file corresponding to the data ID, firstly obtaining all file block IDs in the file, informing a storage server to delete all file block IDs, and then deleting the file. All updated files are updated on the network, and the files to be deleted are also notified to the storage server to be deleted.

The data query module establishes four B's locally at the user ⁺ The tree respectively stores file name information, file type information, file size and file creation date; when the user exits the system, these four B pieces are taken ⁺ The tree is encrypted and written into a file respectively, and then uploaded to a server in a point-to-point network for storage; when the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded ⁺ The tree is read from the file and decrypted; thus, the information of the file held by the user is restored; when a user wants to insert or delete a file, the four B's are aligned ⁺ The tree performs an insert or delete operation, respectively. When the user needs to inquire the file, the related file information needs to be input, and the system respectively sends the information to four B's according to the information ⁺ Searching in the tree, and finally returning the intersection of the searching results of the four trees to the user as a final result.

Further, the method comprises the steps of,

the operation is as follows:

data preparation

The system provides storage services in the form of files, encodes user files into blocks and places them on different nodes, and in addition, the system supports users to view previous versions of data; a data organization structure is designed, which consists of a root data block, a directory block, a version block and a data block. Each user has a root data block, and the root data block contains the file catalogue, the shared file catalogue and the signed personal information owned by the user; the directory block contains pointers to the latest version block of each data file, and the version block contains more than one pointer to the actual data block; the version block also has a backward pointer back pointer to the previous version, and more than one version share the same data block; the pointer is assigned ID by DHT, so long as the ID of a block is obtained, DHT can find the block;

(1) Root data block

The root data block stores all personal information of the user, including root data block ID, user name, hash value of the password, size of the used storage space, user file directory block ID, shared directory block ID and digitally signed information;

(2) Directory block

The directory block contains directory block ID, user name, root data block ID, number of files, total size of files, file name, latest version block ID of files, digital signature, etc. Each user has two directory blocks: a user file directory block and a shared file directory block; when a user uploads a file, if the file is selected not to be shared, the version block of the file is associated with the file directory block of the user, otherwise, the version block of the file is associated with the shared file directory block; the function that the user can check the shared file of the friend is realized by the shared file directory block;

(3) Version block

The version block comprises a version block ID, a user name, a directory block ID, a file name, a file type, a file size, a creation time, a last access time, a redundancy type, a data block number, all data block IDs and a previous version ID; the version blocks correspond to files, that is, each file has a version block; if the user modifies a file which has already been uploaded and then uploads the file again, the system still generates a new version block for the file;

(4) Data block

The data block comprises a data block ID, data block content and a digital signature; a file contains more than one data block, which is the result of the file being encoded; so that one version block also corresponds to more than one data block;

each block only stores one data record, each data record stores the hash value of the last data besides the data, and the data record are put together for hash to obtain a value which is used as the hash value of the block, so that all the data are strung into a chain through the block.

Model development

A node is randomly selected to establish a user public key management file and store public keys of all users. The public key management file is established when the system is started, an ID value is distributed by the structural overlay network DHT of the bottom layer, and the routing protocol sends the file to the corresponding storage node according to the ID value;

the public key management file is stored in the format of an XML file, and the specific format is as follows:

the usernames after the Name tag represents the user Name, the content in the Modulus tag is the coefficient required for calculating the key, the content in the publicExponent tag is the index required for calculating the key, and the public key of the user can be calculated through the two parameter system. When the user registers, the system finds the public key management file through the DHT, downloads the public key management file to the local, fills the contents of the Name, modulus, publicExponent tags, and uploads the contents back to the server in which the contents are stored.

System verification process

The background program runs on 5 servers respectively, the servers are in a local area network, 5 nodes run on each server, and 25 nodes are simulated in total; the foreground interface runs on the PC.

Clipping the system function to obtain a system which does not comprise the blockchain digital signature and SSL and a system which does not comprise the blockchain digital signature and SSL; respectively testing the transmission time of the files with different sizes under a system with three sets of cutting functions and a normal system, wherein the file size is from 3M to 100M; the blockchain digital signature function has the greatest impact on system performance.

The large-scale distributed safe storage system based on the blockchain can interconnect storage resources scattered in regions and on a network to form a large-scale distributed mass storage resource pool, and provides high-reliability, high-availability and safe storage service for users. The system performs redundant storage on the data of the user to ensure high reliability of the data, reasonably places the high availability and access performance of the data in the wide area network range, and can provide data sharing for different users. In addition, the system encrypts and stores the data index of the user, so that the data information of the user is not leaked, SSL security protocol is used for transmission in the data transmission process, the security of the transmission process is ensured, and the block chain digital signature technology is performed on the data blocks to ensure the integrity of the data. The blockchain digital signature technology can provide a more effective solution to the problem of data integrity, and by combining the blockchain and the distributed secure storage technology, the on-chain and off-chain data storage modes are designed, so that the problem that data is easy to tamper is effectively solved, and the data integrity is greatly improved on the premise of ensuring privacy.

The invention has the beneficial effects that

The performance change of the system after the functions of improving the security such as blockchain digital signature, SSL and the like are added is analyzed through experimental comparison. Experimental results show that while the addition of these functions has been shown to affect the performance of the system, the safety of the system is also greatly enhanced. Compared with the traditional storage form, the method has the advantages of being strong in privacy, high in controllability, safe in data sharing and the like, and is easy to expand.

Drawings

FIG. 1 is a schematic diagram of the system topology of the present invention;

FIG. 2 is a block diagram of a large-scale distributed storage system logic implementation;

FIG. 3 is an organizational chart of data for a mass distributed storage system;

FIG. 4 is a schematic diagram of a root data block and directory block organization;

FIG. 5 is a version block and data block organization schematic;

FIG. 6 is a schematic diagram of a file upload blockchain digital signature process

FIG. 7 is a schematic diagram of a file download verification process

Fig. 8 is a graph of file transfer time versus time.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making any inventive effort based on the embodiments of the present invention are within the scope of protection of the present invention.

The system is composed of a large number of nodes distributed in the wide area network, and the nodes are interconnected in a point-to-point mode. The system topology is shown in fig. 1. All servers serve as providers of storage services to form a distributed wide area network storage system; the user as a consumer of the storage service communicates with the entire storage system through the client software to upload, download, share, etc. files.

Overall system design

The system provides the functions mainly as follows: data storage, data querying, data sharing, and related security measures.

And (3) data storage: the system performs redundant storage on the data of the user, distributes the data into the wide area network range through the DHT, on one hand, can ensure the high reliability of the data, and can still restore the data to the user through other backups when the data on some nodes are damaged; on the other hand, the quality of service is ensured, and the data is returned to the user in a shorter time.

And (3) data query: multiple attributes are set for user data, indexes are built on the attributes, so that complex inquiry of the user to the data is supported, and the system supports sub-string inquiry based on character string attributes, section inquiry based on numerical value attributes and joint inquiry among different attributes.

Data sharing: the system sets up the friend function. The user can access the shared data of the friends, and can set partial data of the user as sharing so as to enable the friends to access.

Safety measures are as follows: the user data is distributed in the wide area network range, so that the user is possibly tampered without being informed, and a blockchain digital signature technology is introduced to ensure the integrity of the user data; in order to avoid the disclosure of personal file information of a user caused by the cracking of a login account of the user, encrypting a file index of the user so as to protect the privacy of the user; meanwhile, because the data is not safely transmitted on the wide area network, the system introduces SSL security protocol, and the SSL security protocol is implanted into the point-to-point network to ensure the transmission security of the data. The system logic implementation framework is shown in fig. 2.

Functional module design

Coverage and routing module

The routing module is the basis for a large-scale distributed storage system that maintains connections between nodes. The nodes follow a consistent routing protocol, messages are continuously forwarded between nodes, reaching the corresponding nodes in fewer hops. The system adopts a path routing protocol.

The route adopts a routing strategy with continuously increased prefix matching, and ensures that the route is routed

Completion within jump (b=2 ^b Is the carry of the node identity, b is a system parameter, typically 2 or 4). Each node server has a nodeId of 128 bits, which is determined to be at [0,2 ] ¹²⁸ ) Is usually regarded as a ring, i.e. order 2 ¹²⁸ =0. The nodeId is obtained by hashing the address (IP and port) of the node when it joins the system.

The routing state of the nodes in the path mainly comprises the following three parts: a routing table (routing table), a neighbor node set (neighbor set), and a leaf node set (leaf set). Routing tables are organized in a prefix matching mode, and pointers of nodes at far positions in an address space are stored. Stored in Leaf set are pointers to the L nodes closest to itself in address space to the nodeId. Through leaf set, all nodes in the system are connected in a ring according to the order of the nodeId, which plays a key role in ensuring route convergence of the path. The neighbor set contains some nodes in the network that are very close to the current node network (typically referred to as small network delay). Neighbor set is only useful in performing route optimization and does not contribute to guaranteeing both the route efficiency and route convergence of log (N) hops.

When given the destination address of a message, the path ensures that whichever node is from, the message will eventually be forwarded to the node in the current system that is closest to the destination address. The processing flow after each node receives the message is as follows:

when a message is received by a path node, firstly judging whether the path node is a routing destination or not according to the leaf set, and if so, transferring the message to an upper layer application; if not, it is checked whether the routing endpoint is in leaf set, if so, the message is forwarded to it, and if not, the longest pointer matching the nodeId with the pre-string of the target address is selected in the routing table, and the message is forwarded to the corresponding node. This method ensures that the nodeId of the node receiving the message always gets closer to the destination address (the pre-string match is longer or the number distance is closer) during forwarding, and can prove the convergence of the path route, i.e. the message must be finally sent to the destination node. Since each hop matches the nodeId of the forwarding node to the destination address longer before reaching the destination node, the route is at most

The jump can be completed.

When a node joins or leaves the system, the relevant node is required to perform state maintenance. When a node with nodeId X needs to join the system, first, please ask a node a already in the system to send a message with X as the destination address. Depending on the convergence of the routing algorithm, this message eventually reaches node Z, which is closest to X, of all nodes currently. Node Z gives its leaf set to X, help X initialize its leaf set; while all nodes on the message forwarding path hand over some rows of their own routing table to X. X initializes its own routing table and leaf set with this information. After the initialization is completed, all nodes on the forwarding path of the X notification message and in the leaf set have joined the system, and the node receiving the notification modifies the state information of the nodes accordingly. The whole addition process only needs O (log _B N) messages can be completed.

Each node explicitly detects all pointers every time a period of time, and whether the corresponding node is still in the system or not is judged, and when the pointers are found to be invalid, the corresponding node needs to be repaired. If there are nodes in the leaf set that fail, the current node will repair its own leaf set by interacting with other nodes in the leaf set. If one node in the routing table fails, the current node selects another node from the routing table row where the failed node is located, and fills the empty position left by the failed node with the corresponding item of the routing table. If the row in the routing table of the current node has no available nodes, it will select a node from the next row of the routing table to query its routing table for its corresponding entry, and this process will continue until a pointer is obtained that can replace the failed node.

Data organization module

The file information of the module is file storage in XML format, which is convenient for reasonably arranging user data and organizing user file catalogues according to layers. All the file information processed by the system is managed by the module in a unified way by the user, and is organized orderly according to the hierarchical levels of the root block, the directory block, the version block and the data block. The system provides storage service in the form of files, associates file information with user information, facilitates storage and searching, and simultaneously divides the files into a plurality of blocks and places the blocks on different nodes. Each user has a root block file containing the information of the user at registration, including the ID of the shared file directory block and the ID of the secured file directory block. The realization of the module is divided into the following five parts:

(1) Registration

The system obtains a user name and a password from the input of a user, checks that the user name and the password are not empty, creates a root data block and two directory blocks locally according to the user name and the password, sends a registration command 'Register', then sends the user name, firstly judges whether the user name is registered, prompts the user to Register if the user name is registered, sends the root data block and the two directory blocks to a storage server according to the root data block ID and the directory block ID if the user name is registered, and finally sends a command 'End', which indicates that data transmission is completed. If the command 'Register OK' is received, the registration is successful, and if the command of successful registration is not received, the registration failure is indicated.

(2) Login

The system obtains the user name and password from the user's input, calculates the ID and determines that it is not null. Firstly searching a user's root data block file in a local cookie file according to a user ID, obtaining a password according to a user name, if the file is found and the obtained password is consistent with the password input by the user, correctly logging in the system, if one condition is not satisfied, sending a Login request command Login, then sending the user name, searching the user's root data block file on each node according to the user name, downloading the root data block to the local according to the user name, obtaining the user's password according to the user name, verifying the correctness, and successfully logging in, otherwise, logging in fails. When logging in, according to the set connection time, logging in progress is displayed, and when the set connection time is exceeded, connection timeout is displayed, and logging in fails.

(3) Storage of

After the user logs in the system, the operations such as storage, access and the like can be performed, the file storage in the local system is selected in the data storage of the main interface, the file is selected by the right key, the uploading is selected in the pop-up menu, at the moment, the uploading dialog box is popped up, whether the sharing and encryption method is selected according to the requirement of the user, and the uploading method of the data file is called after confirmation. Firstly, obtaining a path of a temporary folder from configuration files, storing all the temporary files in the temporary folder, and deleting the files in the temporary folder according to requirements after the task execution is completed. Then the ID of the shared directory and the ID of the personal file directory are obtained from the user root data block file, and are used when creating the version block file. Acquiring attribute information of an uploaded file, dividing the file into file blocks, storing the file block information into data block files by using an ID (identity) of each file block, creating a version file, storing the file attribute information into the file, modifying the content of a corresponding directory file according to whether the uploaded file is shared or not, and finally modifying the content of a root directory file. After the local operation is completed, a Store storage command is sent to the server, and then two directory files, a version block file and a data block file are sent.

(4) Downloading

The downloading is that a file information list is returned through the data inquiry module, then a file is selected in the file information list, a menu is popped up by a right key after the selection, the downloading is selected from the menu, the name and the storage position of the downloaded file are selected in the downloading dialog box, and the downloading is started after the confirmation.

The download execution process is as follows: the method comprises the steps of firstly obtaining a version ID of a file from a file information list, obtaining a data ID of the file from the version ID, obtaining all IDs of file blocks from the data ID, interacting with a background, downloading data blocks corresponding to all IDs, merging the data blocks into a file according to the sequence of file segmentation, downloading the file name to a designated position, and completing downloading.

(5) Deletion of

Deletion is also the deletion of files on a query basis. The process comprises the following steps: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks the storage server if the version block file is not local, obtains the dirID and data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file. After the catalog file block is obtained, corresponding file information in the file is deleted from the catalog file block, and the catalog block file is updated; and obtaining a data file corresponding to the data ID, firstly obtaining all file block IDs in the file, informing a storage server to delete all file block IDs, and then deleting the file. All updated files are updated on the network, and the files to be deleted are also notified to the storage server to be deleted.

Data redundancy module

In large-scale distributed storage systems, the dynamics of the nodes are high, and in order to ensure that data is still available in the event of node failure, redundant storage of the data is necessary. There are two main redundancy methods at present: full copy redundancy and error correction code redundancy. Full copy redundancy refers to the preservation of a complete copy of multiple copies of data, which is not lost as long as there is one copy available. The error correcting code refers to dividing the stored data into m parts, then coding and transforming into n (n > m) parts, and obtaining any t (t is larger than or equal to m) parts when recovering the data.

The full copy redundancy is simple and visual, but has the defects of large storage space consumption, poor performance when processing large files, and the like. By using error correction code redundancy, the storage space and maintenance bandwidth in the system can be greatly saved under the condition that the same reliability is obtained as that of duplicate redundancy. Therefore, the system adopts the method of Raptor coding such error correction code redundancy.

One type of code in error correction codes that is suitable for use as network transmissions is known as fountain codes. The fountain code has two characteristics: the source end can generate infinite codes from the original data; the receiving end can recover the original data only by receiving enough codes without requiring the received codes to be sequential. LT coding is the first implementation of fountain coding, raptor coding is improved from LT coding, and a precoding process is added before LT coding, so that higher decoding efficiency is achieved.

As shown in fig. 7, the Raptor coding adopts a multi-layer check precoding technology, the two middle layers of nodes are middle coding check units, the mapping from the input unit to the first layer middle coding check unit adopts an extended hamming code, and the mapping from the first layer middle coding check unit to the second layer middle coding check unit adopts an LDPC code.

The Raptor encoding process can be summarized as follows: firstly, dividing and filling original data, converting the data into source characters with the same size, forming a block by a plurality of source characters, forming a coding unit by one block, then generating intermediate characters by executing coding operation, and finally generating coding characters by LT coding.

The Raptor decoding and the encoding process are similar, the Gaussian elimination is adopted to recover the intermediate character from the encoded character, and then the LT encoding process is carried out on the intermediate character to obtain the original character.

SSL safety communication module

The secure socket layer protocol (SSL, security Socket Layer) is a WEB application-based security protocol proposed by the netscape company, and includes: server authentication, client authentication, data integrity over SSL link, and data confidentiality over SSL link. The SSL security protocol mainly comprises an SSL handshake protocol and an SSL recording protocol, also comprises an SSL modification ciphertext protocol and an SSL warning protocol, and is a protocol family formed by the four protocols. SSL is an optional layer between TCP and application layer protocols. SSL uses public key and traditional encryption technology to realize a secure encryption tunnel above TCP layer, and ensures confidentiality and integrity of information transmission.

The SSL protocol is applied to hypertext transfer for WEB clients and WEB servers, but the SSL protocol is also applicable to point-to-point network environments. Because in handshake protocols, the client and server themselves can be considered a pair of alien nodes. It can be seen that the SSL protocol can be implemented in theory by implanting it between a point-to-point single node and a single node. If the single node and the SSL protocol introduced by the single node can be extended to all nodes in the network, the security of the whole peer-to-peer network resource interaction can be realized. However, the point-to-point development is based on openness and privacy, and if identity authentication is required before each communication node interacts, the development of the point-to-point development is contraband.

Block chain digital signature module

Because the data storage servers are distributed in the whole internet, users cannot effectively monitor the files uploaded by the users, and the data of the users can be illegally tampered by owners or hackers of the servers, so that a blockchain digital signature scheme is proposed. The data sharing of the blockchain technology is a distributed book, and the transaction record is provided with a plurality of copies, so that the problem of distributed data storage is solved first. The basic unit of the block chain storage is a block, the block adopts a chain structure, namely, a newly added block (similar to a row of records of a database) knows what the previous block (the previous row of records) is, the block can be traced back to the root, the mark of the block is the hash value of the block, meanwhile, the chain structure keeps the track generated by the service, and the verification can be carried out according to the previous records when the transaction is newly added, so that the content of the block is not easy to tamper.

The public key management problem caused by the fact that a central server does not exist in a distributed environment is solved by uniformly storing public keys of all users using the distributed storage system and randomly distributing storage positions of the public keys by the DHT. The method enables any server needing the public key management file to find the public key management file through the DHT route like downloading any common file. Meanwhile, the integrity of the data is ensured by regenerating the abstract and comparing the abstract with the original abstract when the data block is downloaded, and the risk of illegal tampering of the data by a user is avoided.

The three-layer data server protection scheme adopted by the traditional storage system is a technical scheme for protecting important data files. The scheme divides the protection of the data server into three layers: the first layer is a remote data terminal for backing up data files; the second layer is camouflage transmission protocol, and transmission is hidden through a UDP, IP, ethernet protocol head; the third layer is honeypot protection of the data server. Comparing the three-layer data server protection scheme with the blockchain-based large-scale distributed secure storage system, the blockchain digital signature scheme can be obtained to have better performance in terms of data protection and expansibility.

Data query module

The data query module realizes the function of returning all data information meeting the query conditions according to the data description information input by the user. This module is one of the core modules of the system, as it is the basis for users to download and delete files. The user must select a file download or delete from the list of files returned by the query module.

The module establishes four B's locally at the user ⁺ The tree holds file name information, file type information, file size, and file creation date, respectively. When the user exits the system, these four B pieces are taken ⁺ The tree is encrypted and written into a file, and then uploaded to a server in the point-to-point network for storage. When the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded ⁺ The tree is read from the file and decrypted. Thus, the information of the file held by the user is restored. In this way, the searching operation of the user on the file is completely carried out locally, and only one network communication is carried out when logging in and logging out, so that the bandwidth overhead of inquiry can be effectively reduced. When a user wants to insert or delete a file, the four B's are aligned ⁺ The tree is inserted or deleted once, and the steps are the same as those of standard B ⁺ The tree operation is consistent and will not be described in detail.

When the user needs to inquire the file, the related file information such as file name, file size and the like needs to be input, and the system respectively obtains four pieces of B according to the information ⁺ A lookup is performed in the tree. And finally, returning the intersection of the search results of the four trees to the user as a final result.

Implementation steps

Data preparation

The system provides storage services in the form of files, encoding user files into blocks and placing on different nodes, and in addition, the system supports users viewing previous versions of data. For this purpose, a data organization structure as shown in fig. 3 is designed. The structure consists of a root data block, a directory block, a version block, and a data block. Each user has a root data block, and the root data block contains personal information such as file catalogues, shared file catalogues, signatures and the like owned by the user. The directory block contains pointers to the latest version block of each data file, and the version block contains a plurality of pointers to the actual data blocks. The version block also has a backward pointer back pointer to the previous version, and multiple versions share the same data block. It should be noted that the pointer here is actually an ID assigned by the DHT, and the DHT can find a block as long as the ID of the block is obtained.

The following describes each block in detail:

(1) Root data block

The root data block stores all personal information of the user, including root data block ID, user name, hash value of password, size of used storage space, user file directory block ID, shared directory block ID, digital signature, etc.

(2) Directory block

The directory block contains directory block ID, user name, root data block ID, number of files, total size of files, file name, latest version block ID of files, digital signature, etc. Each user has two directory blocks: a user file directory block and a shared file directory block. When a user uploads a file, if the file is selected not to be shared, the version block of the file is associated with the user file directory block, otherwise, the file is associated with the shared file directory block. The function that the user can view the shared files of friends is realized by the shared file directory block.

(3) Version block

The version block contains a version block ID, a user name, a directory block ID, a file name, a file type, a file size, a creation time, a last access time, a redundancy type, a number of data blocks, all data block IDs, a previous version ID, and the like. Version blocks correspond to files, that is, one version block for each file. If the user modifies an already uploaded file and then uploads it again, the system will still generate a new version block for it.

(4) Data block

The data block contains a data block ID, data block content and a digital signature. A file contains a plurality of data blocks, which are the result of the file being encoded. One version block also corresponds to a plurality of data blocks.

In the large-scale distributed secure storage system, each block stores only one data record, each data record stores a hash value of the last data besides the data, and the two data records are put together for hash to obtain a value as the hash value of the block, so that all data are strung into a chain through the block. This not only verifies whether the data is used, but also discovers that all blocks need to be modified since the block that was modified when someone wants to modify the information of some of the data, which is almost impossible, and other clients will not accept the modification even if it is.

The privacy protection mechanism of the blockchain digital signature scheme is mainly divided into two aspects of identity privacy protection and data privacy protection. The identity Publisher, user, requester on the chain is replaced by the public key address PK, and the communication parties cannot acquire the information of the real company and personnel of the opposite party. The log file is desensitized before cloud storage and chain release, information related to company confidentiality is hidden by publicher, and final confirmation of the desensitized data still needs User to be carried out for subsequent steps. The identity and data privacy protection mechanism can effectively prevent malicious users from calling log information to conduct targeted attack. In the sharing process, anonymizing is carried out on both communication parties so as to ensure the privacy security of both parties. The request is sent by the request, the User will authorize after receiving the request, permission and Time are used for realizing the authorization operation and the aging limit of the request, ensuring that the authorized person cannot perform the operation exceeding the Permission, and simultaneously, the Permission is revoked after exceeding the aging. The data files are released externally through desensitization processing, and the chain records release and call requests of each data file. Under the conditions that the data file is desensitized and abnormal and the data is wrong, the block chain mechanism can be utilized for tracing, and the use record is inquired.

Model development

The main purpose of the blockchain digital signature scheme technique is to ensure the integrity of the user data. Because the nodes are mutually not trusted in the peer-to-peer network, the data of the user can be tampered with at any time when the data is stored in the machines, and a technology is needed to prompt the user whether the data is tampered with or not, and the digital signature technology is not suitable.

In a large-scale distributed secure storage system, the positions of all nodes are equal, and an absolute authoritative server does not exist to serve as an authentication center, so that public keys of other nodes cannot be obtained through certificates to carry out encryption and decryption. Thus, according to this feature of the distributed storage network, a method is adopted in which a node is randomly selected to create a user public key management file, and public keys of all users are stored. The public key management file is established at system start-up and an ID value is assigned by the underlying structured overlay network DHT, and the routing protocol sends the file to the corresponding storage node according to this ID value. The essence is that the public key management file is treated as a common data file, thus simplifying the workflow of the system.

In the following, taking the document transmission to a distributed storage system as an example, a blockchain digital signature scheme adapting to a point-to-point network environment is proposed. Let a be the user uploading the file, the public key, private key pair (ka 1, ka 2) and B be the server selected by the DHT to receive the document. The system has a public key management file which stores the public keys of all users including user A, and the private key of user A is stored by the user A. In order to realize the function of finding whether user data is tampered or not and considering the variable size of signed content, the scheme directly signs original text content, has large operation content and long time, so that a message abstract is firstly generated for the original text by using an SHA-1 algorithm, then the abstract is signed, and finally the signature and the data are uploaded as a whole.

The blockchain digital signature process when user a uploads a file is shown in fig. 6, which can be described as:

(1) Generating a summary Z1 for the data block P by using an SHA-1 algorithm;

(2) The user A uses the private key Ka1 of the user A to carry out digital signature on the abstract Z1 by using an RSA algorithm to obtain a signature C;

(3) Combining the data block P and the signature C into a new data block PC;

(4) The DHT assigns an ID to this new data block and routes it to the server where it is stored for storage.

When the user A needs to download own files, the DHT finds all the required data blocks, a server storing the data blocks finds a public key management file through the DHT and downloads the public key management file after receiving an instruction for downloading a certain data block, the public key of the user A is found in the file, then the RSA signature C is decrypted by the public key to recover a digest Z1, then the data block P is subjected to a digest Z2 again by using an SHA-1 algorithm, the Z1 and the Z2 are compared, if the data blocks are found to be unequal, the data block P is tampered, and the user can be notified and the downloading of the data block can be stopped.

The verification process when user a downloads the file is as shown in fig. 7: the process can be described as:

(1) After receiving the downloading instruction, the server separates a data block P and a signature C from the data block PC;

(2) Calculating a summary Z2 of the data block P by using an SHA-1 algorithm;

(3) Downloading the public key to a public key management file through the DHT, and taking out the public key of the user A from the public key management file;

(4) Decrypting the signature C by using the public key to decrypt the abstract Z1;

(5) Comparing Z1 with Z2, if not, informing the user A that the data block P is tampered, and stopping downloading the data block P; if equal, the data block is passed to user A.

(6) The node A and the node B generate respective public keys and private keys before communication, the public keys can be disclosed outwards, but the private keys cannot be leaked to a third party.

(7) Node a connects node B and exchanges public keys with each other.

(8) Node a encrypts its own private key to node B using node B's public key and node B similarly encrypts its own private key to node a using node a's public key.

(9) At this time, the nodes of both sides obtain the private key of the opposite side, and the symmetric key K can be generated by multiplying the private key of the opposite side with the private key of the opposite side.

(10) After the downloading is completed, the block chain is used for verification, the current owner signs a digital signature for the former transaction and the next owner by using the private key K, and the signature is attached to the end to make a log record. When a new download is generated, it is broadcast to other participating nodes in the blockchain network.

(11) Propagation of all blockchain transactions, the current owner broadcasts a transaction ticket to the whole network, each node gathers several unverified transaction Hash values into blocks, each of which may contain hundreds or thousands of downloads. The fastest completing node will propagate its own block to other nodes.

(12) The download amount proves that each node obtains the right of creating a new block through a workload proving mechanism equivalent to solving a mathematical problem. Each node performs a calculation of the proof of workload to determine who can verify the transaction, and the node that calculates the result fastest verifies it.

(13) When one node finds a cut, it broadcasts all the time-stamped transactions recorded by the block to the whole network, and the other nodes check the time-stamped transactions by the other nodes of the whole network, and the other nodes can confirm whether the download contained in the block is valid or not, and accept the block after confirming that the download is not repeatedly spent and has valid digital signature, and the block is formally connected to the block chain at the moment and cannot tamper with the data.

(14) The blockchain records, other nodes of the whole network check the correctness of the block accounting, and after no error, the nodes compete for the next block after the legal block, so that a legal accounting block is formed. Once all nodes accept the block, the block which has not been calculated before is invalid, each node can reestablish one block, and the next node calculation work is continued.

(15) The storage log will record each file upload and download call request. Under the condition that the desensitization abnormality occurs to the storage log file and the data is wrong, the block chain mechanism can be utilized for tracing, and the use record is inquired.

System verification process

The background program runs on 5 servers respectively, the servers are in a local area network, 5 nodes run on each server, and 25 nodes are simulated in total; the foreground interface runs on the PC. The configuration of the server and PC is as follows:

(1) Server configuration

Hardware: langchao English letter server and 8G memory

Software: red Hat Enterprise Linux 7.2 operating System, JDK 8

(2) PC configuration

Hardware: i5-8250U CPU 1.80GHz,16G memory, 1000M network card

Software: windows 10, JDK 8

In order to ensure the safety of data in a large-scale distributed storage system, a blockchain digital signature technology and an SSL secure transmission technology are introduced to strengthen the safety of the system, but the addition of the technologies inevitably has a certain influence on the performance of the system. In order to quantify this effect and provide a basis for future improvement of the system, the following performance comparison analysis was performed:

and cutting the system function to obtain a system which does not comprise the blockchain digital signature and SSL and a system which does not comprise the blockchain digital signature and SSL. And comparing and analyzing the file transmission performance of the three systems with that of a normal system.

We test the transmission time of files with different sizes under the system of three sets of cutting functions and the normal system respectively, and the file size is from 3M to 100M. The test results are shown in fig. 8.

As can be seen from fig. 8, the performance of the system having only the blockchain digital signature function is relatively close to that of the normal system, and the system having only SSL and the system having no blockchain digital signature function are relatively close to each other, so it can be concluded that the blockchain digital signature function has the greatest effect on the system performance. It has also been found that as files become larger, the performance gap increases.

The reason for this is analyzed, firstly, the most time-consuming process of the SSL module is a handshake process, but the process is only performed once when a connection is established between nodes, and only data is encrypted without handshake authentication again when a file is transmitted, and the SSL adopts a DES encryption algorithm for encrypting the data, which belongs to a symmetric key algorithm and is faster than the blockchain technology adopted by digital signature. With the increase of the file volume, the data redundancy module divides the file into more blocks, so that the encryption work of the blockchain digital signature and SSL is more and more, and the larger the file volume is, the larger the difference between the performance of the file and the performance of a system without the function is. Although the addition of the blockchain digital signature and the SSL module has a certain influence on the data transmission performance of the system, the system also provides a powerful guarantee for the security of user data in the transmission and storage processes. It is also worth the improvement of the safety performance by replacing part of the performance loss.

The foregoing description is only illustrative of the preferred embodiments of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A large-scale distributed secure storage system based on block chain is characterized in that,

comprising the following steps:

the coverage and routing module adopts a path routing protocol;

The communication module adopts SSL safety protocol;

the block chain digital signature module is used for uniformly storing public keys of all users using the distributed secure storage system and randomly distributing storage positions of the public keys by the DHT;

a data query module; returning all data information meeting the query conditions according to the data description information input by the user; the user must select a file download or delete from the list of files returned by the query module.

2. The storage system of claim 1, wherein the memory is configured to store the data,

the data organization module is divided into the following five parts

(1) Registration

(2) Login

(3) Storage of

(4) Downloading

the download execution process is as follows: firstly, obtaining a version ID of a file from a file information list, obtaining a data ID of the file from the version ID, obtaining all IDs of file blocks from the data ID, interacting with a background, downloading data blocks corresponding to all IDs, merging the data blocks into a file according to the sequence of file segmentation, downloading the file to a designated position according to a designated file name, and completing downloading;

(5) Deletion of

Deletion is also the deletion of files on a query basis.

3. The storage system of claim 1, wherein the memory is configured to store the data,

the deleting process is as follows: the system obtains the file name and ID of the file to be deleted from the file information list, searches the version block file of the file according to the ID, asks for the version block file from the storage server if the version block file is not local, obtains the dirID and the data ID of the file from the version block file, and simultaneously informs the storage server of deleting the version block file; after the file directory block is obtained, corresponding file information in the file is deleted from the file directory block, and the directory block file is updated; obtaining a data file corresponding to the data ID, firstly obtaining all file block IDs in the file, informing a storage server to delete all file block IDs, and then deleting the file; all updated files are updated on the network, and the files to be deleted are also notified to the storage server to be deleted.

4. The storage system of claim 1, wherein the memory is configured to store the data,

the data query module establishes four B's locally at the user ⁺ The tree respectively stores file name information, file type information, file size and file creation date; when the user exits the system, these four B pieces are taken ⁺ The tree is encrypted and written into a file, and then uploaded to a point-to-point networkThe platform server stores the data; when the user logs in, the file is downloaded to the local of the user and the four B are sequentially downloaded ⁺ The tree is read from the file and decrypted; thus, the information of the file held by the user is restored.

5. The storage system of claim 4, wherein the memory is configured to store the data for the plurality of data,

when a user wants to insert or delete a file, the four B's are aligned ⁺ The tree performs an insert or delete operation, respectively.

6. The storage system of claim 5, wherein the memory is configured to store the data for the plurality of data,

when the user needs to inquire the file, the related file information needs to be input, and the system respectively sends the information to four B's according to the information ⁺ Searching in the tree, and finally returning the intersection of the searching results of the four trees to the user as a final result.

7. The storage system of claim 6, wherein the memory is configured to store the data for the plurality of data,

The operation is as follows:

data preparation

The system provides storage services in the form of files, encodes user files into blocks and places them on different nodes, and in addition, the system supports users to view previous versions of data; designing a data organization structure, wherein the structure consists of a root data block, a directory block, a version block and a data block; each user has a root data block, and the root data block contains the file catalogue, the shared file catalogue and the signed personal information owned by the user; the directory block contains pointers to the latest version block of each data file, and the version block contains more than one pointer to the actual data block; the version block also has a backward pointer back pointer to the previous version, and more than one version share the same data block; the pointer is assigned an ID by the DHT, and the DHT can find a block as long as the ID of the block is obtained;

(1) Root data block

(2) Directory block

The directory block comprises a directory block ID, a user name, a root data block ID, the number of files, the total size of the files, a file name, the latest version block ID of the files and a digital signature; each user has two directory blocks: a user file directory block and a shared file directory block; when a user uploads a file, if the file is selected not to be shared, the version block of the file is associated with the file directory block of the user, otherwise, the version block of the file is associated with the shared file directory block; the function that the user can check the shared file of the friend is realized by the shared file directory block;

(3) Version block

(4) Data block

8. The storage system of claim 7, wherein the memory is configured to store the data,

model development

Randomly selecting a node to establish a user public key management file and storing public keys of all users; the public key management file is established when the system is started, an ID value is distributed by the structural overlay network DHT of the bottom layer, and the routing protocol sends the file to the corresponding storage node according to the ID value;

<PKadmin_Block>

<Name>username

</Name>

</PKadmin_Block>

the usernames after the Name label represents the user Name, the content in the Modulus label is the coefficient required by calculating the key, the content in the public label is the index required by calculating the key, and the public key of the user can be calculated through the two parameter systems; when the user registers, the system finds the public key management file through the DHT, downloads the public key management file to the local, fills the contents of the Name, modulus, publicExponent tags, and uploads the contents back to the server in which the contents are stored.

9. The storage system of claim 8, wherein the memory is configured to store the data,

system verification process

The background program runs on 5 servers respectively, the servers are in a local area network, 5 nodes run on each server, and 25 nodes are simulated in total; the foreground interface runs on the PC; clipping the system function to obtain a system which does not comprise the blockchain digital signature and SSL and a system which does not comprise the blockchain digital signature and SSL; and the transmission time of the files with different sizes under a system with three sets of cutting functions and a normal system is respectively tested, the file size is from 3M to 100M, and the influence of the blockchain digital signature function on the system performance is the greatest.