TWI710238B - Synchronous deletion method of distributed storage system - Google Patents

Abstract

The present invention is a synchronous deletion method for a distributed storage system. After a first host accesses at least one first data, a first data list and a second data list are correspondingly built on the blockchain network to Corresponding to the data change of the first host, a second host is updated by the first data list and the second data list. In this way, the indestructible feature of the blockchain technology is used to maintain the continuous existence of the first data list and the second data list, so that the second data accessed by the second host remains the same after the first host is changed.

Description

Synchronous deletion method of distributed storage system

The present invention relates to a network data system and a data processing method thereof, in particular to a synchronous deletion method of a distributed storage system.

A distributed system is a group of computers that are connected to each other through the network to transmit information and communication and coordinate their computing behaviors to form a system as a whole. In this system architecture, computers interact with each other to achieve a common The calculation results, therefore, the engineering data that requires a large number of calculations are divided into small pieces of data, and then calculated separately by multiple computers in the distributed system, and then the dominant computer combines all the calculation results to reach a data conclusion. Examples of distributed systems come from different service-oriented architectures, massively multiplayer online games, and peer-to-peer network applications.

With the evolution of distributed systems and the cloudification of data storage methods, the distributed file system (English: Distributed file system, DFS) has been further developed, which is a data system that allows data to be shared on multiple hosts through the network. Allows multiple users on multiple machines to share data and storage space. In a distributed file system, the client does not directly access the underlying data storage section, but communicates with the server on the network through the specific communication protocol of the network, so that the client and the server can be controlled according to the access List or authorization to restrict access to the data system.

Based on the evolution of peer-to-peer technology, the InterPlanetary File System (IPFS) was developed, which is a peer-to-peer (P2P) hypermedia protocol composed of thousands of computers around the world, and is one of the technologies of distributed networks. In IPFS, this data system, data is stored in the IPFS composed of multiple network nodes on the network. The network nodes are peer-to-peer, and each data can be calculated as a unique hash ( Hash) value, through the distributed hash table (English: distributed hash table, DHT) algorithm, the calculated hash value can be calculated to locate the address of the data on many network nodes, so the IPFS concept is like a big The pile of data is scattered among multiple nodes, and the data access When using DHT algorithm to find the network node address on the network for access. The advantage of IPFS is that even if the original network node terminates its service, is hacked, or the server encounters a natural disaster, users can still find what the user needs from different network nodes.

With the popularity of blockchain, distributed file systems are also developed to match blockchain technology. In particular, the decentralized peer-to-peer P2P architecture concept of IPFS is widely used, but the characteristics of blockchain systems are usually written After entering, it cannot be tampered with and "deleted", but in real applications, the topic of deletion is often needed, and the confirmation of deleted data is also one of the problems of the distributed file system. After the data is distributed and stored, it is difficult to be like traditional centralized The distributed file system confirms each step of access, especially when the data transmission medium of the distributed file system is the network transmission interface, and the network transmission data tampering is difficult to prevent, the distributed file system is based on a decentralized architecture , It is difficult to confirm whether the data of each system node is deleted or not. In addition, many current blockchain-related application systems will be equipped with "distributed storage systems" (such as IPFS, Ethereum Swarm) to store large amounts of data. However, such storage systems have decentralized characteristics, so most of them cannot have "delete data". "Function.

Nowadays, in order to protect the data in the distributed storage system, a variety of protection mechanisms have been developed. For example, the Blockcerts Wallet jointly developed by the Massachusetts Institute of Technology (MIT) Media Lab and the industry for digital diplomas can provide a revocation function for file revocation. It mainly uses the "revocation list" as the core to carry out the certificate revocation function, not to delete the digital file itself. In addition, the data replication between each node in the distributed storage system is also based on the network transmission interface as the data transmission medium. It is difficult to confirm whether the data has been copied.

An object of the present invention is to provide a synchronous deletion method for a distributed storage system, which provides a data update function for the hosts on each network node of the distributed storage system to ensure that the hosts on each network node execute data correctly Sync and delete.

An object of the present invention is to provide a synchronous deletion method for a distributed storage system, which provides a data deletion function for each host of the distributed storage system to ensure that each host executes data deletion correctly.

In order to achieve the above objective, an embodiment of the present invention discloses a synchronous deletion method of a distributed storage system, wherein a first host stores a plurality of first data and a second host stores a plurality of second data, the first host The first original list and the second original list are respectively stored with the second host, and correspond to an index data list of a blockchain content. The first host first adds or deletes at least one transaction data according to a first original list to generate A first update list; then the first host updates the index data list of the blockchain content according to the first update list; updates the index data list according to the first update list and generates a change data list; and, the second host The second original list is compared according to the change data list to delete at least one second change data of the second data. In this way, the user can use the change data list to allow the second host to delete the data, thereby avoiding the inconsistency of the data stored by the first host and the second host.

The present invention provides an embodiment, wherein the first host and the second host are set in the same network domain or different network domains.

The present invention provides an embodiment, wherein in the step of generating a first update list and uploading it to a blockchain, the first host further generates a first verification data and uploads it to an audit control unit, and further Performing providing a second verification data to the audit control unit; and the audit control unit audits the second host according to the first verification data and the second verification data.

The present invention provides an embodiment in which the audit control unit further audits the second host according to a data period data.

The present invention provides an embodiment that further includes comparing the second original list according to the second update list, and when the second original list is different from the second update list, the second host correspondingly deletes the second transaction data.

The present invention provides an embodiment in which the synchronous deletion method further includes a step in which a third host receives the broadcasted blockchain content without performing the comparison of the abnormal data list.

1: Distributed storage system

10: First host

12: The first monitoring program

14: The first storage space

20: second host

22: The second monitoring program

24: second storage space

26: Security module

30: Blockchain content

32: Index data list

34: List of transaction data

36: Execution list

38: Audit Control Unit

40: List of data deadlines

42: Audit List

50: Third host

52: The third monitoring program

54: third storage space

A: Blockchain network

NET1: the first network

NET2: second network

S1: The first safety module

S2: The second safety module

S3: Third Security Module

T1: First original list

T2: second original list

U1: First update list

U2: Second update list

V1: First verification data

V2: Second verification data

Figure 1: It is a flowchart of an embodiment of the present invention; Figure 2: It is a system schematic diagram of an embodiment of the present invention; Figure 3: It is a partial flowchart of an embodiment of the present invention; Figure 4: It is a flowchart of another embodiment of the present invention; Figure 5: It is a system diagram of another embodiment of the present invention; Figure 6: It is a partial flowchart of another embodiment of the present invention; Figure 7: It is a process flow of another embodiment of the present invention Figure; Figure eight: it is a partial flowchart of another embodiment of the present invention; and Figure ninth: it is a system schematic diagram of another embodiment of the present invention.

In order to enable your reviewer to have a further understanding and understanding of the features of the present invention and the effects achieved, the preferred embodiments and detailed descriptions are provided, as follows:

Hereinafter, various embodiments of the present invention will be described in detail through the use of drawings. However, the concept of the present invention may be embodied in many different forms, and should not be construed as being limited to the exemplary embodiments described herein.

The present invention is a synchronous deletion method for a distributed storage system, which provides a user to create a data list on a blockchain network, so that under the distributed storage system architecture, each host can maintain data consistency after synchronization.

First of all, please refer to the first figure, which is a flowchart of an embodiment of the present invention. As shown in the figure, the synchronous deletion method of the distributed storage system of the present invention includes the following steps: Step S1: Store a plurality of data as a plurality of first data of a first host and a plurality of second data of a second host Data, the first host and the second host respectively create a first original list and a second original list; step S3: update at least one first transaction data of the first data according to the first original list to generate A first update list; step S5: update an index data list according to the first update list and generate a change data list in the blockchain content to broadcast the blockchain content; and Step S7: The second host compares the second original list with the change data list to update at least one second change data of the second data.

The following describes the distributed storage system in order to achieve the present invention, please refer to the second figure, which is a system diagram of an embodiment of the present invention, as shown in the second figure; the distributed storage system 1 of the present invention includes a first Host 10, a second host 20, and a blockchain content 30. In addition, the distributed storage system 1 of the present invention may further include a first network NET1, which is an internal network, and the first host 10 and the second host 20 are set in the first network NET1. Among them, the first host 10 has a first monitoring program 12 and a first storage space 14, the second host 20 has a second monitoring program 22 and a second storage space 24, the first host 10 and the second host 20 are respectively There is a first security module S1 and a second security module S2. The first monitoring program 12 is built in the first security module S1, and the second monitoring program 22 is built in the second security module S2. Blockchain The content 30 has an index data list 32 and a transaction data list 34. The first security module S1 and the second security module S2 make the first monitoring program 12 and the second monitoring program 22 non-mutable.

In addition, the blockchain content 30 further has an execution list 36. The first host 10 and the second host 20 are respectively a network node (not shown in the figure) of a blockchain network A to create the blockchain content 30. The blockchain network A is a private chain network, a consortium chain network or a public chain network. The first host 10 and the second host 20 respectively have corresponding network nodes in the blockchain network A, and each has its own exclusive and unique in the first security module S1 and the second security module S2 Private key (Privacy Key).

As shown in step S1, store a plurality of data in the first host 10 as a plurality of first data in the first storage space 14, and store the data in the second host 20 as second data in the second storage space 24. Let the first host 10 and the second host 20 respectively establish a first original list T1 and a second original list T2, and the first original list T1 and the second original list T2 correspond to the block chain content 30 Index data list 32; as shown in step S3, in this embodiment, the first host 10 performs a change with respect to the first data stored in the first storage space 14, for example: adding a file data to the first storage space 14 or One of the data in the first storage space 14 is deleted. Therefore, the first host 10 generates a first update list U1 from its first original list T1 based on the changed data. The first original list T1 corresponds to the first host 10 All the original first data, the first update list U1 corresponds to all the first hosts 10 after the transaction Data, that is, the unchanged data and the changed data of the first data, the first monitoring program 12 then uploads the first update list U1 to the blockchain content on the corresponding network node due to the data change of the first host 10 30. The first monitoring program 12 can further upload a corresponding one of the first verification data V1 to the corresponding node of the blockchain network 30. As shown in step S5, the block chain content 30 is equivalent to the index data list 32 of the block chain content 30 updated due to the first update list U1 through the first monitoring program 12, and is based on the data changed in step S3 and the first The update list U1 generates the transaction data list 34 in the blockchain content 30, so the first host 10 broadcasts the blockchain content 30 through the corresponding network node on the blockchain network A, thereby allowing the users on the same blockchain Other network nodes also have the updated blockchain content 30. Therefore, other networks obtain the updated blockchain content 30, that is, the index data list 32 corresponds to the first storage space 14 of the first host 10 Of all the data stored after the update, the changed data list 34 corresponds to the changed data in the first storage space 14 of the first host 10. As shown in step S7, because the second host 20 corresponds to one of the network nodes of the blockchain network A, and the second monitoring program 22 of the second host 20 monitors the data in the second storage space 24, so The second host 20 obtains the updated blockchain content 30 at the corresponding network node, and compares the second original list T2 with the index data list 32 and the transaction data list 34 of the blockchain content 30 to generate a second update List U2, the second host 20 compares the data in the second storage space 24 with the second original data list T2 and the second update list U2 to delete the data, for example: according to the second original data list T2 and the second update The comparison of the list U2 deletes the corresponding data, wherein the second monitoring program 22 uploads a second verification information V2 to the blockchain content 30 according to the deletion of the data in the second storage space 24.

As shown in the third figure, step S7 further includes: step S72: determining whether to delete the data; step S74: continuing to monitor; and step S76: deleting the data.

As shown in step S72, the second monitoring program 22 monitors all data in the second storage space 24 of the second host 20 for the second original list T2 and the second updated list U2, and determines whether the second storage space 24 deletes the data. When the second update list U2 is not different from the second original list T2, step S74 is executed. When the second update list U2 is different from the second original list T2 and data needs to be deleted, step S76 is executed. As shown in step S74, when the second host 20 does not need to add data, the second host 20 continues to monitor the data stored in the second storage space 24. As shown in step S76, when the second update list U2 is different from the second original list T2 when deleting data, since the second update list U2 lists deleted data, the second host 20 also deletes the corresponding data in the second storage space 24的信息。 Information.

In addition, any host with monitoring authority determines the data update status of the second host 20 according to the execution list 36 of the blockchain content 30, and further determines whether the first host 10 is continuously updated by the execution list 36 at the same time, so as to pass the first monitoring The program 12 retrieves the data list in the first storage space 14. In addition, the transaction data list 34 can also be configured with multiple digitally signed documents. When the transaction data list 34 lists deleted data, the blockchain network 30 will not list the deleted data of the incomplete digitally signed documents in the transaction data list 34. Therefore, Let the second monitoring program 22 not delete the corresponding data.

The flow of the synchronous deletion method of the distributed storage system of another embodiment of the present invention will be described below. Please refer to the fourth figure. The step flow of the present invention includes the following: Step S11: Store a plurality of data as a first host A plurality of first data and a plurality of second data of a second host, the first host and the second host respectively establish a first original list and a second original list; step S13: update according to the first original list At least one first change data of the first data generates a first update list; and step S15: update an index data list according to the first update list and generate a change data list in the blockchain content, To broadcast the blockchain content; Step S17: The second host compares the second original list according to the list of transaction data to delete at least one second transaction data of the second data; Step S19: Provide a second The verification data is sent to the audit control unit; and step S21: the audit control unit audits the second host according to the first verification data and the second verification data.

Please refer to the fifth figure. The difference between the second figure and the fifth figure is that the second host 20 is further provided with a security module 26, and the blockchain content 30 is further provided with an audit control unit 38 and a data expiration list 40. Furthermore, the first host 10 and the second host 20 are respectively located in the first network NET1 and the second network NET2. Therefore, for the first host 10, the second host 20 is located on the external network. In the blockchain network A, any host can obtain the audit authority, and the audit control unit 38 is executed to audit the second host 20 The second storage space 24.

In step S11 to step S17, the execution method is the same as the data deletion technique described in step S1 to step S7 above, so it will not be repeated here. As shown in step S13, add or delete the first storage in the first host 10. When the space 14 changes data, the first monitoring program 12 of the first host 10 will generate the first verification data V1 and synchronize it to the blockchain content 30. As shown in step S19, in step S17, except for the second monitoring program 22 to follow The second updated list U2 is different from the second original list T2 in that it deletes the second data stored in the second storage space 24. The second host 20 further generates a second verification data V2 and synchronizes it to the blockchain. Content 30. As shown in step S21, the audit control unit 38 audits all data in the second storage space 24 of the second host 20 according to the first authentication data V1 and the second authentication data V2. Furthermore, the audit control unit 38 further according to the index data list 32 The data period list 40 is generated with the change data list 34, and all the data stored in the second storage space 24 of the second host 20 are checked through the data period list 40 to confirm whether the second storage space 24 stores expired data.

In step S21, please also refer to the sixth figure. The synchronous deletion method of the distributed storage system of the present invention further includes in step S21: Step S212: Determine whether the data is in the latest storage state; Step S214: Continue monitoring; and Step S216: Delete data synchronously.

In step S212, the audit control unit 38 determines whether the data stored in the second storage space 24 corresponds to the data stored in the first storage space 14 and is in the latest storage state according to the first verification data V1 and the second verification data V2. When the data is in the latest state, step S214 is executed. When it is determined that one of the data is the data that has been changed, step S216 is continued; in step S214, the second monitoring program 24 will continue to monitor the second storage of the second host 20 Space 24. In step S216, after the audit control unit 38 audits the non-updated status on the second host 20, the audit control unit 38 controls to send a request message to the security module 26 to allow the security module 26 to delete data. In addition, the audit control unit 38 can further determine whether one of the data is out of date on the data expiration list 40, and the audit control unit 38 Then it sends a request message to the security module 26 to drive the security module 26 to delete the data and make the data in the second storage space 24 the latest state. For example, the second host 20 deletes all data in the second storage space 24 to The data stored in the second host 20 and the first host 10 are agreed.

In addition, the blockchain network 30 can further have an audit list 42 to access the first verification data V1 and the second verification data V2. Whenever the first host 10 completes a data deletion, the first monitoring program 12 or A security module S1 will generate the first verification data V1 to the blockchain content 30 to be stored in the audit list 42. Whenever the second host 20 completes a data deletion, the second monitoring program 22 or the second security module S2 will generate the second verification data V2 to the blockchain content 30 to be stored in the audit list 42, and the audit control unit 38 can use the audit list 42 to verify whether the data update on the audit data period list 40 or the execution list 36 is actually executed Therefore, manual audits are avoided, manual comparison operations are reduced, and data security is improved. The above-mentioned first data stored in the first storage space 14 and the second data stored in the second storage space 24 are data under the system architecture of the InterPlanetary File System (IPFS).

Please refer to Figure 7, which is a flowchart of another embodiment of the present invention. The step process of the present invention includes the following: Step S31: Store a plurality of data as a plurality of first data of a first host and a plurality of second data of a second host, the first host and the second host are created separately A first original list and a second original list; step S33: update at least one first transaction data of the first data according to the first original list to generate a first update list; and step S35: according to the first data An update list updates an index data list and generates a transaction data list in the blockchain content to broadcast the blockchain content; step S37: the second host compares the second original list according to the transaction data list, To delete at least one second transaction data of the second data; step S39: provide the second verification data to the audit control unit; and step S41: the audit control unit audit the second host according to the first verification data and the second verification data .

Steps S31 to S39 are the same as the above steps S11 to S19, so they will not be described again. The difference between step S41 and step S21 is that step S41 further includes a self-destruct function. Please also refer to the fifth figure. The audit control unit 38 of the present invention audits the second host 20 according to the first verification data V1 and the second verification data V2 on the audit list 42. When the first verification data V1 is different from the second verification data V1 When the data is V2, the audit control unit 38 sends a data deletion request message to the second host 20 to request the second security module S2 or the second monitoring program 22 therein to execute the data deletion accordingly, as shown in the eighth figure, in step S41 further includes: Step S412: Determine whether the data is in the latest storage state; Step S414: Continue to monitor; Step S416: Request to delete the data; Step S418: Whether to complete the deletion; Step S420: Continue to monitor; Step S422: Whether to delete the data has been requested 3 And step S424: drive the host offline.

Steps S412 to S416 are the same as steps S212 to S216, so they will not be repeated. In step S418, the audit control unit 38 further audits whether the second host 20 has completed the synchronization deletion of data, and when it is completed, executes step S420 , When it is not completed, perform step S422; in step S420, the second monitoring program 24 will continue to monitor the second storage space 24 of the second host 20; in step S422, it will be determined that the audit control unit 38 sends a synchronous deletion data Whether the request message has reached 3 times, when it has not reached 3 times, perform step S416, when it has reached 3 times, perform step S424; in step S424, the audit control unit 38 marks the second host 20 response in the execution list 36 Offline, therefore, the second monitoring program 22 will clear the second data stored in the second storage space 24 according to the second host 20 marked in the execution list 36 that should be offline, so that the second host 20 that should be offline stops subsequent actions , And let the second host 20 that should go offline actually go offline on the blockchain network A.

The first host 10 and the second host 20 described above each have a private key corresponding to the blockchain network 30, which is respectively carried by the first host 10 and the second host 20 to be stored in the nodes on the blockchain network 30. Obtain data, avoid data tampering, and provide confirmation of data correctness. In addition, the first The first monitoring program 12 and the second monitoring program 22 in the first security module S1 and the second security module S2 of the host 10 and the second host 20 both have a self-destruct function, which is based on the audit control unit 38 issuing a synchronous deletion After the same request message arrives at the first host 10 or the second host 20 three times, the audit control unit 38 indicates that the host should be offline on the execution list 36, so when the corresponding monitoring program is on the corresponding network node of the blockchain network A When it reads that the host should be offline, the host that should be offline is immediately deleted by the monitoring program in the security module to delete all the data in the corresponding storage space, so that the host that should be offline stops subsequent actions, and the host should be offline The host is indeed offline on blockchain network A.

Please refer to Figure 9, which is a schematic diagram of a system according to another embodiment of the present invention. The difference between the fifth figure and the ninth figure is that the ninth figure further has a third host 50, the third host 50 has a third monitoring program 52 and a third storage space 54, and the third monitoring program 52 is located in a first Three security module S3. Since the third host 50 is a host that should go offline, the third host 50 does not continue to execute the blockchain content 30 received in step S37 after the first host 10 executes step S35 to compare the transaction data list 34 Perform synchronous deletion.

In summary, the present invention is a synchronous deletion method for a distributed storage system, which provides a monitoring program as an automatic auxiliary monitoring storage space in each host, thereby achieving the following effects: 1. Provide corresponding data updates through the data list; 2. .Ensure the consistency of the IPFS content of each node; and 3. Through the automatic audit mechanism, reduce personnel comparison operations and improve security.

However, the above are only the preferred embodiments of the present invention, and are not used to limit the scope of implementation of the present invention. For example, the shapes, structures, features and spirits described in the scope of the patent application of the present invention are equally changed and modified. , Should be included in the scope of patent application of the present invention.

The invention is based on the fact that it is novel, progressive, and available for industrial use. It should meet the patent application requirements stipulated by the Chinese Patent Law. No doubt, I filed an invention patent application in accordance with the law. I hope that the Bureau will grant the patent as soon as possible. prayer.

Claims (7)

A method for synchronous deletion of a distributed storage system. The steps include: storing a plurality of data as a plurality of first data of a first host and a plurality of second data of a second host, the first host and the second host The host separately creates a first original list and a second original list, the first original list and the second original list corresponding to an index data list of a blockchain content; the first host updates according to the first original list At least one first change data of the first data, generate a first update list; update the index data list according to the first update list and generate a change data list in the blockchain content to broadcast the zone Block chain content; and the second host compares the second original list with the transaction data list to delete at least one second transaction data of the second data. Further, the at least one first transaction data corresponds to at least One second transaction data. The synchronous deletion method of the distributed storage system described in the first item of the scope of patent application, wherein the first host and the second host are set in the same network domain or different network domains. For example, in the synchronous deletion method of the distributed storage system described in the scope of patent application, in the step of generating a first update list, the first host further generates a first verification data and uploads it to an audit control unit, The steps of the synchronous deletion method further include: providing a second verification data to the audit control unit; and the audit control unit audits the second host according to the first verification data and the second verification data. For example, in the synchronous deletion method of the distributed storage system described in item 3 of the scope of patent application, in the step of the audit control unit auditing the second host according to the first authentication data and the second authentication data, the audit control unit The second host is further audited based on a data period data. The synchronous deletion method of the distributed storage system described in item 1 of the scope of patent application further includes: When the audit control unit audits the second host according to the first verification data and the second verification data and has sent the same data deletion request message three times, the audit control unit marks the second host on an execution list. Offline; and the second host deletes the second data according to the execution list. For example, the synchronous deletion method of the distributed storage system described in item 1 of the scope of patent application further includes: comparing the second original list according to the second update list, and when the second original list is different from the second update list and needs When deleting the second transaction data, the second host correspondingly deletes the second transaction data. For example, the synchronous deletion method of the distributed storage system described in item 1 of the scope of patent application further includes: a third host receives the broadcasted content of the blockchain without performing the comparison of the abnormal data list.

Images