KR102598227B1 - Data management method for guaranteeing the integrity of large capacity content using distributed storage with immutability and blockchain - Google Patents

Abstract

The present invention relates to a data management method that can ensure the integrity of large amounts of data. This data management method is a process of connecting a client to a preset blockchain system to enable sending and receiving blockchain transactions, a process of the client creating a first transaction for storing the first file, and the first transaction satisfying the guarantee policy. And, when predetermined block creation conditions are met, the process of generating a block containing the first transaction, the process of propagating the created block to each peer node constituting the blockchain system and checking its validity, the first transaction If this is valid, the process of storing the first file in a storage device that provides data immutability by the called smart contract, extracting metadata for the first file, and storing the extracted metadata in the ledger of the blockchain system. Includes a saving process. According to the present invention, the processing speed can be improved while ensuring the integrity of large amounts of data by utilizing blockchain technology and distributed storage technology.

Description

Data management method for guaranteeing the integrity of large capacity content using distributed storage with immutability and blockchain}

The present invention relates to a data management method that can guarantee the integrity of large amounts of data such as video content using distributed storage technology and blockchain technology that provides an immutable data function.

Blockchain is a distributed data storage technology that contains data in blocks and connects them in the form of a chain, and multiple nodes forming a blockchain network each replicate and store data in a distributed ledger at the same time for each transaction. All transaction participants can share information and compare it to prevent data forgery and falsification.

In this way, blockchain technology can inherently provide a data integrity guarantee function and can also overcome the security limitations of centralized data storage.

However, when ensuring the integrity of large amounts of data, such as video content, is processed using blockchain technology, the storage capacity required for each node becomes enormous, transaction processing time is also relatively long, and in the event of a node failure, the ledger of the replaced node is stored. During the recovery process, it is not easy to recover large amounts of data from other nodes through the network.

Accordingly, in areas where integrity management of large amounts of data is required, it is necessary to consider ways to simultaneously utilize distributed storage technology and blockchain technology to provide high processing speed and perfect data integrity.

Therefore, the purpose of the present invention is to provide a data management method that can improve processing speed while ensuring the integrity of large data by utilizing blockchain technology and distributed storage technology.

The data management method according to the present invention for achieving the above purpose includes connecting a client to a preset blockchain system so that it can transmit and receive blockchain transactions, and generating a first transaction for the client to store the first file. Step, if the first transaction satisfies the guarantee policy and predetermined block creation conditions are met, generating a block including the first transaction, each peer constituting the blockchain system Propagating to a node and validating the first transaction; if the first transaction is valid, storing the first file by a called smart contract on a storage device that provides data immutability; and It includes extracting metadata about the file and storing the extracted metadata in the ledger of the blockchain system.

The storage device may be a distributed storage device that is dynamically expandable by a scale-out method, and the ledger may be stored in the distributed storage.

Additionally, a transaction for storing the first file can be created in a decentralized client application running on the client.

Additionally, whether the first transaction satisfies the guarantee policy can be determined by sending a transaction proposal message for the first transaction from the client application to the endorsing peer node, and simulating the transaction at the endoring peer node. When a response message is transmitted to the client application, it can be checked whether the guarantee policy is satisfied based on the proposal response message.

The validation step includes a committing peer node receiving the block from an ordering service node, and the committing peer node checking whether the transactions included in the block satisfy the guarantee policy, then validating each transaction. It may include checking and marking validated transactions as valid, and marking unvalidated transactions as invalid.

In addition, the client generates a second transaction for withdrawing the first file, and when the second transaction satisfies a guarantee policy and a predetermined block creation condition is met, a block containing the second transaction is generated. A step of propagating the generated block to each peer node constituting the blockchain system and checking its validity, and if the second transaction is valid, metadata stored in the ledger by the called smart contract The method may further include searching for a storage location of the first file with reference to and retrieving the first file from the searched storage location.

In order to achieve the above object, the present invention can provide a processor-readable recording medium on which a program for executing the data management method on the processor is recorded.

According to the present invention, instead of storing the entire file in the blockchain ledger, only small-sized metadata is stored in the ledger, and the entire file is stored in a storage device that provides an immutable data function, thereby doubly preventing data forgery and falsification. In addition, the processing speed of blockchain transactions can be improved by reducing the storage capacity in the blockchain ledger. In addition, the storage device has scalability in a scale-out manner and can effectively respond to stored data whose size can continuously increase. Additionally, even if a failure occurs in the nodes that make up the blockchain system, the recovery process can be performed quickly.

Figure 1 is a diagram to explain a general blockchain structure,
Figure 2 is a diagram showing the block structure of Hyperledger Fabric;
3 is a diagram referenced to explain an example of a system to which the data management method according to the present invention is applied;
4 and 5 are flowcharts provided to explain the file storage process in the data management method according to an embodiment of the present invention;
6 and 7 are flowcharts provided to explain the file retrieval process in the data management method according to an embodiment of the present invention, and
Figure 8 is a flowchart provided to explain the failure recovery process of a peer node in the data management method according to an embodiment of the present invention.

In this specification, when a component is referred to as being “connected” or “connected” to another component, the component may be directly connected or connected to the other component, but may be connected to another component in the middle. It should be understood that elements may exist. In addition, other expressions that describe the relationship between components, such as “between” or “neighboring to,” and expressions such as “one component transmits” a signal to another component, should be interpreted similarly. do.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

Figure 1 is a diagram to explain a general blockchain structure.

Referring to Figure 1, a blockchain block consists of a header and a body. The header contains information to create a linked list in a single block, and the body contains the contents of the transactions contained in the block. In the case of Bitcoin, the header may include information such as software version, difficulty, hash value of the previous block, block creation time, Merkle root, and nonce, and the body may contain at least one transaction. You can.

In order to create a block, a process of finding the nonce value is required, and the hash value for the previous block is used to calculate the nonce value of the new block. The process of finding the value of this nonce is expressed as mining in Bitcoin, and is also called a ‘consensus algorithm’. This enables the function of a distributed ledger that allows participants on a blockchain network to hold the same ledger in real time and connects blocks to each other.

The stability of the blockchain is created through the expansion of the chain length through linked lists and the identical ledger held by blockchain network participants. As the length of the chain increases, hacking a previously existing block becomes closer to impossible, and since all nodes hold the transaction data of newly created blocks through a distributed ledger, they do not have the computing power to modify more than half of them. Otherwise, manipulation of blockchain data is impossible.

In terms of its usability, blockchain comes in the form of public blockchain, private blockchain, and consortium blockchain. In the case of a public blockchain, anyone can participate and anyone can view information. Private blockchains can apply limited functions depending on the purpose, such as restrictions on participants and scope of viewing. In addition, the consortium blockchain is an extension of the private blockchain and can provide reliability and scalability by connecting heterogeneous blocks created in different blockchains through a process called ordering by combining different private blockchains.

Figure 2 shows the Hyperledger Fabric block structure.

Hyperledger Fabric is a modular architecture platform for developing blockchain solutions and applications. Hyperledger Fabric was developed as one of Hyperledger's projects, and Hyperledger is a global open source collaborative activity organized for the development of cross-industry blockchain technology under the leadership of the Linux Foundation, with various technical steering committees and organizations. Managed by maintainers.

Referring to Figure 2, the block structure of Hyperledger Fabric protects past transaction information from being damaged by storing the block structure using an encrypted hash function. In addition, the block structure of Hyperledger Fabric is almost the same as that of Bitcoin, but it is unique in that the KVS (Key-Value Store) hash value is also recorded.

The hash value of the previous block is calculated using SHA3 SHAKE256 at 512 bits (64 bytes), and the metadata of the block can be freely entered at each node and is excluded from the hash calculation.

In general, a smart contract refers to a technology in which transactions are automatically concluded when certain conditions are met between participants on a blockchain platform. In Hyperledger Fabric, chain code is a technology that specifies smart contract policies as code. It is a component of purpose and is used in almost the same sense as a smart contract.

In the present invention, a blockchain system based on Hyperledger Fabric is basically used to guarantee data integrity.

Figure 3 is a diagram referenced to explain an example of a system to which the data management method according to the present invention is applied.

Referring to FIG. 3, the system may include a client 100, a blockchain system 200, and a storage device 300.

The client 100 can be connected to the blockchain system 200, create a client transaction, and call the transaction through it. The client 100 can connect to a desired peer node among the first to nth peer nodes 210a to 210n.

The blockchain system 200 is basically a system based on Hyperledger Fabric, but is not necessarily limited thereto.

In the blockchain system 200, the first to nth peer nodes 210a to 210n maintain the blockchain system 200, process transactions, manage the ledger and chain code, and Save.

In the case of Hyperledger Fabric, the ledger consists of a blockchain that stores immutable ordered records in blocks and a state database that maintains the current state, and the ledger is stored in a storage called world state.

Among the first to nth peer nodes 210a to 210n, the endorsing peer node first performs and reviews the requested transaction according to the endorsement policy and attaches an endorsement signature. In other words, the endorsing peer node is a peer node that performs the role of verifying transactions, simulates transactions to be performed by smart contracts, and returns the results to the client application.

Among the first to nth peer nodes (210a to 210n), the committing peer node confirms the transaction and updates the contents to the blockchain if there is no verification problem in the transaction result executed by the endorsing peer node.

In addition, the anchor peer node performs a representative role within the channel, and the leader peer node performs a representative role within the organization.

The ordering service node 220 owns configuration information for the channel (C) 250 and performs the role of manager of the entire system based on this. The ordering service node 220 delivers the transaction to the peer node 210 and performs branching and sorting duties for the mode peer nodes 210 participating in the network.

The channel 250 performs the role of binding different nodes. One channel 250 has one ledger, and all peer nodes 210 within the channel have a copy of the same ledger.

The storage device 300 stores data and provides a function to prevent unauthorized deletion or alteration of the stored data.

Additionally, the storage device 300 has scalability through a scale-out distribution method. The scale-out method refers to a method of expansion by adding equipment. In other words, if there is a limit to capacity or performance with the existing device, you can connect a device with similar specifications to not only increase capacity by the additional amount, but also increase performance by sharing the workload. Users recognize the storage cluster as one system through a distributed file system or global namespace. Due to the scalability of the storage device 300, even if the data managed through the blockchain system 200 increases, it can be dynamically accommodated.

4 and 5 are flowcharts provided to explain the file storage process in the data management method according to an embodiment of the present invention.

Referring to FIG. 4, the client application, which is a decentralized application (DApp) running on the client 100, is connected to the blockchain system 200 and generates a transaction for file storage, A transaction proposal message is transmitted to one or more endorsing peer nodes among the n-th peer nodes (210a to 210n) (S400). The transaction proposal message includes the client ID, the chaincode ID to which the transaction belongs or originates, the payload containing the transaction, and the client signature.

The endorsing peer node that receives the transaction proposal message checks whether the client signature is valid, checks the client's authority to determine whether the submitter is authorized to perform the proposed action on the channel, and then executes the transaction. After simulating and checking the validity of the transaction, the result is sent to the client application as a proposal response message (S405).

The client application checks the received proposal response message to check whether it satisfies the preset guarantee policy (S410), and if it satisfies the guarantee policy, transmits the generated transaction to the ordering service node 220 (S415, S420) ). Guarantee policy refers to the conditions that guarantee a transaction. If, as a result of the judgment in step S415, the guarantee policy is not satisfied, the created transaction is abandoned (S425). When a transaction is abandoned, an option to retry later can be used.

The ordering service node 220 collects transactions received from the client application in order, checks whether they satisfy preset block creation conditions, such as a certain time or a certain number of transactions (S430), and determines whether the block generation conditions are satisfied. In case (S425), the process shown in FIG. 5 is performed. If the block setting conditions set in step S435 are not satisfied, the transaction processing failure is notified to the client application (S440).

Referring to FIG. 5, when the block creation conditions are confirmed to be satisfied, the ordering service node 220 creates a block containing the received transaction and sends the block to all committing peer nodes and endorsing peer nodes belonging to the same channel. Transmit (S445).

The committing peer node checks whether the transactions included in the received block satisfy the guarantee policy, then checks the validity of each transaction included in the block, records the validated transactions as valid, and validates them. If it fails, the transaction is recorded as invalid (S450). Transaction validation marks data read during chaincode execution as valid if it has not changed since execution or approval, and can be applied to the ledger state database. However, if the data read during chaincode execution is changed by another transaction, that transaction is marked as invalid and is not applied to the ledger state database.

When the validity of the transaction created to save the file is verified (S455), the storage target file is stored in the storage device 300 by execution of the called smart contract (S460), and the stored file is stored in the storage device 300. Extract metadata (S465). Then, the extracted metadata is stored in the ledger (S470). If the transaction validity is not verified in step S455, the transaction processing failure is notified to the client application (S440).

When the ledger is updated, the client application is notified of successful transaction processing (S475), and the transaction processing process for file storage ends.

Through this process, files are stored in the storage device 300 where data cannot be changed, and metadata for the stored files are recorded in the ledger of the blockchain system 200 to prevent forgery and alteration of the metadata, thereby preventing the stored files from being altered. Double protection is possible. Additionally, by storing only small-sized metadata in the ledger instead of storing the entire file in the ledger, the processing speed of blockchain transactions can be improved by reducing the storage capacity in the blockchain distributed ledger.

In addition, the blockchain ledger is generally stored in each peer node, but the ledger can also be configured to store a storage device 300 in which data cannot be changed, in which case the integrity of the data stored in the ledger is guaranteed and unauthorized deletion is prevented. can do.

Figures 6 and 7 are flowcharts provided to explain the file retrieval process in the data management method according to an embodiment of the present invention.

Referring to FIG. 6, the client application, which is a decentralized application (DApp; Decentralized Application) running on the client 100, is connected to the blockchain system 200 and generates a transaction for file withdrawal, A transaction proposal message is transmitted to one or more endorsing peer nodes among the peer nodes 210a to 210n (S500).

The endorsing peer node that receives the transaction proposal message checks whether the client signature is valid, checks the client's authority to determine whether the submitter is authorized to perform the proposed action on the channel, and then executes the transaction. After simulating and checking the validity of the transaction, a proposal response message is transmitted to the client application (S505).

The client application checks the proposal response message received from the endoring peer node to check whether it satisfies the preset guarantee policy (S510), and if it satisfies the guarantee policy, transmits the generated transaction to the ordering service node 220. Do it (S520). If the guarantee policy is not satisfied, the created transaction is abandoned (S525).

The ordering service node 220 collects transactions received from the client application in order, checks whether they satisfy preset block creation conditions, such as a certain time or a certain number of transactions (S530), and determines whether the block generation conditions are satisfied. In case (S535), the process shown in FIG. 7 is performed. If the block setting conditions set in step S535 are not satisfied, the transaction processing failure is notified to the client application (S540).

Referring to FIG. 7, when the block creation conditions are confirmed to be satisfied, the ordering service node 220 creates a block containing the received transaction and sends the block to all committing peer nodes and endorsing peer nodes belonging to the same channel. Transmit (S545).

The committing peer node checks whether the transactions included in the transmitted block satisfy the guarantee policy, then checks the validity of each transaction included in the block, records the verified transactions as valid, and confirms the validity. If it fails, the transaction is marked as invalid (S550).

When the validity of the transaction created to withdraw a file is verified (S555), the storage location of the file requested to be withdrawn is searched by referring to the metadata stored in the ledger (S560) by executing the called smart contract (S565) , the file is retrieved from the searched location of the storage device 300 (S570).

When the file is retrieved, the successful transaction processing is notified to the client application (S575), and the retrieved file can be sent to the client application.

Through this process, files stored in the storage device 300, where data cannot be changed, can be retrieved through the blockchain system 200. In addition, when there is a search or inquiry request for files stored in the storage device 300, where data cannot be changed, and metadata stored in the ledger, files and metadata that meet the search or inquiry conditions may be searched and output. .

Figure 8 is a flowchart provided to explain the failure recovery process of a peer node in the data management method according to an embodiment of the present invention.

Referring to Figure 8, when a failure is detected among the peer nodes constituting the blockchain system (S600), the failed peer node is excluded from all channels in which the failed peer node is registered (S605).

Next, after deleting the ledger of the failed peer node (S610), the container of the failed peer node is stopped and excluded from the group membership (S615). Hyperledger Fabric utilizes container technology to provide chaincode services for configuring system logic.

If the replacement peer node replacing the failed peer node is a new node, an MSP (Membership Service Provider) of the replacement peer node is created (S620, S625). Node roles can be assigned through MSP.

Next, a container for the replacement peer node is created and assigned to the group (S630), a ledger is created in the replacement peer node, and the replacement peer node is registered in the channel to share the ledger so that the contents of the ledger can be copied. (S635).

Next, install the chaincode on the alternate peer node (S640) and run the instancite command as needed. Just because a chaincode is installed on a certain peer node does not mean it can be used immediately. To use a chaincode, it needs to be instantiated by informing other peer nodes of the chaincode's interface. Instantization can be done using the instantiate command. there is.

Next, the ledger contents of other peer nodes of the same channel are copied to the replacement peer node (S645). In this case, the large-capacity data managed by the ledger, or both the ledger and the large-capacity data managed by the ledger, are configured to be stored in the storage device 300, so that the ledger contents can be quickly copied. In addition, it can be configured to be managed by an alternative node by only changing the access rights to the ledger without directly copying the ledger contents.

Through this process, if a failure occurs in a peer node that makes up the blockchain system, recovery is achieved by replacing it with another node.

Meanwhile, the storage device 300 is a storage device with a WORM (Write Once Read Many) function added so that data cannot be changed, and can be configured as a hard disk or solid state disk (SSD).

WORM storage devices such as CD-R, DVD-R, etc. record data by physically forming patterns on the surface of the media using a laser, so data cannot be physically tampered with from the moment it is recorded. However, WORM storage based on hard disks and SSDs, which are implemented to support large capacity, high performance, and search usability, do not have a device to protect data with physical properties, so the data protection function is implemented through software methods. I do it. At this time, the software is implemented as fixed code or firmware within the operating system that provides file system services to the storage device, fundamentally blocking any way to change it from outside, preventing data from being altered or deleted by any authority. You can.

Meanwhile, the data management method according to the present invention cannot be applied limited to the configuration of the embodiments described above, but all or part of each embodiment can be selectively selected so that various modifications can be made. It may also be composed in combination.

Additionally, the present invention can be implemented as a computer program on a programmable computer. These computers may include processors, storage devices, input devices, and output devices. To implement the content described in the present invention, program code can be input with a mouse or keyboard input device. These programs can be implemented in a high-level language or an object-oriented language. It can also be implemented as a computer system implemented in assembly or machine code.

Additionally, the subject matter of the present invention is not limited to the use of hardware or software, and is applicable to any other computing or processing environment. It can be implemented with hardware, software, or a combination of hardware and software described in the present invention. The invention may be implemented using circuitry. That is, it can be implemented with one or more programmable logic circuits, that is, application specific integrated circuits (ASICs) or logic circuits (AND, OR NAND gates), or processing devices (eg, microprocessors, controllers).

The present invention can also be implemented as processor-readable code on a processor-readable recording medium. Processor-readable recording media includes all types of recording devices that store data that can be read by a processor. Examples of recording media that can be read by a processor include ROM, RAM, CD-ROM, and optical data storage devices. In addition, the processor-readable recording medium is distributed in a computer system connected to a network, so that the processor-readable code can be stored and executed in a distributed manner.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

100: Client 200: Blockchain system
300: storage device

Claims (10)

A step of connecting with a preset blockchain system so that the client can send and receive blockchain transactions;
the client generating a first transaction for storing a first file;
If the first transaction satisfies the guarantee policy, and if at least one block creation condition is met among the collection of other transactions during a preset time and the collection of a preset number of other transactions, a block containing the first transaction is created. generating step;
Propagating the generated block to each peer node constituting the blockchain system and checking its validity;
If the first transaction is valid, the first file is stored in a storage device that provides data immutability and is dynamically expandable distributed storage by a scale-out method by the called smart contract. step; and
A data management method comprising extracting metadata for the first file and storing the extracted metadata in a ledger of the blockchain system. delete According to paragraph 1,
A data management method, characterized in that the ledger is stored in the storage device. According to paragraph 1,
When a failure is detected in any one of the peer nodes constituting the blockchain system, the step of replacing the failed peer node for which the failure was detected with another node, and the other node transferring the ledger of the failed peer node and synchronizing it How to manage data. According to paragraph 1,
A data management method wherein the blockchain system is a Hyperledger Fabric blockchain system. According to paragraph 1,
A data management method, characterized in that the transaction for storing the first file is created in a decentralized client application. According to clause 6,
Whether the first transaction satisfies the guarantee policy,
The client application transmits a transaction proposal message for the first transaction to the endorsing peer node,
When the endowing peer node simulates the transaction and transmits the result as a proposal response message to the client application,
A data management method characterized by checking whether the guarantee policy is satisfied based on the proposal response message. In clause 7,
The validation steps are:
A committing peer node receiving the block from an ordering service node; and
After the committing peer node checks whether the transactions included in the block satisfy the guarantee policy, the validity of each transaction is checked, and the validated transaction is recorded as valid, and the unvalidated transaction is invalid. A data management method that includes recording steps. According to paragraph 1,
generating, by the client, a second transaction to withdraw the first file;
If the second transaction satisfies an endorsement policy and the block creation condition is met, generating a block including the second transaction;
Propagating the generated block to each peer node constituting the blockchain system and checking its validity; and
If the second transaction is valid, the storage location of the first file is searched by referring to the metadata stored in the ledger by the called smart contract, and the first file is retrieved from the searched storage location of the storage device. A data management method further comprising the steps of: A processor-readable recording medium recording a program for executing the data management method of any one of claims 1 and 3 to 9 on a processor.

Images