KR20230043339A - Data management method for guaranteeing the integrity of large capacity content using distributed storage with immutability and blockchain - Google Patents

Abstract

The present invention relates to a data management method which can ensure the integrity of large amounts of data. The data management method comprises: a process of connecting with a pre-established blockchain system so that the client can send and receive blockchain transactions; a process in which a client creates a first transaction for storing a first file; a process of generating a block containing the first transaction when the first transaction satisfies the guarantee policy and predetermined block creation conditions are met; a process of propagating the created block to each peer node that makes up the blockchain system and checking its validity; a process of storing the first file in a storage device that provides data immutability by a smart contract called when the first transaction is valid; and a process of extracting metadata for the first file and storing the extracted metadata in the ledger of the blockchain system. According to the present invention, the processing speed can be improved while ensuring the integrity of large amounts of data by utilizing blockchain technology and distributed storage technology.

Description

Data management method for guaranteeing the integrity of large capacity content using distributed storage with immutability and blockchain}

The present invention relates to a data management method capable of guaranteeing the integrity of large-capacity data such as video content by using a distributed storage technology that provides a data immutability function and a blockchain technology.

Blockchain is a distributed data storage technology that has a structure that contains data in blocks and connects them in the form of a chain, and multiple nodes forming a blockchain network simultaneously replicate and store data in a distributed ledger. All transaction participants can share information and compare it to prevent data forgery.

As such, blockchain technology can inherently provide data integrity assurance functions and overcome the security limitations of centralized data storage.

However, when guaranteeing the integrity of large-capacity data such as video content is handled by blockchain technology, the storage capacity required for each node becomes enormous, the transaction processing time is relatively long, and the ledger of the replaced node in the event of a node failure In the process of recovery, it is not easy to recover large amounts of data from other nodes through the network.

Therefore, in areas where integrity management of large amounts of data is required, it is necessary to consider a method that can simultaneously utilize distributed storage technology and blockchain technology to provide high processing speed and perfect data integrity.

Accordingly, an object of the present invention is to provide a data management method capable of improving processing speed while guaranteeing the integrity of large amounts of data by utilizing blockchain technology and distributed storage technology.

A data management method according to the present invention for achieving the above object is a step of connecting with a previously set blockchain system so that a client can transmit and receive a blockchain transaction, the client generating a first transaction for storage of a first file Step, if the first transaction satisfies the guarantee policy and a predetermined block generation condition is satisfied, generating a block including the first transaction, each peer constituting the blockchain system propagating to a node to check validity, if the first transaction is valid, storing the first file in a storage device that provides data immutability by a called smart contract, and the first Extracting metadata about a file and storing the extracted metadata in a ledger of the blockchain system.

The storage device may be a distributed storage that is dynamically expandable by a scale-out method, and the ledger may be stored in the distributed storage.

Also, a transaction for storing the first file may be generated by a decentralized client application running on the client.

In addition, whether the first transaction satisfies the guarantee policy is determined by transmitting a transaction proposal message for the first transaction to the endorsing peer node in the client application and simulating the transaction at the endorsing peer node. When a response message is transmitted to the client application, whether the guarantee policy is satisfied may be checked based on the proposal response message.

The step of checking validity may include receiving the block from an ordering service node by a committing peer node, and verifying whether transactions included in the block satisfy a guarantee policy at the committing peer node, and then checking validity for each transaction. Inspection may include marking a transaction that has been validated as valid, and marking a transaction that has not been validated as invalid.

In addition, the client generating a second transaction for fetching the first file, generating a block including the second transaction when the second transaction satisfies a guarantee policy and a predetermined block generation condition is met. Step of propagating the generated block to each peer node constituting the blockchain system to check validity, and if the second transaction is valid, the metadata stored in the ledger by the called smart contract The method may further include searching for a storage location of the first file with reference to and fetching the first file from the searched storage location.

And, in order to achieve the above object, the present invention can provide a processor-readable recording medium in which a program for executing the data management method in a processor is recorded.

According to the present invention, instead of storing all files in a blockchain ledger, only small-sized metadata is stored in the ledger, and all files are stored in a storage device that provides a data immutability function, thereby preventing data forgery and falsification. In addition, it can improve the processing speed of blockchain transactions by reducing the storage capacity in the blockchain ledger. In addition, the storage device has scalability in a scale-out method, so that it can effectively respond to stored data that can continuously increase in size. In addition, even when a failure occurs in a node constituting the blockchain system, a recovery process can be quickly performed.

1 is a diagram for explaining a general blockchain structure;
2 is a diagram showing the block structure of Hyperledger Fabric;
3 is a diagram referenced to explain an example of a system to which a data management method according to the present invention is applied;
4 and 5 are flowcharts provided to explain a file storage process in a data management method according to an embodiment of the present invention;
6 and 7 are flowcharts provided to explain a file retrieval process in a data management method according to an embodiment of the present invention, and
8 is a flowchart provided to explain a process of recovering from a failure of a peer node in a data management method according to an embodiment of the present invention.

In this specification, when a component is referred to as "connected" or "connected" to another component, a component may be directly connected or connected to another component, but other components in the middle It should be understood that elements may be present. In addition, other expressions describing the relationship between components, such as "between" or "adjacent to", and expressions such as "transmitting" a signal from one component to another, should be interpreted similarly. do.

Hereinafter, the present invention will be described in more detail with reference to the drawings.

1 is a diagram for explaining a general blockchain structure.

Referring to FIG. 1, a block of a blockchain is composed of a header and a body. The header includes information for creating a linked list in a single block, and the body includes the contents of transactions included in the block. In the case of Bitcoin, the header may include information such as software version, difficulty, hash value of the previous block, block generation time, merkle root, and nonce, and the body may include at least one transaction. can

To create a block, a process of finding the nonce value is required, and the hash value of the previous block is used to calculate the nonce value of the new block. The process of finding the value of this nonce is expressed as mining in Bitcoin, and is also called a 'consensus algorithm'. This enables the function of a distributed ledger that allows participants on the blockchain network to hold the same ledger in real time, and connects blocks to each other.

The stability of blockchain comes from the expansion of chain length through linked lists and the same ledger held by blockchain network participants. As the length of the chain increases, it becomes almost impossible to hack a previously existing block, and since all nodes hold the transaction data of newly created blocks through a distributed ledger, they do not have the computing power to modify more than half of them. It is impossible to manipulate the data of the blockchain unless it is.

Blockchain has forms such as public blockchain, private blockchain, and consortium blockchain in terms of its utilization. In the case of a public blockchain, anyone can participate and anyone can view information. Private blockchains can apply limited functions depending on the purpose, such as limiting participants and limiting the scope of viewing. In addition, the consortium blockchain, as an extension of the private blockchain, can provide reliability and scalability by connecting heterogeneous blocks created in different blockchains through a process called ordering by combining different private blockchains.

2 shows a block structure of Hyperledger Fabric.

Hyperledger Fabric is a modular architecture platform for developing blockchain solutions and applications. Hyperledger Fabric was developed as one of the projects of Hyperledger, which is a worldwide open-source collaborative activity organized under the leadership of the Linux Foundation to advance cross-industry blockchain technology. Managed by maintainers.

Referring to FIG. 2, the block structure of Hyperledger Fabric uses an encrypted hash function to store the block structure, thereby protecting past transaction information from being damaged. In addition, the block structure of Hyperledger Fabric is almost the same as that of Bitcoin, but it is characterized by the fact that the KVS (Key-Value Store) hash value is also recorded.

The hash value of the previous block is calculated using SHA3 SHAKE256 at 512bit (64bytes), and the metadata of the block can be freely entered in each node, and is excluded from hash calculation.

In general, smart contract refers to a technology in which transactions are automatically concluded when certain conditions are met between participants on a blockchain platform. In Hyperledger Fabric, chain codes specify smart contract policies as codes. It is a component of the purpose, and is used in almost the same sense as a smart contract.

In the present invention, the guarantee of data integrity basically utilizes a blockchain system based on Hyperledger Fabric.

3 is a diagram referenced to explain an example of a system to which a data management method according to the present invention is applied.

Referring to FIG. 3 , the system may include a client 100, a blockchain system 200 and a storage device 300.

The client 100 may be connected to the blockchain system 200, write a client transaction, and call the transaction through it. The client 100 may connect to a desired peer node among the first to nth peer nodes 210a to 210n.

The blockchain system 200 is basically a system based on Hyperledger Fabric, but is not necessarily limited thereto.

In the blockchain system 200, the first to nth peer nodes 210a to 210n maintain the blockchain system 200, process transactions, manage ledgers and chain codes, Save.

In the case of Hyperledger Fabric, the ledger consists of a block chain that stores immutable ordered records in blocks and a state database that maintains the current state, and stores the ledger in a storage called world state.

Among the first to nth peer nodes 210a to 210n, an endorsing peer node first performs a transaction requested according to an endorsement policy, reviews it, and attaches an endorsement signature. That is, the endorsing peer node is a peer node that performs the role of verifying the transaction, simulates the transaction to be performed by the smart contract, and returns the result to the client application.

Among the first to nth peer nodes 210a to 210n, a committing peer node confirms the transaction and updates the contents to the blockchain if there is no verification problem in the transaction result executed by the endorsing peer node.

In addition, an anchor peer node performs a representative role within a channel, and a leader peer node performs a representative role in an organization.

The ordering service node 220 owns the configuration information for the channel (C) 250 and, based on this, performs the managerial role of the entire system. The ordering service node 220 forwards the transaction to the peer node 210, and performs branching and sorting of all peer nodes 210 participating in the network.

The channel 250 serves to bind different nodes together. One channel 250 has one ledger, and all peer nodes 210 in the channel have copies of the same ledger.

The storage device 300 stores data and provides a function of preventing unauthorized deletion or alteration of the stored data.

In addition, the storage device 300 has scalability in a scale-out distribution method. Scale-out refers to a method of expanding by adding equipment. In other words, if there is a limit in capacity or performance with only the existing device, not only the capacity increases by the amount added by connecting devices with similar specifications, but also the performance can be increased by sharing the workload. Users perceive the storage cluster as a system through a distributed file system or global namespace. Due to the scalability of the storage device 300 as described above, even if data managed through the blockchain system 200 increases, it can be dynamically accommodated.

4 and 5 are flowcharts provided to explain a file storage process in a data management method according to an embodiment of the present invention.

Referring to FIG. 4, a client application that is a decentralized application (DApp) running on the client 100 is connected to the blockchain system 200, generates a transaction for file storage, and first through A transaction proposal message is transmitted to one or more endorsing peer nodes among the n-th peer nodes 210a to 210n (S400). The transaction proposal message contains the client ID, the chaincode ID to which the transaction belongs or originates, the payload containing the transaction, and the client signature.

Upon receiving the transaction proposal message, the endorsing peer node checks whether the client signature is valid, checks whether the submitter is authorized to perform the proposed operation in the corresponding channel, etc. After checking the validity of the transaction by simulation, the result is transmitted to the client application as a proposal response message (S405).

The client application checks the received proposal response message and checks whether it satisfies the preset guarantee policy (S410), and transmits the generated transaction to the ordering service node 220 when the guarantee policy is satisfied (S415, S420). ). The assurance policy refers to conditions under which transactions are guaranteed. If, as a result of the determination in step S415, if the guarantee policy is not satisfied, the generated transaction is abandoned (S425). When a transaction is abandoned, an option to retry later may be used.

The ordering service node 220 collects the transactions transmitted from the client application in order, checks whether or not a predetermined block generation condition, such as a certain time or a certain number of transactions, is satisfied (S430), In case S425, the process shown in FIG. 5 is performed. If the block setting condition set in step S435 is not satisfied, the transaction processing failure is notified to the client application (S440).

Referring to FIG. 5, if the block generation condition is satisfied, the ordering service node 220 creates a block including the transmitted transaction and distributes the block to all committing peer nodes and endorsing peer nodes belonging to the same channel. Transmit (S445).

The committing peer node verifies that the transactions included in the received block satisfy the guarantee policy, and then validates each transaction included in the block, records the validated transaction as valid, and confirms that the validation is successful. If it fails, the corresponding transaction is recorded as invalid (S450). Transaction validation, which marks data read during chaincode execution as valid if it has not changed since execution or approval, can be applied to ledger state databases. However, if the data read during chaincode execution is changed by another transaction, that transaction is marked as invalid and is not applied to the ledger state database.

When the validity of the transaction generated for file storage is verified (S455), by executing the called smart contract (S460), the storage target file is stored in the storage device 300, and information about the stored file is stored. Metadata is extracted (S465). Then, the extracted metadata is stored in the ledger (S470). If the validity of the transaction is not verified in step S455, the transaction processing failure is notified to the client application (S440).

When the ledger is updated, the transaction processing success is notified to the client application (S475), and the transaction processing process for file storage ends.

By this process, the file is stored in the storage device 300 where data cannot be changed, and the metadata for the stored file is recorded in the ledger of the blockchain system 200 to prevent forgery and alteration of the metadata, and the stored file Double protection against In addition, by storing only small-sized metadata in the ledger instead of storing all files in the ledger, the capacity to be stored in the blockchain distributed ledger can be reduced and the processing speed of blockchain transactions can be improved.

In addition, although the blockchain ledger is generally stored in each peer node, the ledger can also be configured to store the storage device 300 in which data cannot be changed. In this case, the integrity of the data stored in the ledger is guaranteed and unauthorized deletion is prevented. can do.

6 and 7 are flowcharts provided to explain a file retrieval process in a data management method according to an embodiment of the present invention.

Referring to FIG. 6 , the client application, which is a decentralized application (DApp) running on the client 100, is connected to the blockchain system 200 and generates a transaction for file withdrawal, and first through nth A transaction proposal message is transmitted to one or more endorsing peer nodes among the peer nodes 210a to 210n (S500).

Upon receiving the transaction proposal message, the endorsing peer node checks whether the client signature is valid, checks whether the submitter is authorized to perform the proposed operation in the corresponding channel, etc. After simulating and validating the transaction, a proposal response message is transmitted to the client application (S505).

The client application checks the proposal response message received from the endorsing peer node, checks whether or not the preset guarantee policy is satisfied (S510), and transmits the generated transaction to the ordering service node 220 if the guarantee policy is satisfied (S510). Do (S520). If the guarantee policy is not satisfied, the generated transaction is abandoned (S525).

The ordering service node 220 collects the transactions transmitted from the client application in order, checks whether or not a predetermined block generation condition, such as a certain time or a certain number of transactions, is satisfied (S530), In case (S535), the process shown in FIG. 7 is performed. If the block setting condition set in step S535 is not satisfied, the transaction processing failure is notified to the client application (S540).

Referring to FIG. 7, when the block generation condition is satisfied, the ordering service node 220 creates a block including the transmitted transaction and distributes the block to all committing peer nodes and endorsing peer nodes belonging to the same channel. Transmit (S545).

The committing peer node verifies that the transactions included in the received block satisfy the guarantee policy, and then validates each transaction included in the block, records the verified transaction as valid, and If it fails, the corresponding transaction is marked as invalid (S550).

When the validity of the transaction created for file withdrawal is verified (S555), by the execution of the called smart contract (S560), the storage location of the file requested for withdrawal is searched by referring to the metadata stored in the ledger (S565) , The file is fetched from the searched location of the storage device 300 (S570).

When the file is fetched, the transaction processing success is notified to the client application (S575), and the fetched file can be sent to the client application.

Through this process, a file stored in the storage device 300, which cannot be changed in data, can be retrieved through the blockchain system 200. In addition, files stored in the storage device 300 in which data cannot be changed and metadata stored in the ledger may be searched for and output if there is a search or inquiry request, or the like, files and metadata that meet the search or inquiry conditions. .

8 is a flowchart provided to explain a process of recovering from a failure of a peer node in a data management method according to an embodiment of the present invention.

Referring to FIG. 8 , when a failure is detected among the peer nodes constituting the blockchain system (S600), the failed peer node is excluded from all channels in which the failure is detected (S605).

Next, after deleting the ledger of the failed peer node (S610), the container of the failed peer node is stopped and excluded from group membership (S615). Hyperledger Fabric leverages container technology to provide chaincode services for configuring system logic.

When the replacement peer node replacing the failed peer node is a new node, a Membership Service Provider (MSP) of the replacement peer node is created (S620 and S625). The role of the node can be assigned through the MSP.

Then, after creating a container of the alternative peer node, it is assigned to the group to which it belongs (S630), a ledger is created in the alternative peer node, and the alternative peer node is registered in a channel to share the ledger so that the contents of the ledger can be copied. (S635).

Next, the chaincode is installed on the alternative peer node (S640), and the instantite command is executed as needed. Chaincode is not immediately usable just because it is installed on a certain peer node. To use a chaincode, it needs to be instantiated to notify other peer nodes of the chaincode interface. Instantiation can be done using the instantiate command. there is.

Next, the contents of the ledger of another peer node of the same channel are copied to the alternative peer node (S645). In this case, by configuring the storage device 300 to store all of the large-capacity data managed by the ledger or both the ledger and the large-capacity data managed by the ledger, copying of the contents of the ledger can be performed quickly. In addition, without direct copying of the contents of the ledger, it is also possible to change only the access rights to the corresponding ledger so that the alternative node manages it.

In the event of a failure of a peer node constituting the blockchain system through this process, recovery is performed by replacing it with another node.

Meanwhile, the storage device 300 is a storage device to which a write once read many (WORM) function is added so that data cannot be changed, and may be configured as a hard disk or a solid state disk (SSD).

WORM storage devices such as CD-R and DVD-R record data by physically forming a pattern on the surface of the media using a laser, so that data cannot be physically tampered with from the moment it is recorded. However, since WORM storage based on hard disks and SSDs implemented to support large capacity, high performance, and search usability does not have a device to protect data with physical properties, the function to protect data is implemented through a software method. will do At this time, the software is implemented as fixed code in the operating system that provides file system service to the storage device or implemented as firmware to fundamentally block the way to change it from the outside, preventing data from being tampered with or deleted with any authority. can

On the other hand, the data management method according to the present invention can not be applied limitedly to the configuration of the embodiments described above, but all or part of each of the embodiments can be selectively modified so that various modifications can be made. They may be configured in combination.

In addition, the present invention can be implemented as a computer program on a programmable computer. Such computers may include processors, storage devices, input devices, and output devices. In order to implement the contents described in the present invention, the program code may be input with a mouse or keyboard input device. These programs can be implemented in high-level languages or object-oriented languages. It can also be implemented as a computer system implemented in assembly or machine code.

Further, the subject matter of the present invention is not limited to hardware or software use, but is applicable to any other computing or processing environment. It may be implemented by hardware, software, or a combination of hardware and software described in the present invention. The invention may be implemented using circuitry. That is, it can be implemented with one or more programmable logic circuits, that is, application specific integrated circuits (ASICs) or logic circuits (AND, OR NAND gates) or processing devices (eg, microprocessors, controllers).

The present invention can also be implemented as processor-readable codes on a processor-readable recording medium. The processor-readable recording medium includes all types of recording devices in which data readable by the processor is stored. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, and an optical data storage device. In addition, the processor-readable recording medium is distributed in computer systems connected through a network, so that processor-readable codes can be stored and executed in a distributed manner.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: client 200: blockchain system
300: storage device

Claims (10)

Connecting with a preset blockchain system so that the client can send and receive blockchain transactions;
generating, by the client, a first transaction for storing a first file;
generating a block including the first transaction when the first transaction satisfies a guarantee policy and a predetermined block generation condition is met;
propagating the generated block to each peer node constituting the blockchain system to check validity;
If the first transaction is valid, storing the first file in a storage device that provides unalterable data by a called smart contract; and
A data management method comprising extracting metadata for the first file and storing the extracted metadata in a ledger of the blockchain system. According to claim 1,
The storage device is a data management method, characterized in that the distributed storage dynamically expandable by a scale-out method. According to claim 2,
The data management method, characterized in that the ledger is stored in the distributed storage. According to claim 1,
When a failure is detected in any one of the peer nodes constituting the blockchain system, replacing the failed peer node where the failure is detected with another node, and the other node receiving and synchronizing the ledger of the failed peer node How to manage your data. According to claim 1,
The blockchain system is a data management method, characterized in that the hyperledger fabric blockchain system. According to claim 1,
The data management method, characterized in that the transaction for the storage of the first file is generated in a decentralized client application. According to claim 6,
Whether the first transaction satisfies the guarantee policy,
The client application forwards a transaction proposal message for the first transaction to an endorsing peer node;
When the endorsing peer node simulates the transaction and transmits the result as a proposal response message to the client application,
and checking whether the guarantee policy is satisfied based on the proposal response message. According to claim 7,
Steps to validate are:
receiving, by a committing peer node, the block from an ordering service node; and
After checking whether the transactions included in the block satisfy the guarantee policy at the committing peer node, the validity of each transaction is checked, and the validated transaction is recorded as valid, and the unvalidated transaction is invalid. Data management method comprising the step of recording as. According to claim 1,
generating, by the client, a second transaction for fetching the first file;
generating a block including the second transaction when the second transaction satisfies a guarantee policy and a predetermined block generation condition is satisfied;
propagating the generated block to each peer node constituting the blockchain system to check validity; and
When the second transaction is valid, the storage location of the first file is searched by referring to the metadata stored in the ledger by the called smart contract, and the first file is retrieved from the searched storage location of the storage device. Data management method further comprising the step of doing. A processor-readable recording medium on which a program for executing the data management method according to any one of claims 1 to 9 in a processor is recorded.

Images