WO2021003985A1

WO2021003985A1 - Blockchain data archiving storage method and apparatus, computer device and storage medium

Info

Publication number: WO2021003985A1
Application number: PCT/CN2019/123147
Authority: WO
Inventors: 薄辰龙
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-07-08
Filing date: 2019-12-05
Publication date: 2021-01-14
Also published as: CN110442644A

Abstract

A blockchain data archiving storage method, comprising: periodically detecting data that meets preset archiving conditions in blockchain data to obtain data to be archived, wherein the preset archiving conditions include a height value condition and an access frequency condition; performing fragmentation processing on the data to be archived to obtain fragmented data; identifying the data source type of the fragmented data, and acquiring a distributed storage engine corresponding to the data source type to obtain a target distributed storage engine; and storing the fragmented data in a storage node in the target distributed storage engine.

Description

Block chain data archive storage method, device, computer equipment and storage medium

Cross references to related applications

This application claims the priority of the Chinese patent application filed with the Chinese Patent Office on July 8, 2019, with the application number 2019106116447 and the application titled "Blockchain data archiving and storage methods, devices, computer equipment and storage media", all of which The content is incorporated in this application by reference.

Technical field

This application relates to a blockchain data archiving and storage method, device, computer equipment and storage medium.

Background technique

With the development of data storage technology, there has been a way to record and store massive amounts of data using blockchain technology. Storing data through blockchain technology ensures data security and storage space, which leads to the current blockchain data storage Way or traditional way.

However, the inventor realized that the current blockchain data storage method or traditional method, because the storage space of the blockchain node increases linearly with the increase of the block, the storage space of the block node is limited, and the incremental zone data will gradually affect The response speed of the memory further affects the efficiency of consensus, verification, and read and write operations for high concurrency of node devices, and there is a problem of low data storage efficiency.

Summary of the invention

According to various embodiments disclosed in the present application, a blockchain data archive storage method, device, computer equipment, and storage medium are provided.

A method for archiving and storing blockchain data, the method includes:

Periodically detect the data that meets the preset archiving conditions in the blockchain data to obtain the data to be archived. The preset archiving conditions include height value conditions and access frequency conditions;

Perform fragmentation processing on the archived data to obtain fragmented data;

Identify the data source type of the sharded data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

Store the sharded data to the storage node in the target distributed storage engine.

In one of the embodiments, periodically detecting data that meets preset archiving conditions in the blockchain data, and obtaining the data to be archived includes:

Count the total amount of data between the lowest height value data and the highest height value data in the blockchain node according to the cycle time. The cycle time is less than the time required to generate a data set that meets the preset total amount;

When the total amount of data is greater than the preset total, the data sets that meet the preset total are sequentially obtained from the lowest height value;

Check the access frequency of each data in the data set; and

When each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.

In one of the embodiments, performing fragmentation processing on the data to be archived to obtain fragmented data includes: dividing the data to be archived into at least two groups to obtain at least two groups of grouped data, and adding each grouped data to the at least two data During fragmentation, fragment data is obtained.

In one of the embodiments, the distributed storage engine includes a private storage engine, a federated storage engine, and a public storage engine, identifying the data source type of sharded data, and obtaining the distributed storage engine corresponding to the data source type includes:

When it is recognized that the shard data comes from the private blockchain, modify the entry parameters corresponding to the general private storage engine to obtain the private storage engine;

When it is recognized that the shard data comes from the alliance blockchain based on the smart contract, modify the entry parameters corresponding to the general alliance storage engine to obtain the alliance storage engine; and

When it is recognized that the sharded data comes from the public blockchain, the entry parameters corresponding to the general public storage engine are modified to obtain the target public storage engine.

In one of the embodiments, the storage node storing the fragmented data in the target distributed storage engine includes:

According to the hash algorithm, calculate the hash value of the fragmented data; and

Store the sharded data to the storage node pointed to by the hash value in the target distributed storage engine.

In one of the embodiments, the storage node storing the fragmented data in the target distributed storage engine includes: identifying the data volume or data importance level identifier of the fragmented data, and according to the data Identifies the amount or data importance level, and allocates the fragmented data to the storage nodes in the target distributed storage engine.

Evaluate and classify each storage node to determine the storage performance of each storage node;

Perform weight distribution evaluation on sharded data to determine the storage requirements of sharded data; and

According to the storage performance of each storage node and the storage requirements of each fragmented data, the fragmented data is stored to the storage node in the target distributed storage engine.

In one of the embodiments, the data slice where the sliced data is located includes multiple data packets; after storing the sliced data to the storage node in the target distributed storage engine, it further includes:

Calculate the hash value of each data packet;

According to the hash value of each data packet, construct a Merkle tree corresponding to the data slice where each data packet is located; and

Record the corresponding relationship between each Merkle tree and the storage node where each data fragment is located.

In one of the embodiments, it further includes:

Perform capacity inspection on each storage node; and

When the capacity of each storage node reaches the preset storage capacity, a storage node expansion request is sent.

A block chain data archive storage device, the device includes:

The data detection module is used to periodically detect the data that meets the preset archiving conditions in the blockchain data to obtain the data to be archived. The preset archiving conditions include height value conditions and access frequency conditions;

The data fragmentation module is used for fragmentation processing of the data to be archived to obtain fragmented data;

The distributed storage engine acquisition module is used to identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

The data storage module is used to store the fragmented data to the storage node in the target distributed storage engine.

A computer device, including a memory and one or more processors, the memory stores computer readable instructions, when the computer readable instructions are executed by the processor, the one or more processors execute The following steps:

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the following steps:

The details of one or more embodiments of the application are set forth in the following drawings and description. Other features and advantages of this application will become apparent from the description, drawings and claims.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

Fig. 1 is an application environment diagram of a method for archiving and storing blockchain data according to one or more embodiments.

FIG. 2 is a schematic flowchart of a method for archiving and storing blockchain data according to one or more embodiments.

Fig. 3 is a schematic diagram of a process of archiving and storing blockchain data according to one or more embodiments.

Figure 4 is a schematic diagram of a hash ring in an embodiment.

Fig. 5 is a block diagram of a blockchain data archive storage device according to one or more embodiments.

Fig. 6 is a block diagram of a blockchain data archive storage device according to one or more embodiments.

Figure 7 is a block diagram of a computer device according to one or more embodiments.

Detailed ways

In order to make the technical solutions and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the application, and not used to limit the application.

The blockchain data archiving storage method provided in this application can be applied to the application environment shown in FIG. 1. The blockchain network includes multiple blockchain nodes 102, and the blockchain nodes 102, the control system 104, and the distributed storage system 106 realize pairwise communication through the network. Among them, the blockchain node 102 and the control system 104 can be different computer devices, which include but are not limited to servers, computer hosts, smart phones, tablets, smart wearable devices, etc. Each computer device serves as a node of the blockchain Devices can participate in database recording, and data can be synchronized quickly between various computer devices. Each blockchain node 102 has a corresponding control system 104, and the distributed storage system 106 may include multiple storage devices, where multiple storage devices are assembled through applications or software to provide data storage and access functions externally. The distributed storage system 106 can provide blockchain block data storage services for at least one blockchain node. Among them, the control system 104 takes multiple servers as an example. The server periodically detects the data that meets the preset archiving conditions in the blockchain data to obtain the data to be archived. The data to be archived is fragmented to obtain the fragmented data and identify The data source type of the fragmented data is obtained, the distributed storage engine corresponding to the data source type is obtained, the target distributed storage engine is obtained, and the fragmented data is stored to the storage node in the target distributed storage engine. The above scheme not only realizes the archive processing of the blockchain data, but also realizes the expansion of the blockchain data storage space through a distributed storage method, and meets the storage space requirements of the linear growth of the blockchain data. Among them, the server can be implemented by an independent server or a server cluster composed of multiple servers.

In one of the embodiments, as shown in FIG. 2, a method for archiving and storing blockchain data is provided. Taking the method applied to the server in the control system 104 in FIG. 1 as an example for description, the method includes the following steps:

Step 200: Periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived. The preset archiving conditions include height value conditions and access frequency conditions.

Blockchain, also known as distributed ledger technology, is a new technology in which several computer devices participate in bookkeeping and jointly maintain a complete distributed database (which can be regarded as a ledger). Blockchain data refers to data that are bound into block files (which can be regarded as account pages) in chronological order, and string together to form data in the form of a chain. Specifically, each "block data" in the blockchain data is like the "account page" in the ledger, and each page records several data transaction records, and the pages of the account are bound in chronological order It forms a complete ledger that is a distributed database. The user can initiate a data transaction request through the blockchain node in the terminal blockchain node 102 and submit the data to be processed. After the blockchain node receives the data processing request, the data to be processed performs corresponding data processing, for example, Consensus processing and storage of data with other blockchain nodes. Data archiving is the process of moving data that is no longer frequently used to a separate storage device for long-term preservation. The height value of the block data in the blockchain and the frequency of access can intuitively reflect whether the block data of the blockchain node is frequently used data (hot data) or data that has not been used for a long time (cold data). Therefore, in this embodiment, the data height value condition and the access frequency condition are used as the preset archiving conditions for detecting the data to be archived in the blockchain data. Specifically, the data that meets the preset archiving condition in the blockchain data is detected according to the cycle time, and the data that meets the preset archiving condition is determined as the data to be archived. The preset archiving condition includes a height value condition and an access frequency condition.

Step 400: Perform fragmentation processing on the data to be archived to obtain fragmented data.

After the data to be archived is obtained, the data to be archived can be divided into multiple (at least two) data groups to obtain at least two groups of grouped data, and each grouped data is added to at least two data fragments to obtain the divided data. Fragment data, to ensure that all packet data (data to be archived) exist in at least two data fragments, and each data fragment does not include all packet data. Specifically, to group the data to be archived, the data to be archived can be divided into groups including the same amount of data by means of equal division, or the data to be archived can be divided into groups including different amounts of data by means of random division. Data, and then add each divided data packet to at least two data fragments. When a data fragment is maliciously damaged, the complete data will not be lost, and other data fragments can be used to repair the file.

Step 600: Identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine.

The storage engine is used to store data in MySQL (Structured Query Language, structured query language) in memory using a variety of different technologies. Each of these technologies uses different storage mechanisms, indexing techniques, and locking Level and ultimately provide a wide range of different functions and capabilities, by choosing different technologies, to obtain additional speed or functionality, thereby improving the overall function of the application. Blockchain data is divided by data source type, including public chain data, private chain data, and alliance chain data. Specifically, by identifying the type of blockchain data source, the distributed storage engine corresponding to the data source type can be obtained to obtain the target Distributed storage engine.

Step 800: Store the fragmented data to the storage node in the target distributed storage engine.

In this embodiment, the storage method of the blockchain data is distributed storage, which is to store the blockchain data on multiple independent devices. It adopts a scalable system structure and uses multiple storage servers to share the storage load. It not only improves the reliability, availability and access efficiency of the system, it is also easy to expand. A storage node is an independent device (storage server) used to store data. After the corresponding distributed storage engine is obtained, the fragmented data is stored to the storage node in the target distributed storage engine. Specifically, according to the hash value of the fragment data, the fragment data may be stored in the storage node pointed to by the hash value. It may also be to identify the data volume or data importance level identification of the fragmented data, and allocate the fragmented data to the corresponding storage nodes in the target distributed engine for storage according to the data volume or data importance level identification.

The above-mentioned blockchain data archiving storage method, by periodically detecting the data to be archived and performing fragmentation processing on it, can avoid data redundancy, reduce the computing speed, and can flexibly choose the storage method according to the source of the fragmented data to ensure data security Availability, and assigning fragmented data to storage nodes in the corresponding distributed storage engine can save node storage space, ensure good storage space and high data availability, and improve data processing efficiency.

As shown in FIG. 3, in one of the embodiments, periodically detecting data that meets preset archiving conditions in the blockchain data to obtain the data to be archived includes: Step 220: Counting the lowest height value in the blockchain node according to the cycle time The total amount of data between the data and the data with the highest height value, the cycle time is less than the time required to generate a data set that meets the preset total amount. When the total amount of data is greater than the preset total amount, the lowest height value is obtained in turn For data sets that meet the preset total amount, the access frequency of each data in the data set is detected, and when each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.

In specific implementation, you can check the number of local storage of blockchain nodes, that is, the total amount of data between the lowest height value and the highest height value data (ie the total number of data), if the total amount of data is greater than the expected If the total amount is set, the data of the preset total amount is obtained from the data of the lowest height value. If the data access volume of each preset total amount is less than the preset access frequency, the preset total amount is determined to meet the archiving conditions. , Wherein the cycle time is less than the generation time of the data reaching the preset total amount. The total amount of preset data can be n, which can be set freely according to the data processing capability of the server. The time required to generate the preset total amount of data can be determined by the data flow in the business processing process. Set the detection cycle time to be less than the data generation time of the preset total amount, so that the data that has met the preset total amount can be archived before the preset total amount of data is generated again, so as to avoid the accumulation of data that meets the archiving conditions. Specifically, for example, the lowest height value of the locally stored data is H0, and the highest height value data is H31, then the total amount of locally stored data is H31-H0+1=31, that is, there are 31 between the lowest height value and the highest height value. When the preset total is 20, the total number of data between the lowest height value and the highest height value stored locally (31) is greater than the preset total (20), then the lowest height value Start acquiring data that meets the preset total of 20 data, that is, acquiring data with height values of H0-H19. Further, a list is used to record the data whose access frequency exceeds the preset access frequency in the most recent period and the corresponding height value of the data. The preset access frequency is p, and the data whose access frequency exceeds p can be {X1,...,X30} , When the preset total amount of data H0-H19 are not in {X1,...,X30}, it is determined that the data with the height value of H0-H19 is the data to be archived. In this embodiment, the detection cycle time is set to be less than the time required to generate the preset total amount of data, and the data blocks meeting the preset total amount can be archived before the preset total amount of data is generated again. Avoid the accumulation of local data to be archived, resulting in data redundancy.

As shown in FIG. 3, in one of the embodiments, performing fragmentation processing on the data to be archived to obtain the fragmented data includes: step 420, using a consistent hash algorithm to perform fragmentation processing on the archived data to obtain the fragmented data.

Taking the consistent hash (Hash) algorithm as an example, the consistent hashing ring-cutting algorithm can be used to implement data fragmentation. The hash ring is cut into fragments of the same size, and then these fragments are handed over to different storage. The node is responsible. Specifically, the corresponding Key (key value, K1 as shown in Figure 4) is hashed into the circular hash space through a hash algorithm, and the packet data is calculated through a specific hash function to obtain the hash value corresponding to the data. Hash the hash value onto the hash ring (ie, the Hash ring, as shown in Figure 4) to form fragmented data. When a storage node exits, the fragments it is responsible for do not need to be merged clockwise After that, it is handed over to storage nodes, such as storage nodes a, b, and c as shown in Figure 4, but the entire shard can be handed over to any storage node as a whole more flexibly. In practice, one shard is often used as the smallest data migration and backup unit. It can be understood that, in other embodiments, the fragmentation processing method may also be performed in a manner such as alternate placement or interval division. In this solution, data packets form at least 3 data fragments, and each data fragment is composed of part of the packet data (not including all the packet data), and the number of data packets in each data fragment can be the same or different , It is also necessary to ensure that each data packet is added to at least two data shards to form at least two storage copies. Even if a storage node in the storage system is attacked, the complete data will not be leaked. It can be based on other storage nodes. The copy of data to repair the missing data. By sharding the data, the data processing speed and data throughput are improved, and the amount of data is prevented from being blocked due to excessive data. The consistent Hash can solve the stability problem very well, and all the storage nodes can be arranged at the end. Hash ring.

As shown in Figure 3, in one of the embodiments, the distributed storage engine includes a public storage engine, a federation storage engine, and a private storage engine to identify the data source type of the sharded data, and obtain the distribution corresponding to the data source type Storage engine to obtain the target distributed storage engine includes: step 620, when it is identified that the shard data comes from the private blockchain, modify the entry parameters corresponding to the general private storage engine to obtain the private storage engine, and when the segment is identified When the slice data comes from the alliance blockchain based on the smart contract, modify the entry parameters corresponding to the general alliance storage engine to obtain the alliance storage engine. When it is identified that the slice data originates from the public blockchain, modify the general public storage The entry parameter corresponding to the engine is used to obtain the public storage engine.

Distributed storage engines include public storage engines, alliance storage engines, and private storage engines. Data source types can include private blockchain (private chain), alliance blockchain (consortium chain), and public blockchain (public Chain), private storage engine, consortium storage engine, and public storage engine refer to storage engines used to store the data of the private blockchain, consortium blockchain, and public blockchain, respectively. When the fragmentation processing of the data to be archived is completed and the fragmented data is obtained, the data source type of the fragmented data is identified, and the distributed storage engine corresponding to the data source is obtained. Specifically, when it is recognized that the shard data comes from the private blockchain, the entry parameters corresponding to the general private storage engine are modified to obtain the private storage engine. The private storage engine is used to store the data of the source private blockchain for itself Use; when identifying the sharded data from the alliance blockchain based on the smart contract, modify the entry parameters corresponding to the general alliance storage engine to obtain the alliance storage engine, which is used to store the data from the alliance blockchain , For specific multi-party use; when identifying that the sharded data comes from a public blockchain, modify the entry parameters corresponding to the general public storage engine to obtain the public storage engine, which is used to store the data from the public zone The data of the block chain is used by multiple parties. It can also be receiving a parameter modification request, and building a storage engine corresponding to the target modification parameter according to the target modification parameter carried in the parameter modification request. In this embodiment, the storage method is flexibly selected according to the data source type of the fragmented data to ensure the safety and availability of the data.

As shown in FIG. 3, in one of the embodiments, storing the sharded data to the storage node in the target distributed storage engine includes: step 820, calculating the hash value of the sharding data according to the hash algorithm, and dividing the sharded data The data is stored in the storage node pointed to by the hash value in the target distributed storage engine.

When the corresponding distributed storage engine is obtained, the hash value of each piece of data can be calculated according to the hash algorithm, and the hash value points to the storage address of the piece of data, and the piece of data is stored in the target distributed storage engine The storage node the hash value points to. Taking the private storage engine as an example, according to the storage address pointed to by the hash value of the fragmented data, the fragmented data is stored in the storage node corresponding to the storage address in the private storage engine. In other embodiments, the preset importance level of the fragmented data can also be preset to identify the importance level identifier of each fragmented data. When the importance level of the fragmented data is high, the choice is strong in computing power and fast in calculation. Storage node for storage. It may also be a preset data amount of the fragmented data, identifying the data amount of the fragmented data, and when the amount of data carried by the fragmented data is higher than a preset threshold, the fragmented data is allocated to a storage node with a large capacity. In this embodiment, the allocation of the storage nodes of the fragment data is completed according to the hash value of the fragment data, and the corresponding storage allocation data can facilitate the recording of the correspondence between the allocation data and the storage nodes.

In one of the embodiments, storing the sharded data in the storage node of the target distributed storage engine includes: step 840, evaluating and classifying each storage node, determining the storage performance of each storage node, and performing weighting on the sharded data. Value distribution evaluation, determine the storage requirements of the sharded data, and store the sharded data to the storage node in the target distributed storage engine according to the storage performance of each storage node and the storage requirements of each sharded data.

The storage method of sharded data can also be pre-evaluated and categorized storage nodes. Specifically, the storage nodes are evaluated according to their capacity, computing speed, and computing power to determine the storage performance of the storage nodes; at the same time The importance, data volume, and type of shard data are evaluated for weight distribution, and the storage requirements of shard data are determined. Then, based on the storage performance of each candidate node and the data storage requirements of each shard, peer selection is made, and the sharded data Store to the appropriate storage node in the target distributed storage engine. Among them, storage nodes are nodes that work independently. Specifically, it can be to identify the degree of data confidentiality, data volume, and data type of the fragmented data, and number each storage node according to the computing power, computing speed, and capacity of each storage node. The storage nodes with the fastest speed or the largest capacity are sequentially numbered 1, 2, 3...m, etc., and important data or data with a large amount of data or high data confidentiality are sequentially sent to the storage nodes numbered 1, 2, 3...m Store it. In this embodiment, by slicing the data and sending the data to the corresponding storage node, high data throughput is achieved and the technical cost of the device is reduced.

As shown in FIG. 3, in one of the embodiments, the data shard where the sharded data is located includes multiple data packets; after storing the sharded data to the storage node in the target distributed storage engine, the method further includes: 900. Calculate the hash value of each data group, construct a Merkle tree corresponding to the data segment where each data group is located according to the hash value of each data group, and record where each Merkle tree and each data segment are located Correspondence of storage nodes.

Merkel tree, or Merck tree, hash tree, is a tree-shaped data structure, each leaf node of the tree stores the hash value of the data block. The data fragment where the fragment data is located includes multiple data packets. After the hash value of each data packet is calculated, the meck corresponding to the data fragment where each data packet is located can be established based on the hash value of each data packet For ease of understanding, it can be called a fragmented Merkle tree. Each data fragment corresponds to a fragmented Merkle tree. After obtaining the fragmented Merkle tree corresponding to each data fragment, you can The corresponding relationship between the Merkle tree of each fragment and the storage node where each data fragment is located is recorded. In this embodiment, the search and verification of data can be realized through the Merkle tree, which improves the privacy and security of stored data.

As shown in FIG. 3, in one of the embodiments, the method further includes: step 950, performing capacity detection on each storage node, and sending a storage node expansion request when the capacity of a storage node reaches a preset storage capacity.

During specific implementation, it may also include: detecting the capacity of each storage node according to the cycle time, and when the capacity of a storage node reaches the preset storage capacity, sending a storage node expansion request. Specifically, it may be to send a storage node expansion request to the master node in the storage node. After receiving the storage node expansion request, the master node actively adds other storage nodes to achieve the purpose of expanding the node. Among them, the master node can manage some changes in the cluster, such as creating or deleting an index, adding or removing other nodes, and so on. It may also send a storage node expansion request to the management terminal, and after receiving the storage node expansion request, the management terminal adds other storage nodes to expand the storage capacity. In this embodiment, by periodically detecting the capacity of the storage node to expand the storage space, the problem of data loss caused by insufficient storage space of the storage node can be avoided.

It should be understood that, although the various steps in the flowchart of FIGS. 2-3 are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least some of the steps in Figure 2-3 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. These sub-steps or stages The execution order of is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

In one of the embodiments, as shown in FIG. 5, a blockchain data archive storage device is provided, including: a data detection module 410, a data slicing module 420, a distributed storage engine acquisition module 430, and a data storage module 440 ,among them:

The data detection module 410 is configured to periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived. The preset archiving conditions include height value conditions and access frequency conditions;

The data fragmentation module 420 is configured to perform fragmentation processing on the data to be archived to obtain fragmented data;

The distributed storage engine obtaining module 430 is used to identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine;

The data storage module 440 is used to store the fragmented data to the storage node in the target distributed storage engine.

As shown in FIG. 6, in one of the embodiments, the blockchain data archiving storage device further includes a relationship recording module 450, which is used to calculate the hash value of each data group, and construct a relationship with each data group according to the hash value of each data group. The Merkle tree corresponding to the data segment where the data group is located records the correspondence between each Merkle tree and the storage node where each data segment is located.

As shown in FIG. 6, in one of the embodiments, the blockchain data archive storage device further includes a capacity detection module 460, which is used to perform capacity detection on each storage node. When the capacity of each storage node reaches the preset storage capacity, send Storage node expansion request.

In one of the embodiments, the data detection module 410 is further configured to, when the total amount of data is greater than the preset total amount, sequentially obtain data sets meeting the preset total amount from the lowest height value, and detect the access frequency of each data in the data set , When each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.

In one of the embodiments, the data fragmentation module 420 is further configured to divide the data to be archived into at least two groups to obtain at least two groups of grouped data, and add each grouped data to the at least two data fragments to obtain the divided data. Piece data.

In one of the embodiments, the distributed storage engine acquisition module 430 is also used to modify the entry parameters corresponding to the general private storage engine to obtain the private storage engine when it is identified that the shard data comes from the private blockchain. When the shard data comes from the alliance blockchain based on the smart contract, modify the entry parameters corresponding to the general alliance storage engine to obtain the alliance storage engine. When it is recognized that the shard data comes from the public blockchain, modify the general public Obtain the target public storage engine corresponding to the entry parameters of the storage engine.

In one of the embodiments, the data storage module 440 is further configured to calculate the hash value of the fragmented data according to the hash algorithm, and store the fragmented data to the storage node pointed to by the hash value in the target distributed storage engine.

In one of the embodiments, the data storage module 440 is also used to identify the data volume or data importance level identification of the fragmented data, and allocate the fragmented data to the target distributed storage engine according to the data volume or data importance level identification Storage node.

In one of the embodiments, the data storage module 440 is also used to evaluate and classify each storage node, determine the storage performance of each storage node, evaluate the weight distribution of the shard data, determine the storage requirements of the shard data, and The storage performance of each storage node and the storage requirements of each fragmented data are stored to the storage node in the target distributed storage engine.

For the specific limitation of the blockchain data archive storage device, please refer to the above limitation on the blockchain data archive storage method, which will not be repeated here. Each module in the above-mentioned blockchain data archiving storage device can be implemented in whole or in part by software, hardware and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

In one of the embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. 7. The computer equipment includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the non-volatile storage medium. The database of the computer equipment is used to store the data to be archived in the blockchain. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer-readable instructions are executed by the processor to realize a blockchain data archiving and storage method.

Those skilled in the art can understand that the structure shown in Figure Y is only a block diagram of part of the structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. The specific computer equipment may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

A computer device, including a memory and one or more processors, in which computer readable instructions are stored, and when the computer readable instructions are executed by the processor, the blockchain data archive storage method provided in any embodiment of the present application is implemented A step of.

One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors are provided in any of the embodiments of the present application The steps of the blockchain data archive storage method.

A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing relevant hardware through computer-readable instructions, which can be stored in a non-volatile computer. In a readable storage medium, when the computer-readable instructions are executed, they may include the processes of the above-mentioned method embodiments. Wherein, any reference to memory, storage, database or other media used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Non-volatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not a limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Channel (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction between the combinations of these technical features, they should It is considered as the range described in this specification.

The above-mentioned embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can be made, and these all fall within the protection scope of this application. Therefore, the scope of protection of the patent of this application shall be subject to the appended claims.

Claims

A method for archiving and storing blockchain data, the method comprising:

Periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived, where the preset archiving conditions include height value conditions and access frequency conditions;

Performing fragmentation processing on the data to be archived to obtain fragmented data;

Identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

Storing the fragmented data to a storage node in the target distributed storage engine.
The method for archiving and storing blockchain data according to claim 1, wherein the periodically detecting data that meets preset archiving conditions in the blockchain data to obtain the data to be archived comprises:

Count the total amount of data between the data with the lowest height value and the data with the highest height value in the blockchain node according to the cycle time, and the cycle time is less than the time required to generate a data set meeting the preset total amount;

When the total amount of data is greater than the preset total amount, sequentially acquiring data sets that meet the preset total amount from the lowest height value;

Check the access frequency of each data in the data set; and

When each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.
The method for archiving and storing blockchain data according to claim 1, wherein said performing fragmentation processing on the data to be archived to obtain fragmented data comprises:

Divide the data to be archived into at least two groups to obtain at least two groups of grouped data; and

Each packet data is added to at least two data fragments to obtain fragmented data.
The method for archiving and storing blockchain data according to claim 1, wherein the distributed storage engine comprises a private storage engine, an alliance storage engine, and a public storage engine;

The identifying the data source type of the fragmented data and obtaining the distributed storage engine corresponding to the data source type includes:

When it is recognized that the sharding data comes from a private blockchain, modify the entry parameters corresponding to the general private storage engine to obtain the private storage engine;

When it is recognized that the sharded data is derived from the alliance blockchain based on the smart contract, modify the entry parameters corresponding to the general alliance storage engine to obtain the alliance storage engine; and

When it is recognized that the shard data comes from a public blockchain, the entry parameters corresponding to the general public storage engine are modified to obtain the public storage engine.
The method for archiving and storing blockchain data according to claim 1, wherein the storing of the fragmented data in the storage node of the target distributed storage engine comprises:

Calculate the hash value of the fragmented data according to the hash algorithm; and

Store the fragmented data in the storage node pointed to by the hash value in the target distributed storage engine.
The method for archiving and storing blockchain data according to claim 1, wherein the storing of the fragmented data in the storage node of the target distributed storage engine comprises:

Identify the data volume or data importance level of the fragmented data; and

According to the data amount or the data importance level identifier, the fragmented data is allocated to the storage nodes in the target distributed storage engine.
The method for archiving and storing blockchain data according to claim 1, wherein the storing of the fragmented data in the storage node of the target distributed storage engine comprises:

Evaluate and classify each storage node, and determine the storage performance of each storage node;

Perform weight distribution evaluation on the fragmented data to determine the storage requirements of the fragmented data; and

According to the storage performance of each storage node and the storage requirement of each fragmented data, the fragmented data is stored to the storage node in the target distributed storage engine.
The method for archiving and storing blockchain data according to claim 1, wherein the data fragment in which the fragment data is located includes multiple data packets;

After the storing the fragmented data to the storage node in the target distributed storage engine, the method further includes:

Calculating the hash value of each data packet;

According to the hash value of each data group, construct a Merkle tree corresponding to the data slice where each data group is located; and

The corresponding relationship between each Merkle tree and the storage node where each data fragment is located is recorded.
The method for archiving and storing blockchain data according to claim 1, wherein the method further comprises:

Perform capacity inspection on each storage node; and

When it is detected that the capacity of a storage node reaches the preset storage capacity, a storage node expansion request is sent.
A block chain data archive storage device, the device includes:

The data detection module is used to periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived, where the preset archiving conditions include height value conditions and access frequency conditions;

A data fragmentation module, which is used to perform fragmentation processing on the data to be archived to obtain fragmented data;

The distributed storage engine obtaining module is used to identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

The data storage module is used to store the fragmented data to the storage node in the target distributed storage engine.
The device according to claim 10, wherein the data detection module is further configured to count the total amount of data between the data with the lowest height value and the data with the highest height value in the blockchain node according to the cycle time, the The cycle time is less than the time required to generate a data set that meets the preset total amount. When the total amount of data is greater than the preset total amount, the data sets that meet the preset total amount are obtained sequentially from the lowest height value, and all The access frequency of each data in the data set, when each data in the data set is less than the preset access frequency, then the data in the data set is determined as the data to be archived.
The device according to claim 10, wherein the data fragmentation module is further configured to divide the data to be archived into at least two groups to obtain at least two groups of grouped data, and add each grouped data to at least two groups. In the two data fragments, fragment data is obtained.
A computer device includes a memory and one or more processors. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the one or more Each processor performs the following steps:

Periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived, where the preset archiving conditions include height value conditions and access frequency conditions;

Performing fragmentation processing on the data to be archived to obtain fragmented data;

Identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

Storing the fragmented data to a storage node in the target distributed storage engine.
The computer device according to claim 13, wherein the processor further executes the following steps when executing the computer-readable instruction:

Count the total amount of data between the data with the lowest height value and the data with the highest height value in the blockchain node according to the cycle time, and the cycle time is less than the time required to generate a data set meeting the preset total amount;

When the total amount of data is greater than the preset total amount, sequentially acquiring data sets that meet the preset total amount from the lowest height value;

Check the access frequency of each data in the data set; and

When each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.
The computer device according to claim 13, wherein the processor further executes the following steps when executing the computer-readable instruction:

Calculate the hash value of the fragmented data according to the hash algorithm; and

Store the fragmented data in the storage node pointed to by the hash value in the target distributed storage engine.
The computer device according to claim 13, wherein the processor further executes the following steps when executing the computer-readable instruction:

Evaluate and classify each storage node, and determine the storage performance of each storage node;

Perform weight distribution evaluation on the fragmented data to determine the storage requirements of the fragmented data; and

According to the storage performance of each storage node and the storage requirement of each fragmented data, the fragmented data is stored to the storage node in the target distributed storage engine.
One or more non-volatile computer-readable storage media storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors perform the following steps:

Periodically detect data that meets preset archiving conditions in the blockchain data to obtain data to be archived, where the preset archiving conditions include height value conditions and access frequency conditions;

Performing fragmentation processing on the data to be archived to obtain fragmented data;

Identify the data source type of the fragmented data, obtain the distributed storage engine corresponding to the data source type, and obtain the target distributed storage engine; and

Storing the fragmented data to a storage node in the target distributed storage engine.
18. The storage medium of claim 17, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Count the total amount of data between the data with the lowest height value and the data with the highest height value in the blockchain node according to the cycle time, and the cycle time is less than the time required to generate a data set meeting the preset total amount;

When the total amount of data is greater than the preset total amount, sequentially acquiring data sets that meet the preset total amount from the lowest height value;

Check the access frequency of each data in the data set; and

When each data in the data set is less than the preset access frequency, the data in the data set is determined as the data to be archived.
18. The storage medium of claim 17, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Calculate the hash value of the fragmented data according to the hash algorithm; and

Store the fragmented data in the storage node pointed to by the hash value in the target distributed storage engine.
18. The storage medium of claim 17, wherein the following steps are further executed when the computer-readable instructions are executed by the processor:

Evaluate and classify each storage node, and determine the storage performance of each storage node;

Perform weight distribution evaluation on the fragmented data to determine the storage requirements of the fragmented data; and

According to the storage performance of each storage node and the storage requirement of each fragmented data, the fragmented data is stored to the storage node in the target distributed storage engine.