WO2020258855A1 - 基于区块链的分级存储方法及装置、电子设备 - Google Patents

基于区块链的分级存储方法及装置、电子设备 Download PDF

Info

Publication number
WO2020258855A1
WO2020258855A1 PCT/CN2020/072026 CN2020072026W WO2020258855A1 WO 2020258855 A1 WO2020258855 A1 WO 2020258855A1 CN 2020072026 W CN2020072026 W CN 2020072026W WO 2020258855 A1 WO2020258855 A1 WO 2020258855A1
Authority
WO
WIPO (PCT)
Prior art keywords
node
data
block number
value
key
Prior art date
Application number
PCT/CN2020/072026
Other languages
English (en)
French (fr)
Inventor
陆钟豪
卓海振
俞本权
Original Assignee
创新先进技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 创新先进技术有限公司 filed Critical 创新先进技术有限公司
Priority to US16/790,605 priority Critical patent/US11036720B2/en
Publication of WO2020258855A1 publication Critical patent/WO2020258855A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • G06F16/2228Indexing structures
    • G06F16/2246Trees, e.g. B+trees
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases

Definitions

  • One or more embodiments of this specification relate to the field of blockchain technology, and in particular to a method and device for hierarchical storage based on blockchain, and electronic equipment.
  • Blockchain technology also known as distributed ledger technology, is an emerging technology in which several computing devices participate in "bookkeeping" and jointly maintain a complete distributed database. Because the blockchain technology has the characteristics of decentralization, openness and transparency, each computing device can participate in database records, and the rapid data synchronization between computing devices, the blockchain technology has been widely used in many fields. To apply.
  • This specification proposes a hierarchical storage method based on the blockchain.
  • the data nodes on the Merkle state tree organized into account state data of the blockchain are stored in the database in the form of Key-Value pairs; among them,
  • the key of the Key-Value key-value pair is a two-tuple consisting of the node ID of the data node and the block number marked for the data node;
  • the database includes multi-level data storage;
  • the block number indicates The block where the data node is located when the data is updated; the method includes:
  • the two-tuple of the target data node it is further searched whether the same node ID as the target data node is stored in the target data storage, and the block number in the two-tuple is greater than the Target data node and a data node smaller than the migration threshold; if yes, migrate the target data node to the next level of the target data storage; otherwise, keep the target in the target data storage Data node.
  • the determining a migration threshold based on the block number interval includes:
  • the right interval of the block number interval is an open interval, determining the right end value of the block number interval as the migration critical value
  • the right interval of the block number interval is a closed interval
  • the sum of the right end point value of the block number interval and the block number increment step of the blockchain is determined as the migration critical value.
  • the search is to find whether the target data storage stores the same node ID as the target data node, and the block number in the two-tuple is greater than the target data node and smaller than the migration
  • the critical value data nodes include:
  • the block number in the two-tuple is the largest block number among the block numbers that are less than the migration threshold Data node
  • the method further includes:
  • the key of the Key-Value key-value pair is a two-tuple composed of the block number of the latest block and the node ID of the data node; the Value of the Key-Value key-value pair is the data The data content contained in the node.
  • the Merkle tree is a variant of the Merkle tree incorporating the tree structure of the Trie dictionary tree; the node ID of the data node is a character prefix corresponding to the path from the root node of the Merkle tree to the data node.
  • the Merkle state tree is a Merkle Patricia Tree state tree.
  • the database is a LevelDB database; or a database based on the LevelDB architecture;
  • the database is a Rocksdb database based on the LevelDB architecture.
  • the read and write performance of the storage medium corresponding to the multi-level data storage has performance differences; wherein, the read and write performance of the storage medium corresponding to the high-level data storage is higher than that of the low-level data storage. Read and write performance of storage media.
  • This specification also proposes a hierarchical storage device based on the blockchain.
  • the data nodes on the Merkle state tree organized by the account state data of the blockchain are stored in the database in the form of Key-Value pairs; wherein The key of the Key-Value key-value pair is a two-tuple consisting of the node ID of the data node and the block number marked for the data node; the database includes multi-level data storage; the block number Indicate the block where the data node is located when the data is updated; the device includes:
  • the determining module when any level of target data storage in the database meets the data migration condition, determines the block number interval corresponding to the data node to be migrated to the next level of data storage in the target data storage; based on the The block number interval determines the migration threshold; wherein the migration threshold value is greater than the block number threshold in the block number interval;
  • a search module that iteratively traverses the two-tuple of data nodes stored in the target data storage, and searches for a target data node whose block number in the two-tuple is less than the migration threshold;
  • the migration module based on the two-tuple of the target data node, further searches whether the target data storage stores the same node ID as the target data node, and the block number in the two-tuple Data nodes larger than the target data node and smaller than the migration threshold; if yes, migrate the target data node to the next level data storage of the target data storage; otherwise, keep it in the target data storage The target data node.
  • the determining module :
  • the right interval of the block number interval is an open interval, determining the right end value of the block number interval as the migration critical value
  • the right interval of the block number interval is a closed interval
  • the sum of the right end point value of the block number interval and the block number increment step of the blockchain is determined as the migration critical value.
  • the migration module further:
  • the block number in the two-tuple is the largest block number among the block numbers that are less than the migration threshold Data node; further determining whether the block number in the two-tuple of the data node found is greater than the block number in the two-tuple of the target data node.
  • the device further includes:
  • the storage module determines the data node whose data is updated on the Merkle state tree of the latest block; constructs a Key-Value key-value pair for the data node whose data is updated on the Merkle state tree of the latest block, and combines the Key- Value key-value pairs are stored in the highest level data storage in the database;
  • the key of the Key-Value key-value pair is a two-tuple composed of the block number of the latest block and the node ID of the data node; the Value of the Key-Value key-value pair is the data The data content contained in the node.
  • the Merkle tree is a variant of the Merkle tree incorporating the tree structure of the Trie dictionary tree; the node ID of the data node is a character prefix corresponding to the path from the root node of the Merkle tree to the data node.
  • the Merkle state tree is a Merkle Patricia Tree state tree.
  • the database is a LevelDB database; or a database based on the LevelDB architecture;
  • the database is a Rocksdb database based on the LevelDB architecture.
  • the read and write performance of the storage medium corresponding to the multi-level data storage has performance differences; wherein, the read and write performance of the storage medium corresponding to the high-level data storage is higher than that of the low-level data storage. Read and write performance of storage media.
  • FIG. 1 is a schematic diagram of organizing account state data of a blockchain into an MPT state tree according to an exemplary embodiment
  • Fig. 2 is a schematic diagram of node multiplexing on an MPT state tree provided by an exemplary embodiment
  • Figure 3 is a flowchart of a block chain-based hierarchical storage method provided by an exemplary embodiment
  • Fig. 4 is a schematic structural diagram of an electronic device provided by an exemplary embodiment
  • Fig. 5 is a block diagram of a block chain-based hierarchical storage device provided by an exemplary embodiment.
  • the steps of the corresponding method may not be executed in the order shown and described in this specification.
  • the method includes more or fewer steps than described in this specification.
  • a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step in other embodiments. description.
  • Blockchain is generally divided into three types: Public Blockchain, Private Blockchain and Consortium Blockchain.
  • the most decentralized one is the public chain.
  • the public chain is represented by Bitcoin and Ethereum. Participants who join the public chain can read the data records on the chain, participate in transactions, and compete for the accounting rights of new blocks.
  • each participant ie, node
  • the private chain is the opposite.
  • the write permission of the network is controlled by an organization or institution, and the data read permission is regulated by the organization.
  • the private chain can be a weakly centralized system with strict restrictions and few participating nodes. This type of blockchain is more suitable for internal use by specific institutions.
  • the blockchain is usually composed of several blocks.
  • a time stamp corresponding to the creation time of the block is recorded in these blocks, and all the blocks strictly follow the time stamp recorded in the block to form a time-ordered data chain.
  • the real data generated in the physical world it can be constructed into a standard transaction format supported by the block chain, and then published to the block chain.
  • the node devices in the block chain will make a consensus, and after reaching a consensus ,
  • the node device as the accounting node in the blockchain will package the transaction into the block and carry out persistent storage in the blockchain.
  • Ethereum In the field of blockchain, an important concept is the account; taking Ethereum as an example, Ethereum usually divides accounts into external accounts and contract accounts; external accounts are accounts directly controlled by users; and contracts The account is created by the user through an external account and contains the contract code (ie smart contract).
  • the account types supported by the blockchain can also be further extended, which is not particularly limited in this specification.
  • a structure is usually used to maintain the account status of the account.
  • the status of the account related to the transaction in the blockchain usually changes.
  • the structure of an account usually includes fields such as Balance, Nonce, Code, and storage. among them:
  • the Balance field is used to maintain the current account balance of the account
  • the Nonce field is used for the number of transactions of the account; it is a counter used to ensure that each transaction can be processed and can only be processed once, effectively avoiding replay attacks.
  • the code field is used to maintain the contract code of the account; in actual applications, the code field usually only maintains the hash value of the contract code; therefore, the code field is usually also called the codehash field. For external accounts, this field is empty.
  • the storage field is used to maintain the storage of the account (empty by default). In practical applications, the storage field only maintains the root node of the MPT (Merkle Patricia Trie) tree constructed based on the storage content of the account; therefore, the storage field is usually also called the storageRoot field.
  • MPT Merkle Patricia Trie
  • the code field and storage field shown above are null values.
  • Merkle trees Most blockchain projects usually use Merkle trees; or, based on the data structure of Merkle trees, to store and maintain data.
  • Merkle trees Take Ethereum as an example.
  • Ethereum uses the MPT tree (a variant of Merkle tree) as a form of data organization to organize and manage important data such as account status and transaction information.
  • Ethereum has designed three MPT trees for the data that needs to be stored and maintained in the blockchain, namely the MPT state tree, the MPT transaction tree and the MPT receipt tree.
  • the MPT state tree is the MPT tree organized into account state data (state) of all accounts in the blockchain;
  • the MPT transaction tree is the MPT tree organized into the transaction data (transaction) in the block;
  • the MPT receipt tree is The transaction receipt (receipt) corresponding to each transaction generated after the transaction in the block is executed is organized into an MPT tree.
  • the hash values of the root nodes of the MPT state tree, MPT transaction tree, and MPT receipt tree shown above will all be added to the block header.
  • the MPT transaction tree and the MPT receipt tree correspond to the blocks, and each block has its own MPT transaction tree and MPT receipt tree.
  • the MPT state tree is a global MPT tree, which does not correspond to a specific block, but covers the account state data of all accounts in the blockchain.
  • the organized MPT transaction tree, MPT receipt tree, and MPT state tree will eventually be stored in a Key-Value database (for example, LevelDB) that uses a multi-level data storage structure.
  • a Key-Value database for example, LevelDB
  • each level of data storage can be set to L0, L1, L2, L3...L(n-1) in sequence; for the above
  • L0 stores the latest blocks of data
  • L1 stores the latest blocks of data
  • the read and write performance of storage media corresponding to all levels of data storage may also generally have performance differences; the read and write performance of storage media corresponding to high-level data storage (that is, with a smaller level number) can be higher than that of low-level data storage The read and write performance of the storage medium corresponding to the data storage.
  • high-level data storage can use storage media with higher read and write performance; while low-level data storage can use storage media with low unit cost and larger capacity.
  • the data stored in the database will contain a lot of historical data; moreover, the smaller the block number, the longer the data in the block, and the less important it is. Therefore, in order to reduce the overall storage cost, it is usually necessary to "discriminately treat" data with different block heights;
  • data in a block with a smaller block number can be stored on a storage medium with a lower cost; and data in a block with a larger block number can be stored on a storage medium with a higher cost.
  • Fig. 1 is a schematic diagram of organizing the account state data of the blockchain into an MPT state tree shown in this specification.
  • the MPT tree is an improved Merkle tree variant that combines the advantages of the Merkle tree and the Trie dictionary tree (also called the prefix tree).
  • the MPT tree usually includes three types of data nodes, which are leaf nodes, extension nodes, and branch nodes.
  • Leaf node expressed as a key-value pair of [key, value], where key is a special hexadecimal code.
  • the extended node is also a key-value pair of [key, value], but the value here is the hash value (hash pointer) of other nodes. In other words, link to other nodes through the hash pointer.
  • Branch node because the key in the MPT tree is encoded into a special hexadecimal representation, plus the final value, the branch node is a list of length 17, and the first 16 elements correspond to 16 of the keys Possible hexadecimal characters (a character corresponds to a nibble). If there is a [key, value] pair that terminates at this branch node, the last element represents a value value, that is, the branch node can be either the end of the search path or the intermediate node of the path.
  • the account address is a string of hexadecimal characters.
  • the account state state is a structure composed of the aforementioned Balance, Nonce, Code, and storage fields.
  • the MPT state tree organized according to the account state data in Table 1 is shown in Figure 1.
  • the MPT state tree organized according to the account state data in Table 1 is composed of 4 leaf nodes, 2 branch nodes, and 2 expansion nodes.
  • the prefix field is a prefix field shared by the extended node and the leaf node.
  • the value of the prefix field can be used to indicate the node type in practical applications.
  • the value of the prefix field is 0, which means an extended node that contains an even number of nibbles; as mentioned above, nibble means a half byte and consists of a 4-bit binary. A nibble can correspond to a character that composes the account address.
  • the value of the prefix field is 1, which means that the extended node contains an odd number of nibble(s);
  • the value of the prefix field is 2, which means that the leaf node contains an even number of nibbles
  • the value of the prefix field is 3, which means that the leaf node contains an odd number of nibble(s).
  • the branch node Since the branch node is a prefix node of a parallel single nibble, the branch node does not have the above prefix field.
  • the Shared nibble field in the extended node corresponds to the key value of the key-value pair contained in the extended node and represents the common character prefix between account addresses; for example, all account addresses in the above table have the common character prefix a7.
  • the Next Node field is filled with the hash value (hash pointer) of the next node.
  • the hexadecimal character 0 ⁇ f field in the branch node corresponds to the key value of the key-value pair contained in the branch node; if the branch node is an intermediate node on the search path of the account address on the MPT tree, then the branch node The Value field can be empty. The 0 ⁇ f fields are used to fill the hash value of the next node.
  • the Key-end in the leaf node corresponds to the key value of the key-value pair contained in the leaf node and represents the last few characters of the account address.
  • the key value of each node on the search path from the root node to the leaf node constitutes a complete account address.
  • the Value field of the leaf node is filled with account status data corresponding to the account address; for example, the structure composed of the aforementioned Balance, Nonce, Code, and storage fields can be numbered and then filled into the Value field of the leaf node.
  • node on the MPT state tree as shown in Figure 1 is finally stored in the database in the form of Key-Value key-value pairs;
  • the key in the key-value pair of the node on the MPT state tree is the hash value of the data content contained in the node; the key value of the node on the MPT state tree The Value in the pair is the data content contained in the node.
  • the hash value of the data content contained in the node can be calculated (that is, the hash calculation is performed on the entire node), and the calculated hash value is used as the key.
  • the data content contained in the node is used as the value to generate Key-Value key-value pairs; then, the generated Key-Value key-value pairs are stored in the database.
  • the node on the MPT state tree is stored based on the hash value of the data content contained in the node and the data content contained in the node as the value; therefore, when you need to query the node on the MPT state tree, you can usually base it on the node
  • the hash value of the contained data content is used as the key for content addressing. With “content addressing”, for some nodes with “repetitive content”, they can usually be “multiplexed” to save storage space for data storage.
  • Fig. 2 is a schematic diagram of node multiplexing on an MPT state tree shown in this specification.
  • the balances of the transferor account and transferee account related to the "transfer transaction” (that is, the value of the Balance field of these accounts) are usually also Will change accordingly.
  • the node device After the transaction of the node device in the latest block generated by the blockchain is completed, because the account status in the current blockchain has changed, the node device needs to determine the current account status data of all accounts in the blockchain. Construct an MPT tree to maintain the latest status of all accounts in the blockchain.
  • every block in the blockchain has a corresponding MPT state tree; the MPT state tree maintains that after the transactions in the block are executed, all accounts in the blockchain are up to date The status of the account.
  • the execution of a transaction in the latest block may only cause the account status of some accounts to change; therefore, when updating the MPT state tree, it does not need to be based on all the current accounts in the blockchain.
  • To rebuild a complete MPT state tree only need to update the node corresponding to the account whose state has changed on the basis of the MPT state tree corresponding to the block before the latest block. .
  • the nodes in the MPT state tree corresponding to the accounts whose account status has not changed since these nodes are data updates, the corresponding nodes on the MPT state tree corresponding to the block before the latest block can be directly reused, namely can.
  • Block N+1 updates the MPT state tree
  • the updated nodes are not the original MPT state tree corresponding to Block N.
  • the content update of a small number of nodes in the MPT state tree of Block N+1 can be used to "reuse" most of the nodes in the previous block Block N as an example.
  • the MPT state tree of Block N+1 may also add nodes to the previous block Block N.
  • the newly added node cannot be reused directly from the MPT tree of the previous block Block N, it may be "multiplexed" from the MPT state tree of the earlier block;
  • the new node on the MPT state tree of Block N+1 although it appeared on the MPT state tree of Block N, appears on the MPT state tree of earlier blocks; for example, it appears in Block N-1.
  • the newly added node on the MPT state tree of Block N+1 can directly reuse the corresponding node on the MPT state tree of Block N-1.
  • Another situation is that if the MPT state tree of a block has a new node compared to the MPT state tree of the previous block, the corresponding node on the MPT state tree of the earlier block can be "multiplexed".
  • node multiplexing although the storage space of the database can be saved, there may be complex multiplexing relationships between the nodes on the MPT state tree of each block.
  • the node on the MPT state tree of each block All may be reused by the next block, or several consecutive blocks after the next block; therefore, this complex node multiplexing relationship is bound to cause difficulties in the hierarchical storage of the MPT state tree.
  • pruning Refers to clearing the multiplexing relationship between nodes on the MPT state tree of each block, removing the nodes that record historical state data from the MPT state tree, and retaining the node that records the latest state data. In this case, it is obvious that the demand for hierarchical storage cannot be met.
  • this specification proposes a method for hierarchical storage of a Merkle state tree composed of blockchain account state data.
  • the account state data of the blockchain can still be organized into a Merkle state tree, and then the data nodes on the Merkle state tree are carried out in the form of Key-Value key-value pairs in a database with a multi-level data storage structure Storage;
  • the data structure of the MPT tree can still be used to organize the account state data of the blockchain into the MPT state tree; and for the nodes on the Merkle state tree stored in the form of Key-Value pairs in the database
  • each node can be marked with a block number, which is specifically used to indicate the block number of the block where the node is updated;
  • the transaction in the latest block generated by the blockchain after the transaction in the latest block generated by the blockchain is executed, it can be determined based on the execution result of the transaction in the latest block that the data occurred on the Merkle state tree of the latest block
  • the updated node; among them, the node where the data is updated usually includes the node where the value is updated and the new node.
  • the hash value of the content contained in the data node may no longer be used as the key of the Key-Value key-value pair
  • the key is a two-tuple composed of the node ID of the data node and the block number that can indicate the block where the data node is located when the data is updated.
  • any level of target data storage in the database meets the data migration conditions; for example, when the storage capacity of the target data storage reaches a threshold; first, it can be determined that the target data storage needs to be migrated to the next level of data storage
  • the block number interval corresponding to the node of, and the migration threshold value is determined based on the block number interval; wherein, the migration threshold value is greater than the block number threshold value of the block number interval;
  • the right interval of the block number interval is an open interval
  • the right end value of the block number interval is determined as the migration threshold; for example, suppose the block number interval is [a, b), then b is determined as the above migration critical value; if the right interval of the block number interval is a closed interval, the right end value of the block number interval is incremented by the block number of the blockchain
  • the long sum is determined as the above migration threshold; for example, assuming that the block number interval is [a, b], the block number of the block chain is incremented by 1 (that is, the block number in the block chain is 1 When it is an incremental step size (compactly increasing), then b+1 is determined as the above-mentioned migration critical value.
  • the migration critical value When the migration critical value is determined, iteratively traverse the two-tuple of data nodes stored in the target data storage to find the target data node whose block number in the two-tuple is less than the migration critical value;
  • the target data node When the target data node is found, according to the above-mentioned two-tuple of the target data node, continue to address in the target data storage, and further search whether the target data storage is stored, which has the same Node ID; and, the block number in the above-mentioned two-tuple is greater than the above-mentioned target data node, and is less than the data node of the migration threshold;
  • the target data node can migrate to the next level data storage of the target data store; for example, write the found target data node to the next level data store of the target data store, and if the write succeeds After that, the target data node is cleared from the target data storage; otherwise, the target data node can be kept in the aforementioned target data storage.
  • the nodes on the Merkle state tree stored in the database can be accurately pruned, and the nodes that record historical state data are removed from the Merkle state tree, migrated to the next level of data storage, and stored in this
  • the node that records the latest state data will continue to be stored and retained in the hierarchical data storage, thereby completing hierarchical storage of the Merkle state tree stored in the database.
  • FIG. 3 is a flowchart of a block chain-based hierarchical storage method provided by an exemplary embodiment.
  • the method is applied to blockchain node equipment; wherein, the data nodes on the Merkle state tree organized into the account state data of the blockchain are stored in the database in the form of Key-Value key-value pairs; wherein, The key of the Key-Value key-value pair is a two-tuple composed of the node ID of the data node and the block number marked for the data node; the database includes multi-level data storage; the block number indicates where The block where the data node is located when the data is updated; the method includes the following steps:
  • Step 302 When any level of target data storage in the database meets the data migration condition, determine the block number interval corresponding to the data node to be migrated to the next level of data storage in the target data storage;
  • Step 304 Determine a migration threshold value based on the block number interval; wherein the migration threshold value is greater than the block number threshold value of the block number interval;
  • Step 306 Iteratively traverse the two-tuple of data nodes stored in the target data store, and search for a target data node whose block number in the two-tuple is less than the migration threshold;
  • Step 308 according to the two-tuple of the target data node, further search whether the target data storage has the same node ID as the target data node and the block number in the two-tuple Data nodes larger than the target data node and smaller than the migration threshold; if yes, migrate the target data node to the next level data storage of the target data storage; otherwise, keep it in the target data storage The target data node.
  • the above-mentioned database may specifically be a Key-Value database with a multi-level data storage structure (stored in comparison; for example, in the illustrated embodiment, the above-mentioned database may be a LevelDB database; or, a database based on the LevelDB architecture ; For example, Rocksdb database is a typical database based on LevelDB database architecture.
  • the account state data in the blockchain can be organized into the data structure of the Merkle state tree and stored in the above-mentioned database; for example, the above-mentioned Merkle state tree may be an MPT tree, and the data structure of the MPT tree may be used to divide the block
  • the account state data of the chain is organized into an MPT state tree.
  • the following uses the data structure of the MPT tree to organize the account state data in the blockchain into an MPT state tree as an example to describe the technical solutions of this specification in detail;
  • Merkle tree variants with tree structure will not be listed one by one in this specification.
  • the user client connected to the blockchain can package the data into a standard transaction format supported by the blockchain, and then publish it to the blockchain; and the node devices in the blockchain can be based on The consensus algorithm of, together with other node devices, agrees on these transactions that the user client publishes to the blockchain to generate the latest block for the blockchain;
  • the consensus algorithms supported in the blockchain are usually divided into consensus algorithms in which node devices need to compete for the accounting rights of each round of accounting cycles, and pre-election of accounting nodes for each round of accounting cycles (no need to compete Accounting rights) consensus algorithm.
  • the former is represented by consensus algorithms such as Proof of Work (POW), Proof of Stake (POS), and Delegated Proof of Stake (DPOS); the latter is represented by Practical Byzantine fault tolerance (Practical Byzantine Fault Tolerance, PBFT) and other consensus algorithms are representative.
  • POW Proof of Work
  • POS Proof of Stake
  • DPOS Delegated Proof of Stake
  • PBFT Practical Byzantine fault tolerance
  • PBFT Practical Byzantine Fault Tolerance
  • nodes competing for the right to bookkeeping The device can execute the transaction after receiving the transaction.
  • One of the node devices competing for the right to book may win this round and become the bookkeeping node.
  • the accounting node can package the received transaction with other transactions and generate the latest block, and send the generated latest block to other node devices for consensus.
  • PBFT Practical Byzantine Fault Tolerance
  • the transaction can be executed during or before the process of packaging the transaction with other transactions and generating the latest block.
  • the accounting node packs the transaction together with other transactions to generate a new block, it can send the generated latest block or the block header of the latest block to other node devices for consensus.
  • this round of accounting nodes can package the received transactions and generate the latest block, and the generated latest block or the latest block
  • the header of the block is sent to other node devices for consensus verification. If other node devices receive the latest block or the block header of the latest block, and there is no problem after verification, the latest block can be appended to the end of the original blockchain to complete the blockchain accounting process.
  • the node device in the blockchain executes the transactions packaged in the latest block generated by consensus, the status of the account related to these executed transactions in the blockchain will usually change accordingly. ; Therefore, after the transaction packaged in the latest block is executed, the node device can organize the data structure of the MPT state tree according to the latest account status data of all accounts in the blockchain.
  • the data nodes on the MPT state tree can be used in the form of Key-Value key-value pairs in a multi-level data storage structure Stored in a Key-Value database.
  • the data nodes on the MPT state tree corresponding to the latest several blocks can be stored in the highest level L0 data storage in the aforementioned database by default.
  • the data nodes on the MPT state tree corresponding to the next several blocks can be stored in the next highest level L1 data storage in the above-mentioned database; and so on.
  • the number of blocks corresponding to the MPT state tree stored in each level of data storage is not particularly limited in this specification; for example, it can be specified that the MPT state of the latest N blocks in the highest level L0 data storage can be stored Tree, the next highest level L1 data storage MPT state tree of the next new N blocks; and so on.
  • the block number can be marked; where, the block number marked by the data node on the MPT state tree is specifically used to indicate when the data node is updated.
  • the node device can start a "block status update thread" locally to maintain and update the node status on the MPT status tree.
  • the "block status update thread” can determine the node where the data update occurred on the MPT state tree corresponding to the latest block; among them, the node where the data update occurred, It usually contains the updated node and the newly added node.
  • the “block state update thread” can mark the node whose data has been updated with the block number of the latest block, indicating that the node is in The block corresponding to this block number has been updated.
  • the "block status update thread" can promptly mark the node whose data has been updated on the MPT state tree corresponding to the latest block, Block number, so that by traversing each node on the MPT state tree stored in the above database, it is possible to know the block where the node is located when the data is updated through the block number marked for each node; because of the MPT state tree
  • the value value of each node (especially the leaf node on the MPT tree) after the most recent data update can usually indicate the latest state of the node; therefore, by viewing the block number marked for each node, you can know each node
  • the latest state of the database is generated in which block; so this mechanism of marking the block number for the node can provide a basis for the data migration of the above-mentioned database at all levels of data storage.
  • the "block status update thread" can specifically mark the above-mentioned block number for the node on the MPT state tree in the key of the Key-Value key-value pair corresponding to the node on the MPT state tree.
  • the "block state update thread" can construct Key-Value key-value pairs for these nodes whose data has been updated, and build Key-Value In the key of the key-value pair, mark the block number for the aforementioned node; then store the constructed Key-Value key in the aforementioned database.
  • the hash value of the content contained in the aforementioned data node may no longer be used as the key of the aforementioned Key-Value key-value pair, and
  • the two-tuple composed of the node ID of the data node and the block number that can indicate the block where the data node is located when the data node is updated is used as the key of the Key-Value key-value pair.
  • the Value of the aforementioned Key-Value key-value pair can still be the data content contained in the data node.
  • the above two-tuple can be recorded as [nid, h], nid represents NodeID; represents the block number marked by the node.
  • the hash value of the data content contained in the data node will no longer be used as the key of the data node; instead, the node ID of the data node and the block that can indicate the data node when the data is updated are used.
  • the two-tuple composed of the block number is used as the key of the data node.
  • the addressing method used by the above-mentioned database will also change accordingly; for the above-mentioned database, the hash value of the data content contained in the node will no longer be used as the key for content addressing; instead, , Will use the node ID and the block number of the block where the data node is located when the data is updated for addressing.
  • the node ID of the above data node specifically refers to the unique identifier of the data node in the MPT state tree
  • a variant of Merkle tree that incorporates a tree structure of Trie dictionary tree, such as MPT tree usually uses a character prefix corresponding to the path from the root node of the MPT tree to the data node; for example, For any leaf node shown in Figure 1 and Figure 2, the character prefix corresponding to the path from the root node to the leaf node is usually the account address corresponding to the leaf node; that is, the node ID of the leaf node on the MPT tree, That is, the account address corresponding to the leaf node.
  • the node ID of the above data node can also be specifically when constructing the MPT tree.
  • the node number set for each data node that can uniquely distinguish each data node is not particularly limited in this specification.
  • the node device can also start a "migration thread" locally, which is used to migrate the node data on the MPT state tree stored in the data storage at all levels in the database to the lower data storage.
  • a "migration thread” is used to migrate the node data on the MPT state tree stored in the data storage at all levels in the database to the lower data storage.
  • the above-mentioned "migration thread” can specifically execute a timed task, which can periodically determine whether the data storage at all levels in the database meets the preset data migration conditions;
  • the data migration conditions of the various levels of data storage in the aforementioned database can be set based on actual data migration requirements, which are not particularly limited in this specification;
  • the data migration condition of the data storage at all levels of the database may be specifically that the storage capacity of the data storage at all levels reaches the threshold; or it may also be the block corresponding to the data stored in the data storage at all levels The number reaches the threshold.
  • the above-mentioned "migration thread” determines that any level of target data storage in the above-mentioned database meets the data migration conditions, then the "migration thread” can perform data migration processing for the target data storage, and the part stored in the target data storage
  • the MPT state tree of the block is used as historical data to migrate to the next level of data storage.
  • the "migration thread" can determine that the target data storage needs to be migrated to the block number interval corresponding to the node of the next level data storage;
  • the block number range of the node corresponding to the node that needs to be migrated to the next level of data storage in the target data storage can be based on the block number of the next block after the previous data migration occurred, and the target data storage can be migrated once
  • the maximum number of blocks to be moved is determined; for example, suppose the next block after the last data migration is Block N, and the number of blocks that can be migrated for the target data storage at one time is 30, then the above block number range can be specifically [N, N+29]; Or, it can be [N, N+30).
  • the migration threshold value of this data migration can be determined based on the block number interval ; Wherein, the migration threshold may specifically be a block number threshold greater than the block number interval;
  • the right interval of the block number interval may be an open interval or a closed interval; and when the migration threshold is determined based on the block number interval, the block number When the right interval of the interval is an open interval or a closed interval, the determined migration threshold may also have a certain difference.
  • the right end value of the block number interval may be used as Determined as the above migration threshold
  • the migration threshold determined based on the block number interval is the right end value b of the block number interval.
  • the right end value of the block number interval may be The sum of the incremental step size of the block number of the blockchain is determined as the above migration threshold
  • the migration threshold determined based on the block number interval is the right end of the block number interval The value b+1.
  • the block number of the above-mentioned block chain is incremented by a step length, which can usually be 1. That is, the block number of the block chain is densely incremented according to the step length; for example, the block number is incremented according to 1, 2, 3, 4...increment.
  • the increment step size of the block number of the blockchain can also be an integer greater than 1.
  • the block number can also be incremented in the order of 1, 3, 5, 7..., in this specification It is not particularly limited.
  • the target data stored in the target data can be stored in the same block number interval according to the migration threshold.
  • the MPT state tree of the corresponding block is used as historical data to migrate to the next level of data storage.
  • the aforementioned "migration thread" can iteratively traverse the keys of each node stored in the target data store; that is, iteratively traverse the node ID of each node stored in the target data store and the marked block number.
  • the "migration thread" can continue to address the target data store according to the two-tuple of the target node, Further find out whether the above target data storage is stored, which has the same node ID as the above target node (that is, has the same character prefix); and, the block number in the above two-tuple is greater than the above target node and less than the above migration threshold Value of node;
  • the target node can be used as the historical data of the next level of data storage migrated to the target data storage, the above "migration thread"
  • the found target node can be migrated to the next level data storage of the target data storage;
  • the aforementioned "migration thread” can copy the target node and store the copied target node to the next level of data storage, and then after the copied target node is successfully stored to the next level of data storage, Then remove the target node from the above target data storage.
  • the target node will continue to be retained in the target data storage.
  • the migration threshold determined based on the block number interval is the right end value b of the block number interval; in this case, Suppose the above-mentioned "migration thread" finds a nodeA in the target data store, and the above-mentioned two-tuple of this nodeA is (nid, i); where i ⁇ b; then, the above-mentioned "migration thread” can be based on the two-tuple (nid, i) Addressing, and further find out whether the nodeB whose binary group is (nid, j) is stored in the above-mentioned target data storage; where b>j>i; if there is nodeB, nodeA can be migrated to the above-mentioned target data storage The next level of data storage; otherwise, the nodeA will continue to be reserved in the target data storage.
  • the above-mentioned "migration thread” can iteratively traverse the above-mentioned two-tuple of data nodes stored in the above-mentioned target data store, and iteratively execute the above-mentioned migration process, and finally can only be retained in the above-mentioned target data store.
  • the node corresponding to the largest block number whose block number in the above two-tuple is less than the migration threshold value stored in the above target data storage it usually represents the latest state of the node after the latest update, so through this This method is equivalent to only keeping the latest state of each node in the above target data storage.
  • the “migration thread” mentioned above is The above-mentioned two-tuple of the above-mentioned target node continues to be used as a query index, and when addressing in the above-mentioned target data storage, content addressing is no longer used.
  • the above-mentioned database can provide the function of "querying the maximum key less than a certain value"; the so-called “querying the maximum key less than a certain value” function specifically refers to the function of "querying the maximum key less than a certain value”.
  • the two-tuple key composed of] is used as the query index for addressing, and the process of finding the two-tuple key composed of [nid, h max ] stored in the database; among them, h max is the binary of all nodes stored in the database In the group, the maximum block number among the block numbers smaller than h. That is, when determining h max , you can first find out all the binary groups with block numbers less than h stored in the database; and then take the maximum value of the block numbers in these found binary groups as the above h max .
  • the “migration thread” when it continues to address in the target data store according to the above-mentioned two-tuple of the target node, it can be based on the node ID (denoted as nid) of the target node and the migration threshold.
  • the value (denoted as h) is spliced into a two-tuple [nid, h], and then the spliced two-tuple [nid, h] is used as the query index to further find out whether the two-tuple [nid] is stored in the target data store. , H max ].
  • the block number in the above-mentioned two-tuple of the above-mentioned target node is The maximum block number that is less than the migration threshold.
  • the processing actions corresponding to the aforementioned "block status update thread” and the aforementioned “migration thread” can be executed concurrently; that is, the aforementioned "block status update thread” and the aforementioned “migration thread” can be used for MPT
  • the same node on the state tree performs concurrent contention processing. Therefore, in practical applications, in order to avoid the “block status update thread” and the above “migration thread” from concurrently competing for the same node, which may cause processing errors, technical means can be used to ensure that at any time, only There can be one thread to access the same node in the MPT state tree.
  • this application also provides an embodiment of an apparatus.
  • this specification also provides an embodiment of a blockchain-based hierarchical storage device.
  • the embodiment of the blockchain-based hierarchical storage device in this specification can be applied to electronic equipment.
  • the device embodiments can be implemented by software, or by hardware or a combination of software and hardware.
  • Taking software implementation as an example as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory through the processor of the electronic device where it is located.
  • FIG. 4 it is a hardware structure diagram of the electronic equipment where the blockchain-based hierarchical storage device is located in this specification, except for the processor, memory, network interface, and non-
  • the electronic device in which the device is located in the embodiment may also include other hardware according to the actual function of the electronic device, which will not be repeated here.
  • Fig. 5 is a block diagram of a hierarchical storage device based on blockchain according to an exemplary embodiment of this specification.
  • the block chain-based hierarchical storage device 50 can be applied to the electronic device shown in FIG. 4, and the account state data of the block chain is organized into data nodes on the Merkle state tree to The form of the Key-Value key-value pair is stored in the database; wherein the key of the Key-Value key-value pair is a two-tuple composed of the node ID of the data node and the block number marked for the data node;
  • the database includes multi-level data storage; the block number indicates the block where the data node is located when the data update occurs; the device 50 includes:
  • the determining module 501 when any level of target data storage in the database meets the data migration conditions, determine the block number interval corresponding to the data node to be migrated to the next level of data storage in the target data storage;
  • the block number interval determines a migration critical value; wherein the migration critical value is greater than the block number critical value of the block number interval;
  • the searching module 502 iteratively traverses the two-tuple of data nodes stored in the target data store, and searches for the target data node whose block number in the two-tuple is less than the migration threshold;
  • the migration module 503 based on the two-tuple of the target data node, further searches whether the target data storage stores the same node ID as the target data node, and the block in the two-tuple Data nodes whose numbers are greater than the target data node and less than the migration threshold; if yes, migrate the target data node to the next level of the target data storage; otherwise, in the target data storage Keep the target data node.
  • the determining module 501 the determining module 501:
  • the right interval of the block number interval is an open interval, determining the right end value of the block number interval as the migration critical value
  • the right interval of the block number interval is a closed interval
  • the sum of the right end point value of the block number interval and the block number increment step of the blockchain is determined as the migration critical value.
  • the migration module 503 further:
  • the block number in the two-tuple is the largest block number among the block numbers that are less than the migration threshold Data node; further determining whether the block number in the two-tuple of the data node found is greater than the block number in the two-tuple of the target data node.
  • the device 50 further includes:
  • the storage module determines the data node on the Merkle state tree of the latest block that has data update; constructs the Key-Value key value for the data node on the Merkle state tree of the latest block. Yes, and store the Key-Value key-value pair in the highest-level data store in the database;
  • the key of the Key-Value key-value pair is a two-tuple composed of the block number of the latest block and the node ID of the data node; the Value of the Key-Value key-value pair is the data The data content contained in the node.
  • the Merkle tree is a variant of the Merkle tree incorporating the tree structure of the Trie dictionary tree; the node ID of the data node is the character prefix corresponding to the path from the root node of the Merkle tree to the data node .
  • the Merkle state tree is the Merkle Patricia Tree state tree.
  • the database is a LevelDB database; or a database based on the LevelDB architecture;
  • the database is Rocksdb database based on LevelDB architecture.
  • the read and write performance of the storage medium corresponding to the multi-level data storage has performance differences; among them, the read and write performance of the storage medium corresponding to the high-level data storage is higher than that of the low-level data storage.
  • the read and write performance of the storage medium is higher than that of the low-level data storage.
  • a typical implementation device is a computer.
  • the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email receiving and sending device, and a game control A console, a tablet computer, a wearable device, or a combination of any of these devices.
  • the computer includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
  • processors CPU
  • input/output interfaces network interfaces
  • memory volatile and non-volatile memory
  • the memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer readable media.
  • RAM random access memory
  • ROM read-only memory
  • flash RAM flash memory
  • Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology.
  • the information can be computer-readable instructions, data structures, program modules, or other data.
  • Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, Magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transitory media, such as modulated data signals and carrier waves.
  • first, second, third, etc. may be used in one or more embodiments of this specification to describe various information, the information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other.
  • first information may also be referred to as second information, and similarly, the second information may also be referred to as first information.
  • word “if” as used herein can be interpreted as "when” or “when” or "in response to determination”.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种基于区块链的分级存储方法,区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;key为节点ID和区块号组成的二元组;数据库包括多级数据存储;区块号指示数据节点发生数据更新时所在的区块;包括:当任一级目标数据存储满足了数据迁移条件时,确定目标数据存储中待迁移的区块号区间;基于区块号区间确定迁移临界值;迭代遍历目标数据存储中存储的二元组,查找二元组中的区块号小于迁移临界值的目标数据节点;查找目标数据存储中是否存储了与目标数据节点具有相同的节点ID,以及二元组中的区块号大于目标数据节点且小于迁移临界值的数据节点;如果是,将目标数据节点迁移至下一级数据存储。

Description

基于区块链的分级存储方法及装置、电子设备 技术领域
本说明书一个或多个实施例涉及区块链技术领域,尤其涉及一种基于区块链的分级存储方法及装置、电子设备。
背景技术
区块链技术,也被称之为分布式账本技术,是一种由若干台计算设备共同参与“记账”,共同维护一份完整的分布式数据库的新兴技术。由于区块链技术具有去中心化、公开透明、每台计算设备可以参与数据库记录、并且各计算设备之间可以快速的进行数据同步的特性,使得区块链技术已在众多的领域中广泛的进行应用。
发明内容
本说明书提出一种基于区块链的分级存储方法,所述区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述方法包括:
当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;
基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
可选的,所述基于所述区块号区间确定迁移临界值,包括:
如果所述区块号区间的右区间为开区间,将所述区块号区间的右端点值,确定为所述迁移临界值;
如果所述区块号区间的右区间为闭区间,将所述区块号区间的右端点值与所述区块链的区块号递增步长的和确定为所述迁移临界值。
可选的,所述查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点,包括:
查找所述目标数据存储中存储的与所述目标数据节点具有相同的节点ID,并且所述二元组中的区块号为小于所述迁移临界值的区块号中的最大区块号的数据节点;
进一步确定查找到的该数据节点的所述二元组中的区块号,是否大于所述目标数据节点的所述二元组中的区块号。
可选的,所述方法还包括:
确定最新区块的Merkle状态树上发生数据更新的数据节点;为所述最新区块的Merkle状态树上发生数据更新的数据节点构建Key-Value键值对,并将所述Key-Value键值对存储至所述数据库中的最高级数据存储;
其中,所述Key-Value键值对的key为所述最新区块的区块号和所述数据节点的节点ID组成的二元组;所述Key-Value键值对的Value为所述数据节点包含的数据内容。
可选的,所述Merkle树为融合了Trie字典树的树形结构的Merkle树变种;所述数据节点的节点ID,为所述Merkle树的根节点到该数据节点的路径对应的字符前缀。
可选的,所述Merkle状态树为Merkle Patricia Tree状态树。
可选的,所述数据库为LevelDB数据库;或者基于LevelDB架构的数据库;
可选的,所述数据库为基于LevelDB架构的Rocksdb数据库。
可选的,所述多级数据存储对应的存储介质的读写性能,存在性能差异;其中,级数高的数据存储对应的存储介质的读写性能,高于级数低的数据存储对应的存储介质的读写性能。
本说明书还提出一种基于区块链的分级存储装置,所述区块链的账户状态数据组 织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述装置包括:
确定模块,当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
查找模块,迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
迁移模块,根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
可选的,所述确定模块:
如果所述区块号区间的右区间为开区间,将所述区块号区间的右端点值,确定为所述迁移临界值;
如果所述区块号区间的右区间为闭区间,将所述区块号区间的右端点值与所述区块链的区块号递增步长的和确定为所述迁移临界值。
可选的,所述迁移模块进一步:
查找所述目标数据存储中存储的与所述目标数据节点具有相同的节点ID,并且所述二元组中的区块号为小于所述迁移临界值的区块号中的最大区块号的数据节点;进一步确定查找到的该数据节点的所述二元组中的区块号,是否大于所述目标数据节点的所述二元组中的区块号。
可选的,所述装置还包括:
存储模块,确定最新区块的Merkle状态树上发生数据更新的数据节点;为所述最新区块的Merkle状态树上发生数据更新的数据节点构建Key-Value键值对,并将所述 Key-Value键值对存储至所述数据库中的最高级数据存储;
其中,所述Key-Value键值对的key为所述最新区块的区块号和所述数据节点的节点ID组成的二元组;所述Key-Value键值对的Value为所述数据节点包含的数据内容。
可选的,所述Merkle树为融合了Trie字典树的树形结构的Merkle树变种;所述数据节点的节点ID,为所述Merkle树的根节点到该数据节点的路径对应的字符前缀。
可选的,所述Merkle状态树为Merkle Patricia Tree状态树。
可选的,所述数据库为LevelDB数据库;或者基于LevelDB架构的数据库;
可选的,所述数据库为基于LevelDB架构的Rocksdb数据库。
可选的,所述多级数据存储对应的存储介质的读写性能,存在性能差异;其中,级数高的数据存储对应的存储介质的读写性能,高于级数低的数据存储对应的存储介质的读写性能。
在以上技术方案中,可以实现对数据库中存储的Merkle状态树上的数据节点进行剪枝,将记录了历史状态数据的数据节点从Merkle状态树上移除,迁移至下一级数据存储,并在本级数据存储中继续存储和保留记录了最新状态数据的数据节点,进而完成针对数据库中存储的Merkle状态树的分级存储。
附图说明
图1是一示例性实施例提供的一种将区块链的账户状态数据组织成MPT状态树的示意图;
图2是一示例性实施例提供的一种MPT状态树上的node复用的示意图;
图3是一示例性实施例提供的一种基于区块链的分级存储方法的流程图;
图4是一示例性实施例提供的一种电子设备的结构示意图;
图5是一示例性实施例提供的一种基于区块链的分级存储装置的框图。
具体实施方式
这里将详细地对示例性实施例进行说明,其示例表示在附图中。下面的描述涉及附图时,除非另有表示,不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本说明书一个或多个实施例相一致的所有实施方 式。相反,它们仅是与如所附权利要求书中所详述的、本说明书一个或多个实施例的一些方面相一致的装置和方法的例子。
需要说明的是:在其他实施例中并不一定按照本说明书示出和描述的顺序来执行相应方法的步骤。在一些其他实施例中,其方法所包括的步骤可以比本说明书所描述的更多或更少。此外,本说明书中所描述的单个步骤,在其他实施例中可能被分解为多个步骤进行描述;而本说明书中所描述的多个步骤,在其他实施例中也可能被合并为单个步骤进行描述。
区块链一般被划分为三种类型:公有链(Public Blockchain),私有链(Private Blockchain)和联盟链(Consortium Blockchain)。此外,还有多种类型的结合,比如私有链+联盟链、联盟链+公有链等不同组合形式。其中去中心化程度最高的是公有链。公有链以比特币、以太坊为代表,加入公有链的参与者可以读取链上的数据记录、参与交易以及竞争新区块的记账权等。
而且,各参与者(即节点)可自由加入以及退出网络,并进行相关操作。私有链则相反,该网络的写入权限由某个组织或者机构控制,数据读取权限受组织规定。简单来说,私有链可以为一个弱中心化系统,参与节点具有严格限制且少。这种类型的区块链更适合于特定机构内部使用。
基于区块链的基本特性,区块链通常是由若干个区块构成。在这些区块中分别记录有与该区块的创建时刻对应的时间戳,所有的区块严格按照区块中记录的时间戳,构成一条在时间上有序的数据链条。
对于物理世界产生的真实数据,可以将其构建成区块链所支持的标准的交易(transaction)格式,然后发布至区块链,由区块链中的节点设备进行共识,并在达成共识后,由区块链中作为记账节点的节点设备,将这笔交易打包进区块,在区块链中进行持久化存证。
在区块链领域,有一个重要的概念就是账户(Account);以以太坊为例,以太坊通常将账户划分为外部账户和合约账户两类;外部账户就是由用户直接控制的账户;而合约账户则是由用户通过外部账户创建的,包含合约代码的账户(即智能合约)。
当然,对于一些基于以太坊的架构而衍生出的区块链项目(比如蚂蚁区块链),还可以对区块链支持的账户类型,进行进一步的扩展,在本说明书中不进行特别限定。
对于区块链中的账户而言,通常会通过一个结构体,来维护账户的账户状态。当 区块中的交易被执行后,区块链中与该交易相关的账户的状态通常也会发生变化。
以以太坊为例,账户的结构体通常包括Balance,Nonce,Code和storage等字段。其中:
Balance字段,用于维护账户目前的账户余额;
Nonce字段,用于该账户的交易次数;它是用于保障每笔交易能且只能被处理一次的计数器,有效避免重放攻击。
code字段,用于维护该账户的合约代码;在实际应用中,code字段中通常仅维护合约代码的hash值;因而,code字段通常也称之为codehash字段。对于外部账户而言,该字段为空值。
storage字段,用于维护该账户的存储(默认为空)。在实际应用中,storage字段仅维护基于账户的存储内容构建的MPT(Merkle Patricia Trie)树的根节点;因此,storage字段通常也称之为storageRoot字段。
其中,对于外部账户而言,以上示出的code字段和storage字段为空值。
而大多数区块链项目,通常都会使用Merkle树;或者,基于Merkle树的数据结构,来存储和维护数据。以以太坊为例,以太坊使用了MPT树(一种Merkle树变种),作为数据组织形式,用来组织和管理账户状态、交易信息等重要数据。
以太坊针对区块链中需要存储和维护的数据,设计了三颗MPT树,分别是MPT状态树、MPT交易树和MPT收据树。
MPT状态树,是区块链中所有账户的账户状态数据(state),组织成的MPT树;MPT交易树是区块中的交易数据(transaction),组织成的MPT树;MPT收据树,是区块中的交易执行完毕后生成的与每笔交易对应的交易收据(receipt),组织成的MPT树。以上示出的MPT状态树、MPT交易树和MPT收据树的根节点的hash值,都会被添加至区块头中。
其中,MPT交易树和MPT收据树,与区块相对应,每一个区块都有自己的MPT交易树和MPT收据树。而MPT状态树是一个全局的MPT树,并不与某一个特定的区块相对应,而是涵盖了区块链中所有账户的账户状态数据。
对于组织成的MPT交易树、MPT收据树和MPT状态树,最终都会在采用多级数据存储结构的Key-Value型数据库(比如,LevelDB)中进行存储。
而采用多级存储结构的上述数据库,通常可以被划分为n级数据存储;例如,各级数据存储可以依次设为L0,L1,L2,L3....L(n-1);对于上述数据库中的各级数据存储而言,等级编号越小通常级别越高;例如,L0存储的是最新的若干区块的数据,L1存储的是次新的若干区块数据,依次类推。
其中,各级数据存储对应的存储介质的读写性能,通常也可以存在性能差异;级别高(即等级编号较小的)的数据存储对应的存储介质的读写性能,可以高于级别低的数据存储对应的存储介质的读写性能。
例如,在实际应用中,级别高的数据存储,可以使用读写性能较高的存储介质;而级别低的数据存储,可以使用单位成本低,且容量较大的存储介质。
在实际应用中,随着区块高度的增长,在数据库中存储的数据,会包含很多历史数据;而且,区块号越小的区块中的数据越久远,越不重要。因此,为了降低整体的存储成本,通常需要对不同区块高度的数据进行“区别对待”;
例如,可以将区块号较小的区块中的数据,存储至成本较低的存储介质上;而将区块号较大的区块中的数据,存储在成本较高的存储介质上。
在针对数据库中存储的MPT交易树、MPT收据树和MPT状态树等数据进行分级存储时,由于MPT交易树和MPT收据树,与各个区块相对应,实际上是“区块间无关”的数据;因此,对于MPT交易树和MPT收据树,很容易进行分级存储;例如,直接按照MPT交易树和MPT收据树上的node所属的区块号进行数据迁移即可完成分级存储。
基于此,本说明书将不再具体阐述MPT交易树和MPT收据树的分级存储,而重点阐述MPT状态树的分级存储。
请参见图1,图1为本说明书示出的一种将区块链的账户状态数据组织成MPT状态树的示意图。
MPT树,是一种经过改良的,融合了Merkle树和Trie字典树(也称之为前缀树)两种树形结构的优点的Merkle树变种。
在MPT树中通常包括三种数据节点,分别为叶子节点(leaf node),扩展节点(extension node)和分支节点(branch node)。
叶子节点,表示为[key,value]的一个键值对,其中key是种特殊十六进制编码。
扩展节点,也是[key,value]的一个键值对,但是这里的value是其他节点的hash 值(hash指针)。也就是说通过hash指针链接到其他节点。
分支节点,因为MPT树中的key被编码成一种特殊的16进制的表示,再加上最后的value,所以分支节点是一个长度为17的list,前16个元素对应着key中的16个可能的十六进制字符(一个字符对应一个半字节nibble)。如果有一个[key,value]对在这个分支节点终止,最后一个元素代表一个value值,即分支节点既可以是搜索路径的终止也可以是路径的中间节点。
假设需要组织成MTP状态树的账户状态数据如下表1所示:
Figure PCTCN2020072026-appb-000001
表1
在表1中,账户地址是由若干16进制的字符构成的字符串。账户状态state,是由上述Balance,Nonce,Code和storage等字段构成的结构体。
最终按照表1中的账户状态数据组织成的MPT状态树,参见图1所示;如图1所示,按照表1中的账户状态数据组织成的MPT状态树,是由4个叶子节点,2个分支节点,和2个扩展节点构成。
在图1中,prefix字段为扩展节点和叶子节点共同具有的前缀字段。该prefix字段的取值,在实际应用中可以用于表示节点类型。
prefix字段的取值为0,表示包含偶数个nibbles的扩展节点;如前所述,nibble表示半字节,由4位二进制组成,一个nibble可以对应一个组成账户地址的字符。
prefix字段的取值为1,表示包含奇数个nibble(s)的扩展节点;
prefix字段的取值为2,表示包含偶数个nibbles的叶子节点;
prefix字段的取值为3,表示包含奇数个nibble(s)的叶子节点。
而分支节点,由于其是并列单nibble的前缀节点,因此分支节点不具有上述prefix字段。
扩展节点中的Shared nibble字段,对应该扩展节点所包含的键值对的key值,表示账户地址之间的共同字符前缀;比如,上表中的所有账户地址均具有共同的字符前缀a7。Next Node字段中填充下一个节点的hash值(hash指针)。
分支节点中的16进制字符0~f字段,对应该分支节点所包含的键值对的key值;如果该分支节点为账户地址在MPT树上的搜索路径上的中间节点,则该分支节点的Value字段可以为空值。0~f字段中用于填充下一个节点的hash值。
叶子节点中的Key-end,对应该叶子节点所包含的键值对的key值,表示账户地址的最后几个字符。从根节点搜索到叶子节点的搜索路径上的各个节点的key值,构成了一个完整的账户地址。该叶子节点的Value字段填充账户地址对应的账户状态数据;例如,可以对上述Balance,Nonce,Code和storage等字段构成的结构体进行编号后,填充至叶子节点的Value字段。
进一步的,如图1所示的MPT状态树上的node,最终也是以Key-Value键值对的形式存储在数据库中;
其中,当MPT状态树上的node在数据库中进行存储时,MPT状态树上的node的键值对中的key,为node所包含的数据内容的hash值;MPT状态树上的node的键值对中的Value,为node所包含的数据内容。
也即,在将MPT状态树上的node存储至数据库时,可以计算该node所包含的数据内容的hash值(即对node整体进行hash计算),并将计算出的hash值作为key,将该node所包含的数据内容作为value,生成Key-Value键值对;然后,将生成的Key-Value键值对存储至数据库中。
由于MPT状态树上的node,是以node所包含的数据内容的hash值为Key,node所包含的数据内容为value进行存储;因此,在需要查询MPT状态树上的node时,通常可以基于node所包含的数据内容的hash值作为key来进行内容寻址。而采用“内容寻址”,对于一些“内容重复”的node,则通常可以进行“复用”,以节约数据存储的存储空间。
如图2所示,图2为本说明书示出的一种MPT状态树上的node复用的示意图。
在实际应用中,区块链每产生一个最新区块,则在该最新区块中的交易被执行之后,区块链中与这些被执行的交易相关账户的账户状态,通常也会随之发生变化;
例如,当区块中的一笔“转账交易”执行完毕后,与该“转账交易”相关的转出 方账户和转入方账户的余额(即这些账户的Balance字段的取值),通常也会随之发生变化。
而节点设备在区块链产生的最新区块中的交易执行完毕后,由于当前区块链中的账户状态发生了变化,因此节点设备需要根据区块链中所有账户当前的账户状态数据,来构建MPT树,用于维护区块链中所有账户的最新状态。
也即,每当区块链中产生一个最新区块,并且该最新区块中的交易执行完毕后,导致区块链中的账户状态发生变化,节点设备都需要基于区块链中所有账户最新的账户状态数据,重新构建一颗MPT树。
换句话说,区块链中每一个区块,都有一个与之对应的MPT状态树;该MPT状态树,维护了在该区块中的交易在执行完毕后,区块链中所有账户最新的账户状态。
而需要说明的是,一个最新区块中的交易执行完毕后,可能仅仅会导致部分账户的账户状态发生变化;因此,在更新MPT状态树时,并不需要基于区块链中所有的账户当前的状态数据,重新构建一颗完整的MPT状态树,而只需要在该最新区块之前的区块对应的MPT状态树的基础上,对部分账户状态发生变化的账户对应的node进行更新即可。而对于MPT状态树上与账户状态未发生变化的账户对应的node而言,由于这些node为发生数据更新,可以直接复用该最新区块之前的区块对应的MPT状态树上相应的node即可。
如图2所示,假设表1中的账户状态数据,为Block N中的交易执行完毕后,区块链上所有账户的最新账户状态;基于表1中的账户状态数据组织成的MPT状态树,仍如图1所示。
假设当Block N+1中的交易执行完毕后,导致上述表1中的账户地址为“a7f9365”的账户状态,由“state3”更新为“state5”;此时,在Block N+1更新MPT状态树时,并不需要基于Block N+1中的交易执行完毕后,区块链中所有的账户当前的状态数据,重新构建一颗MPT状态树。
请参见图2,在这种情况下,可以仅将Block N对应的MPT树上(即图1示出的MPT状态树),“key-end”为“9365”的叶子节点中的Value,由“state3”更新为“state5”,并继续更新从root节点到该叶子节点的路径上的所有节点的hash指针;也即,当MPT状态树上的叶子节点发生更新,由于该叶子节点整体的hash值发生更新,那么从根节点到该叶子节点的路径上的所有的节点的hash指针也会随之发生更新。例如,请继续 参见图2,除了需要更新“key-end”为“9365”的叶子节点中的Value值以外,还需要更新该叶子节点的上一个分支节点(Branch Node)的f字段中填充的,指向该叶子节点的哈希指针;进一步的,还可以继续向根节点追溯,继续更新该分支节点的上一个根节点(Root Extension Node)的“Next Node”字段中填充的,指向该分支节点的hash指针。
而除了以上发生更新的节点以外,其它未发生更新的节点,都可以直接复用Block N的MPT状态树上对应的节点即可;
其中,由于Block N对应的MPT树,最终需要作为历史数据进行保留;因此,在Block N+1更新MPT状态树时,对于这些发生更新的node,并不是对Block N对应的MPT状态树上原来的node的基础上,直接进行修改更新,而是在Block N+1对应的MPT树上重新创建这些发生更新的node。
也即,对于与Block N+1对应的MPT状态树上,实际上只需要重新创建少量发生更新的node,对于其它未发生更新的node,可以通过直接复用Block N对应的MPT状态树上对应的节点。
例如,如图2所示,对于Block N+1对应的MPT状态树上,实际上只需要重新创建少量发生更新的node;比如,图2中仅需要重新创建一个作为根节点的扩展节点、一个分支节点和一个叶子节点;对于未发生更新的node,可以通过在该MPT状态树上这些重新创建的node中,添加指向Block N对应的MPT状态树上的相应node的hash指针来完成node的复用。而Block N对应的MPT状态树上那些更新前的node,将作为历史账户状态数据进行保存;比如,图2示出的“key-end”为“9365”,且Value为“state3”的叶子节点,将作为历史数据进行保留。在以上例子中,以Block N+1的MPT状态树上的少量node发生内容更新,可以“复用”上一个区块Block N的大多数node为例进行了说明。而在实际应用中,Block N+1的MPT状态树上也可能会较上一个区块Block N新增node。
在这种情况下,该新增的node虽然无法直接从上一个区块Block N的MPT树中进行复用,但有可能从更早之前的区块的MPT状态树上进行“复用”;
例如,Block N+1的MPT状态树上新增的node,虽然在Block N的MPT状态树上出现过,但出现在更早的Block的MPT状态树上;比如,出现在Block N-1的MPT状态树上;因此,Block N+1的MPT状态树上新增的node,可以直接复用Block N-1的 MPT状态树上对应的node即可。
可见,MPT状态树的节点复用,一共有两种“复用”的情形:
一种情形是,一个区块的MPT状态树上只有少量的节点发生内容更新,则可以“复用”上一个区块的大多数节点;
另一种情形是,一个区块的MPT状态树较上一个区块的MPT状态树新增了节点,则可以“复用”更早之前的区块的MPT状态树上对应的节点。
然而,通过节点复用,虽然可以节约数据库的存储空间,但由于各个区块的MPT状态树上的节点之间,可能存在复杂的复用关系,每个区块的MPT状态树上的node,都可能被下一个区块,或者是下一个区块之后的若干连续的区块进行复用;因此,这种复杂的节点复用关系,势必会对MPT状态树的分级存储造成困难。
例如,在需要将一些节点作为历史数据,从本级数据存储向下一级数据存储迁移时,由于这些节点可能会被下一个区块;甚至是该下一个区块之后的若干个区块复用;而这些节点将会被哪些节点复用,又是无法准确预知的;因此,这就会导致无法对数据库中存储的MPT状态树上的节点进行精确剪枝;其中,所谓剪枝,是指清除各区块的MPT状态树上的节点之间的复用关系,将记录了历史状态数据的节点从MPT状态树上移除,并保留记录了最新状态数据的node。在这种情况下,显然无法满足分级存储的需求。
基于此,本说明书提出一种针对由区块链的账户状态数据组成的Merkle状态树进行分级存储的方法。
在实现时,仍然可以将区块链的账户状态数据组织成Merkle状态树,然后将Merkle状态树上的数据节点,以Key-Value键值对的形式在采用多级数据存储结构的数据库中进行存储;例如,仍然可以采用MPT树的数据结构,将区块链的账户状态数据组织成MPT状态树;而对于在数据库中以Key-Value键值对的形式存储的Merkle状态树上的node而言,可以在这些node对应的Key-Value键值对的key中,为这些node分别标记一个区块号,该区块号具体用于指示该node发生数据更新时所在区块的区块号;
例如,在一种实施方式中,区块链产生的最新区块中的交易执行完毕后,可以基于该最新区块中的交易的执行结果,来确定该最新区块的Merkle状态树上发生数据更新的node;其中,发生数据更新的node,通常包含value值发生更新的node和新增的node。当确定该最新区块的Merkle状态树上发生数据更新的node之后,可以在这些node 对应的Key-Value键值对的key中,为这些发生数据更新的node标记该最新区块的区块号,表示该node在该区块号对应的区块发生了数据更新。
其中,Merkle状态树上的数据节点以Key-Value键值对的形式在数据库中进行存储时,可以不再采用上述数据节点所包含内容的hash值作为上述Key-Value键值对的key,而是采用上述数据节点的节点ID和能够指示该数据节点发生数据更新时所在区块的区块号组成的二元组作为key。
进一步的,当数据库中的任一级目标数据存储满足了数据迁移条件时;比如,当该目标数据存储的存储容量达到阈值;首先,可以确定该目标数据存储中需要迁移至下一级数据存储的node对应的区块号区间,并基于该区块号区间确定迁移临界值;其中,该迁移临界值为大于该区块号区间的区块号临界值;
例如,在一种实施方式中,如果该区块号区间的右区间为开区间,则将该区块号区间的右端点值,确定为上述迁移临界值;比如,假设该区块号区间为[a,b)时,则将b确定为上述迁移临界值;如果该区块号区间的右区间为闭区间,将该区块号区间的右端点值与区块链的区块号递增步长的和,确定为上述迁移临界值;比如,假设该区块号区间为[a,b],区块链的区块号递增步长为1(即区块链中的区块号以1为递增步长致密递增)时,则将b+1确定为上述迁移临界值。
当确定了上述迁移临界值后,迭代遍历上述目标数据存储中存储的数据节点的上述二元组,查找上述二元组中的区块号小于上述迁移临界值的目标数据节点;
当查找到上述目标数据节点之后,根据该目标数据节点的上述二元组,继续在上述目标数据存储中进行寻址,进一步查找上述目标数据存储中是否存储了,与上述目标数据节点具有相同的节点ID;以及,上述二元组中的区块号大于上述目标数据节点,且小于所述迁移临界值的数据节点;
如果是,可以将该目标数据节点迁移至上述目标数据存储的下一级数据存储;例如,将查找到的该目标数据节点写入到该目标数据存储的下一级数据存储,在写入成功后,再将该目标数据节点从目标数据存储中清除;否则,可以在上述目标数据存储中继续保留该目标数据节点。
通过以上技术方案,可以实现对数据库中存储的Merkle状态树上的node进行精确剪枝,将记录了历史状态数据的node从Merkle状态树上移除,迁移至下一级数据存储,并在本级数据存储中继续存储和保留记录了最新状态数据的node,进而完成针对数据库 中存储的Merkle状态树的分级存储。
请参见图3,图3是一示例性实施例提供的一种基于区块链的分级存储方法的流程图。所述方法应用于区块链节点设备;其中,所述区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述方法包括以下步骤:
步骤302,当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;
步骤304,基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
步骤306,迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
步骤308,根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
上述数据库,具体可以是采用多级数据存储结构的Key-Value型数据库(比中进行存储;例如,在示出的一种实施方式中,上述数据库可以为LevelDB数据库;或者,基于LevelDB架构的数据库;比如,Rocksdb数据库就是一种典型的基于LevelDB数据库架构的数据库。
区块链中的账户状态数据,可以被组织成Merkle状态树的数据结构,在上述数据库中进行存储;例如,上述Merkle状态树具体可以是MPT树,可以采用MPT树的数据结构,将区块链的账户状态数据组织成MPT状态树。
以下以采用MPT树的数据结构,将区块链中的账户状态数据组织成MPT状态树为例,对本说明书的技术方案进行详细描述;
其中,需要强调的是,以采用MPT树的数据结构来组织区块链中的账户状态数据,仅为示例性的。
在实际应用中,对于基于以太坊架构而衍生出的区块链项目,除了可以采用诸如MPT树等改良版的Merkle树以外,也可以采用其他形式的类似于MPT树的融合了Trie字典树的树形结构的Merkle树变种,在本说明书中不再进行一一列举。
在本说明书中,接入区块链的用户客户端,可以将数据打包成区块链所支持的标准的交易格式,然后发布至区块链;而区块链中的节点设备,可以基于搭载的共识算法与其它节点设备一起,对用户客户端发布至区块链的这些交易进行共识,以此来为区块链产生最新区块;
其中,区块链中支持的共识算法,通常分为节点设备需要争夺每一轮的记账周期的记账权的共识算法,和预先为每一轮记账周期选举记账节点(不需要争夺记账权)的共识算法。
例如,前者以工作量证明(Proof of Work,POW)、股权证明(Proof of Stake,POS)、委任权益证明(Delegated Proof of Stake,DPOS)等共识算法为代表;后者以实用拜占庭容错(Practical Byzantine Fault Tolerance,PBFT)等共识算法为代表。
对于采用工作量证明(Proof of Work,POW)以及股权证明(Proof of Stake,POS)、委任权益证明(Delegated Proof of Stake,DPOS)等共识算法的区块链网络中,争夺记账权的节点设备,都可以在接收到交易后执行该笔交易。争夺记账权的节点设备中可能其中一个在本轮争夺记账权的过程中胜出,成为记账节点。记账节点可以将收到的交易与其它交易一起打包并生成最新区块,并将生成的最新区块发送至其它节点设备进行共识。
对于采用实用拜占庭容错(Practical Byzantine Fault Tolerance,PBFT)等共识算法的区块链网络中,具有记账权的节点设备在本轮记账前已经商定好。因此,节点设备在接收到交易后,如果自身不是本轮的记账节点,则可以将该交易发送至记账节点。
对于本轮的记账节点,在将该交易与其它交易一起打包并生成最新区块的过程中或者之前,可以执行该交易。记账节点在将该交易与其它交易一起打包生成新区块后,可以将生成的最新区块或者该最新区块的区块头发送至其它节点设备进行共识。
如上所述,无论区块链采用以上示出的哪种共识算法,本轮的记账节点都可以将接收到的交易打包并生成最新区块,并将生成的最新区块或者该最新区块的区块头发送至其它节点设备进行共识验证。如果其它节点设备接收到最新区块或者该最新区块的区块头后,经验证没有问题,可以将该最新区块追加到原有的区块链末尾,从而完成区块 链的记账过程。
在本说明书中,区块链中的节点设备在执行了经过共识产生的最新区块中打包的交易之后,区块链中与这些被执行的交易相关的账户状态,通常也会随之发生变化;因此,节点设备在最新区块中打包的交易执行完毕后,可以根据区块链中所有账户最新的账户状态数据,组织成MPT状态树的数据结构。
其中,根据区块链中所有账户最新的账户状态数据,组织成MPT状态树时,仍然可以采用如图2示出的复用该最新区块之前的区块对应的MPT树上的节点的方式,在本说明书中不再赘述。
当节点设备根据区块链中所有账户最新的账户状态数据,组织成MPT状态树后,可以将该MPT状态树上的数据节点,以Key-Value键值对的形式在采用多级数据存储结构的Key-Value型数据库中进行存储。
例如,在实际应用中,对于最新的若干区块对应的MPT状态树上的数据节点,可以默认存放在上述数据库中的级别最高的L0级数据存储中。对于次新的若干区块对应的MPT状态树上的数据节点,可以存放在上述数据库中的级别次高的L1级数据存储中;以此类推。其中,每一级数据存储所存储的MPT状态树对应的区块数,在本说明书中不进行特别限定;比如,可以规定级别最高的L0级数据存储中存储最新的N个区块的MPT状态树,级别次高的L1级数据存储次新的N个区块的MPT状态树;以此类推。
在本说明书,对于构建的MPT状态树上的数据节点,可以被标记区块号;其中,为MPT状态树上的数据节点所标记的区块号,具体用于指示该数据节点发生数据更新时所在的区块;
例如,如图2所示,以Block N+1对应的MPT状态树上发生数据更新的node为例,为这些node标记的区块号,即为N+1,以表示这些node在Block N+1中的交易执行完毕后,发生了数据更新。
在实现时,节点设备可以在本地启动一个“区块状态更新线程”,来维护和更新MPT状态树上的node的状态。当区块链产生的最新区块中的交易执行完毕后,该“区块状态更新线程”可以确定该最新区块对应的MPT状态树上发生数据更新的node;其中,发生数据更新的node,通常包含value值发生更新的node和新增的node。
例如,在实现时,可以直接根据该最新区块中的交易的执行结果,来确定该最新区块对应的MPT状态树上发生数据更新的node;或者,也可以通过判断MPT状态树 上的node是否为复用的node,来确定该最新区块对应的MPT状态树上发生数据更新的node;比如,如果是复用的node,则表明该node是未发生数据更新的node;
当确定该最新区块对应的MPT状态树上发生数据更新的node之后,该“区块状态更新线程”,可以为这些发生数据更新的node标记该最新区块的区块号,表示该node在该区块号对应的区块发生了数据更新。
通过这种方式,每当区块链产生一个最新区块,“区块状态更新线程”都可以及时的对该最新区块对应的MPT状态树上发生数据更新的node,标记该最新区块的区块号,使得通过遍历上述数据库中存储的MPT状态树上的各个node,就能够通过为各个node标记的区块号,获知到该node发生数据更新时所在的区块;由于MPT状态树上的各个node(尤其是MPT树上的叶子节点)最近一次发生数据更新后的value值,通常可以指示该node的最新状态;因此,通过查看为各个node标记的区块号,则可以获知各个node的最新状态,是在哪个区块产生的;从而这种为node标记区块号的机制,可以对上述数据库的各级数据存储的数据迁移提供依据。
其中,在本说明书中,“区块状态更新线程”,具体可以在MPT状态树上的node对应的Key-Value键值对的key中,为MPT状态树上的node标记上述区块号。
当确定该最新区块对应的MPT状态树上发生数据更新的node之后,“区块状态更新线程”,可以为这些发生数据更新的node构建Key-Value键值对,并在构建的Key-Value键值对的key中,为上述node标记区块号;然后将构建的Key-Value键值在上述数据库中进行存储。
其中,MPT状态树上的数据节点以Key-Value键值对的形式在数据库中进行存储时,可以不再采用上述数据节点所包含内容的hash值作为上述Key-Value键值对的key,而是采用上述数据节点的节点ID和能够指示该数据节点发生数据更新时所在区块的区块号组成的二元组作为上述Key-Value键值对的key。上述Key-Value键值对的Value仍然可以为数据节点所包含的数据内容。例如,上述二元组可以记为[nid,h],nid代表NodeID;代表为该节点标记的区块号。
也即,在本说明书中,将不再使用数据节点所包含的数据内容的hash值,作为数据节点的key;而是采用数据节点的节点ID和能够指示该数据节点发生数据更新时所在区块的区块号组成的二元组作为数据节点的key。
通过这种方式,上述数据库所采用的寻址方式也随之会发生变化;对于上述数据 库而言,将不再采用node所包含的数据内容的hash值作为key来进行内容寻址;取而代之的是,将采用节点ID和数据节点发生数据更新时所在区块的区块号来进行寻址。
其中,需要说明的是,上述数据节点的节点ID,具体是指数据节点在MPT状态树中的唯一标识;
例如,在实际应用中,对应诸如MPT树这种融合了Trie字典树的树形结构的Merkle树变种,通常采用从为MPT树的根节点到该数据节点的路径对应的字符前缀;比如,如图1和图2示出的任意一个叶子节点而言,从根节点到该叶子节点的路径对应的字符前缀通常为叶子节点对应的账户地址;也即,MPT树上的叶子节点的节点ID,即为该叶子节点对应的账户地址。
当然,在实际应用中,除了采用MPT树的根节点到数据节点的路径对应的字符前缀作为该数据节点的节点ID以外,上述数据节点的节点ID,具体也可以是在构建MPT树时,为各数据节点设置的能够唯一区分各个数据节点的节点编号,在本说明书中不进行特别限定。
在本说明书中,节点设备还可以在本地启动一个“迁移线程”,用于将上述数据库中各级数据存储中存储的MPT状态树上的node数据,向下级数据存储进行迁移。
上述“迁移线程”,具体可以执行一个定时任务,可以周期性的确定上述数据库中的各级数据存储是否满足预设的数据迁移条件;
其中,上述数据库中的各级数据存储的数据迁移条件,可以基于实际的数据迁移需求来设置,在本说明书中不进行特别限定;
例如,在实际应用中,上述数据库的各级数据存储的数据迁移条件,具体可以是各级数据存储的存储容量达到阈值;或者,也可以是各级数据存储所存储的数据所对应的区块数达到阈值。
上述“迁移线程”,在确定上述数据库中任一级目标数据存储,满足了数据迁移条件,则该“迁移线程”可以针对该目标数据存储进行数据迁移处理,将该目标数据存储中存储的部分区块的MPT状态树,作为历史数据向下一级数据存储进行迁移。
在实现时,当目标数据存储满足了数据迁移条件时,上述“迁移线程”可以确定该目标数据存储中,需要迁移至下一级数据存储的node对应的区块号区间;
其中,该目标数据存储中需要迁移至下一级数据存储的node对应的区块号区间, 可以基于上一次数据迁移发生后的下一区块的区块号,和该目标数据存储一次能够迁移走的最大区块数来确定;比如,假设上一次数据迁移后的下一个区块是Block N,该目标数据存储一次能够迁移走的区块数为30,那么上述区块号区间具体可以是[N,N+29];或者,也可以是[N,N+30)。
当上述“迁移线程”确定出该目标数据存储中,需要迁移至下一级数据存储的node对应的区块号区间之后,可以基于该区块号区间,来确定本次数据迁移的迁移临界值;其中,该迁移临界值具体可以是一个大于该区块号区间的区块号临界值;
需要说明的是,在实际应用中,上述区块号区间的右区间具体可以是开区间,也可以是闭区间;而在基于上述区块号区间来确定上述迁移临界值时,上述区块号区间的右区间为开区间或者闭区间时,确定出的迁移临界值也可以存在一定的差异。
在示出的一种实施方式中,如果上述区块号区间的右区间为开区间,在基于上述区块号区间来确定上述迁移临界值时,可以将上述区块号区间的右端点值,确定为上述迁移临界值;
例如,假设上述区块号区间为[a,b),那么基于该区块号区间确定出的迁移临界值为该区块号区间的右端点值b。
在示出的另一种实施方式中,如果上述区块号区间的右区间为闭区间,在基于上述区块号区间来确定上述迁移临界值时,可以将上述区块号区间的右端点值与区块链的区块号递增步长的和,确定为上述迁移临界值;
例如,假设上述区块号区间为[a,b],区块链的区块号递增步长为1,那么基于该区块号区间确定出的迁移临界值为该区块号区间的右端点值b+1。
其中,上述区块链的区块号递增步长,通常可以是1;也即,区块链的区块号按照1为步长进行致密递增;比如,区块号按照1、2、3、4…的顺序进行递增。
当然,在实际应用中,区块链的区块号递增步长具体也可以是大于1的整数;比如,区块号也可以按照1、3、5、7…的顺序进行递增,在本说明书中不进行特别限定。
进一步的,当上述“迁移线程”,基于上述区块号区间,确定出本次数据迁移的迁移临界值之后,可以根据该迁移临界值,将该目标数据存储中存储的与上述区块号区间对应的区块的MPT状态树,作为历史数据向下一级数据存储进行迁移。
具体地,上述“迁移线程”,可以迭代遍历上述目标数据存储中存储的各个node 的key;也即,迭代遍历上述目标数据存储中存储的各个node的节点ID和被标记的区块号所组成的上述二元组,来查找上述目标数据存储中存储的所有的node中,上述二元组中的区块号小于上述迁移临界值的目标node。
当查找到一个上述二元组中的区块号小于上述迁移临界值的目标node之后,上述“迁移线程”可以根据该目标node的上述二元组,继续在上述目标数据存储中进行寻址,进一步查找上述目标数据存储中是否存储了,与上述目标node具有相同的节点ID(也即具有相同的字符前缀);以及,上述二元组中的区块号大于上述目标node且小于上述迁移临界值的node;
如果存在,表明该目标node所代表的账户状态,并不是最新的账户状态;此时该目标node,即可以作为迁移至该目标数据存储的下一级数据存储的历史数据,上述“迁移线程”可以将查找到的该目标node,迁移至该目标数据存储的下一级数据存储;
例如,在实现时,上述“迁移线程”可以复制该目标node,并将复制的该目标node存储至下一级数据存储,然后在将复制的该目标node成功存储至下一级数据存储之后,再将该目标node从上述目标数据存储中清除。
反之,如果不存在,表明该目标node所代表的账户状态,为最新的账户状态;在这种情况下,该目标node将会继续在该目标数据存储中进行保留。
例如,举例而言,假设上述区块号区间为[a,b),那么基于该区块号区间确定出的迁移临界值为该区块号区间的右端点值b;在这种情况下,假设上述“迁移线程”在目标数据存储中查找到一个nodeA,该nodeA的上述二元组为(nid,i);其中i<b;那么,上述“迁移线程”可以根据二元组(nid,i)进行寻址,进一步查找上述目标数据存储中是否存储了二元组为(nid,j)的nodeB;其中,b>j>i;如果存在nodeB,可以将nodeA迁移至上述目标数据存储的下一级数据存储;反之,在该目标数据存储中继续保留该nodeA。
而上述“迁移线程”可以通过迭代遍历上述目标数据存储中存储的数据节点的上述二元组,并迭代执行上述迁移过程,最终可以仅在上述目标数据存储中保留,上述二元组中的区块号小于上述迁移临界值的最大区块号所对应的node;而除了该node以外的其它node,都将作为历史数据迁移至下一级数据存储。
由于上述目标数据存储中存储的,上述二元组中的区块号小于上述迁移临界值的最大区块号所对应的node,通常代表了该node在最近一次更新后的最新状态,因此通过这种方式,相当于仅在上述目标数据存储中保留了各个node的最新状态。
其中,需要说明的是,由于在本说明书中,MPT状态树上的node的Key-Value键值对的key,不再是node所包含的数据内容的hash值;因此,上述“迁移线程”在将上述目标node的上述二元组继续作为查询索引,在上述目标数据存储中进行寻址时,将不再采用内容寻址的方式。
在示出的一种实施方式中,上述数据库可以提供“查询小于某值的最大值key”的功能;所谓“查询小于某值的最大值key”功能,具体是指将一个由[nid,h]组成的二元组key作为查询索引进行寻址,查找出数据库中存储的由[nid,h max]组成的二元组key的过程;其中,h max为数据库中存储的所有node的二元组中,小于h的区块号之中的区块号最大值。也即,在确定h max时,可以先查找出数据库中存储的所有区块号小于h的二元组;然后再取查找出的这些二元组中的区块号最大值作为上述h max
在这种情况下,上述“迁移线程”在根据上述目标node的上述二元组继续在上述目标数据存储中进行寻址时,可以根据上述目标node的节点ID(记为nid)和上述迁移临界值(记为h)拼接出二元组[nid,h],然后将拼接出的该二元组[nid,h]作为查询索引,进一步查找上述目标数据存储中是否存储了二元组[nid,h max]。
当查找到二元组[nid,h max]之后,可以进一步确定h max是否大于上述目标node的上述二元组中的区块号;如果是,此时可以确定上述目标数据存储中存储了与上述目标node具有相同的节点ID,以及上述二元组中的区块号大于上述目标node且小于上述迁移临界值的node;反之,说明上述目标node的上述二元组中的区块号即为小于上述迁移临界值的最大区块号,此时可以确定上述目标数据存储中并没有存储与上述目标node具有相同的节点ID,以及上述二元组中的区块号大于上述目标node且小于上述迁移临界值的node。
在以上技术方案中,由于上述目标数据存储中存储的,上述二元组中的区块号小于迁移临界值的node中,只有上述二元组中的最大区块号对应的node能继续保留在该目标数据存储,而其它的node都将作为历史数据迁移至下一级数据存储;
因此,按照上述方式完成数据迁移后,相当于对数据库中存储的MPT状态树上的node进行了一次剪枝,将记录了历史状态数据的node从MPT状态树上移除,迁移至下一级数据存储,并在本级数据存储中继续存储和保留记录了最新状态数据的node,进而完成针对数据库中存储的MPT状态树的分级存储。
其中,需要说明的是,上述“区块状态更新线程”和上述“迁移线程”对应的处 理动作,可以并发执行;也即,上述“区块状态更新线程”和上述“迁移线程”可以对MPT状态树上的同一个node进行并发竞争处理。因此,在实际应用中,为了避免上述“区块状态更新线程”和上述“迁移线程”对同一个node进行并发竞争处理,可能造成的处理错误,可以通过技术手段,来保证在任一时刻,只能有一个线程来访问MPT状态树上的同一个node。
其中,在保证在任一时刻,只能有一个线程来访问MPT状态树上的同一个node所采用的具体技术手段,在本说明书中不进行特别限定;例如,在实现时,可以采用互斥锁技术、单线程技术来实现,在本说明书中不再进行详述。
与上述方法实施例相对应,本申请还提供了装置的实施例。
与上述方法实施例相对应,本说明书还提供了一种基于区块链的分级存储装置的实施例。
本说明书的基于区块链的分级存储装置的实施例可以应用在电子设备上。装置实施例可以通过软件实现,也可以通过硬件或者软硬件结合的方式实现。以软件实现为例,作为一个逻辑意义上的装置,是通过其所在电子设备的处理器将非易失性存储器中对应的计算机程序指令读取到内存中运行形成的。
从硬件层面而言,如图4所示,为本说明书的基于区块链的分级存储装置所在电子设备的一种硬件结构图,除了图4所示的处理器、内存、网络接口、以及非易失性存储器之外,实施例中装置所在的电子设备通常根据该电子设备的实际功能,还可以包括其他硬件,对此不再赘述。
图5是本说明书一示例性实施例示出的一种基于区块链的分级存储装置的框图。
请参考图5,所述基于区块链的分级存储装置50可以应用在前述图4所示的电子设备中,所述区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述装置50包括:
确定模块501,当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
查找模块502,迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
迁移模块503,根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
在本说明书中,所述确定模块501:
如果所述区块号区间的右区间为开区间,将所述区块号区间的右端点值,确定为所述迁移临界值;
如果所述区块号区间的右区间为闭区间,将所述区块号区间的右端点值与所述区块链的区块号递增步长的和确定为所述迁移临界值。
在本说明书中,所述迁移模块503进一步:
查找所述目标数据存储中存储的与所述目标数据节点具有相同的节点ID,并且所述二元组中的区块号为小于所述迁移临界值的区块号中的最大区块号的数据节点;进一步确定查找到的该数据节点的所述二元组中的区块号,是否大于所述目标数据节点的所述二元组中的区块号。
在本说明书中,所述装置50还包括:
存储模块(图5中未示出),确定最新区块的Merkle状态树上发生数据更新的数据节点;为所述最新区块的Merkle状态树上发生数据更新的数据节点构建Key-Value键值对,并将所述Key-Value键值对存储至所述数据库中的最高级数据存储;
其中,所述Key-Value键值对的key为所述最新区块的区块号和所述数据节点的节点ID组成的二元组;所述Key-Value键值对的Value为所述数据节点包含的数据内容。
在本说明书中,所述Merkle树为融合了Trie字典树的树形结构的Merkle树变种;所述数据节点的节点ID,为所述Merkle树的根节点到该数据节点的路径对应的字符前缀。
在本说明书中,所述Merkle状态树为Merkle Patricia Tree状态树。
在本说明书中,所述数据库为LevelDB数据库;或者基于LevelDB架构的数据库;
在本说明书中,所述数据库为基于LevelDB架构的Rocksdb数据库。
在本说明书中,所述多级数据存储对应的存储介质的读写性能,存在性能差异;其中,级数高的数据存储对应的存储介质的读写性能,高于级数低的数据存储对应的存储介质的读写性能。
上述实施例阐明的系统、装置、模块或单元,具体可以由计算机芯片或实体实现,或者由具有某种功能的产品来实现。一种典型的实现设备为计算机,计算机的具体形式可以是个人计算机、膝上型计算机、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件收发设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任意几种设备的组合。
在一个典型的配置中,计算机包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器,随机存取存储器(RAM)和/或非易失性内存等形式,如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括,但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带、磁盘存储、量子存储器、基于石墨烯的存储介质或其他磁性存储设备或任何其他非传输介质,可用于存储可以被计算设备访问的信息。按照本文中的界定,计算机可读介质不包括暂存电脑可读媒体(transitory media),如调制的数据信号和载波。
还需要说明的是,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、商品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、商品或者设备所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括所述要素的过程、方法、商品或者设备中还存在另外的相同要素。
上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。 在一些情况下,在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外,在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中,多任务处理和并行处理也是可以的或者可能是有利的。
在本说明书一个或多个实施例使用的术语是仅仅出于描述特定实施例的目的,而非旨在限制本说明书一个或多个实施例。在本说明书一个或多个实施例和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式,除非上下文清楚地表示其他含义。还应当理解,本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解,尽管在本说明书一个或多个实施例可能采用术语第一、第二、第三等来描述各种信息,但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如,在不脱离本说明书一个或多个实施例范围的情况下,第一信息也可以被称为第二信息,类似地,第二信息也可以被称为第一信息。取决于语境,如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。
以上所述仅为本说明书一个或多个实施例的较佳实施例而已,并不用以限制本说明书一个或多个实施例,凡在本说明书一个或多个实施例的精神和原则之内,所做的任何修改、等同替换、改进等,均应包含在本说明书一个或多个实施例保护的范围之内。

Claims (20)

  1. 一种基于区块链的分级存储方法,所述区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述方法包括:
    当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;
    基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
    迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
    根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
  2. 根据权利要求1所述的方法,所述基于所述区块号区间确定迁移临界值,包括:
    如果所述区块号区间的右区间为开区间,将所述区块号区间的右端点值,确定为所述迁移临界值;
    如果所述区块号区间的右区间为闭区间,将所述区块号区间的右端点值与所述区块链的区块号递增步长的和确定为所述迁移临界值。
  3. 根据权利要求1所述的方法,所述查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点,包括:
    查找所述目标数据存储中存储的与所述目标数据节点具有相同的节点ID,并且所述二元组中的区块号为小于所述迁移临界值的区块号中的最大区块号的数据节点;
    进一步确定查找到的该数据节点的所述二元组中的区块号,是否大于所述目标数据节点的所述二元组中的区块号。
  4. 根据权利要求1所述的方法,所述方法还包括:
    确定最新区块的Merkle状态树上发生数据更新的数据节点;为所述最新区块的Merkle状态树上发生数据更新的数据节点构建Key-Value键值对,并将所述Key-Value 键值对存储至所述数据库中的最高级数据存储;
    其中,所述Key-Value键值对的key为所述最新区块的区块号和所述数据节点的节点ID组成的二元组;所述Key-Value键值对的Value为所述数据节点包含的数据内容。
  5. 根据权利要求1或4所述的方法,所述Merkle树为融合了Trie字典树的树形结构的Merkle树变种;所述数据节点的节点ID,为所述Merkle树的根节点到该数据节点的路径对应的字符前缀。
  6. 根据权利要求5所述的方法,所述Merkle状态树为Merkle Patricia Tree状态树。
  7. 根据权利要求1所述的方法,所述数据库为LevelDB数据库;或者基于LevelDB架构的数据库。
  8. 根据权利要求7所述的方法,所述数据库为基于LevelDB架构的Rocksdb数据库。
  9. 根据权利要求1所述的方法,所述多级数据存储对应的存储介质的读写性能,存在性能差异;其中,级数高的数据存储对应的存储介质的读写性能,高于级数低的数据存储对应的存储介质的读写性能。
  10. 一种基于区块链的分级存储装置,所述区块链的账户状态数据组织成的Merkle状态树上的数据节点,以Key-Value键值对的形式存储在数据库中;其中,所述Key-Value键值对的key为所述数据节点的节点ID和为所述数据节点标记的区块号组成的二元组;所述数据库包括多级数据存储;所述区块号指示所述数据节点发生数据更新时所在的区块;所述装置包括:
    确定模块,当所述数据库中的任一级目标数据存储满足了数据迁移条件时,确定所述目标数据存储中待迁移至下一级数据存储的数据节点对应的区块号区间;基于所述区块号区间确定迁移临界值;其中,所述迁移临界值为大于所述区块号区间的区块号临界值;
    查找模块,迭代遍历所述目标数据存储中存储的数据节点的所述二元组,查找所述二元组中的区块号小于所述迁移临界值的目标数据节点;
    迁移模块,根据所述目标数据节点的所述二元组,进一步查找所述目标数据存储中是否存储了与所述目标数据节点具有相同的节点ID,以及所述二元组中的区块号大于所述目标数据节点且小于所述迁移临界值的数据节点;如果是,将所述目标数据节点迁移至所述目标数据存储的下一级数据存储;否则,在所述目标数据存储中保留所述目标数据节点。
  11. 根据权利要求10所述的装置,所述确定模块:
    如果所述区块号区间的右区间为开区间,将所述区块号区间的右端点值,确定为所述迁移临界值;
    如果所述区块号区间的右区间为闭区间,将所述区块号区间的右端点值与所述区块链的区块号递增步长的和确定为所述迁移临界值。
  12. 根据权利要求10所述的装置,所述迁移模块进一步:
    查找所述目标数据存储中存储的与所述目标数据节点具有相同的节点ID,并且所述二元组中的区块号为小于所述迁移临界值的区块号中的最大区块号的数据节点;进一步确定查找到的该数据节点的所述二元组中的区块号,是否大于所述目标数据节点的所述二元组中的区块号。
  13. 根据权利要求10所述的装置,所述装置还包括:
    存储模块,确定最新区块的Merkle状态树上发生数据更新的数据节点;为所述最新区块的Merkle状态树上发生数据更新的数据节点构建Key-Value键值对,并将所述Key-Value键值对存储至所述数据库中的最高级数据存储;
    其中,所述Key-Value键值对的key为所述最新区块的区块号和所述数据节点的节点ID组成的二元组;所述Key-Value键值对的Value为所述数据节点包含的数据内容。
  14. 根据权利要求10或13所述的装置,所述Merkle树为融合了Trie字典树的树形结构的Merkle树变种;所述数据节点的节点ID,为所述Merkle树的根节点到该数据节点的路径对应的字符前缀。
  15. 根据权利要求14所述的装置,所述Merkle状态树为Merkle Patricia Tree状态树。
  16. 根据权利要求10所述的装置,所述数据库为LevelDB数据库;或者基于LevelDB架构的数据库。
  17. 根据权利要求16所述的装置,所述数据库为基于LevelDB架构的Rocksdb数据库。
  18. 根据权利要求10所述的装置,所述多级数据存储对应的存储介质的读写性能,存在性能差异;其中,级数高的数据存储对应的存储介质的读写性能,高于级数低的数据存储对应的存储介质的读写性能。
  19. 一种电子设备,包括:
    处理器;
    用于存储处理器可执行指令的存储器;
    其中,所述处理器通过运行所述可执行指令以实现如权利要求1-9中任一项所述的 方法。
  20. 一种计算机可读存储介质,其上存储有计算机指令,其特征在于,该指令被处理器执行时实现如权利要求1-9中任一项所述方法的步骤。
PCT/CN2020/072026 2019-06-28 2020-01-14 基于区块链的分级存储方法及装置、电子设备 WO2020258855A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/790,605 US11036720B2 (en) 2019-06-28 2020-02-13 Blockchain-based hierarchical data storage

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910579371.2A CN110334154B (zh) 2019-06-28 2019-06-28 基于区块链的分级存储方法及装置、电子设备
CN201910579371.2 2019-06-28

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/790,605 Continuation US11036720B2 (en) 2019-06-28 2020-02-13 Blockchain-based hierarchical data storage

Publications (1)

Publication Number Publication Date
WO2020258855A1 true WO2020258855A1 (zh) 2020-12-30

Family

ID=68143752

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/072026 WO2020258855A1 (zh) 2019-06-28 2020-01-14 基于区块链的分级存储方法及装置、电子设备

Country Status (3)

Country Link
CN (1) CN110334154B (zh)
TW (1) TWI737157B (zh)
WO (1) WO2020258855A1 (zh)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10789222B2 (en) 2019-06-28 2020-09-29 Alibaba Group Holding Limited Blockchain-based hierarchical data storage
CN110347660B (zh) * 2019-06-28 2020-08-11 阿里巴巴集团控股有限公司 基于区块链的分级存储方法及装置、电子设备
CN110334154B (zh) * 2019-06-28 2020-07-21 阿里巴巴集团控股有限公司 基于区块链的分级存储方法及装置、电子设备
US11036720B2 (en) 2019-06-28 2021-06-15 Advanced New Technologies Co., Ltd. Blockchain-based hierarchical data storage
US11636470B2 (en) 2019-09-25 2023-04-25 Visa International Service Association Key-value map commitments system and method
CN111226209B (zh) 2019-12-05 2024-03-01 支付宝(杭州)信息技术有限公司 在基于区块链的系统中执行映射迭代
CN111209341B (zh) * 2020-01-07 2023-03-14 北京众享比特科技有限公司 区块链的数据存储方法、装置、设备及介质
CN112131304B (zh) * 2020-02-10 2021-05-04 北京天德科技有限公司 一种基于区块链技术的新型计算及存储架构
CN111324613B (zh) * 2020-03-13 2021-03-26 华东师范大学 一种面向联盟链的分片内数据组织管理方法
CN111488611B (zh) * 2020-04-08 2021-06-22 北京瑞策科技有限公司 业务数据区块链的关系数据存储方法及装置
CN111488396B (zh) * 2020-04-08 2021-01-01 北京瑞策科技有限公司 业务数据区块链的数据同步方法及装置
CN111488343A (zh) * 2020-04-08 2020-08-04 北京瑞策科技有限公司 基于业务数据区块链的电商数据上链方法及装置
CN111506577B (zh) * 2020-04-13 2023-08-22 杭州溪塔科技有限公司 区块链上基于多种状态组织方式的数据操作方法和装置
CN111598567B (zh) * 2020-05-07 2023-11-10 深圳创客区块链技术有限公司 防止区块链大算力攻击的方法、装置及存储介质
CN112540969B (zh) * 2020-11-26 2023-07-14 南京纯白矩阵科技有限公司 一种异构区块链间智能合约的数据迁移方法
CN112650832B (zh) * 2020-12-14 2022-09-06 中国电子科技集团公司第二十八研究所 基于拓扑及文献特征的知识关联网络关键节点发现方法
CN112988912B (zh) * 2021-05-07 2021-11-02 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN112988909B (zh) * 2021-05-07 2021-09-28 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN112988910B (zh) * 2021-05-07 2021-09-24 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN112988908B (zh) * 2021-05-07 2021-10-15 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN112905607B (zh) * 2021-05-07 2022-02-01 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN112988761B (zh) * 2021-05-07 2022-04-08 支付宝(杭州)信息技术有限公司 区块链数据存储方法及装置、电子设备
CN113220685B (zh) * 2021-05-11 2022-04-19 支付宝(杭州)信息技术有限公司 智能合约存储内容的遍历方法及装置、电子设备
CN116432239B (zh) * 2023-06-06 2023-11-17 杭州格物智安科技有限公司 一种基于区块链的个人信息服务系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109144414A (zh) * 2018-07-27 2019-01-04 阿里巴巴集团控股有限公司 区块链数据的多级存储方法和装置
CN109669709A (zh) * 2018-11-26 2019-04-23 杭州秘猿科技有限公司 一种区块链升级的数据迁移方法及数据迁移系统
WO2019101229A2 (en) * 2019-03-04 2019-05-31 Alibaba Group Holding Limited Updating blockchain world state merkle patricia trie subtree
CN110334154A (zh) * 2019-06-28 2019-10-15 阿里巴巴集团控股有限公司 基于区块链的分级存储方法及装置、电子设备

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN100585662C (zh) * 2003-06-20 2010-01-27 汤姆森普罗梅特里克公司 利用高速缓存和可高速缓存对象扩展测试驱动应用程序的功能的基于计算机测试的系统和方法
US8589574B1 (en) * 2005-12-29 2013-11-19 Amazon Technologies, Inc. Dynamic application instance discovery and state management within a distributed system
US10812582B2 (en) * 2016-03-10 2020-10-20 Vmware, Inc. Management of applications across nodes using exo-clones
CN107018125B (zh) * 2017-02-17 2019-08-09 阿里巴巴集团控股有限公司 一种区块链系统、数据存储方法及装置
CN107239479B (zh) * 2017-03-28 2020-03-13 创新先进技术有限公司 一种基于区块链的数据存储以及查询的方法及装置
CN108197226A (zh) * 2017-12-29 2018-06-22 山大地纬软件股份有限公司 Mptc账户状态树以及mptc区块链快速检索方法
CN109359222B (zh) * 2018-08-06 2021-07-06 杭州复杂美科技有限公司 数据存储方法及系统、设备和存储介质
CN109165224B (zh) * 2018-08-24 2021-02-19 东北大学 一种在区块链数据库上针对关键字key的索引方法
CN109726229B (zh) * 2018-11-30 2023-10-10 深圳市元征科技股份有限公司 一种区块链数据存储方法及装置
CN109684333B (zh) * 2018-12-24 2021-02-09 杭州复杂美科技有限公司 一种数据存储及裁剪方法、设备和存储介质
CN109919756B (zh) * 2019-02-22 2023-04-18 西南财经大学 基于Merkle树回溯定位技术的转账系统、查验方法及交易方法
CN109903049A (zh) * 2019-03-01 2019-06-18 长沙理工大学 一种区块链交易数据存储方法、装置、设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109144414A (zh) * 2018-07-27 2019-01-04 阿里巴巴集团控股有限公司 区块链数据的多级存储方法和装置
CN109669709A (zh) * 2018-11-26 2019-04-23 杭州秘猿科技有限公司 一种区块链升级的数据迁移方法及数据迁移系统
WO2019101229A2 (en) * 2019-03-04 2019-05-31 Alibaba Group Holding Limited Updating blockchain world state merkle patricia trie subtree
CN110334154A (zh) * 2019-06-28 2019-10-15 阿里巴巴集团控股有限公司 基于区块链的分级存储方法及装置、电子设备

Also Published As

Publication number Publication date
TWI737157B (zh) 2021-08-21
TW202101249A (zh) 2021-01-01
CN110334154B (zh) 2020-07-21
CN110334154A (zh) 2019-10-15

Similar Documents

Publication Publication Date Title
WO2020258855A1 (zh) 基于区块链的分级存储方法及装置、电子设备
WO2020258856A1 (zh) 基于区块链的分级存储方法及装置、电子设备
CN110457319B (zh) 区块链状态数据存储方法及装置、电子设备
WO2020258853A1 (zh) 基于区块链的分级存储方法及装置、电子设备
WO2021017436A1 (zh) 区块链状态数据同步方法及装置、电子设备
CN110471795B (zh) 区块链状态数据恢复方法及装置、电子设备
US11113272B2 (en) Method and apparatus for storing blockchain state data and electronic device
US20200167367A1 (en) Block chain state data synchronization method, apparatus, and electronic device
US11036720B2 (en) Blockchain-based hierarchical data storage
US11386054B2 (en) Blockchain-based hierarchical data storage
CN113220685B (zh) 智能合约存储内容的遍历方法及装置、电子设备
CN114706848A (zh) 区块链数据存储、更新、读取方法及装置、电子设备
CN112988761A (zh) 区块链数据存储方法及装置、电子设备
US11288247B2 (en) Blockchain based hierarchical data storage
CN115221176A (zh) 区块链数据存储方法及装置、电子设备
CN112988911A (zh) 区块链数据存储方法及装置、电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20833669

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20833669

Country of ref document: EP

Kind code of ref document: A1