WO2026016331A1 - 一种构建基于默克尔字典树的世界状态的方法和计算机设备 - Google Patents
一种构建基于默克尔字典树的世界状态的方法和计算机设备Info
- Publication number
- WO2026016331A1 WO2026016331A1 PCT/CN2024/128752 CN2024128752W WO2026016331A1 WO 2026016331 A1 WO2026016331 A1 WO 2026016331A1 CN 2024128752 W CN2024128752 W CN 2024128752W WO 2026016331 A1 WO2026016331 A1 WO 2026016331A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- node
- tree
- state
- key
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
Definitions
- the embodiments in this specification belong to the field of blockchain technology, and in particular relate to a method and computer device for constructing a world state based on a Merkle trie.
- Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms.
- data blocks are sequentially linked together to form a chain-like data structure, and a distributed ledger is cryptographically guaranteed to be immutable and unforgeable. Due to its decentralized, immutable, and autonomous characteristics, blockchain is receiving increasing attention and application.
- a method for constructing a world state based on a Merkle trie, applied to the storage module of a blockchain node, wherein the storage module constructs a tree-structured world state in memory comprising:
- leaf nodes of the tree structure are generated, and the state value is stored in the data file of the persistent storage medium.
- the leaf node includes the position index of the state value in the data file of the persistent storage medium.
- a computer device comprising:
- the solution described in this application provides an integrated Merkle state data implementation. It no longer uses a two-level structure of an upper-level Merkle tree and a lower-level general-purpose KVDB, but instead integrates into a single MerkleDB with separate indexes and data.
- the Merkle tree as the index and contract state data as the data, the read/write process for state data is shortened, reducing overall write amplification and read amplification.
- Figure 1 is a schematic diagram of deploying a smart contract in one embodiment
- Figure 4 is a schematic diagram of a block storage structure in one embodiment
- Figure 5 is a schematic diagram of an MPT tree in one embodiment
- FIG. 6 is a schematic diagram of the modules involved in the transaction processing process and the relationship between CPU, memory and disk in one embodiment
- FIG. 7 is a schematic diagram of the modules involved in the transaction processing process and the relationship between CPU, memory and disk in one embodiment
- Figure 8 is a schematic diagram of the logical relationship of the state database in one embodiment
- Figure 9 is a flowchart of reading status data in one embodiment
- Figure 10 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure
- Figure 11 is a schematic diagram illustrating the reading of status data in conjunction with a tree structure in one embodiment
- Figure 12 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure
- Figure 13 is a schematic diagram illustrating the reading of status data in one embodiment using a tree structure
- Figure 14 is a schematic diagram illustrating the reading of status data in an embodiment using a tree structure
- Figure 15 is a schematic diagram of an embodiment incorporating a tree structure
- Figure 16 is a schematic diagram of an embodiment incorporating a tree structure.
- Figure 17 is a schematic diagram of an embodiment incorporating a tree structure.
- Blockchains are generally classified into three types: public blockchains, private blockchains, and consortium blockchains. In addition, there are various combinations of these types, such as private blockchain + consortium blockchain, consortium blockchain + public blockchain, etc.
- Public blockchains offer the highest degree of decentralization. Represented by Bitcoin and Ethereum, public blockchains allow participants to read data records on the chain, participate in transactions, and compete for the right to record new blocks. Furthermore, participants (represented by nodes on the blockchain) can freely join and leave the network and perform related operations.
- Private blockchains on the other hand, have write permissions controlled by a specific organization or institution, and data read permissions are regulated by that organization. Simply put, a private blockchain can be a weakly centralized system with strictly limited and few participating nodes.
- consortium blockchains fall between public and private blockchains, achieving "partial decentralization.”
- Each node in a consortium blockchain typically has a corresponding entity or organization; participants join the network through authorization and form a consortium of stakeholders to jointly maintain the operation of the blockchain.
- a smart contract on the blockchain is a contract that can be triggered and executed by transactions on the blockchain system. Smart contracts can be defined in the form of code.
- EVM Ethereum Virtual Machine
- bytecode virtual machine code
- Node 1 After Bob sends a transaction containing information about creating a smart contract to the Ethereum network, Node 1's EVM can execute this transaction and generate the corresponding contract instance.
- "0x6f8ae93" represents the address of this contract.
- the transaction's data field can store bytecode, and the transaction's to field is an empty account. After the nodes reach a consensus through the consensus mechanism, the contract is successfully created, and users can subsequently call this contract.
- a contract account corresponding to that smart contract is generated on the blockchain, with a specific address.
- the contract code and account storage are stored in this contract account.
- the behavior of the smart contract is controlled by the contract code, while the smart contract's account storage preserves the contract's state.
- a smart contract enables the creation of a virtual account on the blockchain that contains both contract code and account storage.
- the ⁇ data ⁇ field of a transaction that creates a smart contract can store the bytecode of that smart contract.
- Bytecode consists of a series of bytes, with each byte representing an operation.
- developers can choose to write smart contract code in a high-level language instead of directly writing bytecode.
- the smart contract code written in a high-level language is compiled by a compiler to generate bytecode, which can then be deployed to the blockchain.
- Ethereum supports many high-level languages, such as Solidity, Serpent, and LLL.
- contracts written in it are very similar to classes in object-oriented programming languages.
- a contract can declare various members, including state variables, functions, function modifiers, events, etc.
- State variables are values stored in the smart contract's account store and are used to save the contract's state.
- the EVM of node 1 can execute this transaction and generate the corresponding contract instance.
- the ⁇ from ⁇ field of the transaction is the address of the account initiating the smart contract call, and "0x6f8ae93" in the ⁇ to ⁇ field represents the address of the called smart contract.
- the ⁇ data ⁇ field of the transaction stores the method and parameters for calling the smart contract. Additionally, it may include a ⁇ value ⁇ field to represent the value of Ether in the transaction.
- the value of ⁇ storedData ⁇ may change. Subsequently, a client can view the current value of ⁇ storedData ⁇ through a specific blockchain node (e.g., node 6 in Figure 2).
- Smart contracts can be executed independently on each node of the blockchain network in a prescribed manner. All execution records and data are stored on the blockchain. Therefore, when such a transaction is completed, the blockchain stores an immutable and unlost transaction certificate.
- ⁇ storedData ⁇ in the example above is the state variable, which is stored in the smart contract's account storage.
- accounts typically include two types:
- Contract account Stores the executed smart contract code and the values of the state within the smart contract code; it can typically only be activated by an external account.
- Externally owned account A user's account, such as an Ethereum owner's account.
- External and contract accounts is essentially a mapping from account addresses to account states.
- Account states typically include fields such as Nonce, Balance, Storage root, and CodeHash. Nonce and Balance exist in both external and contract accounts. CodeHash and Storage root attributes are generally only valid for contract accounts.
- Nonce A counter. For external accounts, this number can represent the number of transactions sent from the account address; for contract accounts, it can be the number of contracts created by the account.
- Storage root The hash of the root node of an MPT tree that organizes the storage of state variables for contract accounts.
- CodeHash The hash value of the smart contract code. For contract accounts, this is the hash value of the smart contract; for external accounts, since the smart contract is not included, the CodeHash field can generally be an empty string or a string of all zeros.
- MPT stands for Merkle-Patricia Tree, a tree structure that combines the Merkle Tree and the Patricia Tree (a compressed prefix tree, a more space-efficient Trie tree).
- the Merkle Tree algorithm calculates a hash value for each transaction, then joins pairs of transactions and calculates the hash again, continuing until the top-level Merkle root.
- Ethereum uses an improved MPT tree, such as a 16-ary tree structure, which is often simply referred to as an MPT tree.
- the Ethereum MPT tree data structure includes a state trie.
- the state tree contains key-value pairs (also written as key-value, or simply k-v) representing the storage content corresponding to each account in the Ethereum network.
- the "key” in the state tree can be a 160-bit identifier (such as the address of an Ethereum account or part of its hash value, hereinafter referred to as the account address), distributed across storage from the root node to the leaf nodes of the state tree.
- the "value” in the state tree is generated by encoding the information of the Ethereum account (using the Recursive-Length Prefix encoding (RLP) method).
- RLP Recursive-Length Prefix encoding
- a contract account is used to store the state of a smart contract. After a smart contract is deployed on the blockchain, a corresponding contract account is created. This contract account typically has several states, defined by state variables within the smart contract and updated with new values during the contract's creation and execution.
- a smart contract is usually a contract defined in code within a blockchain environment that can automatically execute its terms. Once an event triggers a clause in the contract (meeting the execution conditions), the code can execute automatically.
- the contract's state is stored in a storage trie.
- the hash value of the storage trie's root node is stored in the aforementioned storage root, thus locking all of the contract's states to this contract account via hash.
- the storage trie is also an MPT tree structure, storing a key-value mapping from state addresses to state values. Information from the root node to the leaf node of the storage trie tree is arranged sequentially to store the address of a state, while the leaf node stores the state's value.
- each block header includes several fields, such as the previous block hash (prev_Hash in the figure), a nonce (in some blockchain systems, this nonce is not random, or the nonce in the block header is not enabled in some blockchain systems), a timestamp, a block number (Block Num), a state root hash (State_Root), a transaction root hash (Transaction_Root), and a receipt root hash (Receipt_Root).
- the Prev Hash in the block header of the next block e.g., block N+1
- points to the previous block e.g., block N
- the previous block e.g., block N
- State_Root, Transaction_Root, and Receipt_Root lock the state set, transaction set, and receipt set, respectively.
- the state set, transaction set, and receipt set are organized in a tree structure. Generally, they can be the same tree structure or different tree structures.
- Ethereum uses the same MPT structure. In some tree structures like Ethereum that include a set of smart contract states, there is a two-level MPT structure: the leaf nodes of the upper-level MPT structure include two types: external accounts and contract accounts; each contract account includes the lower-level MPT structure, and the leaf nodes of the lower-level structure include the values of the state in the contract account.
- FIG 4 is a schematic diagram of a blockchain data storage structure.
- ⁇ state_root ⁇ is the hash value of the root of the MPT tree composed of the states of all accounts in the current block. That is, pointing to ⁇ state_root ⁇ is a state trie in MPT form.
- the root node of this MPT tree is generally an extension node or a branch node, and ⁇ state_root ⁇ typically stores the hash value of this root node.
- the root node can be connected to one or more extension nodes/branch nodes below it. These multi-level tree nodes can be collectively referred to as internal nodes.
- a subset of values from each node in this MPT, from the root node to the leaf node, can be concatenated sequentially to form an account address, which serves as the key.
- the account information stored in the leaf nodes is the value corresponding to this account address, thus forming a key-value pair.
- This key can also be a portion of the SHA3 hash of the address (using a hash algorithm such as SHA3), and its stored value can be an RLP (Account) encoding of the account information.
- the account information is a four-tuple consisting of [nonce, balance, storageRoot, codeHash].
- Contract accounts generally include Nonce, Balance, Storage Root, and CodeHash.
- Nonce is the transaction counter for the contract account
- Balance is the account balance
- Storage root corresponds to another MPT, which links to contract-related state information
- CodeHash is the hash value of the contract code.
- its account information is generally located in a single leaf node. From the root node's Extension Node/Branch Node to each account's Leaf Node, there may be several branch nodes and extension nodes along the way.
- a state trie can be an MPT-style tree, typically a 16-ary tree, meaning each level can have a maximum of 16 child nodes.
- An Extension Node used to store a common prefix, typically has one child node, which can be a Branch Node.
- a Branch Node can have a maximum of 16 child nodes, which may include Extension Nodes and/or Leaf Nodes.
- this MPT-form tree pointed to by ⁇ storage_root ⁇ is the Storage Trie, specifically the hash value of the root node.
- this Storage Trie tree also stores key-value pairs.
- the key represents the address of the state variable, and its value can be the result of processing the state variable's declaration position (counting from 0) according to certain rules, such as SHA3(state variable declaration position) or SHA3(contract name + state variable declaration position).
- the value stores the state variable's value (e.g., an RLP-encoded value).
- this Storage Trie can also be an MPT-type tree, typically a 16-ary tree. That is, a Branch Node can have a maximum of 16 child nodes, which may include Extension Nodes and/or Leaf Nodes. An Extension Node can generally have one child node, which can be either a Branch Node or a Leaf Node.
- Leaf Node Account P in the state Trie in Figure 4 is a contract account whose Storage Root locks all the states stored in the contract. These states are organized as an MPT tree, with a tree structure like the Storage Trie linked by the Storage Root.
- Leaf Node State Variable N is the value of storedData in the aforementioned contract code example, then its key is sha3 (the declaration location of storedData, i.e., line 2 of the code), and its value is s (for simplicity, the encoding format of the value is omitted here, such as RLP, and similarly, it will not be elaborated further).
- the key values are distributed sequentially from the root node to the leaf node (i.e., Leaf Node Variable N) of the storage Trie.
- the address of Account C is the key, and its values are distributed sequentially from the root node to the leaf node (i.e., Leaf Node Variable C) of the state Trie.
- the key of the leaf node is formed by sequentially combining a7 of the shared nibble in the root node A8 (Extension Node), slot 1 of the intermediate node A7 (Branch Node), and key-end 1335 in leaf node A1, which is a711335.
- the key of the leaf node is formed by sequentially combining a7 of the shared nibble in the root node A8 (Extension Node), slot 7 of the intermediate node A7 (Branch Node), d3 of the shared nibbles in node A6 (Extension Node), slot 3 of the intermediate node A5 (Branch Node), and key-end 7 in leaf node A2, which is a77d337.
- leaf node A3 the key of this leaf node is formed by sequentially combining a7 from the shared nibble in root node A8 (Extension Node), slot f from intermediate node A7 (Branch Node), and key-end 9365 in leaf node A3, resulting in a7f9365.
- the leaf nodes of A1, A2, and A3 store information about external accounts, while the leaf node of A4 stores information about contract accounts.
- For each contract account there is a next-level MPT, forming a Storage Trie used to store the state variables of that contract account.
- the key of the leaf node is formed by sequentially combining slot 3 in the root node A10 (Branch Node) with the key-end value 35b2e4 in leaf node A11, which is 335b2e4.
- the key of the leaf node is formed by sequentially combining slot 7 in the root node A10 (Branch Node) with the key-end value c25988 in leaf node A12, which is 7c25988.
- leaf node A16 the key of this leaf node is formed by sequentially combining slot f in the root node A10 (Branch Node), slot a in the shared nibble of the intermediate node A13 (Extension Node), slot 9 in the intermediate node A14 (Branch Node), and key-end 9365 in the leaf node A16, which is fa99365.
- the prefix indicates the type of tree node. For example, 0 indicates an Extension Node containing an even number of shared nibbles, 1 indicates an Extension Node containing an odd number of shared nibble(s), 2 indicates a Leaf Node containing an even number of nibbles, and 3 indicates a Leaf Node containing an odd number of nibble(s).
- the hash value of the entire content of the next tree node is filled into the corresponding position of the previous tree node.
- the database actually stores the key-value mapping for each tree node, where the value includes the content stored in that tree node, and the corresponding key is the hash value of the entire content of that tree node.
- the actual key-value pairs of the tree nodes stored in the database are shown in the following table:
- H() represents hash calculation. This way, the hash value of the next tree node is anchored to the previous tree node. Through this layer-by-layer hashing, the root hash of the entire state trie tree is obtained, and this root hash is locked into the state root field of the block header.
- the blockchain platform's code may include P2P (Peer-to-Peer) modules, consensus modules, execution modules, and storage modules.
- P2P is a way of organizing computer networks; unlike common web networks, P2P is decentralized and distributed.
- P2P modules enable the distributed propagation of data.
- P2P modules allow for the peer-to-peer transmission and reception of receipts.
- Different participants can establish a distributed blockchain network through deployed nodes.
- the ledger constructed using a chain-like block structure is stored on each node (or most nodes, such as consensus nodes) in the distributed blockchain network; this is also known as a decentralized (or multi-centralized) distributed ledger.
- Such blockchain systems need to address the issues of consistency and correctness of the ledger data across multiple decentralized (or multi-centralized) nodes. Each node runs the same blockchain platform program. With a certain level of fault tolerance, the consensus module ensures that all loyal nodes have the same transactions, guaranteeing consistent execution results for the same transactions. The transactions and execution results are then packaged into blocks.
- Current mainstream consensus mechanisms include: Proof of Work (POW), Proof of Stake (POS), Delegated Proof of Stake (DPoS), Practical Byzantine Fault Tolerance (PBFT), and Honey Badger Byzantine Fault Tolerance (HoneyBadgerBFT).
- POW Proof of Work
- POS Proof of Stake
- DoS Delegated Proof of Stake
- PBFT Practical Byzantine Fault Tolerance
- HoneyBadgerBFT Honey Badger Byzantine Fault Tolerance
- the consensus module can also typically generate timestamps for the blocks corresponding to the current transaction set.
- the execution module can execute transactions, including ordinary transfer transactions and transactions involving contracts, either before or after the consensus module completes consensus.
- the execution module can introduce a virtual machine, such as the Ethereum Virtual Machine (EVM), to execute the smart contract code.
- EVM Ethereum Virtual Machine
- This EVM shields the differences in hardware configuration and software environment across nodes, ensuring that the execution process and results of smart contracts are identical across all nodes.
- the sandbox environment prevents the execution of smart contracts from affecting the blockchain platform code, other programs, or the operating system on the host machine.
- nodes can determine the content and order of transactions in a transaction set through a consensus module, and then output a deterministic transaction set as the consensus result to the execution module.
- the execution module generates execution results by executing ordinary transfer transactions/transactions involving contracts and sends them to the storage module.
- the storage module is responsible for storing the execution results in the node's local persistent storage medium.
- a blockchain node physically includes a CPU, memory, and disk.
- the blockchain platform code executed by this node can include a P2P module, a consensus module, an execution module, and a storage module.
- the P2P, consensus, and execution modules generally require CPU and memory to function.
- the storage module can include a tree construction module, a block header generation module, a WAL (Write-Ahead Log) module, and a state database module.
- the tree construction module is used to build a tree (e.g., an MPT tree) based on the state key-value pairs passed from the execution module, such as the aforementioned state trie and storage trie, thus obtaining the key-value pairs of the tree nodes. This generally requires CPU and memory.
- the block header generation module is used to generate block headers based on the root node of the tree constructed by the tree construction module and other data (such as the previous block hash, timestamp, block number, etc.). This generally requires CPU and memory.
- the WAL module persistently stores the leaf node keys (k-v) of the tree generated by the tree-building module before writing them to the state database module. This prevents data loss due to power outages or other unforeseen circumstances during the writing process and allows for data recovery in such cases. It typically requires the participation of CPU, memory, and disk.
- the state database module stores the tree node keys (k-v) constructed by the tree-building module, as shown in Table 1, on a persistent storage device. Since the tree node data will ultimately be written to persistent storage media (such as the disk in the diagram), the state database module generally requires the participation of disk in addition to CPU and memory.
- the aforementioned Merkle tree structures such as Ethereum's MPT and Libra's SMT (Sparse Merkle Tree, similar to MPT), can reside in the tree construction module and be stored in memory, according to the correspondence shown in Table 1.
- the upper-level Merkle tree is a prefix tree (prefix tree), capable of organizing data and obtaining a unique Merkle root for the organized data.
- Leaf nodes can store state values, and the root node to intermediate nodes to leaf nodes implement a lexicographical index of the state keys.
- These tree nodes are encoded as keys according to certain rules, and their contents are encoded as values, ultimately stored in the lower-level database.
- the LSM storage engine is a tiered, ordered, disk-oriented storage engine that borrows from the append-only (rather than modify) nature of the Log. Its core idea is to fully leverage the fact that sequential, batch writes to disk are far more efficient than random writes, sacrificing some read efficiency for maximum write efficiency. Generally, maximizing disk performance involves reading or writing a fixed-size block of data at a time, minimizing random access operations. LSM's design is based on this disk characteristic and assumes sufficient memory. Instead of writing data to disk every time it's updated, it first keeps the latest data in memory. Once a sufficient amount of data has accumulated, it uses merge sort to combine the data in memory with the data on disk and appends it to disk in batches.
- FIG. 7 uses Figure 7 as an example to further illustrate the storage process of the state data module.
- this key-value data is first written to the Write-Ahead Log (WAL) in an append-only manner and stored on the disk.
- WAL Write-Ahead Log
- the tree node key-value data is passed to the state database module.
- writing to the WAL first prevents data loss due to power outages or other factors during the writing of the tree node key-value data generated by the tree building module to the state database module, and allows for data recovery in such cases.
- the state database module After receiving the tree node key-value data, the state database module first stores it in memory, specifically in a MemTable within memory. The MemTable can provide concurrent read and write operations.
- MemTables can exist in memory.
- a certain threshold such as 256MB
- the data in the MemTable can be flushed to the disk.
- this MemTable is converted into an immutable Memtable, meaning the immutable Memtable is set to read-only, and a new MemTable is created to receive newly passed tree node key-value pairs.
- This new MemTable can provide concurrent read and write operations (as shown in the New MemTable diagram).
- the storage engine writes the data in the immutable MemTable to disk.
- KVDB On disk, KVDB stores data in multiple levels of SST (Sorted String Table) files.
- SST originates from the Google Bigtable paper. Bigtable is a distributed data storage system designed by Google to handle massive amounts of data; it's a non-relational database. SST can be used to store data files within Bigtable and is an efficient key-value file storage format. Content stored on persistent storage media can also be called a file.
- SSTs can include multiple levels, such as 3, 4, 5, 6, 7, or more.
- Figure 7 shows a 3-level case, namely level 0, level 1, and level 2.
- the total capacity of the SST in the upper level is significantly smaller than that in the lower level.
- the total capacity of the SST in level 0 is 1GB
- the total capacity of the SST in level 1 is 10GB
- the total capacity of the SST in level 2 is 100GB.
- the capacity of the MemTable is 256MB.
- the tree construction module can be written using the key-value pairs (k-v) of the tree nodes corresponding to blocks, and then written to the MemTable.
- the space occupied by the MemTable may reach 256MB. Then, this MemTable is transformed into an Immutable MemTable, and the data in the Immutable MemTable can be written (flush, which refers to the operation of writing data in memory to disk) to disk. Specifically, the data in the MemTable can be flushed to the Level 0 SST on disk.
- a new MemTable is generated to receive the newly passed-in tree node k-v and to provide concurrent read and write operations.
- each level of disk storage can include multiple SST files.
- the key-value pairs (kv) stored in Level 1 and lower-level SSTs are generally stored in key order, and the key values in different SSTs within the same level are also related in size and do not repeat.
- Level 1 includes 3 SSTs.
- the SST11 on the left contains k1 ⁇ k2 ⁇ k3 ⁇ ... ⁇ k1000, but k1, k2, k3, ... k1000 are not necessarily consecutive.
- the SST12 in the middle contains k1 ⁇ k2 ⁇ k3 ⁇ ... ⁇ k800, but similarly, k1, k2, k3, ... k800 are not necessarily consecutive.
- SST13 on the right contains k1 ⁇ k2 ⁇ k3 ⁇ ... ⁇ k1100, but similarly, k1, k2, k3,...k1100 are not necessarily consecutive. Furthermore, overall, the minimum value of k in the left SST11 ⁇ the maximum value of k in the left SST11 ⁇ the minimum value of k in the middle SST12 ⁇ the maximum value of k in the middle SST12 ⁇ the minimum value of k in the right SST13 ⁇ the maximum value of k in the right SST13. The same applies to each level, Level 2, Level 3, and so on; that is, within each SST in each level, there is a ranking order in the value of k, and among multiple SSTs in each level, there is a ranking order in the range of k.
- the key-value pairs (k-v) within a single SST can be sorted according to the size of the k, while the k values across multiple SSTs do not necessarily have a range-based relationship. This is because the MemTable typically stores multiple k-v values corresponding to each block in ascending order of block number (either from oldest to newest, or from smallest to largest, since newer blocks have larger block numbers and older blocks have smaller block numbers). These k values are the hash values of tree nodes, and changes in the content of tree nodes are highly likely to result in significantly different hash values (i.e., k values).
- Sorting the k-v values within an SST in Level 0 according to the size of the k is efficient and feasible in memory; however, sorting by k values is not strictly prohibited. For example, when the MemTable's capacity reaches 256MB, it may store multiple k-v values corresponding to multiple blocks. This MemTable is then converted into an Immutable MemTable. These k-v values in the Immutable MemTable can be sorted by k and written to the left-hand SST of Level 0. There may be multiple processes of different Immutable MemTables writing to the same SST before filling SST01 on the left side of Level 0, and then SST02 on the right side of Level 0 can be written in a similar manner.
- the key-value pairs (k-v) written to a specific SST in Level 0 by a single Immutable MemTable are sorted, the k-v sets written by different Immutable MemTables within the same SST in Level 0 are generally not sorted by size, and the k-v sets between different SSTs (e.g., left and right SSTs) are also not sorted by size. That is, the ranges of k in the left and right SSTs in Level 0 may overlap.
- the k-v sets written to a single SST in Level 0 may be written after multiple MemTables have filled up, and the ranges of k-v in these multiple write processes generally overlap, the multiple k-v sets stored in each SST in Level 0 generally do not have a specific order, and the ranges between different SSTs in Level 0 generally do not have a specific order.
- the Immutable MemTable sorts the k-v values when writing to the SST, the k-v values stored in the SST will also be sorted according to k.
- a process called “compaction” is used to write the data from Level 0 into Level 1.
- the k-v values in each SST within Level 0 and some or all of the k-v values in Level 1 can be transferred to memory, sorted in memory, and then written to the SSTs of Level 1. Because sorting occurs during compaction, within each SST of Level 1, there is a size order relationship in the value of k, and across multiple SSTs in Level 1, there is a size order relationship in the range of k.
- k-v values are arranged in the order ⁇ k1-v1> ⁇ k2-v2> ⁇ k3-v3>... ⁇ kn-vn>, where k1 ⁇ k2 ⁇ k3 ⁇ ... ⁇ kn, and k1, k2, k3,...kn are not necessarily consecutive.
- the minimum value of k in the left SST ⁇ the maximum value of k in the left SST ⁇ the minimum value of k in the right SST ⁇ the maximum value of k in the right SST, as shown in Figure 8.
- the key-value pairs (k-v) in each SST within Level 0 and the key-v pairs in some SSTs within Level 1 can be loaded into memory for sorting.
- the SSTs within Level 1 can be SSTs whose key-v ranges overlap with those in Level 0; that is, SSTs whose key-v ranges overlap with those in each SST of Level 0.
- Level 1 includes three SSTs: SST11 has a key range of 10b43c032a46 to 2018df9326a7; SST12 has a key range of 21a514c0a7e1 to 284a5df3fca7; and SST13 has a key range of 3a5e55ac3fe1 to 400a51ef1405.
- the key range in each SST within Level 0 is 18c49a405e32 to 24d35f39ab56.
- SST11 and SST12 overlap with the key range in Level 1 and Level 0.
- the key-value pairs in each SST in Level 0 and the key-value pairs in SST11 and SST12 in Level 1 can be transferred into memory and sorted, and then the sorted results can be written back to the SST in Level 1.
- a compaction process is used to write the data from Level 1 to Level 2.
- the key-value pairs (kv) in each SST of Level 1 and some or all of the kv in Level 2 can be first transferred to memory, sorted in memory, and then written to the SSTs of Level 2.
- a compaction process is used to write the data from Level 2 to Level 3.
- the key-value pairs of each SST in Level 2 and some or all SSTs in Level 3 can be first transferred to memory, sorted in memory, and then written back to the SSTs in Level 3. And so on.
- the data stored in the upper-level SST is newer than that in the lower-level SST.
- the newest data is stored in memory
- the second newest data is stored in Level 0, and so on, with the oldest data stored in the SST within the lowest level.
- compaction can be used for garbage collection, such as deleting data marked for deletion.
- Sorted key-value pairs can be efficiently retrieved, for example, using binary search.
- Binary search also known as half-interval search, has the following algorithm: Define ⁇ left ⁇ as the leftmost element position of the sequential list, ⁇ right ⁇ as the rightmost element position, and ⁇ mid ⁇ as (left + right) / 2, which is the middle position of the sequential list; compare the value to be searched with the value at position ⁇ mid ⁇ ; since the list is ordered, if the value to be searched is smaller than ⁇ mid ⁇ , only the first half of the list needs to be searched; otherwise, only the second half needs to be searched (if the two values are equal on the first comparison, the current value's position is returned directly). This process continues until the value is found or it is determined that the value to be searched is not in the list (i.e., the search fails).
- Such storage engines suffer from read amplification and write amplification issues when reading and writing data, which will be explained in detail below.
- the execution module may need to read an existing state during transaction execution.
- external account A transfers a certain amount (e.g., f) of assets to external account B, requiring adjustments to the account balances.
- account A's balance is 'a' and account B's balance is 'b'
- the transaction sets account A's balance to (a-f) and account B's balance to (b+f). Therefore, this transaction requires reading the values of the state variables 'a' and 'b' during execution.
- Another example is a contract call transaction. External account C initiates a call to contract D.
- C initiates a transaction pointing to the address of contract account D, specifying the called function and input parameters in the transaction's data field.
- the logic includes executing branch 1 if true and branch 2 if false. This contract call transaction requires reading the value of state 'd' during execution.
- the execution module can first check if the value of that state variable exists in its own memory. If it does not exist, a read command is sent to the storage module. This read command can include the address of the state to be read, i.e., the state's key value. Upon receiving the read command, the storage module locates the root node of the state tree in the tree construction module. It then matches the state key value to be searched against the root node's shared nibble(s) field (for Extension Nodes) or slot (for Branch Nodes) from the beginning.
- next-level tree node which can be an Internal Node or a Leaf Node; we will use an Internal Node as an example
- it then first searches for the hash in the tree construction module's memory. If it is not found there, it searches in the state database module's memory. If it is still not found there, it searches in the state database module's file. Specifically, the search can begin within the SSTs of Level 0. If the hash value is not found there, the search continues within an SST of Level 1, and so on, down to the next SST of the next lower level, until the next Internal Node pointed to by the hash value is found.
- the remaining fields of the key value of the state address to be read are matched from front to back. If a match is found, the hash value is read from the match point, and the user jumps to the next level tree node pointed to by that hash value. This process is repeated, unpacking Internal Nodes level by level and matching the remaining fields of the key value of the state address to be read from front to back.
- the hash value at the match point serves as the basis for the next search of intermediate or leaf nodes, until a Leaf node is matched, from which the state value is read. During this process, some key-value pairs of tree nodes at each level may not exist in the memory of the tree building module, but rather in the state database module.
- the key can be searched first in the MemTable or Immutable MemTable of the state database module. If it doesn't exist, further searching of the various levels of SST files on disk is required. For Internal Nodes and Leaf Nodes on disk, they need to be loaded into memory and decoded (deserialized) to obtain their contents. Specifically, for example, the search is first performed one by one in the level 0 SST files. If each SST in Level 0 is sorted, a binary search can be performed within each SST based on the hash value of the intermediate/leaf node. Once found, the value (v) is loaded into memory and decoded and matched.
- a binary search is performed in Level 1 based on the hash value of the intermediate/leaf node. If it is not found there either, a binary search is performed in Level 2. If found, the value corresponding to this key is loaded into memory and decoded. This process continues until a leaf node of the tree structure is matched and its value is read. When searching at Level 1, Level 2, and lower levels, since the overall SST at these levels is sorted, a binary search can generally be performed at each level.
- the tree-building module is missing a key-value pair (k-v) for a particular tree node during the process of constructing the key-value pairs of all tree nodes along the path from the state key-v to the root node, it may need to perform a binary search from Level 0, Level 1, Level 2, etc., with a similar principle to the process described above.
- the value of the execution result (a-f) / (b+f) / the value of the contract state d needs to be written to the state database.
- the execution module calls the storage interface of the storage module and outputs the key-value pairs of the state to the storage module.
- the tree-building module in the storage module generates the key-value pairs of the tree nodes from the root to the intermediate and leaf nodes based on the state key-v, writes these key-value pairs to the WAL module, and then writes this series of key-value pairs of tree nodes to the MemTable of the state database module, marking them as dirty data.
- Dirty data typically refers to data that has been written to the MemTable but has not yet been written to persistent storage (such as SSTable). Dirty data can be written to disk during the aforementioned flush process.
- the state database module then writes some key-value pairs, including the tree node key-value pairs, from the MemTable to an SST file on disk.
- This application provides an embodiment of a method for reading blockchain state data.
- This method embodiment can be applied to the storage module of a blockchain node.
- the storage module constructs a tree-structured world state in memory.
- the leaf nodes of the tree structure may include the content of the state data or the position index of the state data within a data file in a persistent storage medium.
- the data file stores the actual content; the index file stores an index pointing to the location of the data file. Its working principle is as follows: data content is written to the data file, recording the data's position within the file (such as filename, offset, and length); then, an entry is created in the index file, containing a key (a unique identifier used for lookup) pointing to a reference in the data file (filename, offset, etc.).
- a key a unique identifier used for lookup
- the location index is used to represent this position.
- the location field can be used to store the position index of the data file containing the state data.
- This position index is, for example, ⁇ 2,750,210>.
- This ⁇ 2,750,210> indicates that the state data is located in "File 2" within the data file, starting at the 750th KB position of "File 2" and having a length of 210KB.
- the logical address range of the index is 0KB to 1023KB.
- the state data of A11 actually occupies the range from the 750th KB to the 959th KB position in File 2.
- this embodiment includes:
- the storage module of the blockchain node receives the instruction to look up the state key and searches for the leaf node corresponding to the state key in the tree structure in memory.
- the execution module can generate execution results by executing ordinary transfer transactions or transactions involving contracts, and then send them to the storage module. Specifically, for example, during contract execution, it may be necessary to read the value of a state variable.
- the execution module can first check if the value of the state variable exists in its own memory. If it does not exist, it sends a read instruction to the storage module. This read instruction can include the address of the state to be read, i.e., the state's key value.
- the storage module includes a tree-building module.
- This module constructs a tree structure representing the world state in memory.
- the individual tree nodes can be loaded from disk and organized into a tree in memory. This process also involves loading leaf nodes.
- An example is given where the tree-building module contains an incomplete tree structure; the cold start loading process is similar. Since memory capacity is generally limited, the LRU (Least Recently Used) algorithm is typically used to evict tree nodes that haven't been accessed for a long time. Therefore, leaf nodes in the tree structure constructed in memory by the tree-building module may be subject to eviction. For the leaf nodes of the world state tree, some may be in memory while others may not. Furthermore, a similar eviction mechanism can be used for the content within tree nodes, such as the state data in leaf nodes; alternatively, state data may not be maintained in memory.
- the tree construction module For the state key to be searched, if the tree construction module has a leaf node corresponding to the state key in memory, it can be found directly in memory. Referring to Figure 5, the key of the corresponding leaf node can be found directly in memory, and the value can be read from it. Then, the state data can be read from the value. If the tree construction module does not have a leaf node corresponding to the state key in memory, or if the leaf node exists but does not contain the state data to be searched, further searching is required in the state database module. For cases where the state database uses an LSM tree, as mentioned earlier, a search can be performed first in the MemTable; if not found, a search can be performed in the files on disk.
- S920 Obtain the position index contained in the leaf node corresponding to the tree node key, where the position index points to the position within the data file in the persistent storage medium.
- the location field can be used.
- the location of leaf node A11 is ⁇ 2, 750, 210>.
- This ⁇ 2, 750, 210> is the location index, which indicates that the state data is located in file 2 of the data file, starting at position 750KB and with a length of 210KB.
- the logical address range of the index is 0KB to 1023KB.
- the state data of A11 actually occupies the range from 750KB to 959KB in file 2.
- Searching for and loading the leaf node from the state database module specifically includes loading the index file and data file of the leaf node.
- the underlying database corresponding to Figure 10 stores the key-value pairs (kv) of all intermediate nodes in the tree structure, as shown in the table below:
- H() represents hash calculation. This anchors the hash value of the next tree node to the previous tree node. Through this layer-by-layer hashing, the root hash of the entire state trie tree is obtained and locked into the state root field of the block header. Assume the key-value pairs in this table are stored in the underlying state database module, saved on disk, and using a similar LSM structure. Thus, similar to the above, after receiving a read command, the storage module can find the root node of the state tree in the tree construction module. It then matches the state key value to be searched against the root node's shared nibble(s) field (for Extension Nodes) or slot (for Branch Nodes) from the beginning.
- the hash of the next level tree node From the matching position, and then first searches for the hash in the memory of the tree construction module. If it is not found there, it searches in the memory of the state database module. If it is still not found there, it searches in the state database module's file. Specifically, the search can begin within the SSTs of Level 0. If the hash value is not found there, the search continues within an SST of Level 1, and so on, down to the next SST of the next lower level, until the next Internal Node pointed to by the hash value is found. After unpacking this Internal Node, the remaining fields of the key value of the state address to be read are matched from front to back.
- the hash value is read from the match point, and the search jumps to the next level tree node pointed to by that hash value. This process is repeated, unpacking Internal Nodes level by level and matching the remaining fields of the key value of the state address to be read from front to back.
- the hash value at the match point serves as the basis for the next search of intermediate or leaf nodes, until a Leaf node is matched, from which the state value is read.
- some key-value pairs of tree nodes at each level may not exist in the memory of the tree building module but are located in the state database module. Therefore, it may be necessary to read them from the state database module.
- the key can be searched first in the MemTable or Immutable MemTable of the state database module.
- the search is first performed one by one in the level 0 SST files. If each SST in Level 0 is sorted, a binary search can be performed within each SST based on the hash value of the intermediate/leaf node. Once found, the value (v) is loaded into memory and decoded for matching. If it is not found in the Level 0 SST, a binary search is performed in Level 1 based on the hash value of the intermediate/leaf node.
- a binary search is performed in Level 2. If found, the value corresponding to this key is loaded into memory and decoded. This process continues until a leaf node of the tree structure is matched and its value is read. When searching at Level 1, Level 2, and lower levels, since the overall SST at these levels is sorted, a binary search can generally be performed at each level.
- the key-value pairs of the found leaf nodes can be loaded into memory.
- the content of the leaf node value (prefix:2, Key-end:35b2e4, location: (2,750,210)) is loaded into memory.
- ⁇ 2,750,210> is the index of the value content in the file.
- S930 Load the value of the data file location in the persistent storage medium pointed to by the location index into memory and read the status data therein.
- the position index ⁇ 2,750,210> points to the location within the data file in the persistent storage medium.
- the value at the location in the data file pointed to by the position index ⁇ 2,750,210> can be loaded into memory and its status data read.
- the location index allows direct location within the data file, enabling the loading of the data file and the reading of status data from it.
- the tree-building module If the tree-building module has the leaf node to be searched in memory, but it does not contain state data, it can directly read the data from the corresponding data file based on the location within the leaf node in memory. This avoids storing the key-value pairs of the leaf node in the persistent storage SST file, which would require multiple binary search operations within the SST.
- the leaf node to be searched is not in the memory of the tree construction module, and it is necessary to search for and load the leaf node from the state database module, it is actually necessary to read a certain amount of content from the disk into memory before searching.
- the key-value pairs of the tree nodes are stored together on disk, and the value of the leaf node includes state data.
- the state data occupies a large space, about 90% or more of the total key-value pair space.
- the index and data are separated.
- the index stores the location of the data file containing the state data, but not the state data itself.
- H(11) corresponds to location (2,750,210)
- H(12) corresponds to (3,350,210)
- H(15) corresponds to (5,760,140)
- H(16) corresponds to location (5,170,210).
- the actual state is stored in the corresponding location in the data file, as shown in Figure 10. Therefore, in the process of searching for leaf nodes, only the index file can be searched. In this way, more index files can be loaded into memory within a unit space for searching.
- the search efficiency in memory is significantly higher than that on disk. Generally, the content to be searched is loaded into memory before searching.
- the implementation in Table 2 can find leaf nodes faster than the implementation in Table 1 because more indexes can be loaded within a unit capacity of memory space.
- the implementation of Table 1 for example, requires reading a batch of key-value pairs into memory. If a leaf node is not found, another batch of key-value pairs needs to be read into memory again for another search. Since the state content occupies a large amount of space, the number of key-value pairs in a unit of memory space is relatively small. Therefore, the number of reads from the disk will generally be more frequent. Since the disk read speed is much slower than memory, the search efficiency of the implementation of Table 1 is low.
- Figure 10 still shows a two-level MPT tree structure, and the structure of the intermediate nodes in each level includes two types of tree nodes: Extension Node and Branch Node.
- a Branch Node can be used as both the intermediate node and the root node, as shown in Figure 11.
- the hash value of the next-level node can be stored in the slot of the previous-level node.
- a two-level tree structure can still be used.
- the leaf nodes in the previous-level tree can store account states (including the states of external accounts and contract accounts), and the leaf nodes in the next-level tree store the contract states. Position indexes can be used in the leaf nodes storing external account states, contract account states, or contract states.
- the leaf nodes of the previous-level tree include the account states in the following four rows: H(M1), H(M2), H(M3), and H(M4) in the table below:
- the location index allows direct access to the data file, enabling the loading of the data file and the reading of state data. Furthermore, the leaf nodes of the next-level tree contain the contract states as shown in rows H(Z1), H(Z2), H(Z3), and H(Z4) in the table above.
- the location in the leaf node can be quickly located within the data file. This avoids the need for multiple binary search operations in the SST based on the leaf node's key (hash value of the content) when storing the key-value pairs of the leaf nodes together in the persistent storage SST file.
- the location in the data file can be directly located based on the location index. Therefore, the content located at the corresponding position in the data file can be directly loaded into memory, thus avoiding the read amplification problem caused by reading and loading a large number of files from multiple levels of SST into memory.
- the leaf node A4 which is also the parent level of the tree, can be stored as shown in Figure 10 or as shown in Figure 11.
- External accounts and contract accounts typically use fixed-length strings as keys, while contract status keys can also be strings of the same length.
- the key for an external account or contract account could be an account address, while the key for the contract status could be a string of the same length.
- the key for an external account or contract account could be the account address hashed using a specific algorithm, such as SHA3; while the key for the contract status could be SHA3(contract name + position of status variable declaration).
- SHA3 specific algorithm
- the key for the contract status could be SHA3(contract name + position of status variable declaration).
- These external accounts, contract accounts, and contract statuses are all strings of the same length and format, collectively referred to as the status key.
- the length of the status key might be 4 bytes, or 32 bits.
- each slot in the intermediate node (such as slots 0, 1, 2, ..., d, e, f in B0, a total of 16 slots) can be filled with the hash value of the next tree node; on the other hand, the position of this slot in the root/intermediate node can represent a 4-bit key.
- the root node and intermediate nodes can store 32 bits, totaling 8 bytes, which is the entire state key.
- the 32 bits of the state key can be stored from the top-level intermediate node to the bottom-level intermediate node and down to the leaf node, with each level storing one half-byte of the key.
- the leaf nodes can store the last 4 bits of the key.
- Figure 11 shows a total of 8 layers from C* to C13D*, all the way to the bottommost M*.
- a compressed prefix and/or infix and/or suffix approach can be used, in which case one or more leaf nodes plus the intermediate nodes above them can be fewer than 8 layers.
- a single-layer tree structure can be used to store account and contract states, as shown in Figure 12. Since the key for a contract state is a hash value, due to the discrete nature of hash algorithms, the state keys for different contract states may follow paths from the root node to different leaf nodes within this single-layer tree structure. Thus, different states of the same contract may have values residing in different leaf nodes at the bottom layer of the tree structure, rather than clustered under their respective contract accounts. In effect, using a single-layer tree structure, both account and contract states are distributed across the bottom-layer leaf nodes.
- prefix and/or infix compression can be performed.
- infix compression for example, if the key of tree node M1 is 0x52d1479e (0x represents each digit in hexadecimal), and the key of tree node Z1 is 0x52d9ab36, then M1 and Z1 share a common infix 2d. Therefore, in tree node C5, besides the 1 and 9 separated by the hexadecimal tree, C5 can also have compressed infix 2d, as shown in the shared nibbles field in Figure 13.
- the key-end in tree node M1 is, for example, 479e
- the key-end in tree node Z1 is, for example, 36.
- prefix and/or infix compression the number of tree levels can be reduced.
- the root node and intermediate nodes are stored in key-value format, where the key is the hash value of the value, similar to the structure in Table 3. In this way, one can start from the upper-level tree node, sequentially match the hash of the path of the state key, and use the hash to find the lower-level tree node until the leaf node of the tree structure corresponding to the state key is found.
- the key of the tree node kv actually stored in the database may not be the hash value of the value, but a part of the value on the state key path.
- it may be a part of the key value from the root node to the current tree node (refer to Figure 12, without compressed prefix/infix, as shown in Table 4), or a part of the key value from the previous tree node (refer to Figure 13, with compressed prefix/infix, as shown in Table 5).
- the search state key is 0x59...36 (leaf node Z1), starting from the root node of the tree structure, we can match the first digit (5) of the state key to the tree node in the database with the key 0x5 (the h (C5) in the value of the tree node with the key 0x5 can be used to verify with the hash value in slot 5 of the root node; if they are the same, it means that the tree node with the key 0x5 is a child node in slot 5 of the 0x tree node); then, we can match the second digit (9) of the state key to the tree node in the database with the key 0x9 (the h (C5D9) in the value of the tree node with the key 0x9 can be used to verify with the hash value in slot 9 of the parent node; if they are the same, it means that the tree node with the key 0x9 is a child node of the 0
- the last digit 6 of the status key can be used to match the tree node in the database with the key 0x6 (the h(Z1) in the value of the tree node with key 0x6 can be used to verify with the hash value on slot 6 in the parent node; if they are the same, it means that the tree node with key 0x6 is a child node of the parent node).
- the system can match the first digit (5) of the state key in the database. If no matching key is found, or if the hash value in the value of the tree node matching the key is inconsistent with h(C5) in the root node, the system will again match the first two digits (52) of the state key in the database. This process is repeated until a match is found.
- the system matches the first three digits (52d) of the state key in the database and finds a tree node with the key 0x52d, and the hash value H(C5) in its value is the same as the hash value in slot 5 of the root node, then a match has been found. Then, the system can match the first four characters (52d9) of the status key in the database. If no matching key is found, or if the hash value of the value in the tree node containing the matched key is inconsistent with h(C5) in the root node, the system can re-match the first two characters (52) of the status key in the database.
- the system matches the first five characters (1-5) of the status key in the database, the first six characters (1-6) of the status key in the database, and so on, until a leaf node is matched.
- the complete state key or the corresponding complete state key can be stored in the leaf nodes.
- the ⁇ key-full ⁇ field of the leaf node can store the complete key of the corresponding state.
- the key-value pair of the leaf node stored in the database has the key that is the complete key of the corresponding state for that leaf node. In this way, when searching for a state key, the key can be directly matched against the key of the leaf node in the database, without needing to match the keys of intermediate nodes. Obviously, this search method is extremely efficient for query operations.
- the complete state key can also be used as the key of the leaf node.
- the location index is stored in the leaf node. This allows for quick location of the leaf node's value in the data file after the leaf node is found. This enables direct loading of the content at the corresponding location in the data file into memory, avoiding the read amplification problem caused by reading and loading large amounts of files from multiple levels of the SST. It also avoids the need for multiple binary search operations within the SST based on the leaf node's key (the hash value of the value) when storing the leaf node's key and value together in the persistent SST file. Furthermore, similarly, if the leaf node to be searched is not present in the memory of the tree-building module, the above embodiments can search only the index file. This allows more index files to be loaded into memory within a unit of space for searching, thus finding the leaf node faster.
- the state root in the block header stores the hash value of the root of the lower-level MPT tree to which it is anchored. This hash value is actually obtained by calculating the hash value layer by layer upwards from the leaf nodes, as described above.
- the storage module of a blockchain node constructs a tree-structured world state in memory. This application provides a method for constructing a world state based on a Merkle trie, including:
- S160 Receive status key-value.
- S162 Based on the lexicographical order rules of the Merkle trie, generate leaf nodes of the tree structure, and store the state value in a data file of the persistent storage medium.
- the leaf nodes include the position index of the state value within the data file of the persistent storage medium.
- the storage module when it receives a key-value pair (k-v) request to write state, it first hashes the key (k) (e.g., using Keccak-256). Alternatively, the state key in S162 could be the same as the state key received in S160, meaning it could be used without hashing. Next, this key is used to perform a lookup in the MPT. This lookup process starts from the root node of the MPT and traverses and matches downwards along the tree path in lexicographical order. During this process, the key is typically compared bit-by-bit (or nibble, typically a single hexadecimal digit) with the path stored in the node.
- bit-by-bit or nibble, typically a single hexadecimal digit
- branch nodes if the path forks at some point
- extension nodes if there is a shared prefix
- leaf nodes which store the final value.
- Branch nodes are used to branch in the tree
- extension nodes are used to optimize storage (through path compression)
- leaf nodes store the actual value. If a complete lexicographical path to the matching key already exists during the lookup, a leaf node is reached. At this point, the value (v) stored in this leaf node can be updated.
- search and matching process finds that only part of the path is matched, it may be necessary to create a new node (which may be a branch node, an extension node, or a leaf node) to complete the remaining path and eventually reach the leaf node, where the value is stored or updated.
- a new node which may be a branch node, an extension node, or a leaf node
- an upward backtracking process begins. This process starts by recalculating the hash value of the modified leaf node. Then, it moves upwards to the parent node, updating the relevant references in the parent node (this reference could be the direct content of the child node, or the hash value of the child node's content, depending on the size of the child node), and recalculating the hash value of the parent node. This process continues until the root node of the MPT is reached.
- each block not only contains transaction information but also stores the hash value of the global state after these transactions are executed through the state root.
- the location stored in the leaf nodes points to a location within the data file of the persistent storage medium.
- different blockchain nodes storing the same value locally may store it at a specific location within a local persistent storage medium. That is, different blockchain nodes storing the same key for the same value are located in different local data files, i.e., different locations.
- a defined anchoring rule between tree nodes needs to be designed.
- the leaf node may include, in addition to the position index of the state value in the data file in the persistent storage medium, the hash value of the corresponding state value content.
- S164 Calculate the hash value of the leaf node based on the content of the leaf node, and store the hash value of the leaf node in the tree node above it.
- leaf node A11 in Figure 15 contains not only the location but also a value hash. This value hash corresponds to the hash value of the state value content, which is the hash value of the value content within the data file in the persistent storage medium pointed to by the location index.
- the hash value of the leaf node A11 stored in slot 3 is calculated based on the value hash content, but does not include the location index of the state value within the data file in the persistent storage medium.
- the hash value of the leaf node A11 stored in slot 3 is calculated by concatenating the prefix, key-end, and value hash in the order of the leaf node A11, but this hash value calculation does not include the location.
- the tree structure may be different due to different locations when different blockchain nodes store value content in the above data file method.
- it also supports SPV capabilities through hash locking.
- the corresponding Merkle tree nodes are modified.
- the contents of these nodes can be written to an index file, as shown in Figure 16.
- a batch of modified tree nodes can be written to an index file in the background according to a set time threshold.
- Data files and index files can be logically managed separately, and physically stored in different files.
- the solution of this application can also update the tree node above the intermediate node to the changes in the hash value of the intermediate node caused by the leaf node.
- the data file is no longer sorted by key.
- the value is read directly from the location, instead of searching for the data location in multiple levels of sst files as in general KVDB, thus reducing read amplification.
- the proposed solution provides an integrated Merkle state data implementation, eliminating the two-tiered structure of an upper-level Merkle tree and a lower-level general-purpose KVDB. Instead, it integrates into a single MerkleDB with separate indexes and data. By using the Merkle tree as the index and contract state data as the data, the read/write process for state data is shortened, reducing overall write amplification and read amplification.
- the entire MerkleState is treated as a dedicated database, storing only Merkle state data. Data and indexes are separated within this database (key-value separation), eliminating the need for a two-layer structure: an upper-level Merkle tree and an underlying general-purpose KVDB. This allows for compaction of large data files without level-based separation. Instead, the garbage ratio of each data file is maintained, and garbage collection is performed in the background based on this ratio. All historical state data on the chain is retained, and garbage is only generated during rollback. Garbage collection is a low-frequency operation, thus avoiding repeated data rewriting caused by compaction in the general-purpose KVDB and reducing write amplification.
- Figure 17 is a complete illustration of another embodiment of this application.
- Figure 17 is similar to Figure 14 in general, with the main difference being that, for example, the intermediate node labeled C5 and its subordinate intermediate nodes labeled C5D1, C5D9, and C5D14 together constitute a LogicalPage.
- the leaf nodes M1, M2, M3, and M4 store key-end data instead of key-full data, and in addition to the position index, the leaf nodes also contain the hash value of the corresponding state value.
- the leaf nodes M1, M2, M3, and M4 can also be key-full data.
- the leaf node may also include the hash value of the state value content and a key-end field.
- the calculation of the hash value of the leaf node based on the content of the leaf node includes the calculation based on the hash value of the state value content of the leaf node and the key-end field, but does not include the position index of the state value in the data file in the persistent storage medium.
- the leaf node may also include the hash value of the state value content and a key-full field.
- the calculation of the hash value of the leaf node based on the content of the leaf node includes the calculation based on the hash value of the state value content of the leaf node and the key-full field, but does not include the position index of the state value in the data file in the persistent storage medium.
- This application also provides a blockchain node, which includes a storage module, a tree-building module, and persistent storage.
- the persistent storage medium includes an index file and a data file.
- the storage module also includes a search unit and a read unit.
- the leaf nodes of the tree structure include an index of the position of state data within the data file in the persistent storage medium.
- the tree-building module is used to build the world state of a tree structure in memory
- the search unit receives an instruction to search for a state key, searches for the leaf node corresponding to the state key in the tree structure in memory, and obtains the location index contained in the leaf node, the location index pointing to the location in the data file in the persistent storage medium;
- the reading unit loads the value of the data file location in the persistent storage medium pointed to by the location index into memory and reads the status data therein.
- the search unit starts from the root node of the tree structure and matches the path of the state key segment by segment until it finds the leaf node of the tree structure corresponding to the state key.
- the search unit starts from the previous level tree node and sequentially matches the hash of the path of the state key, and searches for the next level tree node based on the hash, until it finds the leaf node of the tree structure corresponding to the state key.
- the search unit starts from the previous level tree node and sequentially matches the key of the path of the state key, and searches for the next level tree node based on the matched key, until it finds the leaf node of the tree structure corresponding to the state key.
- the tree node key of the leaf node is searched in the index file in persistent storage, the value in the data file corresponding to the key is loaded into memory, and the position index is obtained from the value.
- the tree node key of the leaf node in the tree structure corresponds to the state key. Therefore, when the leaf node does not exist in memory, the tree node key of the leaf node is retrieved from persistent storage.
- the search unit searches for the tree node key of the leaf node in the persistently stored index file based on the status key.
- the tree structure includes a single-layer tree structure or a two-layer tree structure.
- PLD Programmable Logic Device
- FPGA Field Programmable Gate Array
- HDL Hardware Description Language
- Hardware description languages such as Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language) are commonly used, with VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog being the most prevalent currently.
- VHDL Very-High-Speed Integrated Circuit Hardware Description Language
- Verilog Verilog
- the controller can be implemented in any suitable manner.
- it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.
- computer-readable program code e.g., software or firmware
- ASICs application-specific integrated circuits
- controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320.
- a memory controller can also be implemented as part of the control logic of the memory.
- controller in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
- the systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions.
- a typical implementation device is a server system.
- the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.
- each module can be implemented in one or more software and/or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units.
- the device embodiments described above are merely illustrative.
- the division of units is only a logical functional division; in actual implementation, there may be other division methods.
- multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
- the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.
- These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and/or one or more block diagrams.
- These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and/or one or more block diagrams.
- a computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.
- processors CPU
- input/output interfaces network interfaces
- memory volatile and non-volatile memory
- Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
- RAM random access memory
- ROM read-only memory
- flash RAM flash random access memory
- Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
- PRAM phase-change memory
- SRAM static random access memory
- DRAM dynamic random access memory
- RAM random access memory
- ROM read-only memory
- one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
- computer-usable storage media including, but not limited to, disk storage, CD-ROM, optical storage, etc.
- One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer.
- program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type.
- One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network.
- program modules can reside in local and remote computer storage media, including storage devices.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Bioethics (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
一种构建基于默克尔字典树的世界状态的方法,应用于区块链节点的存储模块,所述存储模块在内存中构建有树结构的世界状态,所述方法包括:接收状态key-value;根据所述默克尔字典树的字典序规则,生成所述树结构的叶子节点,将所述状态value存储在持久化存储介质的数据文件中,所述叶子节点中包括所述状态value在持久化存储介质中数据文件内的位置索引;根据叶子节点中的内容计算叶子节点的hash值,并将叶子节点的hash值存储在其上层的树节点中。
Description
本申请要求于2024年7月18日提交中国国家知识产权局、申请号为2024109696604、申请名称为“一种构建基于默克尔字典树的世界状态的方法和计算机设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
本说明书实施例属于区块链技术领域,尤其涉及一种构建基于默克尔字典树的世界状态的方法和计算机设备。
区块链(Blockchain)是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链系统中按照时间顺序将数据区块以顺序相连的方式组合成链式数据结构,并以密码学方式保证的不可篡改和不可伪造的分布式账本。由于区块链具有去中心化、信息不可篡改、自治性等特性,区块链也受到人们越来越多的重视和应用。
发明内容
本发明的目的在于提供一种构建基于默克尔字典树的世界状态的方法和计算机设备,包括:
一种构建基于默克尔字典树的世界状态的方法,应用于区块链节点的存储模块,所述存储模块在内存中构建有树结构的世界状态,所述方法包括:
接收状态key-value;
根据所述默克尔字典树的字典序规则,生成所述树结构的叶子节点,将所述状态value存储在持久化存储介质的数据文件中,所述叶子节点中包括所述状态value在持久化存储介质中数据文件内的位置索引;
根据叶子节点中的内容计算叶子节点的hash值,并将叶子节点的hash值存储在其上层的树节点中。
一种计算机设备,包括:
处理器;
以及存储器,其中存储有程序,其中在所述处理器执行所述程序时,执行上述方法的操作。
上述实施例中,采用本申请的上述方案,提供了一体化的Merkle状态数据实现,不再分为上层Merkle树、底层为通用KVDB两级结构,而是整体为一个MerkleDB,DB内索引与数据分离。将merkle树作为索引,合约状态数据作为数据,缩短状态数据的读写流程,减少整体的写放大与读放大。
为了更清楚地说明本说明书实施例的技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本说明书中记载的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。
图1是一实施例中部署智能合约的示意图;
图2是一实施例中调用智能合约的示意图;
图3是一实施例中区块存储结构的示意图;
图4是一实施例中区块存储结构的示意图;
图5是一实施例中MPT树的示意图;
图6是一实施例中交易处理过程中涉及的模块及CPU、内存和磁盘关系的示意图;
图7是一实施例中交易处理过程中涉及的模块及CPU、内存和磁盘关系的示意图;
图8是一实施例中状态数据库逻辑关系的示意图;
图9是一实施例中读取状态数据的流程图;
图10是一实施例中结合一种树结构说明读取状态数据的示意图;
图11是一实施例中结合一种树结构说明读取状态数据的示意图;
图12是一实施例中结合一种树结构说明读取状态数据的示意图;
图13是一实施例中结合一种树结构说明读取状态数据的示意图;
图14是一实施例中结合一种树结构说明读取状态数据的示意图;
图15是一实施例中结合一种树结构的示意图;
图16是一实施例中结合一种树结构的示意图。
图17是一实施例中结合一种树结构的示意图。
为了使本技术领域的人员更好地理解本说明书中的技术方案,下面将结合本说明书实施例中的附图,对本说明书实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本说明书一部分实施例,而不是全部的实施例。基于本说明书中的实施例,本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例,都应当属于本说明书保护的范围。
区块链一般被划分为三种类型:公有链(Public Blockchain),私有链(Private Blockchain)和联盟链(Consortium Blockchain)。此外,还有多种类型的结合,比如私有链+联盟链、联盟链+公有链等不同组合形式。其中去中心化程度最高的是公有链。公有链以比特币、以太坊为代表,加入公有链的参与者可以读取链上的数据记录、参与交易以及竞争新区块的记账权等。而且,各参与者(体现为参与者在区块链上的节点)可自由加入以及退出网络,并进行相关操作。私有链则相反,该网络的写入权限由某个组织或者机构控制,数据读取权限受组织规定。简单来说,私有链可以为一个弱中心化系统,参与节点具有严格限制且少。这种类型的区块链更适合于特定机构内部使用。联盟链则是介于公有链以及私有链之间的区块链,可实现“部分去中心化”。联盟链中各个节点通常有与之相对应的实体机构或者组织;参与者通过授权加入网络并组成利益相关联盟,共同维护区块链运行。
不论是公有链、私有链还是联盟链,除了可以支持账户间转移区块链上的原生资产,例如转让比特币、以太币等,还可以提供智能合约的功能。区块链上的智能合约是在区块链系统上可以被交易触发执行的合约。智能合约可以通过代码的形式定义。
以以太坊为例,支持用户在以太坊网络中创建并调用一些复杂的逻辑,这是以太坊区别于比特币区块链技术的最大挑战。以太坊作为一个可编程区块链的核心是以太坊虚拟机(EVM),每个以太坊节点都可以运行EVM。EVM是一个图灵完备的虚拟机,这意味着可以通过它实现各种复杂的逻辑。用户在以太坊中发布和调用智能合约就是在EVM上运行的。实际上,虚拟机直接运行的是虚拟机代码(虚拟机字节码,下简称“字节码”)。部署在区块链上的智能合约可以是字节码的形式。
例如图1所示,Bob将一个包含创建智能合约信息的交易发送到以太坊网络后,节点1的EVM可以执行这个交易并生成对应的合约实例。图中1中的“0x6f8ae93…”代表了这个合约的地址,交易的data字段保存的可以是字节码,交易的to字段为一个空的账户。节点间通过共识机制达成一致后,这个合约成功创建,后续用户可以调用这个合约。
合约创建后,区块链上生成一个与该智能合约对应的合约账户,并拥有一个特定的地址,合约代码和账户存储将保存在该合约账户中。智能合约的行为由合约代码控制,而智能合约的账户存储则保存了合约的状态。换句话说,智能合约使得区块链上产生包含合约代码和账户存储(Storage)的虚拟账户。
前述提到,包含创建智能合约的交易的data字段保存的可以是该智能合约的字节码。字节码由一连串的字节组成,每一字节可以标识一个操作。基于开发效率、可读性等多方面考虑,开发者可以不直接书写字节码,而是选择一门高级语言编写智能合约代码。高级语言编写的智能合约代码,经过编译器编译,生成字节码,进而该字节码可以部署到区块链上。以太坊支持的高级语言很多,如Solidity、Serpent、LLL语言等。
以Solidity语言为例,用其编写的合约与面向对象编程语言中的类(Class)很相似,在一个合约中可以声明多种成员,包括状态变量、函数、函数修改器、事件等。状态变量是存储在智能合约的账户存储中的值,用于保存合约的状态。
如下是以Solidity语言编写的一个简单的智能合约的代码示例1:
此外,如图2所示,仍以以太坊为例,Bob将一个包含调用智能合约信息的交易发送到以太坊网络后,节点1的EVM可以执行这个交易并生成对应的合约实例。图中2中交易的from字段是发起调用智能合约的账户的地址,to字段中的“0x6f8ae93…”代表了被调用的智能合约的地址,交易的data字段保存的调用智能合约的方法和参数。此外,还可以包括value字段,用以表示该交易中以太币的值。调用智能合约后,storedData的值可能改变。后续,某个客户端可以通过某一区块链节点(例如图2中的节点6)查看storedData的当前值。
智能合约可以以规定的方式在区块链网络中每个节点独立的执行,所有执行记录和数据都保存在区块链上,所以当这样的交易完成后,区块链上就保存了无法篡改、不会丢失的交易凭证。
如前所述,上述示例中的storedData即是状态变量,其存储在智能合约的账户存储中。引入智能合约的各种区块链网络中,以太坊为例,通常账户可以包括两种类型:
合约账户(contract account):存储执行的智能合约代码以及智能合约代码中状态的值,通常只能通过外部账户调用激活;
外部账户(Externally owned account):用户的账户,例如以太币拥有者账户。
外部账户和合约账户的设计,实际上是账户地址到账户状态的映射。账户的状态通常包括Nonce、Balance、Storage root、CodeHash等字段。Nonce、Balance在外部账户和合约账户中都存在。CodeHash和Storage root属性一般仅在合约账户上有效。
Nonce:计数器。对于外部账户,这个数字可以代表从账户地址发送的交易数量;对于合约账户,可以是账户创建的合约数量。
Balance:这个地址拥有的以太币的数量。
Storage root:一个MPT树根节点的哈希,这个MPT树对合约账户的状态变量的存储进行组织。
CodeHash:智能合约代码的哈希值。对于合约账户,这是智能合约的哈希值;对于外部账户,由于不包括智能合约,因此CodeHash字段一般可以是空字符串/全0字符串。
MPT全称为Merkle Patricia Tree,是结合了Merkle Tree(默克尔树)和Patricia Tree(压缩前缀树,一种更节省空间的Trie树,字典树)的一种树形结构。Merkle Tree,默克尔树算法对每个交易都计算一个Hash值,然后两两连接再次计算Hash,一直到最顶层的Merkle根。以太坊中采用改进的MPT树,例如是16叉树的结构,通常也简称为MPT树。
以太坊MPT树的数据结构包括状态树(state trie)。状态树中包含以太坊网络中每个账户所对应的存储内容的键值对(key and value pair,也写作key-value,简称k-v或kv)。状态树中的“键”(key)可以是一个的160bits的标识符(例如以太坊账户的地址或地址的hash值的一部分,下面统称为账户地址),这个账户地址分布于从状态树的根节点开始到叶子节点的存储中。状态树中的“值”是通过对以太坊账户的信息进行编码(使用递归长度字典编码(Recursive-Length Prefix encoding,RLP)方法)生成的。如前所述,对于外部账户来说,值包括nonce和balance;对于合约账户来说,值包括nonce、balance、codehash和storageroot。
合约账户用于存储智能合约相关的状态。智能合约在区块链上完成部署后,会产生一个对应的合约账户。这个合约账户一般会具有一些状态,这些状态由智能合约中状态变量所定义并在智能合约创建、执行时产生新的值。所述的智能合约通常是指在区块链环境中以代码形式定义的能够自动执行条款的合约。一旦某个事件触发合约中的条款(满足执行条件),代码即可以自动执行。在区块链中,合约的相关状态保存在storage trie中,storage trie根节点的hash值即存储于上述storageroot中,从而将该合约的所有状态通过hash锁定到该合约账户下。storage trie也是一个MPT树形结构,存储了状态地址到状态值的key-value映射。从storage trie树的根节点到叶子节点中的部分信息顺序排布后用以存储一个状态的地址,该叶子节点中存储状态的值。
如图3所示的一些区块链数据存储中,每一区块的区块头包括若干字段,例如上一区块哈希previous_Hash(图中的Prev Hash),随机数Nonce(在一些区块链系统中这个Nonce不是随机数,或者在一些区块链系统中不启用区块头中的Nonce),时间戳Timestamp,区块号Block Num,状态根哈希State_Root,交易根哈希Transaction_Root,收据根哈希Receipt_Root等。其中,下一区块(如区块N+1)的区块头中的Prev Hash指向上一区块(如区块N),即为上一区块的hash值。通过这种方式,区块链上通过区块头实现了下一区块对上一区块的锁定。其中,State_Root、Transaction_Root和Receipt_Root分别锁定了状态集合、交易集合和收据集合。状态集合、交易集合和收据集合分别以树的形式组织了状态、交易和收据。一般的,可以是相同的树形结构,也可以是不同的树形结构。例如在以太坊中,采用了相同的MPT结构。在以太坊之类的一些包括智能合约的状态集合的树形结构中,包括两级的MPT结构:上一级的MPT结构的叶子节点包括外部账户和合约账户两种类型;其中的每个合约账户包括下一级的MPT结构,下一级的叶子节点中包括合约账户中的状态的值。
图4是一个区块链数据存储的结构示意图。仍以以太坊为例,可以结合图3所示,state_root是当前区块中所有账户的状态组成的MPT树的根的哈希值,即指向state_root的为一颗MPT形式的状态树state trie。这个MPT树的根节点一般为一个扩展节点(Extension Node)或一个分支节点(Branch Node),state_root中存储的一般为这个根节点的hash值。根节点可以与下面一层或多层的Extension Node/Branch Node相连,
这些多层的树节点可以统称为中间节点(Internal Node)。从这个MPT的根节点到叶子节点中每个节点中的一部分值按照顺序串联起来可以构成账户地址并作为key,叶子节点中存储的账户信息为这个账户地址对应的value,这样,构成了key-value键值对。这个key也可以是sha3(Address)后取一部分,即账户地址的hash值(hash算法例如采用sha3算法)的一部分,其存储的值value可以为rlp(Account),即账户信息的rlp编码。其中账户信息是[nonce,balance,storageRoot,codeHash]构成的四元组。如前所述,对于外部账户来说,一般只有nonce和balance两项,而storageRoot、codeHash字段默认存储空字符串/全0字符串。也就是说,外部账户不存储合约,也不存储合约执行后的产生的状态变量。合约账户一般包括Nonce,Balance,Storage root,CodeHash。其中Nonce是该合约账户的交易计数器;Balance是账户余额;Storage root对应另外一个MPT,通过Storage root能链接到合约相关的状态的信息;CodeHash是合约代码的hash值。不论是外部账户还是合约账户,其账户信息一般都位于一个单独的叶子节点(Leaf Node)中。从根节点的Extension Node/Branch Node到每个账户的Leaf Node,可能中间会经过若干个分支节点以及扩展节点。
state trie可以是MPT形式的树,一般是16叉树,即每一层最多可以有16个孩子节点。对于Extension Node,用于存储共同前缀,其一般有1个孩子节点,这个孩子节点可以是Branch Node。对于Branch Node,其最多可以有16个孩子节点,其中可能包括Extension Node和/或Leaf Node。
其中,对于state trie中的一个合约账户来说,其storage_Root指向另一颗同为MPT形式的树,其中存储了合约执行涉及的状态变量(state variable)的数据。这个storage_Root指向的MPT形式的树为Storage Trie,即Storage Trie的根节点的hash值。一般的,这个Storage Trie树存储的也是key-value键值对。key表明状态变量的地址,其取值可以是合约中的状态变量声明的位置(从0开始计数的值)经过一定规则处理后得到的结果,例如是sha3(状态变量声明的位置),或者是sha3(合约名称+状态变量声明的位置)。value用于存储状态变量的取值(例如是经RLP编码的值)。从根节点经中间节点到叶子节点的路径上存储的一部分数据连起来构成key,叶子节点中存储value。前面提到,这个Storage trie也可以是MPT形式的树,一般也是16叉树,即对于Branch Node,其最多可以有16个孩子节点,这些孩子节点可能包括Extension Node和/或Leaf Node。而对于Extension Node,其一般可以有1个孩子节点,这个孩子节点可以是Branch Node或Leaf Node。
例如图4中的state Trie的Leaf Node Account P,该账户是一个合约账户,其Storage Root锁定了该合约存储中的所有状态。这些状态组织为MPT树,树形结构如该Storage Root链接的Storage trie。这个链接的Storage trie中,以Leaf Node State Variable N为例,例如为前述合约代码示例的storedData的值,则其key为sha3(storedData的声明位置,即代码的第2行),其value值为s(为了简洁,这里省略了对value的编码格式,例如是RLP,后续类似,不再赘述)。其中,key的值顺序的分布于storage Trie的根节点到叶子节点(即Leaf Node Variable N)中。
再例如,图4中的state Trie中的Leaf Node Account C,该账户是一个外部账户,其key为sha3(Address C),即账号C的地址的hash值(hash算法例如采用sha3算法),其存储的值value可以为(Account),其中账户信息Account是[nonce,balance]构成的二元组。如前所述,由于Account C为外部账户,因此其账户信息是nonce和balance两项(这里省略了codehash和storage root,以下类似)。例如一个外部账户,其nonce为20,Balance为4550,则Leaf Node State Variable C这个叶子节点中即存储nonce=20,balance=4550。而Account C的地址为key,其值顺序的分布于state Trie的根节点到叶子节点(即Leaf Node Variable C)中。
这些状态,包括外部账户的k-v和合约账户的k-v,最终存储于数据库中。数据库中的存储,并不是直接存储这些账户的状态,即不是直接存储这些账户的k-v,而是存储每个树节点本身的k-v值。
如图5的示例中所示,上一级的MPT结构中,对于叶子节点A1,通过根节点A8(Extension Node)中shared nibble的a7—中间节点A7(Branch Node)的槽位1—叶子节点A1中key-end的1335,顺序组合起来构成该叶子节点的key,即为a711335,在该叶子节点中存储Balance=45.0ETH,Nonce=n1。对于叶子节点A2,通过根节点A8(Extension Node)中shared nibble的a7—中间节点A7(Branch Node)的槽位7—节点A6(Extension Node)中shared nibbles的d3—中间节点A5(Branch Node)中的槽位3-叶子节点A2中key-end的7,顺序组合起来构成该叶子节点的key,即为a77d337,在该叶子节点中存储Balance=1.00WEI,Nonce=n2。对于叶子节点A3,通过根节点A8(Extension Node)中shared nibble的a7—中间节点A7(Branch Node)的槽位f—叶子节点A3中key-end的9365,顺序组合起来构成该叶子节点的key,即为a7f9365,在该叶子节点中存储Balance=1.1ETH,Nonce=n3。对于叶子节点A4,通过根节点A8(Extension Node)中shared nibble的a7—中间节点A7(Branch Node)的槽位7—节点A6(Extension Node)中shared nibbles的d3—中间节点A5(Branch Node)中的槽位9-叶子节点A4中key-end的7,顺序组合起来构成该叶子节点的key,即为a77d397,在该叶子节点中存储Balance=0.12ETH,Nonce=n4,CodeHash=c1,Storage root=s1。s1可以为H(A10),即下一层树的根节点A10的hash至。其中,A1、A2和A3的叶子节点中存储的是外部账户的信息,A4的叶子节点中存储的是合约账户的信息。对于合约账户,其包含下一级MPT,构成Storage Trie,用于存储该合约账户中的状态变量。
如图5的示例中所示,下一级的MPT结构中,对于叶子节点A11,通过根节点A10(Branch Node)中的槽位3—叶子节点A11中key-end的35b2e4,顺序组合起来构成该叶子节点的key,即为335b2e4,在该叶子节点中存储“张三_A=20”,例如表示在合约中定义的A类型数字资产归属于张三的份额为20,即张三的A类资产的余额为20。对于叶子节点A12,通过根节点A10(Branch Node)中的槽位7—叶子节点A12中key-end的c25988,顺序组合起来构成该叶子节点的key,即为7c25988,在该叶子节点中存储“李四_B=20”,例如表示在合约中定义的B类型数字资产归属于李四的份额为50,即李四的B类资产的余额为50。对于叶子节点A15,通过根节点A10(Branch Node)中的槽位f—中间节点A13(Extension Node)中shared nibble的a—中间节点A14(Branch Node)的槽位6—叶子节点A15中key-end的be33,顺序组合起来构成该叶子节点的key,即为fa6be33,在该叶子节点中存储“storedData=s”。对于叶子节点A16,通过根节点A10(Branch Node)中的槽位f—中间节点A13(Extension Node)中shared nibble的a—中间节点A14(Branch Node)的槽位9—叶子节点A16中key-end的9365,顺序组合起来构成该叶子节点的key,即为fa99365,在该叶子节点中存储“王五_A=35”,例如表示在合约中定义的A类型数字资产归属于王五的份额为35,即王五的A类资产的余额为35。
上述MPT树的节点构成中,用前缀prefix表示树节点类型,例如0表示包含偶数个shared nibbles(共享的半字节)的Extension Node,用1表示包含奇数个shared nibble(s)的Extension Node,用2表示包含偶数个nibbles的Leaf Node,用3表示包含奇数个nibble(s)的Leaf Node。
上述节点构成中,下一个树节点的整体内容的hash值,填入上一个树节点的对应位置中。数据库中,实际上存储每个树节点的key-value的映射,其中value包括这个树节点中存储的内容,对应的key是这个树节点整体内容的hash值。这样,数据库中实际存储的树节点k-v如下表:
表1、数据库中实际存储的树节点k-v
上表1中,用H()表示hash计算。这样,下一个树节点的hash值锚定在了上一个树节点中。通过这样的层层hash,得到整颗state trie树的根hash,并将该根hash锁定到了区块头的state root字段中。
在一些区块链系统中,区块链平台的代码可以包括P2P(Peer to Peer,点对点)模块,共识(consensus)模块,执行模块和存储模块。P2P是一种计算机网络的组成方式,与常见的web网络不同,P2P是分散的、去中心化的。P2P模块可以完成数据的分布式传播。对于区块链节点来说,通过P2P模块可以以点对点的方式传播和接收收据。不同参与方通过部署的节点(Node)可以建立一个分布式的区块链网络。利用链式区块结构构造的账本,保存于分布式的区块链网络中的每个节点(或大多节点上,如共识节点)上,这也称为去中心化(或称为多中心化)的分布式账本。这样的区块链系统需要解决去中心化(或多中心化)的多个节点上各自的账本数据的一致性和正确性的问题。每个节点上都运行着相同的区块链平台程序,在一定容错需求的设计下,通过共识模块可以保证所有忠诚节点具有相同的交易,从而保证所有忠诚节点对相同交易的执行结果一致,并将交易及执行结果打包生成区块。当前主流的共识机制包括:工作量证明(Proof of Work,POW)、股权证明(Proof of Stake,POS)、委任权益证明(Delegated Proof of Stake,DPOS)、实用拜占庭容错(Practical Byzantine Fault Tolerance,PBFT)算法,蜜獾拜占庭容错(HoneyBadgerBFT)算法等。共识过程中共识模块一般还可以生成当前交易集合对应的区块的时间戳等。执行模块可以执行交易,包括普通转账交易和涉及合约的交易,可以是在共识模块完成共识之前或之后。对于涉及合约的交易,执行模块可以引入虚拟机来执行智能合约的代码,如以太坊虚拟机(Ethereum Virtual Machine,EVM),从而通过EVM屏蔽各个节点硬件配置和软件环境的差异性,以保证各个节点上执行智能合约的过程和结果是相同的,并通过沙箱环境避免智能合约的执行给主机上的区块链平台代码、其它程序或操作系统带来影响。对于联盟链的一种情形来说,节点之间通过共识模块可以确定一个交易集合中的交易内容和交易顺序,进而将共识结果的一个确定性的交易集合输出至执行模块。执行模块通过执行普通转账交易/涉及合约的交易,生成执行结果,并发送至存储模块。存储模块可以负责将执行结果存储至节点本地的持久化存储介质中。
如图6所示的一个区块链节点中,物理上包括CPU、内存和磁盘等。这个区块链节点所执行的区块链平台代码中,可以包括P2P模块,共识模块,执行模块和存储模块。P2P模块、共识模块和执行模块的功能实现一般需要CPU、内存的参与。存储模块可以包括构建树模块,区块头生成模块,WAL(Write Ahead Log,写前日志)模块,状态数据库模块。其中,构建树模块用于基于执行模块传入的状态k-v构建树(例如是MPT树),如前述的state trie和storage trie,从而得到树节点的k-v,一般需要CPU、内存的参与。区块头生成模块用于根据构建树模块所构建的树的根节点和其它一些数据(如上一区块hash、时间戳、区块号等)生成区块头,一般需要CPU、内存的参与。WAL模块用于构建树模块生成的树的叶子节点k-v写入状态数据库模块之前,持久化存储构建树模块生成的树的叶子节点k-v,以防止构建树模块生成的树的叶子节点k-v写入状态数据库模块的过程中由于断电等情形造成的数据丢失,并在发生这种情况时恢复数据,一般需要CPU、内存和磁盘的参与。状态数据库模块用于将构建树模块所构建的如表1中的树节点k-v存储在持久化存储设备上;由于最终会将树节点数据写入持久化存储介质(例如图中的磁盘),因此状态数据库模块除了CPU、内存外一般还需要磁盘的参与。
从存储结构上来说,上述的Merkle树结构,如以太坊的MPT、Libra的SMT(Sparse Merkle Tree,稀疏默克尔树,类似MPT),以上述表1中的对应关系形式可以位于构建树模块中,并存储于内存中。其中,上层Merkle树为前缀树(字典树),能够实现对数据的组织,并对组织后的数据得到唯一的Merkle根。叶子节点可以保存状态Value,根节点到中间节点到叶子节点实现对状态key的字典序索引。这些树节点按某种规则编码为Key、其内容编码为Value,最终存储于下层数据库中。数据库大多采用LSM(Log-Structured Merge-Tree,日志结构的合并树)类结构的NoSQL Key-Value DB(DataBase,数据库;Key-Value DB也简称为KVDB),位于状态数据库模块中,最终保存于磁盘。具体的,数据库例如是以太坊的levelDB,Libra的RocksDB。这两种KVDB都是基于LSM存储引擎。
LSM存储引擎是一种分层、有序、面向磁盘的存储引擎,其借鉴了Log不断追加(而不是修改)的特点,核心思想是充分利用磁盘批量的顺序写远比随机写高效的特性,放弃部分读效率换取最大化的写操作效率。一般来说,最大化发挥磁盘特性的使用方式是一次性地读取或写入固定大小的一块数据,并尽可能地减少随机的寻址操作。LSM的设计思想就是依据磁盘这个特性,并假定内存足够,不需要每次有数据更新就将其写入磁盘,而是先将最新的数据驻留在内存中,等数据量积累到足够多之后,再使用归并排序的方式将内存中的数据与磁盘中的数据合并,批量追加到磁盘。
以下以图7为例,进一步说明状态数据模块的存储过程。构建树模块生成树节点k-v数据并发送至存储模块后,这些k-v数据先以Append-Only(仅追加)的方式写入WAL,并存储在磁盘上。进而,树节点k-v数据传入状态数据库模块。如前所述,先写入WAL可以防止构建树模块生成的树节点k-v写入状态数据库模块的过程中由于断电等情形造成的数据丢失,并在发生这种情况时恢复数据。状态数据库模块接收到树节点k-v数据后,先存入内存中,具体可以是存入内存中的MemTable中。MemTable可以提供并发读写操作。内存中可以有多个MemTable。当MemTable中的数据量达到一定阈值后,例如达到256MB时,MemTable中的数据可以写入(flush)到磁盘中。为了避免写MemTable的操作对flush造成堵塞,这个MemTable转化为不可更改的Immutable Memtable,即Immutable Memtable设置为只读,并生成一个新的MemTable来接收新传入的树节点k-v。这个新的MemTable可以提供并发读写操作(如图中的New MemTable)。存储引擎将Immutable MemTable中的数据写入磁盘。
磁盘中,KVDB以多个level的SST(Sorted String Table,SSTable,排序字符串表)文件来存储。SST来源于Google Bigtable的论文。BigTable是Google设计的分布式数据存储系统,用来处理海量的数据的一种非关系型的数据库。SST可以用于Bigtable内部数据文件的存储,是一种高效的key-value型文件存储格式。存储在持久化存储介质中的内容也可以称为文件。
SST可以包括多层,例如包括3层、4层、5层、6层、7层或更多。图7中示出了3层的情况,即level 0、level 1、level 2。一般的,上一层的SST的总容量明显小于下一层。作为例子,例如level 0的sst总容量是1GB,level 1的sst总容量是10GB,level2的sst总容量是100GB。假设MemTable的容量是256MB。可以以区块对应的树节点k-v来写入构建树模块,进而写入MemTable。在一个或多个区块对应的树k-v写入MemTable后,MemTable的空间占用可能会达到256MB。进而,这个MemTable转化为Immutable MemTable,并可以将Immutable MemTable中的数据写入(flush,这里指内存中的数据写入到磁盘的操作)到磁盘中。具体的,MemTable中的数据可以flush到磁盘的Level 0的SST中。另一方面,如前所述,生成一个新的MemTable来接收新传入的树节点k-v,并提供并发读写操作。
如前所述,每一层的磁盘存储中,即每个level的磁盘存储中,可以包括多个SST文件。Level 1以及更底层Level的SST中存储的kv,一般按照key的顺序存储,而且同一层的不同SST中的key值也是有大小关系且不重复。例如Level 1中包括3个SST,左边的SST11中包括1000个k-v,例如是<k1-v1><k2-v2><k3-v3>...<k1000-v1000>,并按照k从小到大的顺序排布,例如k1=10b43c032a46,k2=1118d46f41e1,k3=1b44942a3d6e,...,k1000=2018df9326a7。可见,左边SST11中包含的k1<k2<k3<...<k1000,但k1,k2,k3,...k1000之间并不一定连续。类似的,中间的的SST12中例如包括800个k-v,<k1-v1><k2-v2><k3-v3>...<k800-v800>,并按照k从小到大的顺序排布,例如k1=21a514c0a7e1,k2=21b46b994214,k3=21b489571b6d,...,k800=284a5df3fca7。可见,中间的SST12中包含的k1<k2<k3<...<k800,但同样的,k1,k2,k3,...k800之间并不一定连续。类似的,右边的SST13中例如包括1100个k-v,<k1-v1><k2-v2><k3-v3>...<k1100-v1100>,其中的k按照从小到大的顺序排布,例如k1=3a5e55ac3fe1,k2=3a5e55ac3fe2,k3=3afb8ed31526,...,k1100=400a51ef1405。可见,右边的SST13中包含的k1<k2<k3<...<k1100,但同样的,k1,k2,k3,...k1100之间并不一定连续。而且,整体上,左边SST11的k的最小值<左边SST11的k的最大值<中间SST12的k的最小值<中间SST12的k的最大值<右边SST13的k的最小值<右边SST13的k的最大值。Level 2、Level 3...的每一层也是类似,即每一层中每个SST之内,在k的值上具有大小顺序关系,在每一层的多个SST之间,在k的范围上具有大小关系。
Level 0中的1个SST内的k-v可以按照k的大小关系排序,多个SST之间k可以没有范围上的大小关系。这是因为,MemTable一般按照区块由旧到新(或者区块号由小到大的关系,因为新的区块的区块号较大,旧的区块的区块号较小)的顺序存储每个区块对应的多个k-v,而这些k是树节点的hash值,树节点的内容的变化大概率导致树节点的hash值(即k值)很不相同。Level 0中一个SST内的k-v可以按照k的大小来排序,在内存中这样的排序是高效的和可行的;当然,也可以不按照k值大小来排序,这里并不排斥。以排序为例,当MemTable的容量达到256MB时,可能其内存储的是包括多个区块对应多个k-v,这个MemTable转化为Immutable MemTable。Immutable MemTable中的这些k-v,可以按照k的顺序排序后写入Level 0的左边的SST中。可能会有多次的不同Immutable MemTable写入同一SST的过程,才将Level 0左侧的SST01写满,进而可以类似的方式写入Level 0右侧的SST02。尽管单次Immutable MemTable写入Level 0中某一个SST的k-v是经过排序的,但由于Level 0中同一SST中由不同Immutable MemTable写入的k-v集合之间一般是没有大小排序的,不同SST(例如左、右两个SST)之间的k-v也没有大小排序,即Level 0中左、右两个SST中k的范围可能有交叉/重合。同样的,由于Level0中的一个SST中所写入的k-v可以是多次MemTable存满后写入的,而多次MemTable存满后的写入过程中的k-v范围一般都会有重叠,因此Level 0中的每个SST中存储的多个k-v之间一般也没有特定顺序,Level 0中各SST之间一般也没有特定的范围顺序。当然,如果Level 0中由Immutable MemTable单次写入一个SST,则由于Immutable MemTable在写入该SST中时对k-v进行了排序,则该SST中存储的k-v也是按照k来排序的。
进一步的,当Level 0的存储容量达到或接近上限后,通过一个称为“compaction”(压实)的过程来将Level 0的数据写入Level 1中。在这个compaction的过程中,Level0内的各SST中的k-v和Level 1内部分或所有SST的k-v可以先转入到内存中,在内存中完成排序后再写入Level 1的SST中。由于在compaction的过程中进行了排序,所以Level 1中SST内,在k的值上具有大小顺序关系,且在Level 1的多个SST之间,在k的范围上具有大小关系。换句话说,Level 1的每个SST内,k-v按照<k1-v1><k2-v2><k3-v3>...<kn-vn>的方式排布,且k1<k2<k3<...<kn,而k1,k2,k3,...kn之间并不一定连续。而且,整体上,对于相邻的两个SST,左边SST的k的最小值<左边SST的k的最大值<右边SST的k的最小值<右边SST的k的最大值。如图8中所示。
在compaction的过程中Level 0内的各SST中的k-v和Level 1内部分SST的k-v可以转入到内存中完成排序,所述Level 1内部分SST,可以是Level 1内与Level 0中的k-v有范围重叠的SST,即Level 1内与Level 0的各SST中的k-v有范围重叠的SST。具体的,例如上述例子中,Level 1中包括3个SST,SST11中key的范围是10b43c032a46~2018df9326a7,SST12中key的范围是21a514c0a7e1~284a5df3fca7,SST13中key的范围是3a5e55ac3fe1~400a51ef1405。例如Level 0内的各SST中key的范围为18c49a405e32~24d35f39ab56。可见,Level 1内与Level 0中的key范围重叠的有SST11和SST12。这样,在compaction的过程中,Level 0内的各SST中的k-v和Level 1内的SST11、SST12的k-v可以转入到内存中并完成排序,进而将排序后的结果再写入到Level1的SST中。
类似的,当Level 1的存储容量达到上限后,通过compaction的过程来将Level 1的数据写入Level 2中。同样的,在这个compaction的过程中,Level 1中各SST中的k-v和Level 2内部分或所有SST的k-v可以先转入到内存中,在内存中完成排序后再写入Level 2的SST中。类似的,如果在Level 2之下还有Level 3,则当Level 2的存储容量达到上限后,通过compaction的过程来将Level 2的数据写入Level 3中。同样的,
在这个compaction的过程中,Level 2内各SST与Level 3内部分或所有SST的k-v可以先转入到内存中,在内存中完成排序后再写入Level 3的SST中。以此类推。
这样,整体上,上层的SST中存储的数据比下层的新。最新的数据存储于内存中,次新的数据存储于Level 0,...最旧的数据存储于最底层Level内的SST中。此外,通过compaction还可以进行垃圾回收(Garbage Collection),将标记为删除的数据进行删除之类。
排序后的kv,可以高效的进行检索,例如通过二分查找进行高效的检索。二分查找也称为折半查找,算法如下:定义left为顺序表最左端元素位置,right为顺序表右端元素位置,定义mid=(left+right)/2,即顺序表的中间位置;用所查找的值与mid所在位置处的值比较;由于列表有序,若所查找的值比mid小,则只需在表的前半部分查找,否则只需在表的后半部分查找(若第一次比较就发现两值相等则直接返回当前值所在的位置)。以此类推,直至查找到所寻找的值或确定所查找的值不在该列表内为止(即查找失败)。
这样的存储引擎,读、写数据时存在读放大、写放大问题,以下具体介绍。
采用上述存储引擎的区块链系统,在执行模块执行交易的过程中,可能需要读某个已有状态。例如,对于普通转账交易,外部账户A转账一定数额(例如是f)的资产至外部账户B,则需要在账户余额基础上做增减。假设该交易执行前,账户A的余额是a,账户B的余额是b,则该交易执行,是将账户A的余额设置为(a-f)的值,将账户B的余额设置为(b+f)的值。则这个交易在执行时需要读取a和b这两个状态变量的值。再例如,一个调用合约的交易,外部账户C发起对合约D的调用,则C发起指向合约账户D的地址的一笔交易,并在交易的data字段指明调用的函数和输入的参数。假设合约D中被调用的函数的逻辑包括判断合约中的一个全局状态d的值,假设这个d的值是布尔类型,判断逻辑包括,如果是true则执行分支1,如果是false则执行分支2。则这个调用合约的交易执行时需要读取d这个状态的值。
交易执行过程中,无论是上述读取状态变量a、b的值,还是读取状态变量d的值,执行模块都可以首先在执行模块的内存中查找是否存在该状态变量的值,如果不存在,则发送读指令至存储模块。其中的读指令可以包括要读的状态的地址,即状态的key值。存储模块接收到读指令后,在构建树模块中找到状态树树节点的根节点,并将待查找的状态key值与根节点的shared nibble(s)字段(对于Extension Node)或槽位(对于Branch Node)从头开始匹配后,从匹配的位置读取下一层树节点(可以是Internal Node或Leaf Node,以Internal Node为例加以说明)的hash,进而,优先在构建树模块的内存中查找该hash,如果还没有找到则在状态数据库模块的内存中查找,如果仍然没有找到,就需要在状态数据库模块的文件中查找。具体的,可以优先Level 0中的SST内查找,如果还找不到就继续在Level 1的某个SST内查找,以此规律不断往下层Level的某个SST内查找,直至找到该hash值指向的下一Internal Node;解开这个Internal Node后,从中继续从前往后的匹配待读取的状态地址key值剩余部分的字段,如果匹配,则从匹配处读取hash值后,跳转至该hash值指向的下一级树节点。由此不断重复,逐级解开Internal Node并从前往后的匹配待读取的状态地址key值剩余的部分字段,匹配处的hash值作为下一次查找中间节点或叶子节点的依据,直至匹配到Leaf node,从而从Leaf node中读取到状态的value值。这个过程中,各级树节点的k-v有些可能并不存在于构建树模块的内存中,而是位于状态数据库模块中。因此,可能需要从状态数据库模块中读取至内存中操作。可以首先在状态数据库模块的MemTable或Immutable MemTable中查找这个key,如果不存在,就需要进一步查找磁盘上的各级SST文件。对于存在于磁盘上的Internal Node和Leaf node,需要加载到内存中,并解码(反序列化)得到其中的内容。具体的,例如首先在level0的sst文件逐个查找,如果Level 0的每个SST内做了排序,则可以在每个SST内根据中间节点/叶子节点的hash值做二分查找,查找到后将v载入内存并对其解码和匹配。如果Level 0的SST中没有找到,则进一步在Level 1内根据中间节点/叶子节点的hash值做二分查找,如果没有找到则在Level 2内做二分查找。如果找到则将这个k对应的v载入内存并解码。以此类推,直至匹配到树结构的叶子节点并从中读取value的值。上述在Level 1、Level 2以及更底层的Level中的查找时,由于这些Level中整体的SST是经过排序的,所以一般可以在这个Level中做一次二分查找。
由上述读取一个状态k-v的过程,可以看出,这个读取过程需要根据树节点的Hash值在多个不同Level的SST中做二分查找,这可能需要在多个Level的SST中读取大量的文件并加载进内存中,这就带来了读放大的问题。假设数据库中包括Level 0、Level 1、Level 2、...、Level M-1,共M层,则最坏的情况下,读取一个状态的数据需要至少对磁盘进行M次的访问(大于M是因为最坏情况下对Level 0层的N个SST可能要进行大于1次小于等于N次的查找)。
此外,构建树模块如果在根据状态k-v构建到根节点的路径上所有树节点的k-v过程中,缺少某个树节点的k-v,也可能需要从Level 0、Level 1、Level 2、...等中做二分查找,原理与上述过程类似。上述的交易例子中,例如交易执行之后,需要将执行结果(a-f)的值/(b+f)的值/合约状态d的值写入状态数据库中。对于上述要写入数据库的值,执行模块调用存储模块的存储接口,输出状态的key-value值至存储模块。存储模块中的构建树模块根据状态k-v生成树结构的树根至中间节点、叶子节点的树节点k-v,还将这些树节点k-v写入WAL模块,之后再将这一系列树节点的k-v写入状态数据库模块的MemTable中,并标记为脏数据(dirty data)。脏数据通常是指已经被写入MemTable但尚未被写入持久性存储(如SSTable)的数据。脏数据可以在前述flush过程中写入磁盘。状态数据库模块进而将MemTable中包括所述树节点k-v在内的一些k-v写入到磁盘的SST文件中。一旦MemTable被持久化到SSTable,其中的数据就不再是脏数据了,即变成“干净”数据。可见,上述持久化存储一个状态k-v的过程较为复杂,消耗较多的计算、内存资源,并且带来了写放大问题。
上述读过程、写过程的复杂性和读放大、写放大问题,对区块链的高性能要求和大规模应用带来了挑战。
本申请提供一种读取区块链状态数据的方法实施例。该方法实施例可以应用于区块链节点的存储模块。所述存储模块在内存中构建有树结构的世界状态。所述树结构的叶子节点中可以包括所述状态数据的内容或所述状态数据在持久化存储介质中数据文件内的位置索引。
一个例子中,树结构整体可以如图10所示,这也与图5中有一定的类似。特别的是,如果内存中的叶子节点内有状态数据,则可以如图5所示,例如图5中的A11这个叶子节点,在内存中有状态数据,即张三_A=20。如果内存中的叶子节点内没有状态数据,则可以利用其中存储该状态数据在数据文件中的位置索引。
本申请中,将实际数据和索引分离,有助于提高系统的灵活性和性能。这里涉及的基本概念包括数据文件和索引文件。其中,数据文件是存储实际内容的文件;索引文件是存储指向数据文件位置的索引。其工作原理在于:将数据内容写入数据文件,记录数据在文件中的位置(如文件名、偏移量和长度);进而,在索引文件中创建一个条目,包含:键(用于查找的唯一标识符)指向数据文件的引用(文件名、偏移量等)。这样,在数据检索时,可以通过索引文件快速定位数据位置,进而根据位置信息从数据文件中读取实际内容。这样的方式,优势在于:①灵活性,数据和索引可以独立管理和优化;②性能,索引通常较小,可以全部加载到内存,加快查询速度;③空间效率:避免在索引中存储大量重复数据;④更新效率:修改数据时,只需更新索引中的位置信息。
例如图10中,用location来表示这个位置索引。仍然以A11节点为例,如果内存中没有状态数据,则可以利用其包括的location字段存储状态数据所在数据文件中的位置索引,该位置索引例如为<2,750,210>。这个<2,750,210>,可以指明状态数据位于数据文件中的“文件2”中,起始位置为“文件2”的第750KB,长度为210KB。如图10左下角的数据文件所示,假设每个文件的大小为1024KB,索引的逻辑地址范围为0KB~1023KB。这样,A11的状态数据,实际上占据了文件2中从第750KB~第959KB的范围。
如图9所示,该实施例包括:
S910:区块链节点的存储模块接收要查找的状态key的指令,在内存中的所述树结构上查找该状态key对应的叶子节点。
如前述图6中所示,执行模块可以通过执行普通转账交易/涉及合约的交易,生成执行结果,并发送至存储模块。具体的,例如是执行合约的过程中需要读取状态变量的值。执行模块可以首先在执行模块的内存中查找是否存在该状态变量的值,如果不存在,则发送读指令至存储模块。其中的读指令可以包括要读的状态的地址,即状态的key值。
存储模块中包括构建树模块。构建树模块在内存中构建有世界状态的树形结构。如果是区块链节点冷启动的情况,即内存中还没有世界状态的树形结构的情况,则可以从磁盘中加载树形结构的各个树节点,并在内存中组织成一颗树,这个过程也会涉及加载树形结构中的叶子节点。这里以构建树模块中存在一颗不完整树形结构的情况举例说明,冷启动加载的情况与此类似。由于内存的容量一般是有限的,因此一般采用LRU(Least Recently Used,最近最少使用,一种常用的页面置换算法,选择最近最久未使用的页面予以淘汰)算法,将长时间没有访问的树节点从内存中淘汰。这样,构建树模块在内存中构建的树形结构上的各个叶子节点,存在一定被淘汰的可能。对于世界状态树的叶子节点,则可能有些在内存中,有些不在内存中。此外,对于树节点中的内容,例如叶子节点中的状态数据,也可以采用类似的淘汰机制,或者,内存中也可以不保持状态数据。
对于要查找的状态key,如果构建树模块的内存中具有该状态key对应的叶子节点,则可以直接在内存中找到。这种情况,可以参考图5,在内存中直接查找到对应叶子节点的key,从而从中读取到value。进而,可以从value中读取状态数据。如果构建树模块的内存中并不具有该状态key对应的叶子节点,或者尽管具有该叶子节点,但其中并不存在要查找的状态数据,则需要从状态数据库模块中进一步查找。对于状态数据库采用LSM树的情形,如前所述,可以先在MemTable中查找,如果没有找到则在磁盘中的文件内查找。
S920:获取所述树节点key对应的叶子节点中包含的位置索引,所述位置索引指向所述持久化存储介质中数据文件内的位置。
对于构建树模块的内存中尽管具有该要查找的叶子节点,但其中并不存在状态数据,则需要从状态数据库模块中进一步查找。如前所述,如果内存中没有状态数据,则可以利用其包括的location字段,其内容例如为图10中A11这个叶子节点中的location为<2,750,210>。这个<2,750,210>即为位置索引,其指明了状态数据位于数据文件中的文件2中,起始位置为第750KB,长度为210KB。如图10左下角的数据文件所示,假设每个文件的大小为1024MB,索引的逻辑地址范围为0KB~1023KB。这样,A11的状态数据,实际上占据了文件2中从第750KB~第959KB的范围。
对于构建树模块的内存中不具有该要查找的叶子节点,则需要从状态数据库模块中查找并加载该叶子节点。从状态数据库模块中查找并加载该叶子节点,具体包括加载该叶子节点的索引文件和数据文件。假设图10对应的底层数据库中存储了树结构所有中间节点的k-v,如下表:
表2、数据库中实际存储的树节点k-v
上表2中,与表1中类似的,用H()表示hash计算。这样,下一个树节点的hash值锚定在了上一个树节点中。通过这样的层层hash,得到整颗state trie树的根hash,并将该根hash锁定到了区块头的state root字段中。假设该表中的k-v对存储于底层的状态数据库模块中,保存于磁盘,且采用类似的LSM结构。这样,与前述类似的,存储模块接收到读指令后,在构建树模块中可以找到状态树树节点的根节点,并将待查找的状态key值与根节点的shared nibble(s)字段(对于Extension Node)或槽位(对于Branch Node)从头开始匹配后,从匹配的位置读取下一层树节点的hash,进而,优先在构建树模块的内存中查找该hash。如果还没有找到则在状态数据库模块的内存中查找,如果仍然没有找到,就需要在状态数据库模块的文件中查找。具体的,可以优先Level 0中的SST内查找,如果还找不到就继续在Level 1的某个SST内查找,以此规律不断往下层Level的某个SST内查找,直至找到该hash值指向的下一Internal Node;解开这个Internal Node后,从中继续从前往后的匹配待读取的状态地址key值剩余部分的字段,如果匹配,则从匹配处读取hash值后,跳转至该hash值指向的下一级树节点。由此不断重复,逐级解开Internal Node并从前往后的匹配待读取的状态地址key值剩余的部分字段,匹配处的hash值作为下一次查找中间节点或叶子节点的依据,直至匹配到Leaf node,从而从Leaf node中读取到状态的value值。这个过程中,各级树节点的k-v有些可能并不存在于构建树模块的内存中,而是位于状态数据库模块中。因此,可能需要从状态数据库模块中读取。可以首先在状态数据库模块的MemTable或Immutable MemTable中查找这个key,如果不存在,就需要进一步查找磁盘上的各级SST文件。对于存在于磁盘上的Internal Node和Leaf node,需要加载到内存中,并解码(反序列化)得到其中的内容。具体的,例如首先在level0的sst文件逐个查找,如果Level 0的每个SST内做了排序,则可以在每个SST内根据中间节点/叶子节点的hash值做二分查找,查找到后将v载入内存并对其解码和匹配判断。如果Level 0的SST中没有找到,则进一步在Level 1内根据中间节点/叶子节点的hash值做二分查找,如果没有找到则在Level 2内做二分查找。如果找到则将这个k对应的v载入内存并解码。以此类推,直至匹配到树结构的叶子节点并从中读取value的值。上述在Level 1、Level 2以及更底层的Level中的查找时,由于这些Level中整体的SST是经过排序的,所以一般可以在这个Level中做一次二分查找。
这样,可以将查找到的叶子节点的树节点k-v加载到内存中。例如A11,将叶子节点value中的内容prefix:2,Key-end:35b2e4,location:(2,750,210)加载进内存中。其中的<2,750,210>即为value内容在文件中的位置索引。
S930:将所述位置索引指向的持久化存储介质中所述数据文件位置的value值加载到内存并读取其中的状态数据。
仍以A11为例,位置索引<2,750,210>指向所述持久化存储介质中数据文件内的位置。这样,可以将所述位置索引<2,750,210>指向的持久化存储介质中所述数据文件位置的value值加载到内存并读取其中的状态数据,即将数据文件中的文件2中从起始位置第750KB起,长度为210KB的数据加载进内存,从而读取其中的状态数据,即“张三_A=20”。
通过位置索引,可以直接定位到数据文件中的位置,从而可以直接加载其中的数据文件,并从中读取状态数据。
对于构建树模块的内存中尽管具有该要查找的叶子节点,但其中并不存在状态数据的情况,可以根据根据内存中叶子节点内的location在对应的数据文件中直接读取。这样,避免了叶子节点的k-v一并存储于持久化存储的SST文件中,需要多次在SST内进行二分查找的过程。
对于构建树模块的内存中不具有该要查找的叶子节点,需要从状态数据库模块中查找并加载该叶子节点的情况,实际上也需要从磁盘中读取一定量的内容至内存后进行查找。则表1对应的实现方式中,树节点的k-v一并在存盘中存储,且叶子节点的value包括状态数据,例如表1中的H(11)对应内容中的张三_A=20,H(12)对应内容中的李四_B=20,H(15)对应内容中的storedData=s,H(16)对应内容中的王五_A=20。一般来说,状态数据所占空间较大,大约占一个k-v总空间的90%以上。而表2对应的实现方式中,将索引和数据分离,索引中存储状态数据所在的数据文件的位置而不存状态数据本身,状态数据所在的数据文件的位置例如表2中H(11)对应内容中的location(2,750,210),H(12)对应内容中的(3,350,210),H(15)对应内容中的(5,760,140),H(16)对应内容中的location:(5,170,210)。在数据文件中的相应位置才存储实际的状态,如图10中的数据文件和状态数据所示。则在查找叶子节点的过程中,可以只查找索引文件,这样,单位空间内可以加载更多的索引文件至内存进而查找。内存的查找效率明显高于磁盘,一般查找的内容,都会加载进内存后再进行查找。表2的实现方式,由于单位容量的内存空间内可以加载更多的索引,相比于表1的实现方式可以更快的找到叶子节点。而表1的实现方式,例如读取了一批k-v至内存后,如果没有找到叶子节点,需要重新读取另外一批k-v至内存中再次查找,且由于带有占较大空间的状态内容,单位容量的内存空间内的k-v数量相对少,则从磁盘读取的次数一般来说会更频繁,而磁盘的读取速度较内存慢的多,因此表1的实现方式查找效率较低。
上述结合图10说明了图9方法实施例的具体过程。图10中仍然是两级MPT树的结构,且每一级树中间节点的结构包括Extension Node和Branch Node两类树节点。在一种变形中,可以采用Branch Node这样的树节点作为中间节点和树根节点,例如图11中所示。这样,从树根节点经过中间节点至叶子节点,中间经过的分支节点中的槽位的顺序组合可构成状态key中的部分内容。下一级节点的hash值可以存储于上一级节点的槽位中。图11中,仍然可以采用两级树的结构,上一级树中的叶子节点可以存储账户状态(包括外部账户和合约账户的状态),下一级树中的叶子节点存储合约的状态。在存储外部账户状态、合约账户状态或合约状态的叶子节点中,可以采用位置索引。例如,上一级树的叶子节点包括账户状态如下表中的H(M1)、H(M2)、H(M3)、H(M4)这4行:
表3、数据库中实际存储的树节点k-v
通过位置索引,可以直接定位到数据文件中的位置,从而可以直接加载其中的数据文件,并从中读取状态数据。进一步的,下一级树的叶子节点包括的合约状态如上表中的H(Z1)、H(Z2)、H(Z3)、H(Z4)这4行。
数据文件例如图11中所示,数据文件中的内容例如如下:
M1:Nonce:n1,balance:45.0ETH
M2:Nonce:n2,balance:1.00WEI
M3:Nonce:n3,balance:1.1ETH
M4:Nonce:n1,balance:0.12ETH,codehash:c1,storage:s1
Z1:张三_A=20
Z2:李四_B=50
Z3:王五_A=35
Z4:storedData=s
不论是上一级树还是下一级树,通过叶子节点中的location,都可以快速定位到数据文件中的位置。这样,避免了叶子节点的k-v一并存储于持久化存储的SST文件中,需要按照叶子节点的key(内容的hash值)多次在SST进行二分查找的过程。图11的示例中,按照图9中的方式读取区块链状态数据,根据位置索引可以直接定位到数据文件中的位置,因此可以直接将定位到底数据文件中对应位置的内容加载进内存,这样也避免了在多个Level的SST中读取大量的文件并加载进内存所带来的读放大问题。同样作为上一级树的叶子节点A4,可以按照图10中所示的方式存储,也可以按照图11中的方式存储。
外部账户、合约账户例如都是采用固定长度的字符串作为key,合约状态的key可以是同样长度的字符串。具体的,例如外部账户、合约账户的key可以是账户地址,而合约状态的key可以是同样长度的一串字符串。再例如,外部账户、合约账户的key可以是账户地址按照某一hash算法取hash值,例如采用sha3算法;而合约状态的key可以是sha3(合约名称+状态变量声明的位置)。上述外部账户、合约账户和合约状态是具有相同长度和格式的字符串,统称为状态key。例如状态key的长度为4bytes,即32bits。这样,假设根节点(B0)、中间节点(...、C5、...、C13、...、C5D1、...、C5D9、...、C5D14、...、C13D2、...、C13D10、...)每个都是16叉树,则一方面,每个中间节点的槽位(如B0中的0、1、2、...、d、e、f共16个槽位)都可以填入下一树节点的hash值,且另一方面,这个根节点/中间节点的槽位的位置,可以表示4bits的key。4bits构成半字节,这样,根节点和中间节点总计8层可以装入32bits,总计8bytes,即全部的状态key。状态key的32bits,可以从顶层的中间节点至底层的中间节点直到叶子节点,自上至下每层可以存入key中的一个半字节。这样,对于中间节点固定展开的情形,叶子节点中可以存储key中最末尾的4bits。在中间节点固定展开的情况下,图11中从C*到C13D*,一直到最下层的M*,一共是8层。另一种情况下,也可以采用压缩前缀和/或中缀和/或后缀的方式,则某个或某些叶子节点加上之上的中间节点可以少于8层。
此外,也可以采用一层树结构来存储账户状态和合约状态,例如图12中所示。由于合约状态的key是hash值,基于hash算法的离散特性,不同合约状态的状态key在这一层树的结构中可能是从根节点到不同叶子节点的路径。这样,同一个合约的不同状态,其value可能存在于一层树结构底层的不同叶子节点中,而不会再聚集于其所属的合约账户之下。实际上,采用一层树结构,账户状态和合约状态均分散于底层的叶子节点中。
此外,在图12的基础上,如图13中,可以进行前缀和/或中缀进行压缩。中缀压缩的情况,例如M1的树节点key为0x52d1479e(0x表示每一位用16进制数表示),Z1的树节点key为0x52d9ab36,则M1和Z1具有共同的中缀2d,则树节点C5中,除了16叉树所分开的1和9以外,C5中还可以具有压缩中缀2d,如图13中shared nibbles字段所示。后缀压缩的情况,例如用key-end表示,则树节点M1中的key-end例如为479e,树节点Z1中的key-end例如为36。采用前缀和/或中缀压缩,树的层数可以减少。
不论是否压缩,对于查找的状态key,都可以从树结构的根节点开始,逐段匹配状态key的路径,直到查找状态key对应的树结构的叶子节点。一种方式中,根节点和中间节点的存储,采用key-value形式,其中key是value的hash值,类似表3中的结构。这样,可以从上一层树节点开始按序匹配状态key的路径的hash,并根据该hash查找下一层树节点,直到查找到状态key对应的树结构的叶子节点。
此外,数据库中实际存储的树节点k-v,其key也可以不是value的hash值,而是状态key路径上的部分值,例如是根节点到当前树节点的部分key的值(结合图12,没有压缩前缀/中缀,如表4所示),或者是从上一树节点之后的部分key的值(结合图13,有压缩前缀/中缀,如表5所示)。
表4、数据库中实际存储的树节点k-v
按照表4,结合图12,例如查找的状态key为0x59...36(叶子节点Z1),从树结构的根节点开始,可以根据状态key的左起第1位5匹配到数据库中key是0x5的树节点(key为0x5的树节点的value中的h(C5)可以用于与根节点中槽位5上的hash值验证,相同则说明0x5的树节点是0x树节点中槽位5中的孩子节点);进而,可以根据状态key的左起第2位9匹配到数据库中key是0x9的树节点(key为0x9的树节点的value中的h(C5D9)可以用于与父节点中槽位9上的hash值验证,相同则说明0x9的树节点是0x5树节点的孩子节点);以此类推。最后,可以根据状态key的最后一位6匹配到数据库中key是0x6的树节点(key为0x6的树节点的value中的h(Z1)可以用于与父节点中槽位6上的hash值验证,相同则说明0x6的树节点是父节点的孩子节点)。
表5、数据库中实际存储的树节点k-v
按照表5,例如查找的状态key为0x52d9...36(叶子节点Z1),从树结构的根节点开始,可以根据状态key的左起第1位5到数据库中匹配,如果没有匹配到相同的key,或者匹配到key的树节点中的value中的hash值与根节点中h(C5)不一致,则重新根据状态key的左起第1-2位52到数据库中匹配,如此循环,直到匹配到为止。当根据状态key的左起第1-3位52d到数据库中匹配到key是0x52d的树节点,且其value中的H(C5)与根节点中槽位5上的hash值相同,则说明匹配到。进而可以根据状态key的左起第1-4位52d9到数据库中匹配,如果没有匹配到相同的key,或者匹配到key的树节点中的value中的hash值与根节点中h(C5)不一致,则重新根据状态key的左起第1-2位52到数据库中匹配,如果匹配到,且匹配到的树节点value中的H(C5D9)与父节点中槽位9上的hash值相同,则说明匹配到。进一步的,根据状态key的左起第1-5位到数据库中匹配,根据状态key的左起第1-6位到数据库中匹配,...,直至匹配到叶子节点。
在另一种实现中,可以在叶子节点中存储完整的状态key或对应完整状态key。以叶子节点中存储完整的状态key为例,例如图14中所示,叶子节点的key-full字段,可以存储对应状态的完整key。数据库中存储的叶子节点的key-value,其key即为该叶子节点对应状态的完整key。这样,当查找一个状态key时,可以直接按照该key在数据库中匹配叶子节点的key,而不需要匹配中间节点的key。显然的,对于查询操作来说这样的查找方式效率极高。实际上,上述图10-13中,也都可以采用完整的状态key作为叶子节点的key。
上述图9-14的实施例中,叶子节点中存储位置索引location,可以实现在找到叶子节点后通过位置索引快速定位到该叶子节点的value在数据文件中的位置。这样,可以直接将定位到的数据文件中对应位置的内容加载进内存,这样也避免了在多个Level的SST中读取大量的文件并加载进内存所带来的读放大问题。这也避免了叶子节点的k-v一并存储于持久化存储的SST文件中,需要按照叶子节点的key(value内容的hash值)多次在SST内进行二分查找的过程。而且,类似的,对于构建树模块的内存中不具有该要查找的叶子节点,上述实施例可以只查找索引文件,这样,单位空间内可以加载更多的索引文件至内存进而查找,进而可以更快的找到叶子节点。
如前所述,区块头中的state root会存储其锚定的下层MPT树的树根的hash值。这个hash值,实际上是从叶子结点开始按照层层向上计算hash的规则得到,具体如前所述。一般的,区块链节点的存储模块会在内存中构建有树结构的世界状态。本申请提供一种构建基于默克尔字典树的世界状态的方法,包括:
S160:接收状态key-value。
S162:根据所述默克尔字典树的字典序规则,生成所述树结构的叶子节点,将所述状态value存储在持久化存储介质的数据文件中,所述叶子节点中包括所述状态value在持久化存储介质中数据文件内的位置索引。
S162的过程具体可以参见前述内容,这里再次简述如下:
可以像以太坊那样,当存储模块接收到一个写状态的键值对(k-v)请求时,首先会对键(k)进行hash(如Keccak-256)处理。此外,这里S162中的状态key,也可以是S160中接收的状态key,即也可以是不进行hash处理。接下来,使用这个键在MPT中进行查找。这个查找过程从MPT的根节点开始,按照键的字典序,沿着树的路径向下遍历和匹配。在这个过程中,一般会逐位(或逐个半字节,半字节也称为nibble,一般是一位的16进制数)的比较键与节点中存储的路径。在遍历和匹配的过程中,可能会遇到三种类型的节点:分支节点(如果在某一点上路径分叉)、扩展节点(如果有共享前缀)和叶子节点(存储最终的值)。分支节点用于在树中分叉,扩展节点用于优化存储(通过路径压缩),而叶子节点则存储实际的值。如果在查找过程中发现匹配键的完整字典序路径已存在,会到达一个叶子节点。此时,可以更新这个叶子节点中存储的值(v)。如果查找和匹配的过程发现只匹配到部分路径,可能需要创建新的节点(可能是分支节点、扩展节点或叶子节点)来完成剩余的路径,并最终到达叶子节点,并在该叶子节点中存储或更新value值。
一旦叶子节点被更新或创建后,还会开始一个向上回溯的过程。这个过程从修改的叶子节点开始重新计算这个节点的哈希值。然后,向上移动到父节点,更新父节点中相关的引用(这个引用可能是子节点的直接内容,或者是子节点内容的哈希值,取决于子节点的大小),并重新计算父节点的哈希值。这个过程会一直持续到达到MPT的根节点。
最终,当回溯过程达到根节点时,整个MPT的根哈希值会被更新。这个新的根哈希值代表了整个状态树的新状态,它将被包含在下一个区块的区块头中,作为新的状态根。这样,每个区块不仅包含了交易信息,还通过状态根保存了执行这些交易后的全局状态的hash值。
如前所述,为了降低读放大并加快查找效率,叶子节点中存储的如前所述的location是指向所述持久化存储介质中数据文件内的位置。对于区块链这种分布式系统来说,不同区块链节点在本地存储同一value值,很可能存储在本地持久化存储介质中特定的数据文件内的位置,即不同区块链节点存储相同key对应的相同value是位于各自本地不同的数据文件内的位置,即location不同。为了使得各个区块链节点能够构建相同的树结构,从而得到相同的树根和相同的区块hash,并能够提供SPV能力,需要设计一种确定的树节点之间的锚定规则。
具体的,本申请提供出的一种方案可以如图15所示,叶子节点中除了所述状态value在持久化存储介质中数据文件内的位置索引以外,还可以包括对应状态value内容的hash值。
S164:根据叶子节点中的内容计算叶子节点的hash值,并将叶子节点的hash值存储在其上层的树节点中。
其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。例如,对于图15中编号为A11的叶子节点,其除了包含location外,还包含valuehash,该valuehash对应状态value内容的hash值,即是location这一位置索引所指向的持久化存储介质中数据文件内value内容的hash值。
并且,对于A10这一中间节点来说,其槽位3中存储的A11这一叶子节点hash值,是根据包括valuehash内容计算得到的hash值,但不包括所述状态value在持久化存储介质中数据文件内的位置索引location。例如,槽位3中存储的A11这一叶子节点hash值,是根据A11这一叶子节点中的prefix、Key-end和valuehash顺序拼接后计算得到的hash值,但这个hash值的计算并不包括location。
这样,一方面避免了不同区块链节点以上述数据文件方式存储value内容时位置不同可能导致的树结构不同,另一方面还以hash锁定的方式实现对SPV能力的支持。
其中,状态数据写入时修改对应Merkle树节点,修改的Merkle树节点达到一定阀值时,可以将这些节点内容写入索引文件,如图16中所示。此外,Merkle树运行一段时间后,还可以是根据设定的时间阈值周期性的对修改的若干树节点在后台批量写入一个索引文件。
数据文件与索引文件逻辑上可以分开管理,物理上可以存储在不同的文件。
如前所述,由于向上回溯的过程需要从修改的叶子节点开始重新计算这个节点的哈希值,因此,本申请方案还可以对所述叶子节点所引起的中间节点hash值变化更新至该中间节点上层的树节点中。
如图15,数据文件不再按key排序,读取时得到location后,直接从location读取value,而不需要像通用KVDB那样在多个level的sst文件里查找数据所在位置,从而可以减少读放大。
采用本申请的上述方案,提供了一体化的Merkle状态数据实现,不再分为上层Merkle树、底层为通用KVDB两级结构,而是整体为一个MerkleDB,DB内索引与数据分离。将merkle树作为索引,合约状态数据作为数据,缩短状态数据的读写流程,减少整体的写放大与读放大。
将整个MerkleState作为一个专用DB,只存储Merkle状态数据,DB内数据与索引分离(key-value分离),不再分上层Merkle树、底层通用KVDB两层。这样,对于大量数据文件,可以不分level执行compaction,而是维护数据文件的垃圾比,在后台根据垃圾比执行垃圾回收,链上所有历史状态数据需要保留,回退时才会产生垃圾,垃圾回收是低频操作,从而避免通用KVDB里compaction引起的数据反复rewrite,减少写放大。
图17是本申请另一实施例的完整图示。图17与图14整体类似,主要区别在于,例如标号C5的中间节点和其下的标号C5D1、C5D9、C5D14的中间节点共同构成一个LogicalPage,且在叶子节点M1、M2、M3、M4中,存的key-end而非key-full,且叶子节点中除了包含的位置索引,还包含对应状态value内容的hash值。当然,该实施例中,如前所述,叶子节点M1、M2、M3、M4中也可以是key-full。
所述叶子节点中还可以包括状态value内容的hash值和key-end字段,其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值和所述key-end字段计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。
所述叶子节点中还可以包括状态value内容的hash值和key-full字段,其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值和所述key-full字段计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。
本申请还提供一种区块链节点,所述区块链节点包括存储模块,所述存储模块包括构建树模块和持久化存储,所述持久化存储介质中包括索引文件和数据文件;所述存储模块还包括查找单元,读取单元;树结构的叶子节点中包括状态数据在持久化存储介质中数据文件内的位置索引;
所述构建树模块用于在内存中构建树结构的世界状态;
所述查找单元接收要查找的状态key的指令,在内存中的所述树结构上查找该状态key对应的叶子节点;获取所述叶子节点中包含的位置索引,所述位置索引指向所述持久化存储介质中数据文件内的位置;
所述读取单元将所述位置索引指向的持久化存储介质中所述数据文件位置的value值加载到内存并读取其中的状态数据。
所述查找单元从所述树结构的根节点开始逐段匹配状态key的路径,直到查找到状态key对应的树结构的叶子节点。
所述查找单元从所述上一层树节点开始按序匹配状态key的路径的hash,并根据该hash查找下一层树节点,直到查找状态key对应的树结构的叶子节点。
所述查找单元从所述上一层树节点开始按序匹配状态key的路径的key,并根据该匹配到的key查找下一层树节点,直到查找状态key对应的树结构的叶子节点。
当内存中的所述叶子点中不存在状态数据时,获取所述叶子节点包含的位置索引。
当内存中不存在所述叶子节点时,从持久化存储中的索引文件中查找该叶子节点的树节点key,将该key对应的数据文件中的value加载至内存中,从value中获取位置索引。
所述树结构叶子节点的树节点key对应于状态key,则所述当内存中不存在所述叶子节点时从持久化存储中查找该叶子节点的树节点key,
当内存中不存在所述叶子节点时所述查找单元根据该状态key在持久化存储的索引文件中查找叶子节点的树节点key。
所述树结构包括一层树结构或两层树结构。
在20世纪90年代,对于一个技术的改进可以很明显地区分是硬件上的改进(例如,对二极管、晶体管、开关等电路结构的改进)还是软件上的改进(对于方法流程的改进)。然而,随着技术的发展,当今的很多方法流程的改进已经可以视为硬件电路结构的直接改进。设计人员几乎都通过将改进的方法流程编程到硬件电路中来得到相应的硬件电路结构。因此,不能说一个方法流程的改进就不能用硬件实体模块来实现。例如,可编程逻辑器件(Programmable Logic Device,PLD)(例如现场可编程门阵列(Field Programmable Gate Array,FPGA))就是这样一种集成电路,其逻辑功能由用户对器件编程来确定。由设计人员自行编程来把一个数字系统“集成”在一片PLD上,而不需要请芯片制造厂商来设计和制作专用的集成电路芯片。而且,如今,取代手工地制作集成电路芯片,这种编程也多半改用“逻辑编译器(logic compiler)”软件来实现,它与程序开发撰写时所用的软件编译器相类似,而要编译之前的原始代码也得用特定的编程语言来撰写,此称之为硬件描述语言(Hardware Description Language,HDL),而HDL也并非仅有一种,而是有许多种,如ABEL(Advanced Boolean Expression Language)、AHDL(Altera Hardware Description Language)、Confluence、CUPL(Cornell University Programming Language)、HDCal、JHDL(Java Hardware Description Language)、Lava、Lola、MyHDL、PALASM、RHDL(Ruby Hardware Description Language)等,目前最普遍使用的是VHDL(Very-High-Speed Integrated Circuit Hardware Description Language)与Verilog。本领域技术人员也应该清楚,只需要将方法流程用上述几种硬件描述语言稍作逻辑编程并编程到集成电路中,就可以很容易得到实现该逻辑方法流程的硬件电路。
控制器可以按任何适当的方式实现,例如,控制器可以采取例如微处理器或处理器以及存储可由该(微)处理器执行的计算机可读程序代码(例如软件或固件)的计算机可读介质、逻辑门、开关、专用集成电路(Application Specific Integrated Circuit,ASIC)、可编程逻辑控制器和嵌入微控制器的形式,控制器的例子包括但不限于以下微控制器:ARC 625D、Atmel AT91SAM、Microchip PIC18F26K20以及Silicone Labs C8051F320,存储器控制器还可以被实现为存储器的控制逻辑的一部分。本领域技术人员也知道,除了以纯计算机可读程序代码方式实现控制器以外,完全可以通过将方法步骤进行逻辑编程来使得控制器以逻辑门、开关、专用集成电路、可编程逻辑控制器和嵌入微控制器等的形式来实现相同功能。因此这种控制器可以被认为是一种硬件部件,而对其内包括的用于实现各种功能的装置也可以视为硬件部件内的结构。或者甚至,可以将用于实现各种功能的装置视为既可以是实现方法的软件模块又可以是硬件部件内的结构。
上述实施例阐明的系统、装置、模块或单元,具体可以由计算机芯片或实体实现,或者由具有某种功能的产品来实现。一种典型的实现设备为服务器系统。当然,本申请不排除随着未来计算机技术的发展,实现上述实施例功能的计算机例如可以为个人计算机、膝上型计算机、车载人机交互设备、蜂窝电话、相机电话、智能电话、个人数字助理、媒体播放器、导航设备、电子邮件设备、游戏控制台、平板计算机、可穿戴设备或者这些设备中的任何设备的组合。
虽然本说明书一个或多个实施例提供了如实施例或流程图所述的方法操作步骤,但基于常规或者无创造性的手段可以包括更多或者更少的操作步骤。实施例中列举的步骤顺序仅仅为众多步骤执行顺序中的一种方式,不代表唯一的执行顺序。在实际中的装置或终端产品执行时,可以按照实施例或者附图所示的方法顺序执行或者并行执行(例如并行处理器或者多线程处理的环境,甚至为分布式数据处理环境)。术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、产品或者设备不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、产品或者设备所固有的要素。在没有更多限制的情况下,并不排除在包括所述要素的过程、方法、产品或者设备中还存在另外的相同或等同要素。例如若使用到第一,第二等词语用来表示名称,而并不表示任何特定的顺序。
为了描述的方便,描述以上装置时以功能分为各种模块分别描述。当然,在实施本说明书一个或多个时可以把各模块的功能在同一个或多个软件和/或硬件中实现,也可以将实现同一功能的模块由多个子模块或子单元的组合实现等。以上所描述的装置实施例仅仅是示意性的,例如,所述单元的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,装置或单元的间接耦合或通信连接,可以是电性,机械或其它的形式。
本发明是参照根据本发明实施例的方法、装置(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器,使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中,使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品,该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上,使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理,从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
在一个典型的配置中,计算设备包括一个或多个处理器(CPU)、输入/输出接口、网络接口和内存。
内存可能包括计算机可读介质中的非永久性存储器,随机存取存储器(RAM)和/或非易失性内存等形式,如只读存储器(ROM)或闪存(flash RAM)。内存是计算机可读介质的示例。
计算机可读介质包括永久性和非永久性、可移动和非可移动媒体可以由任何方法或技术来实现信息存储。信息可以是计算机可读指令、数据结构、程序的模块或其他数据。计算机的存储介质的例子包括,但不限于相变内存(PRAM)、静态随机存取存储器(SRAM)、动态随机存取存储器(DRAM)、其他类型的随机存取存储器(RAM)、只读存储器(ROM)、电可擦除可编程只读存储器(EEPROM)、快闪记忆体或其他内存技术、只读光盘只读存储器(CD-ROM)、数字多功能光盘(DVD)或其他光学存储、磁盒式磁带,磁带磁磁盘存储、石墨烯存储或其他磁性存储设备或任何其他非传输介质,可用于存储可以被计算设备访问的信息。按照本文中的界定,计算机可读介质不包括暂存电脑可读媒体(transitory media),如调制的数据信号和载波。
本领域技术人员应明白,本说明书一个或多个实施例可提供为方法、系统或计算机程序产品。因此,本说明书一个或多个实施例可采用完全硬件实施例、完全软件实施例或结合软件和硬件方面的实施例的形式。而且,本说明书一个或多个实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本说明书一个或多个实施例可以在由计算机执行的计算机可执行指令的一般上下文中描述,例如程序模块。一般地,程序模块包括执行特定任务或实现特定抽象数据类型的例程、程序、对象、组件、数据结构等等。也可以在分布式计算环境中实践本本说明书一个或多个实施例,在这些分布式计算环境中,由通过通信网络而被连接的远程处理设备来执行任务。在分布式计算环境中,程序模块可以位于包括存储设备在内的本地和远程计算机存储介质中。
本说明书中的各个实施例均采用递进的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于系统实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本说明书的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。
以上所述仅为本说明书一个或多个实施例的实施例而已,并不用于限制本本说明书一个或多个实施例。对于本领域技术人员来说,本说明书一个或多个实施例可以有各种更改和变化。凡在本说明书的精神和原理之内所作的任何修改、等同替换、改进等,均应包含在权利要求范围之内。
Claims (10)
- 一种构建基于默克尔字典树的世界状态的方法,应用于区块链节点的存储模块,所述存储模块在内存中构建有树结构的世界状态,所述方法包括:接收状态key-value;根据所述默克尔字典树的字典序规则,生成所述树结构的叶子节点,将所述状态value存储在持久化存储介质的数据文件中,所述叶子节点中包括所述状态value在持久化存储介质中数据文件内的位置索引;根据叶子节点中的内容计算叶子节点的hash值,并将叶子节点的hash值存储在其上层的树节点中。
- 如权利要求1所述的方法,状态数据写入时修改对应树节点,修改的树节点达到一定阀值时,将这些节点内容写入索引文件。
- 如权利要求1所述的方法,状态数据写入时修改对应树节点,根据设定的时间阈值周期性的对修改的若干树节点写入索引文件。
- 如权利要求2所述的方法,数据文件与索引文件逻辑上分开管理,物理上存储在不同的文件。
- 如权利要求1所述的方法,所述叶子节点中还包括状态value内容的hash值;其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。
- 如权利要求1-5中任一项所述的方法,还包括对所述叶子节点所引起的中间节点hash值变化更新至该中间节点上层的树节点中。
- 如权利要求1-5中任一项所述的方法,所述叶子节点中还包括状态value内容的hash值和key-end字段,其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值和所述key-end字段计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。
- 如权利要求1-5中任一项所述的方法,所述叶子节点中还包括状态value内容的hash值和key-full字段,其中,根据叶子节点中的内容计算叶子节点的hash值,包括根据叶子节点中状态value内容的hash值和所述key-full字段计算,但不包括所述状态value在持久化存储介质中数据文件内的位置索引。
- 如权利要求1-5中任一项所述的方法,所述中间节点除了包含槽位,还包含压缩的共同前缀/中缀。
- 一种计算机设备,包括:处理器;以及存储器,其中存储有程序,其中在所述处理器执行所述程序时,执行权利要求1-6中任一项的操作。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410969660.4A CN118939653A (zh) | 2024-07-18 | 2024-07-18 | 一种构建基于默克尔字典树的世界状态的方法和计算机设备 |
| CN202410969660.4 | 2024-07-18 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2026016331A1 true WO2026016331A1 (zh) | 2026-01-22 |
Family
ID=93359655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2024/128752 Pending WO2026016331A1 (zh) | 2024-07-18 | 2024-10-31 | 一种构建基于默克尔字典树的世界状态的方法和计算机设备 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN118939653A (zh) |
| WO (1) | WO2026016331A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119620939B (zh) * | 2024-11-19 | 2026-04-21 | 杭州高新区(滨江)区块链与数据安全研究院 | 区块链数据的处理方法、装置及计算机设备 |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180329785A1 (en) * | 2017-05-10 | 2018-11-15 | Elastifile Ltd. | File system storage in cloud using data and metadata merkle trees |
| CN115982781A (zh) * | 2022-12-30 | 2023-04-18 | 蚂蚁区块链科技(上海)有限公司 | 一种在区块链中创建账户的方法和区块链节点 |
| CN116467733A (zh) * | 2022-12-31 | 2023-07-21 | 南京理工大学 | 一种基于sgx的安全高可用键值存储方法及系统 |
| CN118797105A (zh) * | 2024-07-19 | 2024-10-18 | 蚂蚁区块链科技(上海)有限公司 | 一种树结构的世界状态的存储方法、恢复方法及计算机设备 |
| CN118797106A (zh) * | 2024-07-19 | 2024-10-18 | 蚂蚁区块链科技(上海)有限公司 | 一种构建基于树结构的世界状态的方法及计算机设备 |
| CN118964351A (zh) * | 2024-07-26 | 2024-11-15 | 蚂蚁区块链科技(上海)有限公司 | 一种对叶子页/中间页进行伸缩的方法及计算机设备 |
| CN119415010A (zh) * | 2024-07-31 | 2025-02-11 | 蚂蚁区块链科技(上海)有限公司 | 一种查找基础页和/或增量页的方法及计算机设备 |
-
2024
- 2024-07-18 CN CN202410969660.4A patent/CN118939653A/zh active Pending
- 2024-10-31 WO PCT/CN2024/128752 patent/WO2026016331A1/zh active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180329785A1 (en) * | 2017-05-10 | 2018-11-15 | Elastifile Ltd. | File system storage in cloud using data and metadata merkle trees |
| CN115982781A (zh) * | 2022-12-30 | 2023-04-18 | 蚂蚁区块链科技(上海)有限公司 | 一种在区块链中创建账户的方法和区块链节点 |
| CN116467733A (zh) * | 2022-12-31 | 2023-07-21 | 南京理工大学 | 一种基于sgx的安全高可用键值存储方法及系统 |
| CN118797105A (zh) * | 2024-07-19 | 2024-10-18 | 蚂蚁区块链科技(上海)有限公司 | 一种树结构的世界状态的存储方法、恢复方法及计算机设备 |
| CN118797106A (zh) * | 2024-07-19 | 2024-10-18 | 蚂蚁区块链科技(上海)有限公司 | 一种构建基于树结构的世界状态的方法及计算机设备 |
| CN118964351A (zh) * | 2024-07-26 | 2024-11-15 | 蚂蚁区块链科技(上海)有限公司 | 一种对叶子页/中间页进行伸缩的方法及计算机设备 |
| CN119415010A (zh) * | 2024-07-31 | 2025-02-11 | 蚂蚁区块链科技(上海)有限公司 | 一种查找基础页和/或增量页的方法及计算机设备 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118939653A (zh) | 2024-11-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2026016330A1 (zh) | 一种构建基于树结构的世界状态的方法及计算机设备 | |
| WO2026016332A1 (zh) | 一种树结构的世界状态的存储方法、恢复方法及计算机设备 | |
| WO2026016329A1 (zh) | 一种读取树结构的世界状态的方法及计算机设备 | |
| CN114706848A (zh) | 区块链数据存储、更新、读取方法及装置、电子设备 | |
| CN118860288A (zh) | 一种查找基础页/增量页的方法及计算机设备 | |
| CN118964351A (zh) | 一种对叶子页/中间页进行伸缩的方法及计算机设备 | |
| CN115640356A (zh) | 区块链系统中的交易执行方法、共识节点和区块链系统 | |
| CN119415010A (zh) | 一种查找基础页和/或增量页的方法及计算机设备 | |
| CN115982781A (zh) | 一种在区块链中创建账户的方法和区块链节点 | |
| CN115665176A (zh) | 区块链系统中节点类型的转换方法、装置及区块链系统 | |
| CN118939653A (zh) | 一种构建基于默克尔字典树的世界状态的方法和计算机设备 | |
| WO2026011612A1 (zh) | 一种读取区块链状态数据的方法和区块链节点 | |
| CN119166722A (zh) | 一种收敛失效文件位置的方法和回收重写页的方法及计算机设备 | |
| WO2026016333A1 (zh) | 一种构建和读取基于树结构的世界状态的方法及计算机设备 | |
| CN118568175A (zh) | 一种生成n叉树的方法和系统 | |
| CN115964442A (zh) | 一种区块链中的账户状态访问方法和区块链节点 | |
| Jordan et al. | Efficient Forkless Blockchain Databases | |
| CN120255811A (zh) | 在区块链系统中存储数据的方法和区块链节点 | |
| CN115658807B (zh) | 区块链系统中的共识方法、共识节点和区块链系统 | |
| CN115577044B (zh) | 区块链系统中的交易执行方法、共识节点和区块链系统 | |
| CN120295973A (zh) | 区块链系统中状态数据的索引重建方法及装置 | |
| CN116188160A (zh) | 在区块链系统中执行交易的方法和区块链节点 | |
| CN120196609A (zh) | 区块链系统中状态数据的管理方法及装置 | |
| CN116186788A (zh) | 一种从区块链查询数据的方法和装置 | |
| CN118964352A (zh) | 一种生成基础页/增量页的方法及计算机设备 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24947710 Country of ref document: EP Kind code of ref document: A1 |