GB2606194A

GB2606194A - Methods and devices for pruning stored merkle tree data

Info

Publication number: GB2606194A
Application number: GB2106104.9A
Authority: GB
Inventors: Patrick Coughlan Steven; Zhang Wei
Original assignee: Nchain Licensing AG
Current assignee: Nchain Licensing AG
Priority date: 2021-04-29
Filing date: 2021-04-29
Publication date: 2022-11-02
Also published as: GB202106104D0

Abstract

A method for storing Merkle tree data in a page-structured memory having a defined page size for storage and retrieval of data includes storing, in the page-structure memory, hash values for a Merkle tree constructed from an ordered set of transaction identifiers in a subtree-based format 1202. The hash tree is partitioned into subtrees including upper subtrees and lower subtrees. When it is determined that the Merkle tree data is to be pruned 1204, at least a portion of the plurality of lower subtrees is deleted from the page-structured memory 1210. The transaction identifiers may correspond to a block in a blockchain. The decision to prune the data may be based on the age of the block or determining that the frequency of related Merkle proof requests is below a threshold. If the leaf nodes, i.e. the transaction identifiers, are to be pruned, the identifiers may be stored elsewhere.

Description

METHODS AND DEVICES FOR PRUNING STORED MERKLE TREE

DATA

TECHNICAL FIELD

[0001] The present disclosure relates to blockchain networks and, in particular, to methods and devices for storing and retrieving Merkle tree data.

BACKGROUND

[0002] A blockchain refers to a form of distributed data structure, wherein a duplicate copy of the blockchain is maintained at each of a plurality of nodes in a distributed peer-to-peer (P2P) network (referred to below as a "blockchain network") and widely publicised. The blockchain is formed from a chain of blocks of data, wherein each block includes one or more transactions. Each transaction, other than so-called "coinbase transactions", points back to a preceding transaction in a sequence which may span one or more blocks up until one or more coinbase transactions. Coinbase transactions are discussed below. Transactions that are submitted to the blockchain network are included in new blocks. New blocks are created by a process often referred to as "mining", which involves each of a plurality of the nodes competing to perform "proof-of-work", i.e. solving a cryptographic puzzle based on a representation of a defined set of ordered and validated pending transactions waiting to be included in a new block of the blockchain. It should be noted that the blockchain may be pruned at a node, and the publication of blocks can be achieved through the publication of mere block headers.

1-00031 The transactions in the blockchain are used to perform one or more of the following: to convey a digital asset (i.e. a number of digital tokens), to order a set of journal entries in a virtualised ledger or registry, to receive and process timestamp entries, and/or to time-order index pointers. A blockchain can also be exploited in order to layer additional functionality on top of the blockchain. Blockchain protocols may allow for storage of additional user data or indexes to data in a transaction. There is no pre-specified limit to the maximum data capacity that can be stored within a single transaction, and therefore increasingly more complex data can be incorporated. For instance this may be used to store an electronic document in the blockchain, or audio or video data.

[0004] Nodes of the blockchain network (which are often referred to as "miners") perform a distributed transaction registration and verification process, which will be described in detail below. In summary, during this process a node validates transactions and inserts them into a block template for which they attempt to identify a valid proof-of-work solution. Once a valid solution is found, a new block is propagated to other nodes of the network, thus enabling each node to record the new block on the blockchain. In order to have a transaction recorded in the blockchain, a user (e.g. a blockchain client application) sends the transaction to one of the nodes of the network to be propagated. Nodes which receive the transaction may race to find a proofof-work solution incorporating the validated transaction into a new block. Each node is configured to enforce the same node protocol, which will include one or more conditions for a transaction to be valid. Invalid transactions will not be propagated nor incorporated into blocks. Assuming the transaction is validated and thereby accepted onto the blockchain, then the transaction (including any user data) will thus remain registered and indexed at each of the nodes in the blockchain network as an immutable public record.

[0005] The node who successfully solved the proof-of-work puzzle to create the latest block is typically rewarded with a new transaction called the "coinbase transaction" which distributes an amount of the digital asset, i.e. a number of tokens. The detection and rejection of invalid transactions is enforced by the actions of competing nodes who act as agents of the network and are incentivised to report and block malfeasance. The widespread publication of information allows users to continuously audit the performance of nodes. The publication of the mere block headers allows participants to ensure the ongoing integrity of the blockchain.

[0006] In an output-based" model (sometimes referred to as a UTXO-based model), the data structure of a given transaction includes one or more inputs and one or more outputs. Any spendable output includes an element specifying an amount of the digital asset that is derivable from the proceeding sequence of transactions. The spendable output is sometimes referred to as a UTXO ("unspent transaction output"). The output may further include a locking script specifying a condition for the future redemption of the output. A locking script is a predicate defining the conditions necessary to validate and transfer digital tokens or assets. Each input of a transaction (other than a coinbase transaction) includes a pointer (i.e. a reference) to such an output in a preceding transaction, and may further include an unlocking script for unlocking the locking script of the pointed-to output. Consider that a pair of transactions may be labelled a first and a second transaction (or "target" transaction). The first transaction includes at least one output specifying an amount of the digital asset, and having a locking script defining one or more conditions of unlocking the output. The second, target transaction has at least one input, including a pointer to the output of the first transaction, and an unlocking script for unlocking the output of the first transaction.

[0007] in such a model, when the second, target transaction is sent to the blockchain network to be propagated and recorded in the blockchain, one of the criteria for validity applied at each node will be that the unlocking script meets all of the one or more conditions defined in the locking script of the first transaction. Another will be that the output of the first transaction has not already been redeemed by another, earlier valid transaction. Any node that finds the target transaction invalid according to any of these conditions will not propagate it (as a valid transaction, but possibly to register an invalid transaction) nor include it in a new block to be recorded in the blockchain.

[0008] An alternative type of transaction model is an account-based model. In this case each transaction does not define the amount to be transferred by referring back to the UTXO of a preceding transaction in a sequence of past transactions, but rather by reference to an absolute account balance. The current state of all accounts is stored by the nodes separate to the blockchain and is updated constantly.

[0009] In order for the blockchain network to be practically useful to large numbers of participants, an end user device may operate a client application (sometime referred to as a "wallet" or Simplified Payment Verification (SPV) software). Such a client application lacks the functionality of a full blockchain node and does not have a full copy of the blockchain. The end user device, through its client application, may send a request to a blockchain node for data proving that a particular transaction is present in a blockchain, i.e. that it has been included in a block, meaning it has been "confirmed". It would be advantageous to have improved methods and devices for providing such data and enabling end user devices or other nodes to prove inclusion of a transaction in a block and/or to prove other features of a block.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Reference will now be made, by way of example, to the accompanying drawings which show example embodiments of the present application and in which: [0011] FIG. 1 shows an example system for implementing a blockchain; [0012] FIG. 2 illustrates an example transaction protocol; [0013] FIG. 3A shows an example implementation of a client application; [0014] FIG. 3B shows an example of a user interface for the client application; [0015] FIG. 4 illustrates example node software for a blockchain node; [0016] FIG. 5A shows an example of a Merkle tree; [0017] FIG. 5B shows an example of a "partial" Merkle tree; [0018] FIG. 6 shows an example of a Merkle path; and [0019] FIG_ 7A shows one example partitioning of a Merkle tree into suhtrees; [00M] FIG. 7B shows another example partitioning of the Merkle tree into subtrees; [0021] FIG. 8 shows in flowchart form, one example method for storing Merkle tree data in a page-based memory; [0022] FIG. 9 shows a simplified example method for responding to a Merkle proof query based on subtree-based Merkle tree data storage; [0023] FIG. 10 illustrates an example of storing Merkle tree in a subtree-hased format with a paired root node; [0024] FIG. 11 shows an example of pruned Merkle tree data; [0025] FIG. 12 shows, in flowchart form, one example method of paining Merkle tree data; [0026] FIG. 13 illustrates a combined subtree and upper Merkle proof data for the subtree; [0027] FIG. 14 shows an example of a composite subtree and its upper Merkle proof data; [0028] FIG. 15 shows a further example of a composite subtree and its upper Merkle proof data; and [0029] FIG. 16 shows, in flowchart form, an example method for storing Merkle tree data in a composite subtree format.

[0030] Like reference numerals are used in the drawings to denote like elements and features.

DETAILED DESCRIPTION OF EXAMPLES

[0031] In one aspect, there may be provided a computer-implemented method of storing Merkle tree data in a page-structured memory, the page-structured memory having a defined page size for storage and retrieval of data. The method may include storing, in the page-structure memory, hash values for a Merkle tree constructed from an ordered set of transaction identifiers in a subtree-based format, wherein the Merkle tree is partitioned into a plurality of subtrees including a plurality of upper subtrees and a plurality of lower subtrees; determining that the stored Merlde tree data is to be pruned; and deleting at least a portion of the plurality of lower subtrees from the page-structured memory.

[0032] In some implementations, the ordered set of transaction identifiers correspond to a block in a blockchain. In some cases, determining includes determining that a time elapsed since creation of the block has exceeded a threshold value. In some cases, determining includes determining that a block height of the blockchain above the block exceeds a threshold height.

[0033] In some implementations, determining includes determining that a frequency of Merkle proof requests over a time window is lower than a minimum threshold number.

[0034] In some implementations, determining includes determining that a time since last access of the portion of the plurality of lower subtrees to reply to a Merkle proof query exceeds a stale time.

[0035] In some implementations, deleting at least the portion of the plurality of lower subtrees includes determining whether the lower subtrees include the ordered set of transaction identifiers and, if so, then storing the ordered set of transaction identifier in the page-structured memory.

[0036] In some implementations, storing includes storing each subtree in a respective page in the page-structured memory, and wherein deleting includes deleting data from pages containing the portion of the plurality of lower subtrees.

[0037] In some implementations, the portion of the plurality of lower subtrees includes a lowest layer of subtrees stored for the Merkle tree.

[0038] In some implementations, the method may further include receiving a Merkle proof query relating to a specified transaction identifier; identifying, based on a layer height of a lowest layer of the stored Merkle tree, a range of transaction identifiers needed to build a lower Merkle path to the lowest layer; obtaining the transaction identifiers in the range; and constructing, from the obtained transaction identifiers, a portion of the Merkle tree to obtain hash values for a partial Merkle proof along the lower Merkle path.

[0039] In some cases, the method then further includes retrieving, from the page-structured memory, one or more pages of hash values for a remaining Merkle proof for the specified transaction identifier along an upper Merkle path; and sending a response containing hash values providing the Merkle proof.

[0040] In some implementations, obtaining the transaction identifiers in the range includes obtaining the transaction identifiers in the range from a blockchain node.

[0041] In some implementations, obtaining the transaction identifiers in the range includes obtaining the transaction identifiers from the page-structured memory.

[0042] In another aspect, there may be provided a computing device implementing a node on a block-chain. The computing device may include memory, one or more processors, and computer-executable instructions that, when executed, cause the processors to carry out one or more of the methods described herein.

[0043] In yet another aspect, there may be provided a computer-readable medium storing processor-executable instructions, the processor-executable instructions including instructions that, when executed by one or more processors, cause the processors to carry out at least one of the methods described herein.

[0044] Other example embodiments of the present disclosure will be apparent to those of ordinary skill in the art from a review of the following detailed description in conjunction with the drawings.

[0045] In the present application, the term "and/or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any one of the listed elements alone, any sub-combination, or all of the elements, and without necessarily excluding additional elements.

S

[0046] In the present application, the phrase "at least one of... or..." is intended to cover any one or more of the listed elements, including any one of the listed elements alone, any sub-combination, or all of the elements, without necessarily excluding any additional elements, and without necessarily requiring all of the elements.

Example System Overview [0047] Figure 1 shows an example system 100 for implementing a blockchain 150. The system 100 may include a packet-switched network 101, typically a wide-area internetwork such as the Internet. The packet-switched network 101 includes a plurality of blockchain nodes 104 that may be arranged to form a peer-to-peer (P2P) network 106 within the packet-switched network 101. Whilst not illustrated, the blockchain nodes 104 may be arranged as a near-complete graph. Each blockchain node 104 is therefore highly connected to other blockchain nodes 104.

[0048] Each blockchain node 104 includes computer equipment of a peer, with different ones of the nodes 104 belonging to different peers. Each blockchain node 104 includes a processing apparatus implemented by one or more processors, e.g. one or more central processing units (CPUs), accelerator processors, application specific processors and/or field programmable gate arrays (FPGAs), and other equipment such as Application Specific Integrated Circuits (ASICs). Each node also includes memory, i.e. computer-readable storage in the form of a non-transitory computer-readable medium or media. The memory may include one or more memory units employing one or more memory media, e.g. a magnetic medium such as a hard disk; an electronic medium such as a solid-state drive (SSD), flash memory or EEPROM; and/or an optical medium such as an optical disk drive.

[0049] The blockchain 150 includes a chain of blocks of data 151, wherein a respective copy of the blockchain 150 is maintained at each of a plurality of blockchain nodes 104 in the distributed or blockchain network 160. As mentioned above, maintaining a copy of the blockchain 150 does not necessarily mean storing the blockchain 150 in full. Instead, the blockchain 150 may be pruned of data so long as each blockchain node 150 stores the blockheader (discussed below) of each block 151. Each block 151 in the chain includes one or more transactions 152, wherein a transaction in this context refers to a kind of data structure. The nature of the data structure will depend on the type of transaction protocol used as part of a transaction model or scheme. A given blockchain will use one particular transaction protocol throughout. In one common type of transaction protocol, the data structure of each transaction 152 has at least one input and at least one output. Each output specifies an amount representing a quantity of a digital asset as property, an example of which is a user 103 to whom the output is cryptographically locked (requiring a signature or other solution of that user in order to be unlocked and thereby redeemed or spent). Each input points hack to the output of a preceding transaction 152, thereby linking the transactions.

[0950] Each block 15! also includes a block pointer 155 pointing back to the previously created block 151 in the chain so as to define a sequential order to the blocks 151. Each transaction I 52 (other than a coinbase transaction) has a pointer hack to a previous transaction so as to define an order to sequences of transactions (N.B. sequences of transactions 152 are allowed to branch). The chain of blocks 151 goes all the way back to a genesis block (Gb) 153 which was the first block in the chain. One or more original transactions 152 early on in the chain 150 pointed to the genesis block 153 rather than a preceding transaction.

[0051] Each of the blockchain nodes 104 is configured to forward transactions 152 to other blockchain nodes 104, and thereby cause transactions 152 to be propagated throughout the network 106. Each blockchain node 104 is configured to create blocks 151 and to store a respective copy of the same blockchain 150 in their respective memory. Each blockchain node 104 also maintains an ordered set 154 of transactions 152 waiting to be incorporated into blocks 151. The ordered set 154 is often referred to as a "mempool". This term herein is not intended to limit to any particular blockchain, protocol or model. It refers to the ordered set of transactions which a node 104 has accepted as valid and for which the node 104 is obliged not to accept any other transactions attempting to spend the same output.

[0052] In a given present transaction 152j, the (or each) input includes a pointer referencing the output of a preceding transaction 152i in the sequence of transactions, specifying I 0 that this output is to be redeemed or "spent" in the present transaction 152j. In general, the preceding transaction could be any transaction in the ordered set 154 or any block 151. The preceding transaction 152i need not necessarily exist at the time the present transaction 152] is created or even sent to the network 106, though the preceding transaction 152i will need to exist and be validated in order for the present transaction to be valid. Hence "preceding" herein refers to a predecessor in a logical sequence linked by pointers, not necessarily the time of creation or sending in a temporal sequence, and hence it does not necessarily exclude that the transactions 152i, 152] be created or sent out-of-order (see discussion below on orphan transactions). The preceding transaction I52i could equally be called the antecedent or predecessor transaction.

[0053] The input of the present transaction I52j also includes the input authorisation, for example the signature of the user 103a to whom the output of the preceding transaction 152i is locked. In turn, the output of the present transaction I52j can be cryptographically locked to a new user or entity I 03b. The present transaction 152j can thus transfer the amount defined in the input of the preceding transaction 152i to the new user or entity I 03b as defined in the output of the present transaction 152j. In some cases a transaction 152 may have multiple outputs to split the input amount between multiple users or entities (one of whom could be the original user or entity 103a in order to give change). In some cases a transaction can also have multiple inputs to gather together the amounts from multiple outputs of one or more preceding transactions, and redistribute to one or more outputs of the current transaction.

[0054] According to an output-based transaction protocol such as bitcoin, when an entity, such as a user or machine, 103 wishes to enact a new transaction 152j, then the entity sends the new transaction from its computer terminal 102 to a recipient. The entity or the recipient will eventually send this transaction to one or more of the blockchain nodes 104 of the network 106 (which nowadays are typically servers or data centres, but could in principle be other user terminals). It is also not excluded that the entity 103 enacting the new transaction 152j could send the transaction to one or more of the blockchain nodes 104 and, in some examples, not to the recipient. A blockchain node 104 that receives a transaction checks whether the transaction is valid according to a blockchain node protocol which is applied at each of the blockchain nodes Ii 104. The blockchain node protocol typically requires the blockchain node 104 to check that a cryptographic signature in the new transaction 152] matches the expected signature, which depends on the previous transaction 152i in an ordered sequence of transactions 152. In such an output-based transaction protocol, this may include checking that the cryptographic signature or other authorisation of the entity 103 included in the input of the new transaction 152j matches a condition defined in the output of the preceding transaction 152i which the new transaction assigns, wherein this condition typically includes at least checking that the cryptographic signature or other authorisation in the input of the new transaction 152j unlocks the output of the previous transaction I 52i to which the input of the new transaction is linked to. The condition may be at least partially defined by a script included in the output of the preceding transaction I 52i. Alternatively it could simply he fixed by the blockchain node protocol alone, or it could be due to a combination of these. Either way, if the new transaction I 52j is valid, the blockchain node 104 forwards it to one or more other blockchain nodes 104 in the blockchain network 106. These other blockchain nodes 104 apply the same test according to the same blockchain node protocol, and so forward the new transaction 152j on to one or more further nodes 104, and so forth. In this way the new transaction is propagated throughout the network of blockchain nodes 104.

[0055] In an output-based model, the definition of whether a given output (e.g. UTXO) is assigned is whether it has yet been validly redeemed by the input of another, onward transaction 152j according to the blockchain node protocol. Another condition for a transaction to be valid is that the output of the preceding transaction 152i which it attempts to assign or redeem has not already been assigned/redeemed by another transaction. Again if not valid, the transaction 152j will not be propagated (unless flagged as invalid and propagated for alerting) or recorded in the blockchain 150. This guards against double-spending whereby the transactor tries to assign the output of the same transaction more than once. An account-based model on the other hand guards against double-spending by maintaining an account balance. Because again there is a defined order of transactions, the account balance has a single defined state at any one time.

1-00561 In addition to validating transactions, blockchain nodes 104 also race to be the first to create blocks of transactions in a process commonly referred to as mining, which is supported by "proof-of-work". At a blockchain node 104, new transactions are added to an ordered set 154 of valid transactions that have not yet appeared in a block 151 recorded on the blockchain 150. The blockchain nodes then race to assemble a new valid block 151 of transactions 152 from the ordered set of transactions 154 by attempting to solve a cryptographic puzzle. Typically this inlcudes searching for a "nonce" value such that when the nonce is concatenated with a representation of the ordered set of transactions 154 and hashed, then the output of the hash meets a predetermined condition. For example, the predetermined condition may he that the output of the hash has a certain predefined number of leading zeros. Note that this is just one particular type of proof-of-work puzzle, and other types are not excluded. A property of a hash function is that it has an unpredictable output with respect to its input. Therefore this search can only be performed by brute force, thus consuming a substantive amount of processing resource at each blockchain node 104 that is trying to solve the puzzle.

14110571 The first blockchain node 104 to solve the puzzle announces this to the network 106, providing the solution as proof which can then be easily checked by the other blockchain nodes 104 in the network (once given the solution to a hash it is straightforward to check that it causes the output of the hash to meet the condition). The first blockchain node 104 propagates a block to a threshold consensus of other nodes that accept the block and thus enforce the protocol rules. The ordered set of transactions 154 then becomes recorded as a new block 151 in the blockchain 150 by each of the blockchain nodes 104. A block pointer 155 is also assigned to the new block 151n pointing back to the previously created block 151n-1 in the chain. A significant amount of effort, for example in the form of hash, required to create a proof-of-work solution signals the intent of the first node 104 to follow the rules of the blockchain protocol. Such rules include not accepting a transaction as valid if it assigns the same output as a previously validated transaction, otherwise known as double-spending. Once created, the block IS! cannot be modified since it is recognized and maintained at each of the blockchain nodes 104 in the blockchain network 106. The block pointer 155 also imposes a sequential order to the blocks 151. Since the transactions 152 are recorded in the ordered blocks at each blockchain node 104 in a network 106, this therefore provides an immutable public ledger of the transactions.

[0058] Note that different blockchain nodes 104 racing to solve the puzzle at any given time may be doing so based on different snapshots of the ordered set of yet to be published transactions 154 at any given time, depending on when they started searching for a solution or the order in which the transactions were received. Whoever solves their respective puzzle first defines which transactions 152 are included in the next new block I 5 In and in which order, and the current set 154 of unpublished transactions is updated. The blockchain nodes 104 then continue to race to create a block from the newly-defined outstanding ordered set of unpublished transactions 154, and so forth. A protocol also exists for resolving any "fork" that may arise, which is where two blockchain nodes 104 solve their puzzle within a very short time of one another such that a conflicting view of the blockchain gets propagated between nodes 104. In short, whichever prong of the fork grows the longest becomes the definitive blockchain 150. Note this should not affect the users or agents of the network as the same transactions will appear in both forks.

[0059] According to the bitcoin blockchain (and most other blockchains) a node that successfully constructs a new block 104 is granted the ability to assign an accepted amount of the digital asset in a new special kind of transaction which distributes a defined quantity of the digital asset (as opposed to an inter-agent, or inter-user transaction which transfers an amount of the digital asset from one agent or user to another). This special type of transaction is usually referred to as a "coinbase transaction", but may also be termed an "initiation transaction". It typically forms the first transaction of the new block 151n. The proof-of-work signals the intent of the node that constructs the new block to follow the protocol rules allowing this special transaction to be redeemed later. The blockchain protocol rules may require a maturity period, for example 100 blocks, before this special transaction may be redeemed. Often a regular (non-generation) transaction 152 will also specify an additional transaction fee in one of its outputs, to further reward the blockchain node I 04 that created the block 15 In in which that transaction was published. This fee is normally referred to as the "transaction fee", and is discussed blow.

[0060] Due to the resources involved in transaction validation and publication, typically at least each of the blockchain nodes 104 takes the form of a server that includes one or more physical server units, or even whole a data centre. However in principle any given blockchain node 104 could take the form of a user terminal or a group of user terminals networked together.

[0961] The memory of each blockchain node 104 stores software configured to run on the processing apparatus of the blockchain node 104 in order to perform its respective role or roles and handle transactions 152 in accordance with the blockchain node protocol. It will be understood that any action attributed herein to a blockchain node 104 may be performed by the software run on the processing apparatus of the respective computer equipment. The node software may be implemented in one or more applications at the application layer, or a lower layer such as the operating system layer or a protocol layer, or any combination of these.

[1111962] Also connected to the network 101 is the computer equipment 102 of each of a plurality of parties 103 in the role of consuming users. These users may interact with the blockchain network but do not participate in validating, constructing or propagating transactions and blocks. Some of these users or agents 103 may act as senders and recipients in transactions. Other users may interact with the blockchain 150 without necessarily acting as senders or recipients. For instance, some parties may act as storage entities that store a copy of the blockchain 150 (e.g. having obtained a copy of the blockchain from a blockchain node 104).

[11963] Some or all of the parties 103 may be connected as part of a different network, e.g. a network overlaid on top of the blockchain network 106. Users of the blockchain network (often referred to as "clients") may be said to be part of a system that includes the blockchain network; however, these users are not blockchain nodes I 04 as they do not perform the roles required of the blockchain nodes. Instead, each party 103 may interact with the blockchain network 106 and thereby utilize the blockchain 150 by connecting to (i.e. communicating with) a blockchain node 106. Two parties 103 and their respective equipment 102 are shown for illustrative purposes: a first party 103a and his/her respective computer equipment 102a, and a second party 103b and his/her respective computer equipment 102b. It will be understood that many more such parties 103 and their respective computer equipment 102 may be present and

IS

participating in the system 100, but for convenience they are not illustrated. Each party 103 may be an individual or an organization. Purely by way of illustration the first party 103a is referred to herein as Alice and the second party 103b is referred to as Bob, but it will be appreciated that this is not limiting and any reference herein to Alice or Bob may be replaced with "first party" and "second "party" respectively.

[0964] The computer equipment 102 of each party 103 comprises respective processing apparatus including one or more processors, e.g. one or more CPUs, GPUs, other accelerator processors, application specific processors, and/or FPGAs. The computer equipment 102 of each party 103 further includes memory, i.e. computer-readable storage in the form of a non-transitory computer-readable medium or media. This memory may include one or more memory units employing one or more memory media, e.g. a magnetic medium such as hard disk; an electronic medium such as an SSD, flash memory or EEPROM; and/or an optical medium such as an optical disc drive. The memory on the computer equipment 102 of each party 103 stores software such as a respective instance of at least one client application 105 arranged to run on the processing apparatus. It will be understood that any action attributed herein to a given party 103 may be performed using the software run on the processing apparatus of the respective computer equipment 102. The computer equipment 102 of each party 103 includes at least one user terminal, e.g. a desktop or laptop computer, a tablet, a smartphone, or a wearable device such as a smartwatch. The computer equipment 102 of a given party 103 may also include one or more other networked resources, such as cloud computing resources accessed via the user terminal.

[0065] The client application 105 may be initially provided to the computer equipment 102 of any given party 103 on suitable computer-readable storage medium or media, e.g. downloaded from a server, or provided on a removable storage device such as a removable SSD, flash memory key, removable EEPROM, removable magnetic disk drive, magnetic floppy disk or tape, optical disk such as a CD or DVD ROM, or a removable optical drive, etc. [0066] The client application 105 includes at least a "wallet" function. This has two main functionalities. One of these is to enable the respective party 103 to create, authorise (for example sign) and send transactions 152 to one or more bitcoin nodes 104 to then be propagated throughout the network of blockchain nodes 104 and thereby included in the blockchain 150. The other is to report back to the respective party the amount of the digital asset that he or she currently owns. In an output-based system, this second functionality includes collating the amounts defined in the outputs of the various 152 transactions scattered throughout the blockchain 150 that belong to the party in question.

[0967] It will be understood that whilst the various client functionality may be described as being integrated into a given client application 105, this is not necessarily limiting and instead any client functionality described herein may instead be implemented in a suite of two or more distinct applications, e.g. interfacing via an API, or one being a plug-in to the other. More generally the client functionality could be implemented at the application layer or a lower layer such as the operating system, or any combination of these. The following will be described in terms of a client application 105 but it will be appreciated that this is not limiting.

[0068] The instance of the client application or software 105 on each computer equipment 102 is operatively coupled to at least one of the blockchain nodes -104 of the network 106. This enables the wallet function of the client 105 to send transactions 152 to the network 106. The client 105 is also able to contact blockchain nodes 104 in order to query the blockchain 150 for any transactions of which the respective party 103 is the recipient (or indeed inspect other parties' transactions in the blockchain 150, since in embodiments the blockchain 150 is a public facility which provides trust in transactions in part through its public visibility). The wallet function on each computer equipment 102 is configured to fon ulate and send transactions 152 according to a transaction protocol. As set out above, each blockchain node 104 runs software configured to validate transactions 152 according to the blockchain node protocol, and to forward transactions 152 in order to propagate them throughout the blockchain network 106. The transaction protocol and the node protocol correspond to one another, and a given transaction protocol goes with a given node protocol, together implementing a given transaction model. The same transaction protocol is used for all transactions 152 in the blockchain 150. The same node protocol is used by all the nodes 104 in the network 106.

[0069] When a given party 103, say Alice, wishes to send a new transaction 152j to be included in the blockchain 150, then she formulates the new transaction in accordance with the relevant transaction protocol (using the wallet function in her client application 105). She then sends the transaction 152 from the client application 105 to one or more blockchain nodes 104 to which she is connected. E.g. this could be the blockchain node 104 that is best connected to Alice's computer 102. When any given blockchain node 104 receives a new transaction 1521, it handles it in accordance with the blockchain node protocol and its respective role. This may include first checking whether the newly received transaction 152j meets a certain condition for being "valid", examples of which will be discussed in more detail shortly. In some transaction protocols, the condition for validation may be configurable on a per-transaction basis by scripts included in the transactions 152. Alternatively the condition could simply be a built-in feature of the node protocol, or he defined by a combination of the script and the node protocol.

[0070] On condition that the newly received transaction I52j passes the test for being deemed valid (i.e. on condition that it is "validated"), any blockchain node 104 that receives the transaction 152j will add the new validated transaction 152 to the ordered set of transactions 154 maintained at that blockchain node 104. Further, any blockchain node 104 that receives the transaction 152j will propagate the validated transaction 152 onward to one or more other blockchain nodes 104 in the network 106. Since each blockchain node 104 applies the same protocol, then assuming the transaction 152j is valid, this means it will soon be propagated throughout the whole network 106.

[0971] Once admitted to the ordered set of transactions 154 maintained at a given blockchain node 104, that blockchain node 104 will start competing to solve the proof-of-work puzzle on the latest version of their respective ordered set of transactions 154 including the new transaction 152 (recall that other blockchain nodes 104 may be trying to solve the puzzle based on a different ordered set of transactions 154, hut whoever gets there first will define the ordered set of transactions that are included in the latest block 151, and eventually a blockchain node 104 will solve the puzzle for a part of the ordered set 154 which includes Alice's transaction I52j). Once the proof-of-work has been done for the ordered set 154 including the new transaction 1521, it immutably becomes part of one of the blocks 151 in the blockchain 150. Each transaction 152 includes a pointer back to an earlier transaction, so the order of the transactions is also immutably recorded.

[0072] Different blockchain nodes 104 may receive different instances of a given transaction first and therefore have conflicting views of which instance is 'valid' before one instance is published in a new block 151, at which point all blockchain nodes 104 agree that the published instance is the only valid instance. If a blockchain node 104 accepts one instance as valid, and then discovers that a second instance has been recorded in the blockchain 150 then that blockchain node 104 must accept this and will discard (i.e. treat as invalid) the instance which it had initially accepted (i.e. the one that has not been published in a block IS!).

[0073] An alternative type of transaction protocol operated by some blockchain networks may be referred to as an "account-based" protocol, as part of an account-based transaction model. In the account-based case, each transaction does not define the amount to be transferred by referring back to the UTXO of a preceding transaction in a sequence of past transactions, but rather by reference to an absolute account balance. The current state of all accounts is stored, by the nodes of that network, separate to the blockchain and is updated constantly. In such a system, transactions are ordered using a running transaction tally of the account (also called the position"). This value is signed by the sender as part of their cryptographic signature and is hashed as part of the transaction reference calculation. In addition, an optional data field may also be signed the transaction. This data field may point back to a previous transaction, for example if the previous transaction ID is included in the data field.

UTXO-based Model [0074] Figure 2 illustrates an example transaction protocol. This is an example of a UTXO-based protocol. A transaction 152 (abbreviated "Tx-) is the fundamental data structure of the blockchain 150 (each block 151 containing one or more transactions 152). The following will be described by reference to an output-based or "UTXO" based protocol. However, this is not limiting to all possible embodiments. Note that while the example UTXO-based protocol is described with reference to bitcom, it may equally be implemented on other example blockchain networks.

[0075] In a UTXO-based model, each transaction ("Tx") 152 is a data structure having one or more inputs 202, and one or more outputs 203. Each output 203 may include an unspent transaction output (UTXO), which can be used as the source for the input 202 of another new transaction (if the UTXO has not already been redeemed). The UTXO includes a value specifying an amount of a digital asset. This represents a set number of tokens on the distributed ledger. The UTXO may also contain the transaction ID of the transaction from which it came, amongst other information. The transaction data structure may also include a header 20!, which may include an indicator of the size of the input field(s) 202 and output field(s) 203. The header 201 may also include an ID of the transaction. in some embodiments, the transaction ID is the hash of the transaction data (excluding the transaction ID itself) and stored in the header 201 of the raw transaction 152 submitted to the nodes 104.

[0076] Say Alice I 03a wishes to create a transaction I 52j transferring an amount of the digital asset in question to Bob 103b. In Figure 2 Mice's new transaction 152j is labelled "Tx/. It tales an amount of the digital asset that is locked to Alice in the output 203 of a preceding transaction 152i in the sequence, and transfers at least some of this to Bob. The preceding transaction 152i is labelled "Tx0" in Figure 2. Tvo and Tx-] are just arbitrary labels. They do not necessarily mean that Tx° is the first transaction in the blockchain 151, nor that Tvi is the immediate next transaction in the pool 154. Tx/ could point back to any preceding (i.e. antecedent) transaction that still has an unspent output 203 locked to Alice.

[0077] The preceding transaction Tx0 may already have been validated and included in a block 151 of the blockchain 150 at the time when Alice creates her new transaction Ti,, or at least by the time she sends it to the network 106. it may already have been included in one of the blocks 151 at that time, or it may be still waiting in the ordered set 154 in which case it will soon be included in a new block 151. Alternatively Tvo and Tx/ could be created and sent to the network 106 together, or 71to could even be sent after Tx/ if the node protocol allows for buffering "orphan" transactions. The terms "preceding" and "subsequent" as used herein in the context of the sequence of transactions refer to the order of the transactions in the sequence as defined by the transaction pointers specified in the transactions (which transaction points back to which other transaction, and so forth). They could equally be replaced with "predecessor" and "successor", or "antecedent" and "descendant", "parent" and "child", or such like. It does not necessarily imply an order in which they are created, sent to the network 106, or arrive at any given blockchain node 104. Nevertheless, a subsequent transaction (the descendent transaction or "child") which points to a preceding transaction (the antecedent transaction or "parent") will not be validated until and unless the parent transaction is validated. A child that arrives at a blockchain node 104 before its parent is considered an orphan. It may be discarded or buffered for a certain time to wait for the parent, depending on the node protocol and/or node behaviour.

109781 One of the one or more outputs 203 of the preceding transaction Tx0 is a particular UTXO, labelled here UTX00. Each UTXO includes a value specifying an amount of the digital asset represented by the UTXO, and a locking script which defines a condition which must be met by an unlocking script in the input 202 of a subsequent transaction in order for the subsequent transaction to be validated, and therefore for the UTXO to be successfully redeemed. Typically the locking script locks the amount to a particular party (the beneficiary of the transaction in which it is included). That is, the locking script defines an unlocking condition, typically include a condition that the unlocking script in the input of the subsequent transaction include the cryptographic signature of the party to whom the preceding transaction is locked.

199791 The locking script (aka scriptPubKey) is a piece of code written in the domain specific language recognized by the node protocol. A particular example of such a language is called "Script" (capital S) which is used by the blockchain network. The locking script specifies what information is required to spend a transaction output 203, for example the requirement of Alice's signature. Unlocking scripts appear in the outputs of transactions. The unlocking script (aka scriptSig) is a piece of code written the domain specific language that provides the information required to satisfy the locking script criteria. For example, it may contain Bob's signature. Unlocking scripts appear in the input 202 of transactions.

[0080] So in the example illustrated, UTX00 in the output 203 of Txo includes a locking script [Checksig PA] which requires a signature Sig PA of Alice in order for UTX00 to be redeemed (strictly, in order for a subsequent transaction attempting to redeem UTX00 to be valid). [Checksig PA] contains a representation (i.e. a hash) of the public key PA from a public-private key pair of Alice. The input 202 of Tx] includes a pointer pointing back to Tro (e.g. by means of its transaction ID, Tx/Do, which in some embodiments is the hash of the whole transaction Tx0). The input 202 of Tx) includes an index identifying UTX00 within Txo, to identify it amongst any other possible outputs of Tx0. The input 202 of Tx) further includes an unlocking script <Sig PA> which has a cryptographic signature of Alice, created by Alice applying her private key from the key pair to a predefined portion of data (sometimes called the "message" in cryptography). The data (or "message") that needs to be signed by Alice to provide a valid signature may be defined by the locking script, or by the node protocol, or by a combination of these.

[0081] When the new transaction Tx/ arrives at a blockchain node 104, the node applies the node protocol. This may include running the locking script and unlocking script together to check whether the unlocking script meets the condition defined in the locking script (where this condition may include one or more criteria). In some embodiments this may involve concatenating the two scripts: <Sig PA> <PA> II [Checksig PA] [0082] where "II" represents a concatenation and "<...>" means place the data on the stack, and "[...1" is a function carried out by the locking script (in this example a stack-based language). Equivalently the scripts may be run one after the other, with a common stack, rather than concatenating the scripts. Either way, when run together, the scripts use the public key PA of Alice, as included in the locking script in the output of Tx°, to authenticate that the unlocking script in the input of TA 7 contains the signature of Alice signing the expected portion of data. The expected portion of data itself (the "message") also needs to be included in order to perform this authentication. In embodiments the signed data includes the whole of Tx, (so a separate element does not need to be included specifying the signed portion of data in the clear, as it is already inherently present).

[0083] The details of authentication by public-private cryptography will be familiar to a person skilled in the art. Basically, if Alice has signed a message using her private key, then given Alice's public key and the message in the clear, another entity such as a node 104 is able to authenticate that the message must have been signed by Alice. Signing typically includes hashing the message, signing the hash, and tagging this onto the message as a signature, thus enabling any holder of the public key to authenticate the signature. Note therefore that any reference herein to signing a particular piece of data or part of a transaction, or such like, can in embodiments mean signing a hash of that piece of data or part of the transaction.

[0084] If the unlocking script in Txr meets the one or more conditions specified in the locking script of Tx() (so in the example shown, if Alice's signature is provided in Tx/ and authenticated), then the blockchain node 104 deems Txr valid. This means that the blockchain node 104 will add Txr to the ordered set of transactions 154. The blockchain node 104 will also forward the transaction Tx/ to one or more other blockchain nodes 104 in the network 106, so that it will be propagated throughout the network 106. Once Tx] has been validated and included in the blockchain 150, this defines UTX00 from Tx0 as spent. Note that Tx/ can only be valid if it spends an unspent transaction output 203. If it attempts to spend an output that has already been spent by another transaction 152, then Tx/ will be invalid even if all the other conditions are met. Hence the blockchain node 104 also needs to check whether the referenced UTXO in the preceding transaction Txo is already spent (i.e. whether it has already formed a valid input to another valid transaction). This is one reason why it is important for the blockchain 150 to impose a defined order on the transactions 152. In practice, a given blockchain node 104 may maintain a separate database marking which UTX0s 203 in which transactions 152 have been spent, hut ultimately what defines whether a UTXO has been spent is whether it has already formed a valid input to another valid transaction in the blockchain 150.

[0085] If the total amount specified in all the outputs 203 of a given transaction 152 is greater than the total amount pointed to by all its inputs 202, this is another basis for invalidity in most transaction models. Therefore such transactions will not be propagated nor included in a block 151.

[0086] Note that in UTXO-based transaction models, a given UTXO needs to be spent as a whole. It cannot "leave behind" a fraction of the amount defined in the UTXO as spent while another fraction is spent. However, the amount from the UTXO can be split between multiple outputs of the next transaction. For example, the amount defined in UTX00 in Tx0 can be split between multiple UTX0s in Tx/. Hence if Alice does not want to give Bob all of the amount defined in U7X00, she can use the remainder to give herself change in a second output of Tx], or pay another party.

[0087] In practice Alice will also usually need to include a fee for the bitcoin node that publishes her transaction 104. If Alice does not include such a fee, Tx0 may be rejected by the blockchain nodes 104, and hence although technically valid, may not be propagated and included in the blockchain 150 (the node protocol does not force blockchain nodes 104 to accept transactions 152 if they don't want). In some protocols, the transaction fee does not require its own separate output 203 (i.e. does not need a separate UTXO). Instead any difference between the total amount pointed to by the input(s) 202 and the total amount of specified in the output(s) 203 of a given transaction 152 is automatically given to the blockchain node 104 publishing the transaction. For example, say a pointer to UTX00 is the only input to Txi, and Tx] has only one output U7X03. If the amount of the digital asset specified in UTX00 is greater than the amount specified in UTX02, then the difference may be assigned by the node 104 that publishes the block containing UTX03. Alternatively or additionally however, it is not necessarily excluded that a transaction fee could be specified explicitly in its own one of the UTX0s 203 of the transaction 152.

[0088] Alice and Bob's digital assets consist of the UTX0s locked to them in any transactions 152 anywhere in the blockchain 150. Hence, typically, the assets of a given party 103 are scattered throughout the UTX0s of various transactions 152 throughout the blockchain 150. There is no one number stored anywhere in the blockchain 150 that defines the total balance of a given party 103. It is the role of the wallet function in the client application 105 to collate together the values of all the various UTX0s which are locked to the respective party and have not yet been spent in another onward transaction. It can do this by querying the copy of the blockchain 150 as stored at any of the bitcoin nodes 104.

[0089] Note that the script code is often represented schematically (i.e. not using the exact language). For example, one may use operation codes (opcodes) to represent a particular function. "OP..." refers to a particular opcode of the Script language. As an example, OP RETURN is an opcode of the Script language that when preceded by OP FALSE at the beginning of a locking script creates an unspendable output of a transaction that can store data within the transaction, and thereby record the data immutably in the blockchain 150. For example, the data could include a document which it is desired to store in the blockchain.

[0090] Typically an input of a transaction contains a digital signature corresponding to a public key PA. In embodiments this is based on the ECDSA using the elliptic curve secp256k1. A digital signature signs a particular piece of data. In some embodiments, for a given transaction the signature will sign part of the transaction input, and some or all of the transaction outputs. The particular parts of the outputs it signs depends on the SIGHASH flag. The SIGHASH flag is usually a 4-byte code included at the end of a signature to select which outputs are signed (and thus fixed at the time of signing).

[0091] The locking script is sometimes called "scriptPubKey" referring to the fact that it typically includes the public key of the party to whom the respective transaction is locked. The unlocking script is sometimes called "scriptSig" referring to the fact that it typically supplies the corresponding signature. However, more generally it is not essential in all applications of a blockchain 150 that the condition for a UTXO to be redeemed includes authenticating a signature. More generally the scripting language could be used to define any one or more conditions. Hence the more general terms "locking script" and "unlocking script" may be preferred.

[0092] As shown in Figure 1, the client application on each of Alice and Bob's computer equipment 102a, 1206, respectively, may include additional communication functionality This additional functionality enables Alice 103a to establish a separate side channel 301 with Bob 103b (at the instigation of either party or a third party). The side channel 301 enables exchange of data separately from the blockchain network. Such communication is sometimes referred to as "off-chain" communication. For instance this may be used to exchange a transaction 152 between Alice and Bob without the transaction (yet) being registered onto the blockchain network 106 or making its way onto the chain 150, until one of the parties chooses to broadcast it to the network 106. Sharing a transaction in this way is sometimes referred to as sharing a "transaction template". A transaction template may lack one or more inputs and/or outputs that are required in order to form a complete transaction. Alternatively or additionally, the side channel 30! may be used to exchange any other transaction related data, such as keys, negotiated amounts or terms, data content, etc. [0093] The side channel 301 may be established via the same packet-switched network 10! as the blockchain network 106. Alternatively or additionally, the side channel 301 may be established via a different network such as a mobile cellular network, or a local area network such as a local wireless network, or even a direct wired or wireless link between Alice and Bob's devices 102a, 102b. Generally, the side channel 301 as referred to anywhere herein may include any one or more links via one or more networking technologies or communication media for exchanging data "off-chain", i.e. separately from the blockchain network 106. Where more than one link is used, then the bundle or collection of off-chain links as a whole may be referred to as the side channel 301. Note therefore that if it is said that Alice and Bob exchange certain pieces of information or data, or such like, over the side channel 301, then this does not necessarily imply all these pieces of data have to be send over exactly the same link or even the same type of network.

Client Software [0994] Figure 3A illustrates an example implementation of the client application 105 for implementing embodiments of the presently disclosed scheme. The client application 105 may include a transaction engine 401 and a user interface (UI) layer 402. The transaction engine 401 is configured to implement the underlying transaction-related functionality of the client 105, such as to formulate transactions 152, receive and/or send transactions and/or other data over the side channel 301, and/or send transactions to one or more nodes 104 to be propagated through the blockchain network 106, in accordance with the processes discussed above.

[0095] The UI layer 402 is configured to render a user interface via a user input/output (I/0) means of the respective user's computer equipment 102, including outputting information to the respective user 103 via a user output means of the equipment 102, and receiving inputs back from the respective user 103 via a user input means of the equipment 102. For example the user output means could include one or more display screens (touch or non-touch screen) for providing a visual output, one or more speakers for providing an audio output, and/or one or more haptic output devices for providing a tactile output, etc. The user input means could include for example the input array of one or more touch screens (the same or different as that/those used for the output means); one or more cursor-based devices such as mouse, trackpad or trackball; one or more microphones and speech or voice recognition algorithms for receiving a speech or vocal input; one or more gesture-based input devices for receiving the input in the form of manual or bodily gestures; or one or more mechanical buttons, switches or joysticks, etc. [0096] Note that whilst the various functionality herein may be described as being integrated into the same client application 105, this is not necessarily limiting and instead they could be implemented in a suite of two or more distinct applications, e.g. one being a plug-in to the other or interfacing via an API (application programming interface). For instance, the functionality of the transaction engine 401 may be implemented in a separate application than the UI layer 402, or the functionality of a given module such as the transaction engine 401 could be split between more than one application. Nor is it excluded that some or all of the described functionality could be implemented at, say, the operating system layer. Where reference is made anywhere herein to a single or given application 105, or such like, it will be appreciated that this is just by way of example, and more generally the described functionality could he implemented in any form of software.

[0097] Figure 3B gives a mock-up of an example of the user interface (UI) 500 which may be rendered by the UI layer 402 of the client application 105a on Alice's equipment 102a. It will be appreciated that a similar UI may be rendered by the client 105b on Bob's equipment 102b, or that of any other party.

[0098] By way of illustration Figure 3B shows the UI 500 from Alice's perspective. The UI 500 may include one or more UI elements 501, 502, 502 rendered as distinct UI elements via the user output means.

[0099] For example, the Ul elements may include one or more user-selectable elements 501 which may he, such as different on-screen buttons, or different options in a menu, or such like. The user input means is arranged to enable the user 103 (in this case Alice I03a) to select or otherwise operate one of the options, such as by clicking or touching the UI element on-screen, or speaking a name of the desired option (N.B. the term "manual" as used herein is meant only to contrast against automatic, and does not necessarily limit to the use of the hand or hands).

[0100] Alternatively or additionally, the UI elements may include one or more data entry fields 502. These data entry fields 502 are rendered via the user output means, e.g. on-screen, and the data can be entered into the fields through the user input means, e.g. a keyboard or touchscreen. Alternatively the data could be received orally for example based on speech recognition.

[0101] Alternatively or additionally, the UI elements may include one or more information elements 503 output to output information to the user. For example, the information could be rendered on screen or audibly.

[0102] It will be appreciated that the particular means of rendering the various UI elements, selecting the options and entering data is not material. The functionality of these UI elements will be discussed in more detail shortly. It will also be appreciated that the UI 500 shown in Figure 3B is only a schematized mock-up and in practice it may include one or more further UI elements, which for conciseness are not illustrated.

Node Software [0103] Figure 4 illustrates an example of the node software 450 that is run on each blockchain node 104 of the network 106, in the example of a UTX0-or output-based model. Note that another entity may run node software 450 without being classed as a node 104 on the network 106, i.e. without performing the actions required of a node 104. The node software 450 may contain, but is not limited to a protocol engine 451, a script engine 452, a stack 453, an application-level decision engine 454, and a set of one or more blockchain-related functional modules 455. Each node 104 may run node software that contains, but is not limited to, all three of: a consensus module 455C (for example, proof-of-work), a propagation module 455P and a storage module 455S (for example, a database). The protocol engine 401 is typically configured to recognize the different fields of a transaction 152 and process them in accordance with the node protocol. When a transaction 1521 (Txj) is received having an input pointing to an output (e.g. UTX0) of another, preceding transaction I 52i (Txmi_i), then the protocol engine 451 identifies the unlocking script in Txf and passes it to the script engine 452. The protocol engine 451 also identifies and retrieves Txi based on the pointer in the input of Txj. Txi may be published on the blockchain 150, in which case the protocol engine may retrieve Tx i from a copy of a block 15! of the blockchain 150 stored at the node 104. Alternatively, Txi may yet to have been published on the blockchain 150. In that case, the protocol engine 451 may retrieve Txi from the ordered set 154 of unpublished transactions maintained by the node104. Either way, the script engine 451 identifies the locking script in the referenced output of Txi and passes this to the script engine 452.

[0104] The script engine 452 thus has the locking script of Txi and the unlocking script from the corresponding input of Txj. For example, transactions labelled Txo and Txi are illustrated in Figure 2, but the same could apply for any pair of transactions. The script engine 452 runs the two scripts together as discussed previously, which will include placing data onto and retrieving data from the stack 453 in accordance with the stack-based scripting language being used (e.g. Script).

[0105] By running the scripts together, the script engine 452 determines whether or not the unlocking script meets the one or more criteria defined in the locking script -i.e. does it "unlock" the output in which the locking script is included? The script engine 452 returns a result of this determination to the protocol engine 451. if the script engine 452 determines that the unlocking script does meet the one or more criteria specified in the corresponding locking script, then it returns the result "true". Otherwise it returns the result -false".

[0106] In an output-based model, the result "true" from the script engine 452 is one of the conditions for validity of the transaction. Typically there are also one or more further, protocol-level conditions evaluated by the protocol engine 451 that must be met as well; such as that the total amount of digital asset specified in the output(s) of Txj does not exceed the total amount pointed to by its inputs, and that the pointed-to output of Txi has not already been spent by another valid transaction. The protocol engine 451 evaluates the result from the script engine 452 together with the one or more protocol-level conditions, and only if they are all true does it validate the transaction Txj. The protocol engine 451 outputs an indication of whether the transaction is valid to the application-level decision engine 454. Only on condition that Txj is indeed validated, the decision engine 454 may select to control both of the consensus module 455C and the propagation module 455P to perform their respective blockchain-related function in respect of Txj. This may include the consensus module 455C adding Txj to the node's respective ordered set of transactions 154 for incorporating in a block 151, and the propagation module 455P forwarding Txj to another blockchain node 104 in the network 106. Optionally, in embodiments the application-level decision engine 454 may apply one or more additional conditions before triggering either or both of these functions. For example, the decision engine may only select to publish the transaction on condition that the transaction is both valid and leaves enough of a transaction fee.

[0107] Note also that the terms "true" and "false" herein do not necessarily limit to returning a result represented in the form of only a single binary digit (bit), though that is certainly one possible implementation. More generally, "true" can refer to any state indicative of a successful or affirmative outcome, and "false" can refer to any state indicative of an unsuccessful or non-affirmative outcome. For instance in an account-based model, a result of "true" could be indicated by a combination of an implicit, protocol-level validation of a signature and an additional affirmative output of a smart contract (the overall result being deemed to signal true if both individual outcomes are true).

[0108] Other variants or use cases of the disclosed techniques may become apparent to the person skilled in the art once given the disclosure herein. The scope of the disclosure is not limited by the described embodiments but only by the accompanying claims.

[0109] For instance, some embodiments above have been described in terms of a bitcoin network 106, bitcoin blockchain 150 and bitcoin nodes 104. However it will he appreciated that the bitcoin blockchain is one particular example of a blockchain 150 and the above description may apply generally to any blockchain. That is, the present invention is in by no way limited to the bitcoin blockchain. More generally, any reference above to bitcoin network 106, bitcoin blockchain 150 and bitcoin nodes 104 may be replaced with reference to a blockchain network 106, blockchain 150 and blockchain node 104 respectively. The blockchain, blockchain network and/or blockchain nodes may share some or all of the described properties of the bitcoin blockchain 150, bitcoin network 106 and bitcoin nodes 104 as described above.

[0110] In some embodiments of the invention, the blockchain network 106 is the bitcoin network and bitcoin nodes 104 perform at least all of the described functions of creating, publishing, propagating and storing blocks 151 of the blockchain 150. It is not excluded that there may be other network entities (or network elements) that only perform one or some but not all of these functions. That is, a network entity may perform the function of propagating and/or storing blocks without creating and publishing blocks (recall that these entities are not considered nodes of the preferred bitcoin network I 06).

[0111] In some other embodiments of the invention, the blockchain network 106 may not be the bitcoin network. in these embodiments, it is not excluded that a node may perform at least one or some but not all of the functions of creating, publishing, propagating and storing blocks 151 of the blockchain 150. For instance, on those other blockchain networks a "node" may be used to refer to a network entity that is configured to create and publish blocks 151 but not store and/or propagate those blocks 151 to other nodes.

[0112] Even more generally, any reference to the term "bitcoin node" 104 above may be replaced with the term "network entity" or "network element", wherein such an entity/element is configured to perform some or all of the roles of creating, publishing, propagating and storing blocks. The functions of such a network entity/element may be implemented in hardware in the same way described above with reference to a blockchain node 104.

Merkle Proofs [0113] As described above, mining nodes group transactions into a block. The payload of the block contains an ordered set of the transactions, including the coinbase transaction. Each block has a block header that includes various data fields, including a Merkle root. The Merkle root may be considered a summary or "fingerprint" of the data in the payload, i.e. of the ordered set of transactions. The Merkle root is determined through building a Merkle tree.

[0114] A Merkle tree for an ordered set of elements is built through hashing each element and recursively creating a next layer by concatenating adjacent pairs of hashed elements and hashing the concatenated hashed elements. A simplified example of a Merkle tree 500 is shown in Figure 5A.

[0115] In this example, the Merkle tree 500 relates to an ordered set of only eight elements, for ease of illustration. The Merkle trees in other examples may be smaller or, more commonly, much larger. Layers of the Merkle tree 500 are labelled from the bottom up, with layer 0 being the base layer, and layer 3, in this example, being a Merkle root 502. The Merkle tree 500 is formed by first creating the base layer through hashing of the elements. That is, each base layer node 504 is a hash of the corresponding element in that position. In the case of bitcoin, each base layer node 504 is a hash (double-SHA256, in the case of bitcoin) of its corresponding transaction. The doubleSHA256 hash of a transaction is also its transaction identifier, TxID. Accordingly, in the case of bitcoin, each of the base layer nodes 504 is a Tx1D corresponding to the transaction in that position in the ordered set of transactions in a block.

[0116] To build the layer 1 nodes of the Merkle tree 500, the base layer nodes are grouped into pairs. Within each pair, the elements are concatenated and then hashed to find the value of the parent node at the layer above. For example, if a base layer pair 506 includes MID9 and Tx/Di, then the two TxID are concatenated as TrIMIITilD1, and the resulting value is hashed to calculate element 508.

[0117] Layer 2 elements are calculated as hashes of concatenated pairs of layer I elements, and so on. For example, element 512 is calculated from a hash of the concatenated pair of element 510 and element 508. The building of the layer through calculating intermediate hashes for the nodes at intermediate layers of the Merkle tree 500 continues until it results in a single element at a top layer, referred to as the Merkle root 502.

[0118] It will be appreciated that not every block with have a full set of transactions that includes exactly 2' elements. Such cases may result in a "partial" Merkle tree. Figure 5B illustrates an example in which there are five elements at layer 0, e.g. five transactions in a block. While the first four elements can be paired, the last transaction indicated by reference numeral 520 has no corresponding paired element. It will be appreciated that the missing paired element at the base layer only arises in the case where the last element is a left-hand member of the pair. For the purposes of building the Merkle tree in this situation, a "copy" of the left-hand element is used as the right-hand element; that is, the Tx/Di indicated by numeral 520 is concatenated with itself and is hashed to find the value of its parent element at node 522. Likewise, because node 522 has no corresponding right-hand element in its pairing, the element at node 522 is concatenated with itself and hashed to find the value of its parent element at node 524 in layer 2.

[0119] The Merkle root 502 is inserted into the header of a block to serve as a fingerprint of the transactions within the block.

[0120] Merkle trees are useful to enable lightweight nodes, such as client applications on end user devices, to determine whether a certain transaction is present in a block on the blockchain without necessarily having to download the entire block or the full blockchain. A client application may determine whether a transaction is present in a block based on a Merkle proof. A Merkle proof involves tracing a path (the Merkle path) from the transaction to the Merkle root through the Met-Ide tree to confirm that the transaction was included in the block. The term "Merkle proof' may refer to the paired nodes within the Met-Ide tree adjacent to nodes on the path that are needed to calculate the parent node on the Merkle path, i.e. the paired adjacent Met-Ide tree hashes/nodes needed to calculated the nodes along the Merkle path in order to confirm the Merkle root.

[0121] A node may request Merkle proof data relating to a transaction from which it can perform a Merkle proof, e.g. trace the Merkle path. In one example, the requested data may be provided in the form of an ordered set of hashes corresponding to the paired hashes within the Merkle tree required along the path in order to arrive at the Merkle root, along with binary signals indicating whether the calculated and/or provided hash is a left-hand or right-hand element of each pairing.

[0122] For the purpose of the discussion herein, the following conventions may be used.

An element that is calculated using hashing of available data (e.g. transaction data or concatenated elements of a Merkle tree) may be indicated using c[n], where n refers to the layer of the tree and the bottom or base layer is at n = 0. An element of the tree, e.g. an intermediate hash, provided by another node for the purposes of the Merkle proof may be indicated using p[n]. That intermediate hash may be referred to as a "paired hash", or as having come from a "paired intermediate node" or "intermediate paired node" or the like, in the sense that each node has a corresponding paired node used in determining the parent hash value.

[0123] Figure 6 shows an example of a Merkle path through a Merkle tree 600. The Merkle path in this case relates to validating the presence of transaction Tx in a block. To complete the Merkle proof using this Merkle path, a computing device has the transaction Tx (or at least the TOD) and an ordered set of hashes (intermediate hashes) corresponding to the needed elements to form pairs along the Merkle path. In this example, the ordered set includes p[0], p[i], p[2].

[0124] The computing device also needs a mechanism for determining whether the provided intermediate hashes are left-hand or right-hand elements of a pairing. This can be signalled using binary flags in some cases. The binary flag may indicate, for each layer, whether the calculated hash (or provided hash) is the left-hand or right-hand element. In some cases, instead of signalling whether the calculated hash (or provided hash) at each layer is the left-hand or right-hand element, the position of the calculated hash in the bottom or base layer may be signalled, i.e. its index in the ordered set of elements. In some implementations, these may be effectively equivalent. For example, bitwise signalling of the left-hand (0) or right-hand (1) path tracing the calculated elements of layers 2 to 0 moving down the Merkle path from a Merkle root 602, results in the bits [0, I, 0]. Equivalently, the index i=2 of the c[0] element may be signalled, which in binary is 010. Other implementations may use different signalling or coding.

[0125] To carry out the Merkle proof, the computing device determines the c[0] element by hashing (in this example, double-SHA256) the transaction Tx to obtain the TxID. In some cases, it may already have the TxID. It then concatenates it with the first provided hash in the ordered set of hashes, p[0]. The computing device determines from the index i or from the bit signalling whether the calculated hash c[0] is aleft-hand or right-hand element and concatenates accordingly. It then hashes the concatenation to determine c[ I]. It carries on with this process until it determines c[3], which it then compares to the Merkle root 602 found in the block header. If it matches, then the computing device has determined that the transaction Tx is included in the block. If not, then the transaction Tx is not in the block (or the Merkle proof hashes it was provided contain an error).

[0126] The position of an element in the ordered set of elements in a layer may be referred to as an "index" and may represented by i or by index0f(c[n]) when referring to the index of a calculated element within the nth layer. At the bottom or base layer, the index i ranges from 0 to the count of transactions in the block less one.

Subtree-based Merkle Tree Data Storage and Query Processing [0127] As the network grows, it may Incorporate dedicated "Merkle path servers" for storing Mende tree data and responding to queries from SPY wallets or other nodes seeking Merkle path data for performing Merkle proofs. In some cases, that server functionality may be implemented using standalone dedicated servers (i.e. networked computing devices) within the blockchain network. In some cases, that functionality may be provided by existing nodes or servers in the blockchain networks, such as mining nodes, validation nodes, full nodes, or other nodes of the blockchain network. In some implementations, the functionality may be incorporated into the node software 450 (FIG. 4) run by one or more nodes. In some implementations, the functionality may be embodied as a standalone software or software-implemented service separate from nodes implementing other blockchain functions or operations.

[0128] Irrespective of the network architecture or implementation model, one or more computing nodes on the blockchain network stores Merkle tree data so as to respond to Merkle proof queries by providing Merkle proof data. In one implementation, the node simply stores the block data for the blockchain in memory, i.e. the ordered sets of TxIDs. In response to any query, the node retrieves the full TxID data for a block from memory and then calculates the full Merkle tree from the TxID data in order to identify the Merkle proof data and provide it in reply to the query. It will be appreciated that this minimizes storage requirements in that only the TxID data is retained in memory, but is very costly in terms of quantity of data retrieved from memory (i.e. the full block data for every query which, with large trees, may require a large number of input/output operations) and very costly in computational burden (i.e. calculation of the full Merkle tree for every query).

[0129] Another alternative is for the node to store the entire calculated Merkle tree for each block. In other words, the node stores all the intermediate hashes for the nodes that make up the Merlde tree. In this manner, when a query is received, the node can retrieve the relevant paired intermediate nodes along the Merlde path in order to provide a reply to the query. While this is more costly in terms of memory usage, it reduces the computational burden on the node.

[0130] A practical challenge to implementation of such a system is determining how to store the Merkle tree data. This is particularly acute in the case where the node employs page-based memory structures, i.e. page-structured memory, such as solid state drive (SSD) memory, in which a read operation functions to return a full page of stored data. Each read operation is costly in terms of both time and computational energy. It would be advantageous to reduce the number of read operations. If the Merkle tree data is stored as a concatenated set of hashes level-by-level the required hashes for tracing a Merkle path may be stored across many pages of memory and may necessitate a large number of read operations.

[0131] The page size for SSD memory (or other types of page-structured memory) may be fixed by the hardware implementation of the memory. As will be described below, it may be possible, in some cases, to structure the storage of Merkle tree data so as to reduce the number of page retrieval operations. in some cases, the data structure and the method of storing Merkle tree data may be based on the page size.

[0132] In one aspect of the present application, the server may be configured to store Merkle tree data by partitioning the Merkle tree into non-overlapping subtrees, each having a subtree root, multiple layers, and leaf nodes. Each subtree may be stored in respective page in memory. In response to a query, the requested TxID is identified as belonging to a particular lowest-level subtree. The server then retrieves that subtree, its parent subtree(s), and any adjacent subtrees needed to trace the Merkle path, each of which is a single page retrieval.

[0133] in another aspect, of the present application, the server may be configured to store less than the full Merkle tree data. This may be particularly useful in the case of older blocks for which the likelihood of receiving a Merkle proof query is lower. in such implementations, the server may store only an upper portion of the tree. in some cases, the server may store an upper portion of the tree and the lowest layer of the tree, i.e. the set of TxiDs at layer 0. in response to a Merkle proof query, the server may retrieve the needed portion of the set of TxiDs for building a Merkle path to the upper portion. The TAD data may be stored at the server or may be available from an external source that stores block data. The server further retrieves the upper portion data, or at least such part of the upper portion as is needed to further complete the Merkle path for the query. in reply to the query, the server sends the Merkle proof data. In some instances, the server may only send the Merkle proof data for the upper portion and the requesting device may obtain the TxID data necessary for calculating the lower portion of the Merkle path from some other source. In some cases, the server may be configured to prune the stored Merkle tree data over time, periodically removing the lowest stored layer of stored subtrees as the corresponding block ages.

[0134] In yet another aspect, the storage of subtrees In respective pages may be modified to further reduce input-output operations and improve memory usage by storing partial Merkle proof data for the path above a subtree. That is, the storage of Merkle tree data may be based on storing subtrees for a set of the lowest layers and each subtree may further store in the same page of memory at least a portion of the Merkle tree data for the Merkle proof above that subtree. That Merkle proof data includes hash values for intermediate paired nodes for the nodes in the path above the subtree. In such implementations, the data for tracing the Merkle path for a TxTD may be retrieved in a single page read operation.

[0135] In some implementations, the server may store in the same page of memory a composite subtree made up of a main subtree and a secondary subtree, both at the lowest level of the Merkle tree) where the secondary subtree is smaller (has fewer TiciDs) than the large tree. Depending on the process used to partition the Merkle tree into composite subtrees, in some cases the smaller second subtree may share at least one common intermediate parent node with the larger first subtree: in other words, their respective Merkle paths may converge at some point below the Merkle root.

[0136] Reference is now made to Figures 7A and 7B, which show one simplified example of a Merkle tree 700. In this simplified example, the Merkle tree 700 contains 32 nodes at its lowest level, level 0. The total number of nodes is 63. For the purposes of the discussion herein each node is presumed to be represented by a hash value of length 32 bytes.

[0137] Figure 7A shows a partitioning of the Merkle tree 700 into subtrees on the basis of a page size of I 28 bytes. Such a page could contain at most 4 nodes. The dashed lines indicate the grouping of nodes into subtrees. The hash values representing the nodes of a subtree may be concatenated and stored in a page in memory. In this example, a subtree of 3 nodes occupies 75% of the memory allocated for the page.

[0138] Figure 7B shows a partitioning of the Merkle tree 700 into subtrees on the basis of a page size of 256 bytes. Such a page could contain at most 8 nodes, i.e. 8 32-byte hash values. The dashed lines indicate the grouping of nodes into subtrees. The hash values representing the nodes of a subtree may be concatenated and stored in a page in memory. In this example, a subtree of 7 nodes occupies 87.5% of the memory allocated for the page.

[0139] It will be appreciated that in many implementations, much larger Merkle trees may be used. For example, a 1 terrabyte (TB) block may contain, for example, about 4.3 billion transactions, each having a TxID length of 32 bytes. A 4.3 billion transaction Merkle tree would have 33 levels including the top level containing the Merkle root. A Merkle tree, with ri transactions in layer zero, contains 2n-I nodes including layer 0. With no redundancy and 32 byte hash values, such a tree would require almost 275 GB of storage. With a page size of about 4kB, this would mean over 67 million pages of memory. A page size of 4096 bytes would accommodate subtrees of 127 nodes, i.e. with n = 64 TxiDs and a depth of 7 levels including level 0 and the subtree root node.

[0140] In some cases, it will be appreciated that stored data for the Merkle tree may exclude storage of the Merkle root since it is not data needed to complete a Merkle path; rather, the Merkle root is available to the SVP wallet from the block header for the block containing the transaction for which the Merkle path is being sought. The Merkle root is the value that the wallet is seeking to confirm by calculating the Merkle path for a transaction. When the calculated root matches the root in the block header, then the wallet has confirmed that the transaction is validly present in that block. For the purposes of the present application, the Merkle tree will be described as having a number of layers that includes the Merkle root, although it will be appreciated that the top layer containing only the Merkle root may not be stored at the Merkle path server.

[0141] Referring still to Helm. 7A and 7B, it will be noted that only some of the pages need to be retrieved to trace the Merkle path for a given TxID. For instance, in each case the shaded node indicates a TAD for which the Merkle path is sought. The dashed lines in heavier line width indicate the pages retrieved for obtaining the data for that Merkle path.

[0142] The storage of the Merkle trees in the manner shown, that is by partitioning the tree into subtrees based on the page size, may reduce the number of pages retrieved to trace a Merkle page versus storing the tree in a level-by-level format, e.g. storing the hash values in a width first, bottom-up progression. For example, if the tree of FIG. 7B were stored in a level-by-level format using 256 byte pages then each page could store eight node hash values. It would take 4 pages to store level 0, 2 pages to store level 1, 1 page to store level 2, and one page to store levels 3, 4 and 5 (the root) together. The total number of pages would be 8, which is fewer than the 9 pages shown in FIG. 7B; however, to retrieve a Merkle path it would take retrieval of 4 pages whereas in the example shown in FIG. 7B the full Merkle path is obtained by retrieving three pages. Accordingly, the described storage technique costs one additional page of storage, but results in retrieving Merkle path data using one less page retrieval.

[0143] It will be appreciated that the above-described storage process results in some wasted memory space, specifically 32 bytes per page. Nevertheless, the savings in input-output operations may be justified to speed the look-up of Merkle path data. Moreover, with larger page sizes the relative wasted memory space is small. For instance, in the case of a 4096 byte page size, the wasted memory space is only 32 bytes, which is less than 0.8%. In addition, as will he described further below, in some embodiments data may be stored in the excess 32 bytes that realizes greater efficiencies. In cases where the subtree root node is not stored, an excess 64 bytes is available.

[0144] Reference is now made to FIG. 8, which shows in flowchart form, one example method 800 for storing Merkle tree data in a page-based memory. The method 800 may be implemented by a network-connected computing device, which may be a blockchain node. The network-connected computing device may be a Merkle path server in some cases. The method 800 may be implemented by way of processor-executable instructions that, when executed by a processor, cause the processor to carry out the described operations. The operations may involve memory access functions, signal or data reception or transmission functions, display operations, and other operations involving components coupled to the processor. The instructions may be embodied in one or more software modules, applications, routines, etc. [0145] The method 800 includes receiving TxID data, as indicated by operation 802. The TxID data may include an ordered set of TxID identifiers. This may include receiving block data relating to a mined block on a blockchain network. It may include receiving an ordered set of transactions from which TxIDs may be calculated.

[0146] In operation 804, the computing device builds a Merkle tree from the TxID data.

Building the Merkle tree may include concatenating pairs of adjacent nodes and hashing the concatenation to determine the hash value of the parent node to that pair, as described above.

[0147] In operation 806, the computing device partitions the Merkle tree into non-overlapping subtrees. Each node in the Merkle tree belongs to one subtree. In one embodiment, the lowest level, level 0, containing the TxID values is not included in the subtrees on the basis that the TitilD values are available from another source. in one embodiment, the root node for the Merkle tree is not included in a subtree, meaning the top-most subtree may have a truncated top.

[0148] In some implementations, operation 806 includes determining the subtree size based on a memory page size. The computing device may include a page-structured memory, like an SSD memory, having a defined page size. The subtree size may be selected such that it is of a maximum size provided that the concatenated hash values of the nodes of the subtree fit within the defined page size.

[0149] In operation 808, the computing device stores each respective subtree within a respective page of the memory.

[0150] Reference is now made to FIG. 9, which shows a simplified example method 900 for responding to a Merkle proof query based on the Merkle tree storage resulting from the method 800 of FIG. 8. The method 900 may be implemented by a network-connected computing device, which may be a blockchain node. The network-connected computing device may be a Merkle path server in some cases. The method 900 may be implemented by way of processor-executable instructions that, when executed by a processor, cause the processor to carry out the described operations. The operations may involve memory access functions, signal or data reception or transmission functions, display operations, and other operations involving components coupled to the processor. The instructions may be embodied in one or more software modules, applications, routines, etc. The terms Merkle proof query" and "Merkle path query" may be used interchangeably in this description.

[0151] In operation 902, the computing device receives a Merkle path query relating to a TxID from a remote device. The Merkle path query may include the TAD in some cases. It may include the full transaction in some cases. It may include a block identifier and a TAD index pointing to the location of the TAD in the ordered set of TADs in some cases. If the query provides the full transaction, then the computing device derives the TxID through hashing (double SHA256 in case of Bitcoin).

[0152] In operation 904, the computing device identifies the block containing the TAD and its index position in the ordered set of TADs. This data may be provided in the Merkle path query, in some cases, and the computing device may simply confirm that the TAD exists in the expected location.

[0153] In operation 906, the computing device identifies and retrieves data from a page of memory storing the subtree containing the TAD. The identification and retrieval of the correct page may be based on the index determined in operation 904. In some cases, an index to the tree may be stored in memory. In some cases, the known size of the tree and the known size of the subtree stored in each respective pages, and the structure or order in which the subtrees making up the tree are stored page-by-page, will make it apparent to the computing device which page to retrieve based on the index of the TxID.

[0154] In operation 908, based on the index of the TxID, computing device proceeds to retrieve each page containing subtree that is a parent to the subtree containing the TAD or that is adjacent and needed to determine the Merkle path for the TxID. Identification of these subtrees (and their corresponding pages) is determined based on the index of the TxID.

[0155] From the retrieved pages, the computing device extracts that hash values for the nodes that enable calculation of the Merkel path in operation 910. That Merkle proof data is then transmitted to the remote device in reply to the Merkle path query in operation 912. The response to the query provides the hash values to enable the remote device to compute the Merkle path from the TxID to the root node to verify the TAD is contained in the mined block.

[0156] It will be appreciated that tracing the Merkle path based on subtree storage involves retrieving a first subtree containing the transaction of interest. That first subtree has a subtree root node at its top level. To trace the path to the parent node, the computing device must also retrieve an adjacent subtree (to left or right, depending on the index of the first subtree) so that it has the hash value of the corresponding paired node to the subtree root node of the first subtree.

[0157] In another aspect of the present application, the subtree-based data structure for storing Merkle tree data may be modified to exploit the excess 32 (or 64) bytes available in paged memory to store the hash value of the paired node to the subtree. This would eliminate the need to retrieve adjacent subtrees in order to trace a Merkle path and will reduce the overall number of page retrieves required. To illustrate, reference is now made to Figure 10, which shows a simplified example Merkle tree 1000 containing 32 TxIDs at level 0. This Merkle tree 1000 has the same structure as the Merkle trees 700 of FIGs. 7A and 7B. The Merkle tree 1000 is partitioned into subtrees containing three levels, i.e. 7 nodes, on the basis of a page size of 256 bytes, just as in FIG. 7B. However, in this case, the page storing each respective subtree also stores the paired root node adjacent to that subtree.

[0158] In particular, a first subtree 1002 contains seven nodes, shown in darker shading, with a first subtree root node 1004. The first subtree root node 1004 has adjacent to it a paired root node at the same level (level 2). The paired root node is a second subtree root node 1006 for a second subtree 1008. Accordingly, in a single 256 byte page of memory, the computing device stores the seven hash values of nodes in the first subtree 1002 and the hash value of the second subtree root node 1006. Likewise, in a second page of memory, the computing device stores the seven hash values of the nodes in the second subtree 1008 and the hash value of the first subtree root node 1004.

[0159] It will be appreciated that the subtree root nodes, such as the first subtree root node 1004 and the second subtree root node 1006, are each stored twice: once in the page containing that subtree of which they are a part and again in the page containing the adjacent subtree with which they are paired. However, given that there were an excess 32 bytes per page when storing a subtree, the repeated storage of the subtree root node does not cost any additional memory.

Moreover, when tracing a Merkle path through the Merkle tree 1000 the computing device does not need to retrieve the adjacent subtree to find the paired subtree root node since it is already stored in the page containing the subtree of interest. In some cases, when storing the first subtree 1002, the computing device may omit the first subtree root node 1004 from the stored hash values of the subtree since it may not be required as part of the hash values for the Merkle proof for TxIDs in the first subtree 1002.

[0160] With this process and structure of Merkle tree data storage, a Merkle path may be traced for the Merkle tree 1000 shown in FIG. 10 in only two page retrievals. The savings in page retrievals may be more significant with larger trees and larger subtree sizes. In one example, a Merkle tree may contain about 4 billion transactions at layer 0 and may have 33 layers. Using 4 kB page sizes holding 128 hashes per page, subtrees of 7 layers may be stored in each page. If the paired adjacent subtree root node is included with each subtree, then a Merkle path may be traced in 4 page retrievals, whereas a traditional layer-by-layer storage structure would need about 27 page retrievals.

[0161] With this data structure, tracing a Merkle path would involve retrieving the lowest level page containing a subtree that includes the transaction-of-interest and then each page containing a parent subtree up to the top level, without needing to retrieve any pages containing adjacent subtrees.

[0162] From the above description it will be appreciated that storing Merkle tree data in a subtree format may lead to advantages in reducing the number of page retrievals for tracing a Merkle path. In some cases with large Merkle trees the number of page retrievals may be reduced by almost an order of magnitude. In some cases, it may be possible to structure the Merkle tree data storage to enable retrieval of Merkle path data with a single page retrieval.

[0163] 'The most extreme option is to store the full Merkle path data for every Tx1D individually. However, this would be prohibitively costly in terms of memory space. For example, if a block of 1TB contains about 4.3 billion transactions, the full Merkle tree with no redundancy can be stored in about 275 GB. However, if we were to store the full Merkle path data (-32 hashes) for each TxID, it would require about 4.3 TB. The excess 4TB are redundant storage of upper nodes of the tree. In fact, the closer to the Merkle tree root, the more repetitively the hash values of those nodes are stored.

[0164] Thus, the proposed page-based subtree-partitioned data structure provides an advantageous balance between the number of page retrievals and the amount of memory required for storage.

Pruning [0165] It will be appreciated from the above description that upper level subtrees will be retrieved more frequently than lower level subtrees, since every query will require retrieval of the top subtree and only queries involving the subset of TxIDs within a specific lowest level subtree (or adjacent if the adjacent subtree root nodes are not stored in each page) will involve retrieve of that subtree. If, over time, the likelihood of a Merkle proof query for a block declines then it may become inefficient to continue to store the full Merkle tree data for that block. Accordingly, the computing device, such as a Merkle tree server, may implement a process for paining the Merkle tree data over time. This may result in additional calculation burden for the server in responding to a Merkle proof query, but will reduce the long-term storage cost.

[0166] Reference is now made to FIG. Ii, which shows an example simplified Merkle tree I 100 after a pruning operation. When originally stored in a Merkle path server, the hash values for all nodes of the Merkle tree 1100 may have been stored in subtree format. That is, the Merkle tree 1100 would have been partitioned into a plurality of subtrees, including a set of subtrees at a lower level of the Merkle tree 1100 and one or more (parent) subtrees at a higher level of the Merkle tree I 100, as described in the examples above. Each subtree may have been stored in a respective page of the page-structured memory. The storage of each subtree may or may not have included its adjacent paired subtree root node or additional paired Merkle proof nodes for determining the Merkle path, as is described elsewhere herein.

[0167] At some trigger point, the computing device, e.g. the Merkle path server, may have determined that the stored Merkle tree data was to be pruned and it deleted at least one portion of subtree data. In particular, the Merlde path server may have deleted a lowest layer of subtrees still stored in memory. In this simplified illustrative example, the Merkle tree 1100 is partitioned into three layers of subtrees, where each subtree contains three nodes. The layers of subtrees may have originally included a set of subtrees containing nodes at layers 0 and 1. That layer of subtrees may have been deleted from memory, leaving only the subtrees containing nodes from layers 2 and 3 and the top subtree containing nodes from layers 4 and 5.

[0168] In some cases, the deleted portion of Merkle tree data may include one or more layers within a set of subtrees. For instance, a layer of subtrees may have within it multiple layers of nodes. The pruning operation may include deleting one or more layers of nodes from within a layer of subtrees. Such an operation may further include consolidating the remaining subtree node data into fewer pages of memory. In some examples, the pruning operation could delete layer 1, or layer I and layer 2, or layers I, 2, and 3, leaving layer 0 available.

[0169] It will be appreciated that if the Merlde path server deletes the lower layers of the subtree data from memory then in order to respond to a Merkle proof query it must calculate the portion of the Mende tree from the TxID of interest up to the remaining lowest layer of subtrees. For this purpose it requires at least some of the TxIDs at layer 0 so as to determine the hashes of the intermediate nodes that provide the Merkle proof along the Merkle path. In one implementation, when deleting the lowest layer of subtrees the Merlde path server also the stores a full ordered list of TxIDs for the block/tree; in other words, it may retain the layer 0 data containing the TxIDs.

[0170] In another implementation, the ordered set of TxID data may he available from another source external to the Merkle path server. For instance, the TxIDs may he available from another blockchain node, such as a full node or a dedicated block-information server.

[0171] In yet another implementation, the requesting node may provide the TxID information. Note that not all TxID's are needed to calculate the lower portion of the Merkle path; only those Txlif s required to complete the path up to the remaining stored subtrees. if, for example, there are five layers of data (including layer 0) missing from the Merl& path server such that the stored set of subtrees begins at layer 5, then the Merkle path server needs 32 Tx1D's to calculate the lower level portion of the Merkle path, e.g. five layers of data = 25 Tx1Ds required. Those TxIDs may be provided by the requesting node with the path query, if the requesting node is aware of the number of layers missing. The requesting node may be aware of the number of layers missing if that data is published by the Merkle path server or if the subtree sizes and pruning schedule is deterministically set by an applicable protocol, for example.

[0172] The Tx1Ds needed will be known by the Merkle path server based on the number of missing lowers layers and the index of the TxID of interest in the path query. Based on that number of missing lower layers and the index, the Merkle path server may respond to a query by requesting from the requesting node the range of TxIDs needed to determine the path data. Or the Merkle path server may request those TxIDs from an external server, such as a full blockchain node or other source of ordered TxID data.

[0173] In another implementation, instead of deleting one or more layers of nodes, the server may selectively delete nodes or entire subtrees based on a history of non-use. That is, for example, if the time since last access of a page exceeds a stale time, then that page of subtree data may be pruned/deleted. The time since last access may indicate the last time that page was read from memory in order to respond to a Merlde proof query.

[0174] Reference is now made to FIG. 12, which shows, in flowchart form, one example method 1200 for storing and pruning Merkle tree data. The method 1200 may be implemented by a network-connected computing device, which may be a blockchain node. The network-connected computing device may be a Merkle path server in some cases. The method 1200 may be implemented by way of processor-executable instructions that, when executed by a processor, cause the processor to carry out the described operations. The operations may involve memory access functions, signal or data reception or transmission functions, display operations, and other operations involving components coupled to the processor. The instructions may be embodied in one or more software modules, applications, routines, etc. [0175] In operation 1202, the Mende path server obtains TxID data for a block, builds the corresponding Merlde tree, partitions the Merkle tree into multiple layers of subtrees, and stores the subtrees in memory. The subtrees may be stored in paged memory, with one subtree stored in each page of memory. The Merkle tree may be associated with a block from a blockchain, and each leaf of the Merkle tree may be a TAD in the ordered set of TxIDs that make up the block.

[0176] At operation I 204, the Merkle path server determines whether to prune the stored Merkle tree data. The determination may be based on a number of possible factors. For example, in one implementation, the pruning may occur based on the age of the block. That is, the Merkle path server may be configured to prune a layer of Merkle tree data if more than a threshold number of days has elapsed since creation of the block. The threshold may be 14 days, a month, 6 months, a year, or any other suitable timeframe. The threshold may be selected based on when the block is likely to be the subject of significantly fewer Merkle proof query requests due to its age. Multiple thresholds may be set that result in progressively pruning further layers of the tree from the bottom up.

[0177] In another implementation, the pruning may occur based on the block height. Block height may be a proxy for age in some blockchain systems where the mining of new blocks is configured to take approximately a set average amount of time, e.g. about 10 minutes in bitcoin. Accordingly, the thresholds may be set based on the block being at a certain depth in the block chain from the current block, e.g. if X thousand blocks are added to the chain above the stored block, then a layer of subtree data may be pruned.

[0178] In yet another implementation, the pruning may occur based on a frequency of Merkle proof requests. That is, if the number of Mende proof requests over a window of time, i.e. an average number of requests, falls below some threshold value, then the Merkle path server may determine that a layer of subtrees may be pruned. Different lower threshold values may he set for pruning successive layers of the stored Mende tree data. For instance, if fewer than N requests are received over the course of (for instance) a month, then the lowest layer may be pnmed. if, after than, fewer than N/2 requests are received over the course of (for instance) a month, then the next lowest layer may be pruned, and so on. The frequency of requests may be Merkle proof request for any TxID in the Merkle tree, or may refer to frequency of requests for a range of TxIDs, such as those in a particular subtree.

[0179] In yet a further implementation, the pruning may occur based on a time of last access of a page exceeding a stale time. That is, determining whether the time since the page was last read in order to reply to a Merkle proof query exceeds a stale time.

[0180] In yet a further implementation, the pruning may be based on a combination of factors, such as age of the block, block height, frequency of proof queries relating to that block, time of last access, and/or storage space availability or limitations at the Merkle path server.

[0181] If the Merkle path server determines in operation 1204 that a pruning operation is to occur, then in operation 1206 it may determine whether the pruning operation relates to a portion of Mende tree data that includes layer 0 containing the TxIDs. If so then it may store the ordered set of TxIDs in memory in operation 1208, or avoid deleting the ordered set of TxIDs, if they are not otherwise available to the Merkle path server.

[0182] In operation I 210, the Merkle path server deletes a portion of the Merkle tree data; e.g. the lowest available layer of subtree data. in operation 1212, it may update an index page, if any.

[0183] When the Merkle path server receives a Merkle proof query from a requesting node, the requesting node may provide a TxID location or index in the ordered set. In some cases, the requesting node provides a copy of the TxID or a copy of the full transaction with the query. The Merkle path server, based on the TxID index and the layer height of the available stored subtree data, identifies the range of TxIDs needed to determine the lower portion of the Merkle path. It then retrieves those TxIDs from its own memory or from an external source and calculates the lower portion of the Merkle tree from those TxIDs. From that calculated lower portion of the Merkle tree and the TxID index, the Merkle path server can identify the hash values that make up the lower portion of the Merkle proof in answer to the query. It then retrieves the pages from memory that store the parent subtrees needed to obtain the Merkle proof data for tracing the upper portion of the Merkle path up to the Merkle root. From those parent subtrees, the Merkle path server can identify the hash values that make up the upper part of the Merkle proof. The server then groups the hash values from the calculated lower portion and the hash values from the retrieved upper portion and generates and sends a response to the MerIde proof query providing those hash values.

[0184] It will be appreciated that pruning of the Merkle tree imposes a greater calculation burden on the Merkle path server, and still requires that it either maintain or retrieve TxID data, but as the block ages the infrequency of queries may make it inefficient to store full Merkle tree data. Pruning of the lower portion of the tree eliminates the storage of the most infrequently accessed portion of the stored Merkle tree data, and may be justified in the case of older lessfrequently-queried Merkle trees.

Single Page Retrieval of Illerkle Path Data [0185] As discussed above, storage of the entire Merkle path for a TxID is prohibitively costly in terms of memory requirements, even if it does lead to a single page retrieval. Nevertheless, it may be possible, if the page sizes permit, to achieve an advantageous balance of memory costs and page retrieval through use of a composite tree/path data structure that enables single page path retrieval. In this aspect of the present application, single page retrieval of a Merkle proof is enabled through storing, in a single page of memory, hash values of a lowest level subtree (or a portion of the subtree including at least the TOD values) and storing upper Merkle path paired hash values for completing the Merkle proof from the top of the subtree to the Merkle root.

[0186] If a given page size accommodates a maximum subtree size of a certain number of layers, the tree may be partitioned so as to create a set of subtrees one layer smaller than the maximum. Consider, for instance, a 7-layer Merkle tree of 64 TODs having 127 total nodes. A 512 byte page has capacity for 16 32-byte hash values, which would accommodate a maximum subtree size of 4 layers (15 total nodes). Instead, the subtrees at the lowest level may be formed using 3 layers (7 total nodes), which leaves sufficient space to store the paired hash values to trace the remainder of the upper portion of the Merkle path. In particular, the subtree would have the hash values for subtree nodes at layers 0, 1, and (possibly) 2, and the page may also then store the hash values for paired nodes in the upper portion of Merkle path at levels 2, 3, 4, and 5. This would enable retrieval of full Merkle proof data for any of the four Tx1Ds in the subtree.

[0187] FIG. 13 illustrates an example of a 32-TxID Merkle tree 1300. In this example, we may presume the page size in memory is 512 bytes, e.g. sufficient room for 16 total 32-byte hashes. We could store a 4 level subtree containing 15 nodes. However, in this example, we store a 3 level subtree 1302 containing 7 nodes, and then also store the hash values of the nodes needed to trace the Merkle path for that subtree up to the root, i.e. the hash values of the paired intermediate nodes, such as node 1304 at level 2, node 1306 at level 3, and node 1308 at level 4.

[0188] This results in a total of 10 hash values stored in the 512 byte page, leaving 6x32 = 192 bytes of wasted space, but permitting the determination of a Merkle path for any TxID in the subtree 1302 based on a single page retrieval. Because the upper portion of the Merkle path is common to all Tx1Ds in the subtree 1302, the path can be determined without needing to retrieve a separate page of data relating to upper subtrees.

[0189] In some cases, the root of the subtree 1302 may be omitted and calculated from the stored level 1 values, which would result in storing only 9 hash values. Taken a further step, the level 1 values may be omitted, such that the system only stores the TxIDs for the subtree 1302 at level 0, relying on the computing device to calculate the subtree 1302 from the TxIDs, and the stored hash values of the paired intermediate nodes for that subtree 1302 are used to enable calculation of the remainder of the path, which would reduce the overall storage to 7 total hash values.

[0190] It will be appreciated that whilst this enables single page retrieval of Merkle path data, the data structure may be wasteful in terms of memory usage efficiency. In the example of FIG. 13, if 10 hash values are stored in a 512 byte page, 192 bytes are unused, which is 37.5% of the memory space. Si

[0191] In another example, a 64 TxID Merkle tree contains 127 nodes and 7 levels. A three-level subtree of 7 nodes requires 4 additional nodes along its upper Merkle path meaning the full subtree and upper Merkle proof data may be stored using 11 hashes. This would leave 5 unused slots in a 512 byte page, or 31.25%. If the subtrees are four-level subtrees containing 15 nodes, then the upper path requires an additional three hash values, for a total of 18 hash values. This necessitates a larger page size of 1024 bytes, and leaves 32-18 = 14 empty 32-byte slots, which is 43.75%.

[0192] In a further example, a 128 TxID Merkle tree contains 255 nodes and 8 levels. A four-level subtree of 15 nodes requires 4 additional nodes to complete its Merkle path, requiring a storage space of a total of I 9 hash values.

[0193] In accordance with another aspect of the present application, it may be possible to both achieve single-page Merkle path retrieval for subtrees and improve on the memory usage efficiency through using a composite Merkle tree data structure. in particular, the storage of Merkle tree data may be structured so as to store a first lower level subtree and its upper path paired nodes hash values together in a memory page with a second lower level subtree of smaller size and the second subtree's upper path paired node hash values. In some cases one or more of the paired hash values may be omitted on the basis that they can be calculated from the stored data. In some cases, one or more of the intermediate hash values from one or both of the subtrees may be omitted on the basis that it can be calculated from the stored data.

[0194] Reference is now made to FIG. 14 which shows a 6 level Merkle tree 1400 with 32 TxIDs at level 0 and a total of 63 nodes. If the memory page size is 512 bytes, then up to 16 hash values may be stored in a page. This could accommodate a four level subtree of 15 nodes; however in this example, the lower level of the Merkle tree is partitioned into subtrees (beginning on the left hand side in this example) of one fewer levels. That is, the partitioning results in a three level first subtree 1402 containing 7 nodes. Calculating the upper portion of the Merkle path requires paired intermediate nodes 1404, 1406, and 1408. Accordingly, storing this subtree and the hash values for paired intermediate nodes for its upper path requires 10 hash values. In a 512 byte page, this leaves the possibility of storing 6 additional hash values.

[0195] In this example, a smaller second subtree 1410 is stored. In this case, the smaller second subtree 1410 is a two-level subtree containing three values. The upper portion of the Merkle path for the two Tx1Ds in this second subtree 1410 requires hash values from the nodes 1412, 1414, 1416, and 1418. Note, however this would necessitate capacity to store seven hash values, but only space for six remains in the page. One of the upper hash values may be omitted if it can be determined from other information already stored in the page. For instance, the hash value for node 1418 is easily calculated from the second subtree 1410 root node, its paired node 1404, and the paired node 1406 to their parent node. Accordingly, the page could omit the value for node 1418, which may be calculated if a Merkle path for one of the TxIDs in the second subtree 1410 is sought.

[0196] It will be noted that in this example, the larger first subtree 1402 is located on the left-hand side of the tree and the smaller subtree 1410 is located on the right-hand side of the tree. A partitioning process may employ a greedy partitioning algorithm to consume nodes on the left hand side of the tree as larger subtrees and to pair them with smaller subtrees beginning on the right hand side of the tree. By consuming the tree from either side, some element of symmetry is maintained. However, in some implementations, the partitioning process for grouping large and small subtrees may group them closer together such that the large and small subtree share at least part of a common upper Merkle tree path. In this manner fewer upper path node hash values may need to be stored. The greedy partitioning mechanism in such a case may, to preserve tree structure and avoid gaps, alternate large-small and small-large as it moves from left to right across the lower part of the Merkle tree. Using the example shown, it may first partition a large 7 node subtree containing TxID indices 0 to 3 and pair it with a small three node subtree containing TAD indices 4 and 5. The small three node subtree leaves another small subtree adjacent to it containing indices 6 and 7, so the next large 7 node subtree may be fanned to contain TxID indices 8-11 and may be paired with the small three node subtree containing TxID indices 6 and 7. In each pairing the two subtrees. Merkle paths will converge at a common intermediate node one or two levels above the large subtree's root node.

[0197] In this example in which the subtree alternate, the subtree sizes may need to be selected such that the number of leaves in the large subtree is exactly twice the number of leaves in the smaller subtree. In this manner the large-small-small-large pattern of consuming leaves of the Merkle tree will consume the equivalent of three-large subtree's number of leaves with each iteration.

[0198] The composite subtree structure is an example of a non-uniform partitioning of the lower part of the Merkle tree. If we consider the Merkle tree as having 2' leaves (TxilDs in the case of bitcoin), then the length in number of hashes of the Merkle proof for one of the TA-Ds is n, where the proof includes the TxTD but excludes the Merkle root.

[0199] As described, the lower part of the tree may be partitioned into subsets or subtrees.

A subset s may be designated Sd where d indicates the size of the subset, specifically that the subset or subtree includes 2" leaves. As an example the first subtree 1402 in FIG. 14 is of size d=2, since it has four leaves. The index i may be used to number the subsets/subtrees from 0, 1, ... in, where in is the total number of subsets/subtrees into which the lower part of the Merkle tree is partitioned. Accordingly, the notation Sd,, may be used to identify a subset/subtree.

[0200] The size, in terms of number of hash values, of storing a composite Merkle proof for TxIDs in subtree Sdj is n -d -1 + 2'1+1. This calculation presumes storage of hash values for all nodes of the subtree plus storage of hash values need to calculate the remaining nodes of the Merkle path from the subtree to the root node. Accordingly, for a tree with 32 TxilDs (e.g. 2" = 32: n = 5) and a subtree of size 22 = 4 (d = 2), the composite tree and path requires storage of size 5 -2 -I + 23 = 10 hash values.

[0201] If we take a large Merkle tree, such as n=32, with approximately 4.3 billion leaves, then the partitioning of the lower level results in the following relationship between subtree size and number of hashes required to store the subtree and its upper Medde path data: d number of leaves Size (number of hashes) I 2 34 2 4 37 3 8 44 4 16 59 32 90 6 64 153 7 128 280 8 256 535 [0202] Based on the above, and taking as an example a memory page size of about 4 kB (specifically 4096 bytes) that can accommodate 128 hashes of 32-bytes each, it will be appreciated that the largest subtree storable in the page is d = 5, i.e. a 32-leaf subtree. This still leaves empty memory space for up to 128-90 = 38 hashes. Accordingly, one could pair the d = 5 subtree with a smaller d = 2 subtree of four total leaves. This results in a total usage of 90+37 = 127 of the 128 available slots in the memory (assuming no common upper Merkle path nodes).

[0203] If, using a greedy algorithm, one begins consuming 32-leaf subtrees from the left-hand side of the tree and paired 4-leaf subtrees from the right-hand side of the tree, then each page of memory consumes 36 leaf nodes of the Merkle tree. Given 232 total number of leaves, this results in 119304647 total pages of memory storing composite trees, plus one additional page to store the remaining small subtree of four leaves. The additional page may further store indexing data. That is, the storage of this example of composite Merkle data would involve 119,304,648 4kB pages, or about 489 GB of memory. Advantageously, the Merkle proof of any TxID may be obtained in a single page retrieval. If we were to store the entire Merkle tree, it would take about 256 GB and would require many (approximately 27) page retrievals to trace a Merkle path.

[0204] In some embodiments, it may be advantageous, depending on the interrelationship between the page size and the subtree sizes, to forego storing one or more of the intermediate nodes of one or both of the subtrees in a composite pairing, and instead rely on the computing device to calculate those intermediate hash values for the missing intermediate node(s) from the available data.

[0205] As an example, reference is now made to FIG. 15, which shows an example Merkle tree 1500 and an illustrative composite subtree. In this example, the Merkle tree contains 32 TxIDs (leaves) meaning that n=5. A first subtree 1502 is formed with d = 2, which features seven nodes including the four leaf nodes; however, in this example the hash value for the root node 1504 of the subtree is not stored. Instead, the computing device can compute that value using the stored values of nodes 1506 and 1508. Similarly, a smaller second subtree 1510 is formed with d = 1, which features three nodes including two leaf nodes; however, the root node 1512 is not included since it can be calculated from the two leaf nodes in the second subtree 1510.

[0206] This example, in which the root nodes of the suhtrees are omitted, saves two hash values, meaning the composite Merkle path data may be stored using I 5 hash values instead of the 17 hash values shown in FIG. 14, for instance. This would advantageously permit storage of this composite subtree structure in a 512 byte page, which would only accommodate up to 16 hash values. In yet a further refinement, the hash values for nodes 1520 and 1522 may be omitted in some implementations since those values can he calculated from the values below them. This would bring the storage cost down to 13 total hash values. Similarly, further internal nodes of the suhtrees may he omitted, such as nodes 1506 and 1508 to save yet further storage space, bringing the storage cost down to 11 total hash values. It will be appreciated that at this extreme, the memory only stores leaf nodes of the composite subtrees and the hash values of any intermediate nodes for paired nodes along the upper portion of the Merkle paths that cannot otherwise be calculated from the stored values.

[0207] In any implementation, a balance may be struck between the number of internal node hash values stored and the number of hash calculations imposed on the computing device, depending on the implementation and the page size restrictions. Different depths of Merkle trees and different page size restrictions may impose interrelated constraints for which an optimal combination of subtree sizes and stored vs. calculated internal nodes may be determined. An optimization expression may be utilized that maximizes memory page usage efficiency (i.e. percentage of page used).

[0208] Reference will now be made to FIG. 16, which shows, in flowchart form, a simplified example method 1600 for storing Merkle tree data in composite subtree form. The method 1600 may be implemented by a network-connected computing device, which may be a blockchain node. The network-connected computing device may be a Merkle path server in some cases. The method 1600 may be implemented by way of processor-executable instructions that, when executed by a processor, cause the processor to carry out the described operations. The operations may involve memory access functions, signal or data reception or transmission functions, display operations, and other operations involving components coupled to the processor. The instructions may be embodied in one or more software modules, applications, routines, etc. [0209] The method 1600 may include receiving the TxID data for a block and building the corresponding Merkle tree through calculation of all of the intermediate hash values to arrive at the Merkle root, as indicated by operation 1602. The Merkle path server may obtain the ordered set of TxIDs from a blockchain node in some cases. In some instances, the Merkle path server may obtain raw transaction data for the block and may calculate the TxIDs from the transaction data. In some cases, the Merkle path server may further verify its calculation of the Merkle tree by comparing the calculated Merkle root to the Merkle root value specified in the block header.

102101 In operation 1604, the Merkle path server may determine the composite subtree size. That is, the Merkle path server may determine the size of the large subtree and the size of the small subtree making up the composite subtree data structure. The selection of the composite subtree size may be prescribed by a governing protocol in some cases. The Merkle path server may have a fixed composite subtree size based on its memory page size. The Mende path server may be configured to select a composite subtree size based on the depth of the Merkle tree and the its memory page size in some cases. In selecting a subtree size for the large and small subtrees, the Merkle path server may be further configured to determine which intermediate nodes of the subtrees and/or the upper Merkle paths from the subtrees may be omitted from the composite subtree data structure. An optimization expression that takes into account page size and Merlde tree depth may be used to determine the composite subtree size and data structure in some cases. Various constraints may he placed on the selection, such as, for example, a minimum memory page usage percentage, a maximum number or percentage of intermediate node calculations per query, etc. [0211] In operation 1606, the Merlde path server partitions the lower portion of the Merkle tree based on the determined composite subtree size. In some implementations, the partitioning is done on a greedy basis beginning on both sides of the tree, with the large subtree consuming nodes from one side and the small tree consuming nodes from the other side. In some implementations, the partitioning is done on a greedy basis from one side of the tree with both subtree sizes. In such as case, if the sizing of subtrees (in terms of number of leaf nodes) is such that the large tree has twice the leaves as the small tree, then the partitioning may occur in the order large-small-smalllarge across the base of the Merkle tree. If the sizing is such that the large tree has three times the leaves as the small tree, then the partitioning may occur in the order large-small-small-small-largelarge (or some variation thereof that groups three small subtree together, such as large-large-largesmall-small-small).Extensions of the concept to other proportions will be understood by those skilled in the art.

[0212] The number of leaves of the Merkle tree may not by evenly partitioned such that the last set of leaves (TxiDs) is less than the composite subtree size, in which case the final partition may be a subtree of sufficient size to contain the remaining leaves.

[0213] The composite subtrees are then each stored in a respective page of memory in operation 1608, together with the hash values of the intermediate nodes in the upper portion of the Merlde tree needed to trace a Merkle path from the large and small subtree up to the Merkle root. In some cases, the Merkle path from the large subtree may converge with the Merkle path from the small subtree, thereby reducing the total number of hash values from the upper paths that need to be stored due to the duplication. In some cases, the Merkle path server may have determined in operation 1604 that an intermediate node hash value needed for the Merkle proof of one of the subtrees may be obtained from the stored data, such as data relating to the other subtree, in which case the composite subtree data structure may exclude that intermediate node hash value in expectation that it can be calculated from the stored data if needed.

[0214] In response to a path query from a requesting node, the Merkle path server identifies the relevant page of memory storing the composite subtree containing the TxID that is the subject of the query based on the Tx1D index. It retrieves that page of memory to read the hash values of the composite subtree. The Merlde path server makes any calculations required to fill in any needed and missing intermediate node hash values (if any) to trace the Merkle path for the TxID of interest, and then extracts the hash values providing the Merkle path proof for the TxID of interest. A response is then sent to the requesting node providing the hash values of the Merkle proof [0215] The various embodiments presented above are merely examples and are in no way meant to limit the scope of this application. Variations of the innovations described herein will be apparent to persons of ordinary skill in the art, such variations being within the intended scope of the present application. In particular, features from one or more of the above-described example embodiments may be selected to create alternative example embodiments including a sub-combination of features which may not be explicitly described above. In addition, features from one or more of the above-described example embodiments may be selected and combined to create alternative example embodiments including a combination of features which may not be explicitly described above. Features suitable for such combinations and sub-combinations would be readily apparent to persons skilled in the art upon review of the present application as a whole. The subject matter described herein and in the recited claims intends to cover and embrace all suitable changes in technology.

Claims

WHAT 15 CLAIMED IS: 1. A computer-implemented method of storing Merkle tree data in a page-structured memory, the page-structured memory having a defined page size for storage and retrieval of data, the method comprising: storing, in the page-structure memory, hash values for a Merkle tree constructed from an ordered set of transaction identifiers in a subtree-based format, wherein the Merkle tree is partitioned into a plurality of subtrees including a plurality of upper subtrees and a plurality of lower subtrees; determining that the stored Merkle tree data is to be pruned; and deleting at least a portion of the plurality of lower subtrees from the page-structured memory.
The method of claim 1, wherein the ordered set of transaction identifiers correspond to a block in a blockchain.
3. The method of claim 2, wherein determining includes determining that a time elapsed since creation of the block has exceeded a threshold value.
4. The method of claim 2 or claim 3, wherein determining includes determining that a block height of the blockchain above the block exceeds a threshold height.
5. The method of any one of claims 1 to 4, wherein determining includes determining that a frequency of Merkle proof requests over a time window is lower than a minimum threshold number.
6. The method of any one of claims 1 to 4, wherein determining includes determining that a time since last access of the portion of the plurality of lower subtrees to reply to a Merkle proof query exceeds a stale time.
7. The method of any one of claims Ito 6, wherein deleting at least the portion of the plurality of lower subtrees includes determining whether the lower subtrees include the ordered set of transaction identifiers and, if so, then storing the ordered set of transaction identifier in the page-structured memory.
8. The method of any one of claims 1 to 7, wherein the storing includes storing each subtree in a respective page in the page-structured memory, and wherein deleting includes deleting data from pages containing the portion of the plurality of lower subtrees.
9. The method of any one of claims 1 to 8, wherein the portion of the plurality of lower subtrees includes a lowest layer of subtrees stored for the Merkle tree.
10. The method of any one of claims 1 to 9, further comprising: receiving a Merkle proof query relating to a specified transaction identifier; identifying, based on a layer height of a lowest layer of the stored Merkle tree, a range of transaction identifiers needed to build a lower Merkle path to the lowest layer: obtaining the transaction identifiers in the range; and constructing, from the obtained transaction identifiers, a portion of the Merkle tree to obtain hash values for a partial Merkle proof along the lower Merkle path.
11. The method of claim 10, further comprising: retrieving, from the page-structured memory, one or more pages of hash values for a remaining Mende proof for the specified transaction identifier along an upper Merkle path; and sending a response containing hash values providing the Merkle proof.
12. The method of claim 10 or claim 11, wherein obtaining the transaction identifiers in the range includes obtaining the transaction identifiers in the range from a blockchain node.
13. The method of claim 10 or claim 11, wherein obtaining the transaction identifiers in the range includes obtaining the transaction identifiers from the page-structured memory.
14. A computing device, the computing device including: one or more processors; memory; computer-executable instructions stored in the memory that, when executed by the one or more processors, cause the processors to carry out the method claimed in any one of claims 1 to 13.
15. A computer-readable medium storing processor-executable instructions, the processor-executable instructions including instructions that, when executed by one or more processors, cause the processors to carry out the method claimed in any one of claims I to 13.