WO2017109140A1

WO2017109140A1 - Decentralized, tamper-resistant, asset-oriented database system and method of recording a transaction

Info

Publication number: WO2017109140A1
Application number: PCT/EP2016/082494
Authority: WO
Inventors: Trent Lorne Mcconaghy; Rodolphe MARQUES; Andreas MÜLLER
Original assignee: Bigchaindb Gmbh
Priority date: 2015-12-22
Filing date: 2016-12-22
Publication date: 2017-06-29

Abstract

A blockchain database, which combines the benefits of traditional distributed databases with new benefits from blockchain technology. It builds on a traditional distributed database that has scalable read/write throughput, scalable capacity, a query language, efficient querying, and permissioning. It adds in new "blockchain" benefits of decentralized shared control, tamper-resistance, and creation & movement of digital assets.

Description

DECENTRALIZED. TAMPER-RESISTANT. ASSET-ORIENTED

DATABASE SYSTEM AND METHOD OF RECORDING A TRANSACTION FIELD

[0001] The present disclosure relates generally to blockchains. More particularly, the present disclosure relates to blockchain technology and databases.

BACKGROUND

[0002] The introduction of Bitcoin has triggered a new wave of decentralization in computing. Bitcoin illustrated a novel set of benefits: decentralized control, where "no one" owns or controls the network; immutability, where written data is tamper-resistant ("forever"); and the ability to create & transfer assets on the network, without reliance on a central entity.

[0003] The initial excitement surrounding Bitcoin stemmed from its use as a token of value, for example as an alternative to government-issued currencies. As people learned more about the underlying blockchain technology, they extended the scope of the technology itself (e.g. smart contracts), as well as applications (e.g. intellectual property). With this increase in scope, single monolithic "blockchain" technologies are being re- framed and refactored into building blocks at four levels of the stack:

1 . Applications

2. Decentralized computing platforms ("blockchain platforms")

3. Decentralized processing ("smart contracts") and decentralized storage (file systems, databases), and decentralized communication

4. Cryptographic primitives, consensus protocols, and other algorithms

[0004] We can frame a traditional blockchain as a database (DB), in the sense that it provides a storage mechanism. If we measure the Bitcoin blockchain by traditional DB criteria, it's terrible: throughput is just a few transactions per second (tps), latency before a single confirmed write is 10 minutes, and capacity is a few dozen GB.

Furthermore, adding nodes causes more problems: with a doubling of nodes, network traffic quadruples with no improvement in throughput, latency, or capacity. It also has essentially no querying abilities: a NoQL1 database.

[0005] Therefore, improvements in blockchain technology are desirable. SUMMARY

[0006] In a first aspect, the present disclosure provides a computer-implemented method of recording a transaction, the transaction being an electronic transaction. The method comprising: at a node of a distributed database comprising a plurality of nodes, the plurality of nodes including voting nodes: placing the transaction in a backlog containing other transactions, the backlog being the same for all the nodes of the plurality of nodes; forming a block of transactions from at least one of the transactions in the backlog, the block of transactions being in an undecided state when formed; writing the block of transactions at an end of a blockchain; at each voting node, after the block of transactions has been written to the blockchain, voting on a validity of the block of transactions; and writing the votes to the distributed database.

[0007] In some embodiments, the undecided state changes to a decided valid state when the block of transactions has been determined valid by a quorum of votes of the voting nodes; and the undecided state changes to a decided invalid state when the block of transactions has been determined invalid by a quorum of votes of the voting nodes.

[0008] In some embodiments, the backlog is a first database table and the blockchain is a second database table.

[0009] In some embodiments, the backlog is first database container and the blockchain is a second database container.

[0010] In some embodiments, the votes are written in the blockchain.

[0011 ] In some embodiments, the votes are written in one of a database table or a container, the database table and the container being distinct from the blockchain.

[0012] In some embodiments, the node in which transaction is placed in the backlog is a selected node, the computer-implemented method further comprising receiving the transaction at any one of the plurality of nodes of the decentralized database and assigning the transaction to the selected node.

[0013] In some embodiments, assigning the transaction to the selected node is preceded by randomly selecting, from the plurality of nodes, the selected node as the node in which to place the transaction.

[0014] In some embodiments, the transaction placed in the selected node is a permissible transaction; and placing the transaction in the backlog is preceded by determining that the transaction is permissible.

[0015] In some embodiments, determining that the transaction is permissible includes: at least one of determining if the transaction is in a pre-defined format and determining if the transaction is a double-spend transaction, the transaction being permissible when in the pre-defined format and when it is not a double-spend transaction.

[0016] In some embodiments, the method further comprises: when the transaction is a double-spend transaction or when the transaction in not in a pre-defined format, rejecting the transaction.

[0017] In some embodiments, forming the block of transactions includes:

determining, one transaction at a time and in an order of the transactions listed in the backlog, a validity of the transactions in the backlog; and forming the block with valid transactions.

[0018] In some embodiments, writing the block of transactions at the end of the blockchain is preceded by determining if the block meets a pre-determined criteria for being written to the blockchain.

[0019] In some embodiments, determining if the block meets the pre-determined criteria for being written to the blockchain includes at least one of: determining a number of transactions in the block; and determining a time lapse from the time a last transaction was placed in the block, the block being ready for being written to the blockchain when the number of transactions in the in the blocks meets a pre-determined number criteria or, when the time lapse meets a pre-determined time lapse criteria.

[0020] In some embodiments, the method further comprises, when the block of transactions has been determined invalid by the quorum of votes of the voting nodes: changing the state of the block of transactions from the undecided state to a decided invalid state; returning instances of the transactions of the block of transactions to the backlog for processing; and maintaining the block of transactions in the blockchain.

[0021] In some embodiments, the transaction and the other transactions comprised in the backlog each have a timestamp; and placing the transaction in the backlog includes placing the transaction at a last entry position in the backlog, the other transactions in the backlog each having an earlier timestamp than the transaction at the last entry position.

[0022] In some embodiments, voting on a validity of the block of transactions includes determining if the transactions depend from a transaction present in an undecided block.

[0023] In some embodiments, wherein voting on the validity of the block of transactions includes voting 'valid' when the block depends from a decided block and when each transaction in the block of transactions is a valid transaction and otherwise, voting 'invalid'. [0024] In further aspect, the present disclosure provides a decentralized database system comprising: a distributed database having processors and nodes, the nodes being interconnected to each other, the nodes including voting nodes; a computer-readable medium having recorded thereon instructions to be executed by the processors to carry out a method of recording a transaction, the transaction being an electronic transaction, the method comprising: at a node of the distributed database: placing the transaction in a backlog containing other transactions, the backlog being the visible to all the nodes; forming a block of transactions from at least one of the transactions in the backlog, the block of transactions being in an undecided state when formed; writing the block of transactions at an end of a blockchain; at each voting node, after the block of transactions has been written to the blockchain, voting on a validity of the block of transactions; and writing the votes to the distributed database.

[0025] In some embodiments, the undecided state changes to a decided valid state when the block of transactions has been determined valid by a quorum of votes of the voting nodes; and the the undecided state changes to a decided invalid state when the block of transactions has been determined invalid by a quorum of votes of the voting nodes.

[0026] In some embodiments, the backlog is a first database table and the blockchain is a second database table.

[0027] In some embodiments, the backlog is first database container and the blockchain is a second database container.

[0028] In some embodiments, the votes are written in the blockchain.

[0029] In some embodiments, the votes are written in one of a database table or a container, the database table and the container being distinct from the blockchain.

[0030] In some embodiments, the node at which the transaction is placed in the backlog is a selected node, the method further comprising receiving the transaction at any one of the nodes of the decentralized database and assigning the transaction to the selected node.

[0031] In some embodiments, assigning the transaction to the selected node is preceded by randomly selecting, from the plurality of nodes, the selected node as the node in which to place the transaction.

[0032] In some embodiments, the transaction placed in the selected node is a permissible transaction; and placing the transaction in the backlog is preceded by determining that the transaction is permissible. [0033] In some embodiments, determining that the transaction is permissible includes: at least one of determining if the transaction is in a pre-defined format and determining if the transaction is a double-spend transaction, the transaction being permissible when in the pre-defined format and when it is not a double-spend transaction.

[0034] In some embodiments, the method further comprises: when the transaction is a double-spend transaction or when the transaction in not in a the pre-defined format, rejecting the transaction.

[0035] In some embodiments, forming the block of transactions includes:

[0036] In some embodiments, writing the block of transactions at the end of the blockchain is preceded by determining if the block meets a pre-determined criteria for being written to the blockchain.

[0037] In some embodiments, determining if the block meets the pre-determined criteria for being written to the blockchain includes at least one of: determining a number of transactions in the block; and determining a time lapse from the time a last transaction was placed in the block, the block being ready for being written to the blockchain when the number of transactions in the in the blocks meets a pre-determined number criteria or, when the time lapse meets a pre-determined time lapse criteria.

[0038] In some embodiments, the method further comprises, when the block of transactions has been determined invalid by the quorum of votes of the voting nodes: changing the state of the block of transactions from the undecided state to a decided invalid state; returning instances of the transactions of the block of transactions to the backlog for processing; and maintaining the block of transactions in the blockchain.

[0039] In some embodiments, the transaction and the other transactions comprised in the backlog each have a timestamp; and placing the transaction in the backlog includes placing the transaction at a last entry position in the backlog, the other transactions in the backlog each having an earlier timestamp than the transaction at the last entry position.

[0040] In some embodiments, voting on a validity of the block of transactions includes determining if the transactions depend from a transaction present in an undecided block.

[0041] In some embodiments, voting on the validity of the block of transactions includes voting 'valid' when the block depends from a decided block and when each transaction in the block of transactions is a valid transaction and otherwise, voting 'invalid'.

[0042] In a third aspect, there is provided a distributed database system comprising: a distributed database having processors and nodes, the nodes being interconnected to each other, the nodes including voting nodes; a computer-readable medium having recorded thereon instructions to be executed by the processors to carry out: a method of recording the transaction, comprising: at a node of the distributed database: placing the transaction in a backlog containing other transactions, the backlog being the visible to all the nodes; forming a block of transactions from at least one of the transactions in the backlog, the block of transactions being in an undecided state when formed; writing the block of transactions at an end of a blockchain; at each voting node, after the block of transactions has been written to the blockchain, voting on a validity of the block of transactions, the voting at each voting node to decentralize a control of the distributed database; cryptographically signing the transaction, the block of transactions, and the votes, in order to provide tamper-resistance to the distributed database system; and writing the votes to the distributed database; wherein the transaction is one a 'creation transaction' to create an asset and a 'transfer transaction' to transfer the asset.

[0043] Other aspects and features of the present disclosure will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments in conjunction with the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Embodiments of the present disclosure will now be described, by way of example only, with reference to the attached Figures.

[0045] Fig. 1 shows an embodiment of the blockchain database system in accordance with the present disclosure.

[0046] Fig. 2 shows the architecture of an embodiment of the blockchain database of the present disclosure.

[0047] Fig. 3 shows another embodiment of the architecture of the blockchain database of the present disclosure.

[0048] Fig. 4 shows different stage of transaction processing in accordance with the present disclosure.

[0049] Fig. 5 shows how the blockchain database of the present disclosure stores invalid blocks in accordance with some embodiments of the present disclosure. [0050] Fig. 6 shows an example of how multiple machines are configured in accordance with the present disclosure.

[0051] Fig. 7 shows an example of a blockchain in accordance with an

embodiment of the present disclosure.

[0052] Fig. 8 shows how the total capacity of the blockchain database system of the present disclosure depends on the number of nodes.

[0053] Fig. 9 shows a flowchart of an embodiment of a method of writing transactions to a blockchain, in accordance with the present disclosure

[0054] Fig. 10 shows an embodiment of a blockchain database of the present disclosure where the votes are stored in a table (or container) distinct from the blockchain table (or container).

DETAILED DESCRIPTION

[0055] Rather than trying to scale up blockchain technology, the blockchain database of the present disclosure starts with a "big data" distributed database (DB), and adds blockchain characteristics. This avoids the technology choices that plague Bitcoin, such as full replication.

[0056] The blockchain DB of the present disclosure is built on top of an enterprise-grade distributed DB, from which the present blockchain DB inherits high throughput, high capacity, a full-featured NoSQL query language, efficient querying, and permissioning. Nodes can be added to the network underlying the blockchain DB to increase throughput and capacity.

[0057] Since the distributed DB, which can be referred as a big data DB, has its own built-in consensus algorithm to tolerate benign faults, the present blockchain DB exploits that solution directly and lets the distributed DB consensus algorithm decide which transactions to write, and what the block order is. Private, peer-to-peer

communication is permitted between the nodes except via the DB's built-in

communication, for great savings in complexity and for reduced security risk. This has for effect that malicious nodes cannot transmit one message to part of the network and different message to other parts of the network. Every time a node "speaks," all the other nodes can listen.

[0058] In the context of the present disclosure, a node is machine or set of closely-linked machines running a Big Data database Server (e.g. MongoDB or

RethinkDB), a blockchain server in accordance with the present disclosure, and the related software. (A "machine" might be a bare-metal server, a virtual machine or a container.) Each node is controlled by one person or organization.

[0059] A set of nodes can connect to each other to form a cluster. Each node in the cluster runs the same software. A cluster contains one logical datastore. A cluster may have additional machines to do things such as cluster monitoring.

[0060] The people and organizations that run the nodes in a cluster belong to a federation (i.e. another organization). A federation must have some sort of governance structure to make decisions. If a cluster is run by a single company, then the federation is just that company. A cluster is just a bunch of connected nodes. A federation is an organization which has a cluster, and where each node in the cluster has a different operator.

[0061] The blockchain community often describes blockchains as "immutable." If we interpret that word literally, it means that blockchain data is unchangeable or permanent. However, in the context of the present disclosure, we interpret the word "immutable" to mean tamper-resistant.

[0062] The blockchain database of the present disclosure can achieve strong tamper-resistance in the following ways:

1 . Replication. All data is sharded and shards are replicated in several (different) places. The replication factor can be set by the federation. The higher the replication factor, the more difficult it becomes to change or delete all replicas.

2. Internal watchdogs. All nodes monitor all changes and if some unallowed change happens, then appropriate action is taken. For example, if a valid block is deleted, then it is put back.

3. External watchdogs. Federations may opt to have trusted third-parties to monitor and audit their data, looking for irregularities. For federations with publicly- readable data, the public can act as an auditor.

4. Cryptographic signatures are used throughout the blockchain DB of the present disclosure as a way to check if messages (transactions, blocks and votes) have been tampered with enroute, and as a way to verify who signed the messages. Each block is signed by the node that created it. Each vote is signed by the node that cast it. A creation transaction is signed by the node that created it, although there are plans to improve that by adding signatures from the sending client and multiple nodes. Transfer transactions can contain multiple fulfillments (one per asset transferred). Each fulfillment will typically contain one or more signatures from the owners (i.e. the owners before the transfer). 5. Full or partial backups of the database may be recorded from time to time, possibly on magnetic tape storage, other blockchains, printouts, etc.

6. Strong security. Node owners can adopt and enforce strong security policies.

7. Node diversity. Diversity makes it so that no one thing (e.g. natural disaster or operating system bug) can compromise enough of the nodes.

[0063] High-Level view

[0064] The blockchain DB of the present disclosure has the following features:

1 . Decentralized control, where "no one" owns or controls a network; 2. Immutability, where written data is tamper-resistant; and

3. The ability to create & transfer assets on the network, without reliance on a central entity.

[0065] Decentralized control is achieved via a DNS-like federation (DNS: Domain

Name System) of nodes with voting permissions. Other nodes can connect to read and propose transactions; this makes it a super-peer P2P network (P2P: Peer-to-peer). The voting operates at a layer above the DB's built-in consensus. Quorum is a majority of votes or any other pre-agree portion/number of votes. For speed, each block is written before a quorum of nodes validates and votes on the block. Chainification actually happens at voting time. Every block has an id equal to the hash of its transactions, timestamp, voters list and public key of its creator-node. It also has a cryptographic signature and a list of votes. A block doesn't include the hash (id) of the previous block when it first gets written. Instead, votes get appended to the block over time, and each vote has a "previous block" attribute equal to the hash (id) of the block coming before it. Immutability / tamper-resistance is achieved via several mechanisms, including: shard replication, reversion of disallowed updates or deletes, regular database backups, and cryptographic signing of all transactions, blocks and votes. Any entity with asset-issuance permissions can issue an asset; an asset can only be acquired by new owners if they fulfill its cryptographic conditions. This means hackers or compromised system admins cannot arbitrarily change data, and there is no single-point-of-failure risk.

[0066] Architecture

[0067] Figure 1 shows an embodiment of the blockchain DB system 50 of the present disclosure. The blockchain DB system 50 has server nodes 51 to which clients can connect. The blockchain DB system 50 further has nodes (network nodes) 55 that define a decentralized blockchain database. The decentralized blockchain database has two tables (database tables) defined therein. One of the tables is a transaction table, the other table is a blockchain table. The following describes these in more detail.

[0068] Figure 2 shows the architecture of an embodiment of the blockchain DB system 50 in accordance with the present disclosure. The blockchain DB system 50 presents its application programming interface (API) to clients connected to the server nodes 51 of Figure 1 as if it is a single blockchain database. However, under the hood, there are actually two distributed databases, S (transaction set or "backlog") shown at reference numeral 52 and C (blockchain) shown at reference numeral 54, connected by a Blockchain DB Consensus Algorithm (BCA). The databases S 52 and C 54 are defined in the nodes 55 of Figure 1. The BCA runs on each signing node (the signing nodes are can be some or all of the server nodes 51 ) underlying the blockchain database. Non-signing nodes may connect to the blockchain DB; depending on permissions they may be able to read, issue assets, transfer assets, and more.

[0069] Each of the distributed DBs S 52 and C 54 is an off-the-shelf big data DB. There is no need to interfere with the internal workings of each DB; in this way, it is possible to leverage the scalability properties of the DBs, in addition to features like revision control and benefits like battle-tested code. Each DB is running its own internal Paxos-like consensus algorithm for consistency among the drives.

[0070] The DB S 52 holds the "backlog" transactions, which is an unordered set of transactions S. When a transaction comes in, it gets validated by the receiving node (one of the server nodes 51 of Figure 1 ) and if it is valid (according to that node), then it gets stored in S 52. (Identical transactions arriving later will be rejected.) The receiving node (one of the nodes 55 of Figure 1 ) is configured to randomly assign the valid transaction to one of the other nodes for that other node to further process the transaction.

[0071] There are N signing nodes. Sk = {tk,i, tk,2, . . . } is the set of transactions assigned to node k. With reference to Figure 1 , the signing nodes are the server nodes 51 .

[0072] Node k runs the (BCA) and processes transactions from S as follows: It moves transactions from the unordered set Sk into an ordered list, creates a block for the transactions, and puts the block into the second database C 54. C is an ordered list of blocks where each block has reference to a parent block and its data, that is, a blockchain.

[0073] A signing node (a server node 51 ) can vote on whether it considers a block valid or invalid. To decide, the signing node checks the validity of every transaction in the block, and if it finds an invalid transaction, then the signing node votes that the block is invalid. If the signing node finds no invalid transactions, then it votes that the block is valid.

[0074] Each block starts out as undecided, with no votes from signing nodes.

Once there is majority of positive (valid) votes for a block, or a majority of negative (invalid) votes, the block goes from undecided to decided valid or decided invalid, respectively, and voting on the block stops. Once it is decided, it can be treated as "etched into stone."

[0075] A block B in the blockchain C 54 has an ID, timestamp, the actual transactions, and vote information. Each of these is described in more detail elsewhere in this document.

[0076] Behavior

[0077] The flow of transactions from a client to a given server node is now described. Each server node (for example, server node 51 in Figure 1 ) has its own view of the transaction backlog S 52 and of the blockchain C 54.

[0078] Figure 3 and subsequent figures illustrate the high-level architecture where each card 56 is a physical machine. The client machines 58 are on the left. Clients are connected to the Blockchain server node(s) 51 , shown on the right. Any client (client machine 58) can send transactions to any blockchain DB server node. The server nodes

51 are voting nodes. As shown at Figure 3, at the middle section, the backlog S 52 starts empty and the block chain C 54 starts with only a genesis block 62. When a client submits a transaction, the receiving node assigns it to one of the server nodes 51 (Figure 1 ), possibly itself, and stores it in the backlog S 52. Subsequently, as shown at the right of Figure 3, the clients (client machines 58) have inserted transactions into the backlog S

52 and, in this example, the transactions have been assigned to nodes Si , S2 and S3. In this example, Si has been assigned transaction #A, #G and #H; S2 the transactions #D and #C; and S3 the transactions #B and #E. Any client can send transactions to any blockchain DB server node 51 . The transaction #A has been requested by one of the clients 58 shown in the Figure. Nothing has yet been stored on the chain C yet (besides the genesis block).

[0079] Figure 4 left shows a state where node S1 has processed all the transactions assigned to it. Si has taken the transactions #A, #G, and #H from the backlog S 52, created a block to hold 64 them, then written the block 64 onto the blockchain C 54. The block 64 points to the previous block, which, in this case, is the genesis block 62. [0080] Figure 4, at the right shows that S3 has processed all of its assigned transactions too, and written them as a block 66 in chain C 54.

[0081] When a block is first written to C 54, it starts as undecided; that is, the block is in an undecided state. Each server node 51 (Figure 1 ) may vote positively (for) or negatively (against) for each block. A block should only be voted positively if all previous blocks are not undecided (i.e., when they are not in the undecided state), and all transactions in the block itself are valid. As soon as there is a majority of positive votes for a block, or a majority of negative votes, the block goes from the undecided state to a decided valid state or a decided invalid state, respectively.

[0082] In this example, the block created by Si gets voted on, and becomes decided valid. Then, the block from S3 gets voted on, and becomes decided invalid. In Figure 4 right, we depict the distinction as a clear background for decided valid, versus a shaded background for decided invalid.)

[0083] While the overall block 66 was considered invalid, some of the transactions in the invalid block may have actually been valid, and so, the blockchain DB of the present disclosure gives them another chance. Figure 5 shows how: transactions #B and #E get re-inserted into backlog S 52 for new consideration. Figure 5 also shows how the blockchain DB approaches storage of invalid blocks. There's a block in the chain C 54 that is invalid (block 66). However, the blockchain DB of the present disclosure doesn't remove the block. There is no need to remove the block 66, as the block 66 is already marked invalid (decided invalid state), disk space is not a problem, and it is faster and simpler to keep all blocks there in C 54. Similarly, voting doesn't stop after a block becomes decided, because it's faster for each node to simply vote than the extra step to check whether voting is necessary.

[0084] Figure 6 shows how multiple machines are configured. Figure 6 left shows that more than one client may talk to a given node. In this example, at Figure 6 left, clients 1 , 2 and x all talk to the same node. This node assign the transaction of Client 1 (transaction #A) to Si, the transaction of Client 2 (transaction #B) to S3, and the transaction of client x (transaction #D) to S2. Figure 6 right shows that there is more than one node, though each node has a view into the backlog S 52 and the chain C 54.

[0085] Transaction Model

[0086] Transactions are the most basic kind of record stored by the blockchain

DB of the present disclosure. There are two kinds of transactions: creation transactions and transfer transactions. A creation transaction initiates the records of an asset in the blockchain DB, including some description of what it is, a list of its initial owners, and the conditions that must be fulfilled by anyone wishing to transfer it. A transfer transaction transfers ownership of the asset to new owners, and assigns new spending conditions.

[0087] In the blockchain DB of the present disclosure, a transaction can be represented by a JSON document with the following structure:

"id": "<hash of transaction , excluding signatures >",

"version": "<version number of the transaction model >",

"transaction": {

"fulfillments": ["<list of fulfillments >"],

"conditions": ["<list of conditions >"],

"operation": "<string >",

"timestamp": "<timestamp from client >",

"data": {

"hash": "<hash of payload >",

"payload": "<any JSON document >"

}

• id: The hash of everything inside the serialized transaction body (see below), with one wrinkle: for each fulfillment in fulfillments, fulfillment is set to null. The id is also the database primary key.

• version: Version number of the transaction model, so that software can support different transaction models.

• fulfillments: List of fulfillments. Each fulfillment contains a pointer to an unspent asset and a crypto-fulfillment that satisfies a spending condition set on the unspent asset. A fulfillment is usually a signature proving the ownership of the asset.

• conditions: List of conditions. Each condition is a crypto-condition that needs to be fulfilled by a transfer transaction in order to transfer ownership to new owners. • operation: String representation of the operation being performed (currently either

"CREATE" or "TRANSFER"). It determines how the transaction should be validated.

• timestamp: Time of creation of the transaction in UTC. It's provided by the client.

• hash: The hash of the serialized payload. · payload: Can be any JSON document. It may be empty in the case of a transfer transaction.

[0088] Full explanations of transactions, conditions and fulfillments are beyond the scope of this paper but are available online.

[0089] Block Model

[0090] A block can be represented by a JSON document with the following structure:

"id": "<hash of block >",

"block": {

"timestamp": "<block -creation timestamp >",

"transactions": ["<list of transactions >"],

"node_pubkey": "<public key of the node creating the block >", "voters": ["<list of federation nodes public keys >"]

}.

"signature": "<signature of block >",

"votes": ["<list of votes >"]

• id: The hash of the serialized block. This is also a database primary key; that's how we ensure that all blocks are unique.

• block:

- timestamp: Timestamp when the block was created. It's provided by the node that created the block.

- transactions: A list of the transactions included in the block.

- node pubkey: The public key of the node that created the block.

- voters: A list of public keys of federation nodes. Since the size of the federation

may change over time, this will tell us how many nodes existed in the federation when the block was created, so that at a later point in time we can check that the block received the correct number of votes.

• signature: Signature of the block by the node that created the block. (To create the signature, the node serializes the block contents and signs that with its private key.)

• votes: Initially an empty list. New votes are appended as they come in from the nodes. Vote Model [0092] Each node must generate a vote for each block, to be appended to that block's votes list. A vote can have the following structure:

"node_pubkey": "<the public key of the voting node >",

"vote": {

"voting_for_block": "<id of the block the node is voting for >", "previous_block": "<id of the block previous to this one >",

"is_block_valid": "<true|false >",

"invalid_reason":

"<None|DOUBLE_SPEND|TRANSACTIONS_HASH_MISMATCH|

N O D ES P U B KE YS M IS M ATC H " ,

"timestamp": "<timestamp of the voting action >"

}.

"signature": "<signature of vote >"

[0093] Block Validity and Blockchain Pipelining

[0094] Figure 7 shows an example of a blockchain C that has Blocks Bi to Ββ.

Block Bi is the genesis block with a null transaction.

[0095] Blocks are written to C in an order decided by the underlying DB. This means that when a signing node inserts a block into C, it cannot provide a vote at the same time (because a vote includes a reference to the previous block, but the previous block isn't known yet). Only after the write is fully-committed does the block order become clear.

[0096] Nodes vote on blocks after the order of the blocks becomes clear. When a block is created, it starts off undecided (undecided state). As soon as there is a majority of positive votes for a block, or a majority of negative votes, the block goes from undecided to decided valid or decided invalid, respectively.

[0097] Note that, unlike a typical block chain, the block model of blockchain DB of the present disclosure doesn't have a reference to the previous block. Instead, it has a list of votes, and each vote has a reference to the previous block (i.e. the block that the voting node considered the previous block, based on its view of the change feed). We say that "chainification happens at voting time." [0098] Normally, all votes will reference the same previous block, but it's possible that different votes may claim that different blocks are the previous block. This can happen, for example, if a node goes down. When it comes back up, there's no reliable way to recover the order of the new blocks it missed while down.

[0099] If a node goes down, it won't accumulate a long list of new blocks that it must vote on when it comes back up. That's because the list of expected voters (nodes) for a block is set when that block is created. If a node is down when a block is created, then it won't be put on the list of expected voters for that block.

[00100] In Figure 7, Block B2 has received three votes of five possible. In this example, all three votes are positive. Since the majority of nodes voted that the block is valid, the block is considered decided valid.

[00101] Block B3 has received five votes of five possible. There was a positive vote, then negative, then positive, then two more negative votes. Since the majority of nodes voted that the block is invalid, the block is considered decided invalid. This block can stay in the chain because all the votes show that it is invalid. It will be ignored when validating future transactions. By keeping the block in place, we can quickly progress the chain to child blocks.

[00102] Block B4 is undecided because it does not yet have a majority of invalid or valid votes from nodes. Voting on B4 continues. It is important that despite B4 being undecided, it still has a child block B5. This is possible because the DB's built-in consensus algorithm determines the order of the blocks, and we have logically separated writing blocks from voting. "Forking" is not a risk as it is not even in the vocabulary of the DB, let alone supported in code. The reason is that every node is working on the same blockchain (instead of every node working on their own replica of the blockchain which may be different from the other nodes) and every node communicates through the database which is basically an open broadcast channel (instead of communicating individually with each node). Because of this, any node can try to add a block, but only one becomes the child to B4; the rest follow according to the built-in consensus algorithm. It is a single railroad track where the location of the next plank is based on previous planks. We do not have to stop at adding a single plank after an undecided block— we can keep aggressively laying track, such as block B6 in Figure 7.

[00103] When there are undecided parent blocks, we need to do one more thing to prevent double-spending: any transaction put into a new block must not depend on transactions in an undecided block. For example, inputs of a new transaction must not be in inputs of any undecided blocks. This is enforced in two ways: when creating new blocks on undecided blocks, such double-spend transactions are not allowed, and when voting, any block containing such transactions is voted invalid.

[00104] We call this "blockchain pipelining" because this behavior is reminiscent of pipelining in microprocessors. There, the microprocessor starts executing several possible instructions at once. Once the microprocessor has worked out the proper order for the instructions, it collates the results as output and ignores useless results. As with microprocessor pipelining, blockchain pipelining gives significant speed benefits.

[00105] Blockchain Consensus Algorithm (BCA)

[00106] The Blockchain Consensus Algorithm (BCA) of the present disclosure can be a state machine that runs on each "signing" node (server). This section outlines the BCA using Python-like pseudocode.

[00107] Main Loop

[00108] Before starting the mainl_oop() on each signing node, the databases S 52 and C 54 must be created and initialized. One of the initialization steps is to write a genesis block to C.

[00109] Listing 1 has high-level pseudocode for the BCA. It shows the mainLoop() running on signing node k. Every signing node runs the same mainLoop().

[00110] Line 4 emphasizes that there is equal access by all the nodes to the databases S and C. The BCA operates by moving data from transaction set S to blockchain C, and occasionally in the other direction as well.

Listing 1

1 def mainLoop (): # Pseudocode for signing node k

2 # Assume S and C exist and are initialized,

3 # and C contains a genesis block.

4 global S, C # tx set and blockchain globally visible

5 while True:

6 S = assignTransactions(S, k)

7 Sk , C = addBlock (Sk , C, k)

8 C = voteOnBlocks(C, k)

[00111] Line 5 is the start of the main loop. All remaining pseudocode is part of this loop, which runs continuously until the node is shut down.

[00112] Line 6 accepts transactions into S and assigns them to nodes, line 7 moves unordered transactions from S into ordered, grouped-by-block transactions in C, and line 8 is where the node votes on undecided blocks.

[00113] The pseudocode of Listing 1 is written as if there is a single process, but each major step can actually be a separate, independent process. In fact, there may be multiple processes doing each step; this helps performance tremendously. Listing 2 shows the pseudocode for multiple processes.

Listing 2

1 def mainLoopParallel ():

2 start => 1 assignTransactionLoop () processes 3 start => 1 addBlockLoop () processes

4 start => 1 voteLoop () processes

5

6 def assignTransactionLoop ():

7 while True:

8 S = assignTransactions(S, k)

9

10 def addBlockLoop ():

1 1 while True:

12 Sk , C = addBlock (Sk , C, k)

13

14 def voteLoop ():

15 while True:

16 C = voteOnBlocks(C, k) [00114] Assigning Transactions

[00115] Listing 3 algorithms are for assigning transactions, as follows: Listing 3 assignTransactions() is the main routine that groups the two major steps: accepting and assigning incoming transactions (line 2), and reassigning old transactions (line 3).

[00116] Listing 3 assignNewTransactions() shows how a node accepts an incoming transaction from a client and assigns it to another node. The receiving node first checks if the transaction is valid. If it's invalid, an error message is sent back to the client. If it's valid (according to the receiving node), it gets randomly assigned to one of the other nodes or to itself. We assign transactions to a node rather than allowing nodes to grab transactions, because assignment greatly reduces double-spend detections in the block chain building side, and therefore helps throughput.

[00117] In the algorithm, line 7 accepts transactions and loops through them; line 8 checks the validity of the transaction; line 9 chooses which node to assign the transaction to, with uniform probability; line 1 1 records the assign time (see the next algorithm for why); and line 12 actually assigns the transaction to the chosen node. Listing 3

1 def assignTransactions(S, k):

2 S = assignNewTransactions(S, k)

3 S = reassignOldTransactions(S, k)

4 return S

5

6 def assignNewTransactions(S, k):

7 for each new tx , t from outside:

8 if t is valid: # defined later

9 i <" U({0, 1 , . . ., k-1 , k+1 , . . ., N-1 })

10 # i is chosen randomly from all nodes but this one (k)

1 1 t.assign_time = time()

12 Si = Si u t

13 else:

14 # inform the sending -client why t is not valid

15 return s

16

17 def reassignOldTransactions(S, k):

18 for Sj in {S1 , S2 , . . .}:

19 for each tx , t, in Sj:

20 if (time() - t.assign_time) > old_age_t.hr:

21 i <~ U({0, 1 , . . ., k-1 , k+1 , . . ., N-1 })

22 t.assign_time = time()

23 Si = Si u t

24 Sj = Sj - 1

25 return S

26 [00118] Listing 3 reassignOldTransactionsQ re-assigns transactions that are too old. Transactions can get old if a node goes down, is running slowly, is acting maliciously, or is not performing its duties more generally. This routine ensures transactions assigned to a node don't get stuck in limbo, by re-assigning old-enough transactions to different nodes. It loops through all assigned transactions (lines 18-19), and if a transaction is old enough (line 20) a new node is randomly chosen for it (line 21 ), a new assign time is set (line 22), and the transaction is re-assigned from the old node (node j) to the new node (node i). For this routine to work, it also needs the unassigned-transactions to have an assign time, which is done in assignNewTransactionsQ (line 1 1 )).

[00119] Adding and Voting on Blocks

[00120] Listing 4 addBlock() creates and adds a (non-genesis) block to C, and ends with a set of transactions to postpone.

Listing 4

I def addBlock(Sk , C, k):

2 Tpostpone ^— { }

4 Btaii = most recent block in C

6 for t in Sk:

7 if dependsOnUndecidedBlock(t, Btail):

8 Tpostpone ^— Tpostpone ^ t

9 elif transactionValid(t, Btaii):

10 Tnew.append(t)

I I id = sha3 hash of {Tnew, ...}

12 votes = [ ]

13 Bnew = Block(id, T_new, votes, ...)

14 add Bnew to C # Consensus algorithm will determine order

15 Sk = 0

16 Sk = Sk u Tpostpone

17 return Sk , C

18

[00121] Lines 2 - 3 initialize the routine's main variables - the block to add B_new, and the transactions to postpone adding until later T_postpone.

[00122] Lines 4 - 17 create a block and adds it to C, in an order determined by C's consensus algorithm. Line 4 updates its pointer Btaii to the most recent block in C. It is important to grab Btaii here rather than computing it on-the-fly, in case new blocks are added while the rest of the routine is running. Line 5 initializes the ordered list of transactions to be added to the block, and lines 7 - 10 add them one at a time. If a transaction t depends on an undecided block (risking double-spend) it will be postponed to another block by being added to T_p0st one (lines 7 - 8). Otherwise, if it is considered valid, then it is added to T_new (lines 9 - 10). Otherwise, it will be discarded. Lines 1 1 - 14 create the block and add it to C.

[00123] Listing 4 lines 15 - 16 occur once the block has been successfully added. With new transactions now in C, those transactions can be removed from S, as line 15 does by clearing Sk. Line 16 reconciles by adding back any postponed transactions, for example any transactions that risked being double-spends due to being added after an undecided block.

[00124] Listing 5 shows a routine for voting on blocks. voteOnBlocks() is the routine for node k to vote on blocks that it hasn't yet voted on.

Listing 5

1 def voteOnBlocks(C, k):

2 B = oldest block in C that node k has not voted on

3 while B:

4 vote = transactionsValid(B)

5 B.V[k] = vote

6 if B is decided and invalid: copy txs from B back into S 7 B = (child block of B) or 0

8 return C

9

[00125] Note that a node actually votes on blocks that may have already been decided, because it's faster to vote than to first query whether the block is decided. Lines 3 - 8 iterate from the oldest block that node k hasn't voted on (found in line 2) to the newest block (when temporary variable goes to 0 in line 7). For each block, line 4 computes a Boolean of whether all transactions in the block B are valid, and line 5 stores this in B's votes variable B.V, signed by node k. Line 6 gives potentially valid transactions another chance.

[00126] Transaction Validity

[00127] Listing 6 shows Routines for transaction validity.

Listing 6

1 def transactionsValid(T, Bi): 2 # are all txs valid?

3 for t in T:

4 if not transactionValid(t, Bi):

5 return False

6 return True

7

8 def transactionValid(t, Bi):

9 # Is tx valid in all blocks up to and including Bi?

10 # (Ignore Bi+1 , Bi+2, . . .)

1 1 if t is ill-formed, commits double -spend, etc.

12 return False

13 if dependsOnUndecidedBlock(t, Bi)

14 return False

15 return True

16

17 def dependsOnUndecidedBlock(t, Bi):

18 # returns True if any of the inputs of t are in a block

19 # that is not voted enough (enough x's or p's)

20 # in [B0 , B1 , . . . , Bi]. Ignores [Bi+1 , Bi+2, . . .]

21

[00128] transactionsValid() is the top-level routine to simply loop through all the transactions supplied in the transaction list T (lines 3 - 6), and if any transaction is found to be invalid (line 4) the routine returns False.

[00129] transactionValid() measures (determines) whether a transaction is valid, based on traditional blockchain validity measures (ill-formed, double-spend, etc.) in lines 1 1 - 12 and also based on whether it depends on an undecided block (lines 13 - 14). dependsOnUndecidedBlock() clarifies what it means to depend on an undecided block.

[00130] Consensus Algorithm Checklist

[00131] Block construction order. When a node finalizes the creation of a block, that block must not allow any more transactions to be added to it. This is to ensure that blocks created after the block can check that their transactions don't double-spend assets spent by previous blocks' transactions. This wouldn't be possible if a block could get more transactions added to it after block finalization. [00132] Hashing votes. Is there transaction malleability because votes are not hashed? This may look like a problem, because a block's hash can be propagated to its child block before all its votes are in. A preliminary answer would be to have a second chain of hashes that actually includes the votes. But the solution can be simpler than that: a hash without votes is fine because the votes are digitally signed by the signing nodes, and therefore not malleable.

[00133] Dropping transactions. If a node went down or, more generally misbehaved, transactions assigned to that node might not be handled. To address this, we added a way to re-assign transactions if the previous node assignment got stale: algorithm reassignOldTransactions() in Listing 3.

[00134] Denial of service. Are there any transactions that can be repeatedly called by aggressor clients or a malicious server node, which tie up the network? To the inventor's knowledge, this is not an issue any more than with a traditional web service.

[00135] Client transaction order. We must try to ensure that transactions sent from the same client in a particular order are processed in that order— or at least with a bias to that order. When a client sends two transactions to the same node, that receiving node could change their order before writing them to the backlog. That wrong ordering might be preserved by the distributed database changelog, so all other nodes would see the same wrong ordering. At the time of writing, the inventors are considering several possible solutions, including using multi-node consensus on client-provided timestamps.

[00136] Database built-in communication vulnerability. The nodes communicate using the big data DB's own built-in consensus algorithm like Paxos to tolerate benign failures. Is this a vulnerability? The answer is that many nodes would have to be affected for it to have any major consequences.

[00137] Double spends. Are there any ways to double-spend? This is a useful question to keep asking at all stages of development. In this regard the blockchain DB of the present disclosure does exactly the same as the Bitcoin network. All past transactions are checked to make sure that input was not already spent. This can be fast for the blockchain DB because it can use an optimized query of the underlying DB.

[00138] Malicious behavior. Questions: How does the blockchain DB of the present disclosure detect that a node has bad (Byzantine) behavior? Does it discourage bad behavior? How? Answers: Overall, it's a simpler problem because of the federation model. Bad behavior can be detected when a node's vote on a block is different than the majority. There are many possible ways to discourage bad behavior, from manual punishment decided by the federation, to needing to post a security deposit (bond) and automatically losing it upon bad behavior.

[00139] Admin becoming god. Does the system administrator have any powers that allow them to play "god", and thus constitute a single point of failure? The inventors were careful to limit the power of the system administrator to even less than a voting node. So to our knowledge, the system administrator cannot play god because all write transactions (including updating software) need to be voted on by the federation.

[00140] Offline nodes. Q: What happens if a voting node goes offline? If many go offline? A: One or a few offline nodes is fine, as a quorum (the number of nodes needed to

decide a block) is still online. If there are many offline nodes, then a block could get stuck in an undecided state. (Transactions depending on transactions in the undecided block would also get stuck in the backlog.) Our solution is to wait until enough nodes come back online to vote on and decide the block. It is a reasonable solution, because all consensus algorithms require some minimum proportion of nodes to be online to work. An alternative solution would be to limit the amount of time that a block can be undecided before being marked decided invalid, so that all its transactions can be copied back to the backlog for reconsideration.

[00141] Chaining blocks rather than transactions. Q: Why do we chain together blocks, rather than chaining together transactions? A: There are several reasons. First, it's easier to write 1000 blocks per second (each containing up to 1000 transactions) than it is to write 1 million transactions per second (to the blockchain database). Second, each voting node only has to append a vote (data structure) to each block, rather than to each transaction, saving a lot of storage space. (If a voting node votes yes for a block, then we can conclude that it found all the contained transactions to be valid.) Lastly, when constructing a vote, the signing node must compute a cryptographic signature. That takes time. We save time by doing that only once per block (rather than per transaction).

[00142] Transaction Validity, Incentivization, and Native Assets

[00143] There are many things to check when determining if a transaction is valid. Signatures must be valid. Certain fields must be present (and no others). Various values must have the correct syntax. If the transaction is to create or register a new asset, then the same asset must not already exist. If the transaction is to transfer an asset, then the asset must exist, the transfer transaction must be requested by the current owner (who must sign it with their private key), not by a previous owner and not by a non-owner. "You can only spend what you have." [00144] Every voting node checks the validity of every transaction (so it can decide how to vote on the transaction's block). Recall that the consensus for the blockchain DB of the present disclosure is federation-based. A node gets to vote on a transaction based on whether it has been given a voting node role. Contrast this to a POW (Proof of Work) model, where the probability of a node voting is proportional to its hash power, which assuming all miners have state-of-the-art hardware is equivalent to electricity spent; or to POS (Proof of Stake) where probability of a node voting is proportional to how much money it has.

[00145] Traditionally, blockchains have held two types of assets. "Native assets," like Bitcoins or Litecoins, are built into the core protocol. The consensus uses these assets to measure transaction validity and to reward voting by native-asset transaction fees and mining rewards. Second are non-native "overlay assets" in overlay protocols sitting above the core protocol (e.g. SPOOL). However, this traditional approach to native assets and reward has weaknesses:

· Overlay Asset Double-Spend. Traditional blockchains' consensus models do not account for overlay assets. There is nothing at the core protocol level to prevent a double-spend of an overlay asset.

• Native Asset Friction to Network Participation. Traditional blockchain voting nodes need to get paid in the native asset, so any new participants in the network must acquire the native asset, typically on an exchange, before being able to conduct a transaction. Acquiring the native asset is especially difficult on newer blockchains with native assets that are not yet available on many exchanges. This is a high barrier to entry when compared to traditional web services, where any new participant can conduct a transaction by paying in a standard currency like U.S. dollars with a standard payment method like a credit card.

[00146] The blockchain DB of the present disclosure overcomes these issues as follows:

• Native consensus voting on every asset. Every transaction keeps track of which asset it is operating on, chaining back to the transaction that issued the asset. Every asset is "native" in the sense that it's used to measure transaction validity.

This overcomes the issue of "asset overlay double-spend."

• Low friction to network participation. Like a traditional web service, the network operator sets the terms on how to join the network and conduct transactions— but in this case the network operator is the collective will of the voting nodes. The voting nodes also determine how users pay for transactions. [00147] Incentivization and Security

[00148] In POW and POS blockchains, the network, incentive model, and security of the network are inextricably linked. Security is intrinsic to the system. But, as is known, both POW and POS have scalability challenges. Though it's a double edged sword: when incentives are intrinsic to the system too, there is motivation to game the system. An example is the emergence of mining pools to benefit the most from Bitcoin's built-in incentive (mining rewards).

[00149] In a federation like the blockchain DB of the present disclosure, the security of each node and aggregate security over the entire network are extrinsic. This means the rules for confidentiality, availability, and integrity are outside the core network design. For instance, if all nodes have weak rules for security, the network will be breached. By contrast, if a minimum fraction of the network nodes have reasonable security standards, the network as a whole can withstand attacks. Extrinsic incentives can have benefits: in a private deployment, the network participants are motivated simply by the benefits of being part of the network (e.g. lower costs, lower fraud, new functionality). Extrinsic incentives can work in a public deployment too, for similar reasons: the voting nodes may have their own reasons for an open, public database to survive, for example a mandate as a nonprofit, and this makes the interests aligned.

[00150] Implementation Details of Some Embodiments

[00151] Choice of distributed DB

[00152] The design of the blockchain DB of the present disclosure is flexible enough to be built on top of a wide variety of existing distributed DBs. Of course, one had to be selected to build on. To select which, we first did benchmarking, then added additional criteria.

[00153] There are > 100 DBs to choose from. This was our shortlist: Cassandra, HBase, Redis, Riak, MongoDB, RethinkDB and ElasticSearch. Each of these DBs uses Paxos or a Paxos descendant such as Raft.

[00154] First, we did preliminary performance investigation of the DBs in our shortlist: Each had 15 - 105 writes/s per thread, 290 - 1000 serial reads/s per thread, and 80 - 400 reads/s per thread.

[00155] While there was some variation in performance among the DBs, the key thing to notice is that performance is per thread: performance improves as the number of threads increases. This is different than traditional blockchain technologies, where performance stays flat or worsens. [00156] Given that all DBs tested had good scalability properties, we realized that other criteria were even more important. In particular:

1 . Consistency. Distributed DBs must make a trade-off between performance and consistency (in the CAP theorem sense, not ACID sense). For a blockchain, consistency means trustworthy ordering of transactions, so we prefer DBs with strong consistency guarantees.

2. Automatic Change Notifications. One way for a node to find out if a change has happened in a DB is to ask it on a regular basis (i.e. polling), but that's not as efficient as having the DB automatically notify the node of changes. We wanted a

DB with automatic change notifications as a standard feature.

Automatic change notifications bring another benefit: they improve tamper- resistance (beyond what a chain of hashes offers). If a hacker somehow manages to delete or update a record in the data store, the hashes change (like any blockchain). In addition, a datastore with automatic change notifications would notify all the nodes, which can then immediately revert the change and restore the hash integrity.

[00157] Of the options considered, RethinkDB performed well. It has strong consistency guarantees and it offers automatic change notifications ("changefeeds") as a standard feature. Therefore, we built the first version of the blockchain DB of the present disclosure on top of RethinkDB. As will be understood by the skilled worker, any other suitable distributed database is to considered within the scope of the present disclosure.

[00158] RethinkDB is a JSON (NoSQL) database with a flexible query language. It is optimized for scalable realtime feeds, which is useful for collaborative apps, streaming analytics, multiplayer games, realtime marketplaces, and connected devices / loT (Internet of Things). It is written in C++, is open source, and has a vibrant development community.

[00159] In the future, we envision a variety of distributed databases being "blockchain-ified" according to the approach of this paper. Every relational database, document store and graph store might someday have a blockchain version.

[00160] Blockchain DB Capacity

[00161] Each node in the RethinkDB cluster adds its own storage capacity to the total database capacity. [00162] For example, if each RethinkDB node were run on a d2.8xlarge instance on Amazon Web Services (AWS), then each of those instances could contribute its (24→i2000) GB = 48000 GB of storage to the database. 32 nodes would have 32→■ 48000 = 1536000 GB total capacity, i.e. more than a petabyte. (This calculation assumes no replication. A replication factor of R would decrease the total capacity by a factor of R.)

[00163] For quick reference, Figure 8 shows how total capacity depends on the number of nodes.

[00164] Serialization

[00165] Before we can hash or sign a JSON message (e.g. a transaction payload), we must convert it to a string in a standard way (i.e. with the same result, regardless of the programming language or computer architecture used). That is, we must serialize the JSON message in a standard way. Fortunately, there is a standard: RFC 7159.

[00166] We can do JSON serialization using the dumps() function in python- rapidjson, a Python wrapper for rapidjson (a fast and RFC 7159-compliant JSON parser/generator written in C++). Listing shows how it is called:

Listing 7

import rapidjson

rapidjson. dumps(data ,

skipkeys=False ,

ensure_ascii=False

sort_keys=True)

The parameters are define as:

• data is the JSON message, stored in a Python diet

• skipkeys = False: Ensures that all keys are strings

• ensure ascii = False: The RFC recommends UTF-8 encoding for maximum interoperability. By setting ensure ascii to False we allow Unicode characters and force the encoding to UTF-8

• sort keys = True: The output is sorted by keys

[00167] Cryptographic Hashes

[00168] All hashes can be calculated using the SHA3-256 algorithm. We store the hex-encoded hash in BigchainDB. A Python implementation example, using pysha3, is shown at Listing 8: Listing 8

1 import hashlib

2 # monkey patch hashlib with sha3 functions

3 import sha3

4 data = "message"

5 tx_hash = hashlib. sha3_256(data).hexdigest () [00169] Keys and Signatures

[00170] In the present embodiment, we use the Ed25519 public-key signature system for generating public/private key pairs (also called verifying/signing keys). Ed25519 is an instance of the Edwards-curve Digital Signature Algorithm (EdDSA). As of April 2016, EdDSA was in "Internet-Draft" status with the IETF but was already widely used.

[00171] The blockchain DB embodiment of the present disclosure uses the the ed25519 Python package, overloaded by the cryptoconditions library.

[00172] All keys are represented using base58 encoding by default.

[00173] Transaction Encryption

[00174] Normally, transactions stored in BigchainDB aren't encrypted, but users can encrypt the payload if they want, using the encryption technology of their choice. (The payload of a transaction can be any valid JSON, up to some maximum size as explained below.) Other aspects of the transaction, such as the current owner's public key and the new owner's public key, aren't encrypted and can't be encrypted.

[00175] Figure 9 shows a flowchart of an embodiment of a method to write transactions to a blockchain, in accordance with the present disclosure. The method starts at 100. At action 102, transactions are received at server nodes, for example, at server nodes 51 shown at Figure 2. The transactions are received at the server nodes from clients connected thereto.

[00176] At action 104, the permissibility of the transaction is determined. That is, the node that receives a transaction verifies if the transaction is valid in accordance with the node. Verifying the permissibility (also referred to as "superficial validation") can include verifying if the format of the transaction is in a required pre-determined format, verifying that the transaction does not double-spend, etc. When any transaction received at a node is determined to not be permissible, the non-permissible transaction is rejected at action 106. The permissibility of each transaction is determined at the node that receives the transaction. The actions 104 and 106 are shown in dotted lines to indicate that they are optional.

[00177] When a transaction is determined to be permissible, the node that has received the transaction assigns, at action 108, the transaction for further processing (for validation) to any one of the nodes of the network. Referring to Figure 1 , any server node 51 that has received a transaction and determined that transaction to be permissible, assigns the permissible transaction to any of the nodes 51 . A node can even assign the permissible transaction to itself, for further processing (validation) of the transaction. Upon being assigned to a node for further processing, the transaction is placed in a backlog (for example, backlog S 52 at Figure 2) visible by all the nodes (e.g., the server nodes 51 of Figure 1 ) in the network. The underlying distributed database notifies all the other nodes about the change to S (i.e. that there is a new transaction), along with the contents of the transaction.

[00178] When received at a node for further processing, a transaction (for example, a permissible transaction) is placed in a queue, behind previous permissible transactions. The transactions placed ahead in the queue are processed first. When the transaction arrives at the head of the queue, the node verifies, at action 1 10, if the transaction is valid. If the transaction is not valid, it is rejected at action 109. The validity of the transaction is verified taking into account all the transactions in the backlog. The assigned node (the node to which the transaction is assigned) must check to see if the transaction commits a double-spend and, at action 1 1 1 , if it depends on a transaction in an undecided block. If the transaction does depend on a transaction in an undecided block, then it must go back to waiting for consideration for inclusion in a block. This is shown at action 1 12. In some embodiments, actions 109 and 1 10 & 1 1 1 and 1 12 are not required, i.e. they are optional.

[00179] When a transaction is determined to be valid, the transaction is placed in, or appended to, a block. This is shown at action 1 14. Subsequently, at action 1 16, determination is made of whether or not the block is ready for being written to the blockchain. This can be referred to as a "ready to write block criteria". The criteria for writing the block to the block chain can include the number of transactions in the block and the length of time the block has been sitting there, waiting for a transaction. When the number of transactions in the block is at a p re-determined value (for example, 1000 transactions) and/or when length of time the block has spent waiting for a transaction to be written to the block since the previous transaction reaches a pre-determined duration (for example, 5 seconds), the block is written to the blockchain (for example, the blockchain C 54 of Figure 2). This is to ensure that a block doesn't wait forever for new transactions.

[00180] When the "ready to write block criteria" is met, the block is written to the blockchain (for example, the blockchain C 54 of Figure 2) at action 1 18. This action occurs after the genesis block is created at action 90. If the block is the first block after the genesis block, it is placed immediately after the genesis block. If the block is not the first block written to the blockchain, it is placed immediately after the immediately previous block written to the blockchain. Writing the block to the blockchain occurs at action 1 18. At this point the block is in an undecided state.

[00181] At action 120, each signing node (for example server nodes 51 at Figure 1 ) votes to decide if the block is in a decided valid state or in a decided valid state. The votes are received at action 122 and are appended to the block.

[00182] At action 124, determination is made of a quorum of vote having been met. That is, valid votes received are counted and compared to a pre-determined quorum value. If quorum is met, the block has a sufficient number of votes to change its state from the undecided state to the decided valid state or to the decided invalid state.

[00183] When quorum has not been met and the block remains undecided, at action 126, the method loops back to action 122.

[00184] At action 128, it is determined if the block has received a sufficient number of valid votes. If so, the state of the block is changed to the decided valid state at action

130. This ensures that any transaction depending from a transaction present in the block that has been decided valid can be processed (action 1 10).

[00185] Otherwise, the state of the block is changed to the decided invalid state at action 132 and, at action 134, the transactions in the block are return to the backlog for processing.

[00186] Figure 10 relates to the writing of votes in a table (vote table 502) that is distinct form the blockchain table, shown at reference numeral 500 in the present example. Both the vote table 502 and the blockchain table 500 are formed in the blockchain database of the present disclosure. The blockchain table 500 includes block of transactions 400, block of transactions 402 and block of transactions 404. The vote table 502 is shown with voting block 406, voting block 408 and voting block 409. Voting block 406 has received four valid votes (✓) and two invalid votes (x) for block 400. Voting block 408 has received three valid votes (✓) and one invalid vote (x) for block B. Voting block 409 has received three valid votes (✓) and two invalid votes (x) for block 404. [00187] The lines 414 are to indicate that the block 408 contains a hash of block 406. The lines 415 are to indicate that the block 409 contains a hash of block 408.

[00188] The lines 410 indicate that each vote in block 406 contains a hash of block 400. The lines 412 indicate that each vote in block 408 contains a hash of block 402. The lines 413 indicate that each vote in block 409 contains a hash of block 404.

[00189] Of note, in this embodiment, each vote in block 408 points to block 402 and to block 406. And, each vote in block 409 points to block 404 and to block 408. This is what establishes the order of the transaction blocks. As will be understood by the skilled worker, Figure 10 can be interpreted as being a Merkel tree.

[00190] In the preceding description, for purposes of explanation, numerous details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that these specific details are not required. In other instances, well-known electrical structures and circuits are shown in block diagram form in order not to obscure the understanding. For example, specific details are not provided as to whether the embodiments described herein are implemented as a software routine, hardware circuit, firmware, or a combination thereof.

[00191] Embodiments of the disclosure can be represented as a computer program product stored in a machine-readable medium (also referred to as a computer-readable medium, a processor-readable medium, or a computer usable medium having a computer-readable program code embodied therein). The machine-readable medium can be any suitable tangible, non-transitory medium, including magnetic, optical, or electrical storage medium including a diskette, compact disk read only memory (CD-ROM), memory device (volatile or non-volatile), or similar storage mechanism. The machine- readable medium can contain various sets of instructions, code sequences, configuration information, or other data, which, when executed, cause a processor to perform steps in a method according to an embodiment of the disclosure. Those of ordinary skill in the art will appreciate that other instructions and operations necessary to implement the described implementations can also be stored on the machine-readable medium. The instructions stored on the machine-readable medium can be executed by a processor or other suitable processing device, and can interface with circuitry to perform the described tasks.

[00192] The above-described embodiments are intended to be examples only. Alterations, modifications and variations can be effected to the particular embodiments by those of skill in the art. The scope of the claims should not be limited by the particular embodiments set forth herein, but should be construed in a manner consistent with the specification as a whole.

Claims

WHAT IS CLAIMED IS:

1 . A computer-implemented method of recording a transaction, the transaction being an electronic transaction, the method comprising:

at a node of a distributed database comprising a plurality of nodes, the plurality of nodes including voting nodes:

placing the transaction in a backlog containing other transactions, the backlog being the same for all the nodes of the plurality of nodes; forming a block of transactions from at least one of the transactions in the backlog, the block of transactions being in an undecided state when formed;

writing the block of transactions at an end of a blockchain; at each voting node, after the block of transactions has been written to the

blockchain, voting on a validity of the block of transactions; and

writing the votes to the distributed database.

2. The computer-implemented method of claim 1 , wherein:

the undecided state changes to a decided valid state when the block of

transactions has been determined valid by a quorum of votes of the voting nodes; and the

the undecided state changes to a decided invalid state when the block of

transactions has been determined invalid by a quorum of votes of the voting nodes.

3. The computer-implemented method of claim 1 or 2, wherein the backlog is a first database table and the blockchain is a second database table.

4. The computer-implemented method of claim 1 or 2, wherein the backlog is a first database container and the blockchain is a second database container.

5. The computer-implemented method of any preceding claim, wherein the votes are written in the blockchain.

6. The computer-implemented method of any preceding claim, wherein the votes are written in one of a database table or a container, the database table and the container being distinct from the blockchain.

7. The computer-implemented method of any preceding claim, wherein the node at which the transaction is placed in the backlog is a selected node, the computer- implemented method further comprising receiving the transaction at any one of the plurality of nodes of the decentralized database and assigning the transaction to the selected node.

8. The computer-implemented method of claim 7, wherein assigning the transaction to the selected node is preceded by randomly selecting, from the plurality of nodes, the selected node as the node in which to place the transaction.

9. The computer-implemented method of claim 8, wherein:

the transaction placed in the selected node is a permissible transaction; and placing the transaction in the backlog is preceded by determining that the

transaction is permissible.

10. The computer-implemented method of claim 9, wherein determining that the transaction is permissible includes:

at least one of determining if the transaction is in a pre-defined format and

determining if the transaction is a double-spend transaction, the transaction being permissible when in the pre-defined format and when it is not a double- spend transaction.

1 1 . The computer-implemented method of claim 10, further comprising: when the transaction is a double-spend transaction or when the transaction in not the pre-defined format, rejecting the transaction.

12. The computer-implemented method of any preceding claim, wherein forming the block of transactions includes:

13. The computer-implemented method of any preceding claim, wherein writing the block of transactions at the end of the blockchain is preceded by determining if the block meets a pre-determined criteria for being written to the blockchain.

14. The computer-implemented method of claim 13 wherein determining if the block meets the pre-determined criteria for being written to the blockchain includes at least one of:

determining a number of transactions in the block; and

determining a time lapse from the time a last transaction was placed in the block, the block being ready for being written to the blockchain when the number of transactions in the in the blocks meets a pre-determined number criteria or, when the time lapse meets a pre-determined time lapse criteria.

15. The computer-implemented method of any preceding claim further comprising, when the block of transactions has been determined invalid by the quorum of votes of the voting nodes:

changing the state of the block of transactions from the undecided state to a

decided invalid state;

returning instances of the transactions of the block of transactions to the backlog for processing; and

maintaining the block of transactions in the blockchain.

16. The computer-implemented method of any preceding claim wherein:

the transaction and the other transactions comprised in the backlog each have a timestamp; and

placing the transaction in the backlog includes placing the transaction at a last entry position in the backlog, the other transactions in the backlog each having an earlier timestamp than the transaction at the last entry position.

17. The computer-implemented method of any preceding claim, wherein voting on a validity of the block of transactions includes determining if the transactions depend from a transaction present in an undecided block.

18. The computer-implemented method of any preceding claim, wherein voting on the validity of the block of transactions includes voting 'valid' when the block depends from a decided block and when each transaction in the block of transactions is a valid transaction and otherwise, voting 'invalid'.

19. A decentralized database system comprising:

a distributed database having processors and nodes, the nodes being

interconnected to each other, the nodes including voting nodes;

a computer-readable medium having recorded thereon instructions to be executed by the processors to carry out a method of recording a transaction, the transaction being an electronic transaction, the method comprising: at a node of the distributed database:

placing the transaction in a backlog containing other transactions, the backlog being the visible to all the nodes;

forming a block of transactions from at least one of the transactions in the backlog, the block of transactions being in an undecided state when formed;

writing the block of transactions at an end of a blockchain; at each voting node, after the block of transactions has been written to the blockchain, voting on a validity of the block of transactions; and writing the votes to the distributed database.

20. The system of claim 19, wherein:

the undecided state changes to a decided valid state when the block of

the undecided state changes to a decided invalid state when the block of

21 . The system of claim 19 or 20, wherein the backlog is a first database table and the blockchain is a second database table.

22. The system of claim 19 or 20, wherein the backlog is a first database container and the blockchain is a second database container.

23. The system of any one of claims 19 to 22, wherein the votes are written in the blockchain.

24. The system of any one of claims 19 to 23, wherein the votes are written in one of a database table or a container, the database table and the container being distinct from the blockchain.

25. The system of any one of claims 19 to 24, wherein the node at which the transaction is placed in the backlog is a selected node, the method further comprising receiving the transaction at any one of the nodes of the decentralized database and assigning the transaction to the selected node.

26. The system of claim 25, wherein assigning the transaction to the selected node is preceded by randomly selecting, from the plurality of nodes, the selected node as the node in which to place the transaction.

27. The system of claim 26, wherein:

transaction is permissible.

28. The system of claim 27, wherein determining that the transaction is permissible includes:

at least one of determining if the transaction is in a pre-defined format and

29. The system of claim 28, further comprising: when the transaction is a double- spend transaction or when the transaction in not in a the pre-defined format, rejecting the transaction.

30. The system of any one of claims 19 to 29, wherein forming the block of transactions includes:

determining, one transaction at a time and in an order of the transactions listed in the backlog, a validity of the transactions in the backlog; and

forming the block with valid transactions.

31 . The system of any one of claims 19 to 30, wherein writing the block of

transactions at the end of the blockchain is preceded by determining if the block meets a pre-determined criteria for being written to the blockchain.

32. The system of claim 31 wherein determining if the block meets the pre-determined criteria for being written to the blockchain includes at least one of:

determining a number of transactions in the block; and

33. The system of any one of claims 19 to 32 further comprising, when the block of transactions has been determined invalid by the quorum of votes of the voting nodes: changing the state of the block of transactions from the undecided state to a

decided invalid state;

maintaining the block of transactions in the blockchain.

34. The system of any one of claims 19 to 33 wherein:

35. The system of any one of claims 19 to 34, wherein voting on a validity of the block of transactions includes determining if the transactions depend from a transaction present in an undecided block.

36. The system of any one of claims 19 to 35 wherein voting on the validity of the block of transactions includes voting 'valid' when the block depends from a decided block and when each transaction in the block of transactions is a valid transaction and otherwise, voting 'invalid'.

37. A distributed database system comprising:

a distributed database having processors and nodes, the nodes being

interconnected to each other, the nodes including voting nodes;

a computer-readable medium having recorded thereon instructions to be executed by the processors to carry out:

a method of recording the transaction, comprising:

at a node of the distributed database:

writing the block of transactions at an end of a blockchain;

at each voting node, after the block of transactions has been written to the blockchain, voting on a validity of the block of transactions, the voting at each voting node to decentralize a control of the distributed database;

cryptographically signing the transaction, the block of transactions, and the votes, in order to provide tamper-resistance to the distributed database system; and

writing the votes to the distributed database;

wherein the transaction is one a 'creation transaction' to create an asset and a 'transfer transaction' to transfer the asset.