WO2020113314A1 - Système et procédé d'augmentation d'applications de base de données avec une technologie de chaîne de blocs - Google Patents

Système et procédé d'augmentation d'applications de base de données avec une technologie de chaîne de blocs Download PDF

Info

Publication number
WO2020113314A1
WO2020113314A1 PCT/CA2019/051700 CA2019051700W WO2020113314A1 WO 2020113314 A1 WO2020113314 A1 WO 2020113314A1 CA 2019051700 W CA2019051700 W CA 2019051700W WO 2020113314 A1 WO2020113314 A1 WO 2020113314A1
Authority
WO
WIPO (PCT)
Prior art keywords
blockchain
database
data
transaction
application
Prior art date
Application number
PCT/CA2019/051700
Other languages
English (en)
Inventor
Yuming QIAN
Original Assignee
Zeu Crypto Networks Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zeu Crypto Networks Inc. filed Critical Zeu Crypto Networks Inc.
Priority to CA3121919A priority Critical patent/CA3121919C/fr
Priority to KR1020217020788A priority patent/KR20210135477A/ko
Priority to EP19894326.8A priority patent/EP3891621A4/fr
Priority to CN201980090660.4A priority patent/CN113396407A/zh
Priority to US17/299,736 priority patent/US20220019575A1/en
Priority to JP2021532074A priority patent/JP2022511084A/ja
Publication of WO2020113314A1 publication Critical patent/WO2020113314A1/fr
Priority to IL283696A priority patent/IL283696A/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2379—Updates performed during online database operations; commit processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2308—Concurrency control
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2365—Ensuring data consistency and integrity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/60—Protecting data
    • G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/50—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols using hash chains, e.g. blockchains or hash trees
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00—Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/56—Financial cryptography, e.g. electronic payment or e-cash

Definitions

  • the present application relates generally to a blockchain system, and in particular to augmenting database applications with blockchain technology.
  • Enterprise applications have traditionally used a tiered or layered architecture, typically made up of a presentation layer, a business layer and a data storage or persistence layer.
  • the presentation layer includes software components that provide the user interface to facilitate user interaction.
  • the business logic layer contains software components that implement the business rules or business logic that are applied to data.
  • the storage layer contains software components used for storing persistent data and provision of related data access services.
  • the data storage layer typically includes a relational database management system (RDBMS) or a NOSQL database to implement the data storage service.
  • RDBMS relational database management system
  • NOSQL database NOSQL database
  • Relational database management systems that are suitable for enterprise class applications typically support transactions that should guarantee validity, even in the event of errors. Transactions are individual, indivisible operations that may occur concurrently.
  • a transaction processing system manages concurrent processing of transactions, enables the sharing of data, ensures the integrity of data and manages the prioritization of transaction execution.
  • Database transactions are thus characterized by a set of properties called atomicity, consistency, isolation, and durability (ACID for short).
  • Atomicity means that all changes to data are performed as if they are a single operation.
  • Consistency requires that data is in a consistent state when a transaction starts and when it ends.
  • Isolation means that the intermediate state of a transaction is invisible to other transactions.
  • Durability implies that after a transaction successfully completes, changes to data persist and are not undone, even in the event of a system failure.
  • Blockchain transactions work via consensus.
  • Blockchain technology maintains a reliable record of transactions by means of collective participation and consensus among participants.
  • Blockchain has often been understood and described as a distributed ledger technology (DLT), jointly maintained by multiple networked devices called nodes.
  • DLT distributed ledger technology
  • Another challenge preventing conventional database applications from adopting blockchain technology is blockchain transaction completion performance.
  • a consensus mechanism exists by which each blockchain transaction needs to wait for most of the nodes to confirm the transaction before writing the transaction into a block in the chain.
  • a blockchain transaction thus often lasts for a few seconds or even minutes before it can be finally committed.
  • the data writing operation committing a database transaction is often instantaneous and as a result a general trading database system has response times that are measured in milliseconds.
  • a method of processing a transaction simultaneously in a database and a blockchain includes modifying a first table the database in accordance with the transaction, and composing and inserting into the blockchain, a data operation record corresponding to the transaction, for consensus voting.
  • the method further includes, upon a successful result from the consensus voting, committing the transaction by modifying a second table in the database corresponding to the transaction and otherwise rolling back the transaction thereby leaving the second table unchanged.
  • a method of synchronizing transactions in a database transactions with transactions in a blockchain includes receiving a write command, in response to receiving the write command writing to a temporary working table; and submitting a corresponding write request to the blockchain for consensus voting.
  • the method further includes upon receiving a result of the consensus voting, modifying the query table in accordance with the write command but otherwise leaving the query table unchanged.
  • a system for synchronizing a database and a blockchain includes a processor executing instructions stored in memory so that, upon receiving a write request having data to be written to the database being received from an application, the processor performs the following steps of: writing the data into a working table in the database; submitting a data operation record to the blockchain for consensus voting; and writing the data into a query table in the database upon receiving indication of a successful result for the consensus voting, and otherwise leaving the query table unchanged.
  • a system for augmenting an application using a database with consensus voting in a blockchain The application runs on a node on the blockchain.
  • the system includes a database proxy receiving application commands from the application and translating them into database commands; a set of temporary tables in the database for storing local transactions before the blockchain completes the consensus voting; a set of query tables for recording data after the blockchain returns result of the consensus voting; a database log tracking module to track the changes in the database; a blockchain writer module to write data to the blockchain; and a blockchain listener module to monitor events and changes in the blockchain.
  • the blockchain listener module and the blockchain writer module are in communication with the blockchain.
  • the database proxy is in communication with the application and the database.
  • the blockchain listener module is in communication with the database.
  • FIG. 1 is a simplified schematic diagram of a system, exemplary of an embodiment of the present invention
  • FIG. 2 is simplified schematic diagram of a working table in the database of
  • FIG. 1 is a diagrammatic representation of FIG. 1 ;
  • FIG. 3 is simplified schematic diagram of a query table in the database of
  • FIG. 1 is a diagrammatic representation of FIG. 1 ;
  • FIG. 4 is a flowchart of activities undertaken by a device executing application of FIG. 1 ;
  • FIG. 5 is a flowchart of an exemplary process for executing a transaction utilizing the device executing application of FIG. 1;
  • FIG. 6 is a flowchart of an exemplary process to recover and synchronize the database data after a new block is added to the blockchain of FIG. 1 ;
  • FIG. 7 is a flowchart summarizing the steps for migrating an database existing application into an application that interacts with both its database and a blockchain.
  • Embodiments of the present invention address issues related to migrating database applications to blockchain applications. These issues include the migration of synchronous trading operations to asynchronous trading operations. This mismatch among the primary reasons that necessitate business layer code to be rewritten.
  • a key advantage of blockchain technology is to ensure tamper-evident transactions, but the architecture and design of the blockchain is predetermined. For blockchain applications, only incremental operations could be implemented. Blockchains are not suitable for random queries or random condition modification. In fact, making random modifications difficult, if not impossible, is a desirable property of blockchains which aim to ensure immutable audit trail without a trusted intermediary institution. Unfortunately, this property also restricts the flexibility and scope of application development.
  • a blockchain application does not need to save all the data into the blockchain. Rather only some of the non-repudiation of the stored data needs to be saved into the blockchain. Thus most blockchain applications are still dependent on relational databases, to store a variety of local information. For example, user login authentication is likely to still be done on a traditional relational database management system or NOSQL database which makes synchronization of the database and blockchain.
  • a “blockchain” is a tamper-evident, shared digital ledger that records transactions in a public or private peer-to-peer network of computing devices.
  • the ledger is maintained as a growing sequential chain of cryptographic hash-linked blocks.
  • A“node” is a device on a blockchain network.
  • the device is typically be a computer having a processor interconnected to a processor readable medium including memory, having processor readable instructions thereon.
  • FIG. 1 depicts a schematic diagram of a software system 100, exemplary of an embodiment of the present invention.
  • a client application 101 which is a software application, exchanges data with a database 111.
  • the database 111 is a relational database management system and includes a plurality of tables including the working tables 103, the private tables 104 and the query tables 105 that are accessible through a database proxy service 102.
  • a blockchain listener service 108 is in data communication with tables 104, tables 105 and a revoke callback service 109.
  • a blockchain 110 made up of a plurality of nodes 110a to 110e interconnected together by a data network, implements consensus voting to process blockchain transactions which will be synchronized with application 101.
  • a transaction tracker component 106 in data communication with the private tables 104 tracks transactions.
  • the client application 101 is a database application, i.e. , a database driven software application.
  • a database application i.e. , a database driven software application.
  • a blockchain system such as system 100
  • different organizations perform different responsibilities, and thus each one of nodes 110a to 110e may run multiple applications. However, these different applications read from and write to a common information store.
  • Private tables 103 are local private tables which keep data locally. The data in tables 103 does not need to be synchronized to blockchain for consensus.
  • Query tables 105 contain persistent data stored in database 111 , and may contain data suitable for use by applications such as application 101.
  • Working tables 104 are temporary tables that are created as copies of corresponding original tables that resemble the query tables, with additional status or state fields or columns appended to help manage potential transaction conflicts, and coordinate persisting data in to the database 111 with consensus voting in blockchain
  • the database proxy service 102 in the depicted embodiment, is a proxy service for facilitating access to the database 111.
  • the database proxy service 102 receives requests from application 101. After receiving a request from the application 101 , the database proxy service 102 determines whether the request is a read request or a write request and translates the request into an appropriate structured query language (SQL) statement suitable for an RDBMS such as database 111.
  • SQL structured query language
  • the read or write requests may be translated into appropriate stored procedures or an application programming interface (API) calls using a predefined interface for low level data access for the particular database 111.
  • API application programming interface
  • Read requests related to blockchain data are forwarded to the query tables 105 while write requests are forwarded to working tables 103.
  • Local data read and write requests are forwarded to the private tables 104.
  • the application 102 performs a data operation requiring synchronization with the blockchain 110
  • the related operation is first directed to the working tables 103 by the proxy service 102, and the transaction log is generated after the working tables 103 perform the transaction operation, which is monitored by the transaction tracker component 106.
  • all the tables in the working tables 103 have a“Status” column to record the current record status as illustrated in FIG. 2.
  • Transaction state or status values for transactions that have yet to be confirmed by the blockchain 110 are said to be in a pending state or pending status, which includes status values“Pending insert”,“Pending update” and“Pending delete”.
  • the status is changed to“Committed” after a consensus is successfully reached in the blockchain. If the blockchain consensus operation fails, the relevant records in working tables 103 and related local tables will be rolled back, and the status column will be marked as“Committed”.
  • Query tables 105 contain data that results from consensus voting in blockchain system 110.
  • FIG. 3 schematically illustrates an example of a query table
  • a blockchain write service component 107 writes the extracted data change record, into the blockchain 110 to initiate a consensus operation.
  • the write service component 107 reads the data change record filtered by the query tables 105, then calls a blockchain smart contract, initiates a write request to the blockchain 110, and writes the changed one or more data records to the blockchain 110.
  • component 107 may use different blockchain application programming interface (API) for writing data.
  • API application programming interface
  • the content of the query tables 105 is recovered by blockchain writer module 107 from transaction logs recorded in blockchain system 110.
  • Transaction tracker component 106 is a log tracking component for monitoring logs generated by the database in real time and filtering the data change records of the private tables 104.
  • the blockchain listener service component 108 retrieves the block content, restores the data change log, converts to data change SQL statement and then plays back or executes the corresponding change operation in the query tables 105. If an operation conflicts with other operations which have already been initiated, the operation is discarded. If the current node identifier (node ID) is the same node ID in which change record is generated, the previous transaction information needs to be found in the log, and the original data will be restored in the worksheet or working tables 103. Meanwhile, rollback information is written back to the blockchain 110 to ensure data consistency.
  • Callback Service 109 generates a call back to application 101, whenever consensus voting in blockchain 110 is completed. If the transaction is generated through the current node, the blockchain listener module 108 generates a call back to application 101 and notifies the consensus result, both when it denotes success and failure.
  • a pre-defined callback API which is defined in configuration files.
  • the predefined API will be called when local node in blockchain 110 gets the result of consensus voting.
  • a message queue (MQ) or a shared database event table may be used. The result will be sent to the MQ or shared event table, and application 101 checks the MQ or event table to get notification.
  • Operational log data is recorded on the blockchain system 110.
  • the blockchain 110 is a bridge that connects all nodes 110a to 110e in different locations and performs consensus voting.
  • the order in which the blockchain 110 initiates consensus is used as a criterion for resolving blockchain transaction conflicts.
  • Each block in the blockchain is already synchronized across all nodes. Data modification will be presented as transaction log in blockchain 110.
  • application 101 converts the business operation into one or more database operations that form part of a database transaction.
  • the application 101 uses the database interface or proxy 102 for the database transaction.
  • FIG. 4 depicts a flowchart of activities undertaken by a device executing application 101.
  • a database transaction is initiated.
  • the underlying business operation may involve local database operations and blockchain operations that require consensus on the blockchain 110.
  • step 406 the application 101 modifies the working tables 103.
  • a consensus operation is required in the blockchain 110 before committing the transaction in the database 111.
  • step 408 data tracking module 106 tracks the changes in database working tables 103 and composes an operation record.
  • step 410 a write data operation record is inserted into the blockchain 110 through the blockchain writer module 107 to perform consensus.
  • step 411 after a period of time, the blockchain 110 completes the consensus, and a new block is generated and synchronized to the current node 110a.
  • the application 101 receives the consensus result and determines whether each record successfully completed the consensus. If the consensus is successfully completed (step 412), then the operation record in the query table 105 (step 414) is used to restore the SQL statement, perform the database operation and resume the record.
  • step 416 if the current node 110a is consistent with the data operation initiation node, the application 101 changes the record status in the working tables 103 and commits the transaction.
  • step 418 application 101 is notified by call back service 109 that the transaction has successfully completed the blockchain consensus. Otherwise application 101 copies the data in the query tables 105 to the corresponding working tables 103.
  • step 412 If the blockchain consensus fails (step 412), application 101 rolls back the database transaction and related records in database 111 are restored to the state before the transaction was initiated (step 420) and a notification is sent from call back service 109 that the transaction blockchain consensus failed (step 422).
  • the application 101 first accesses the local database 111.
  • update performance or speed of the blockchain 110 is much slower than the update speed of the database 111 , in one embodiment, asynchronous update is used to ensure that the blockchain content is consistent with the database content.
  • the two sets of tables in the database 111 are designed to handle data that needs to be synchronized with the blockchain 110.
  • the first set of tables includes local temporary working tables 103 used to save data temporarily updated by the database.
  • the second set includes query tables 105 that are used to restore data records from the blockchain 110 to the database.
  • a simple proxy in the form of database proxy service 102, is provided between the application 101 and database 111 for read- write separation, mapping read requests to query tables, and write operations to temporary tables.
  • Working tables 103 are copies of original tables having one more status/state columns added or appended thereto.
  • One of the added status column represents the transaction state the respective record or row.
  • this status column e.g., the last column of working table 103 shown in FIG. 2, may have four status values. In other embodiments, a larger or fewer number of status values or state values may be employed.
  • a status value of “Committed” means the data had finished consensus while a status value of “Insert pending” means the record is generated by an insert SQL command and is waiting for the consensus voting result.
  • a status value of“Update pending” means the row or record is modified by an update SQL and is awaiting the result of consensus voting.
  • a status value of “Delete pending” means the row is modified by a delete SQL, but not yet actually deleted and thus just marked for deletion while waiting for the consensus voting result.
  • the log monitor After the user of application 101 modifies the contents of the temporary table, the log monitor synchronizes the data modification log information and writes the log information to the blockchain 111.
  • the node that completes the local transaction operation writes the relevant data to the blockchain 110 through a blockchain adapter having a smart contract.
  • blockchain 110 exposes an API for a client to access the smart contract.
  • the adapter layer comprises smart contracts which can be invoked via the API to modify the blockchain data internally, and forms part of blockchain.
  • Each node in blockchain 110 may include smart contract module (not shown). If local data fails to be synchronized to the blockchain 110 for any one of a variety of reasons, then after the local node is restored, the blockchain adapter automatically restarts sending the relevant logs to the blockchain 110 for consensus from the position last interrupted.
  • the blockchain adaptation program monitors the blockchain changes and synchronizes the log data in the blockchain to the query tables according to the data recovery rules.
  • the blockchain adaptation program monitors the blockchain changes and synchronizes the log data in the blockchain to the query tables according to the data recovery rules.
  • the record can be restored into the query tables.
  • pending transaction data will be written to working table 103 before consensus but maintained in a pending status.
  • the status or state in the working table 103 is changed to a confirmed status. The transaction is rolled back from working table 103 if consensus is not reached.
  • the transaction originator node will generate a new rollback record and write it into the blockchain.
  • the transaction initiating node continues to determine whether the data in the working tables is a pending state that needs to be rolled back. If the status corresponding to the record is in a pending state, the current record is overwritten with its previous value and the status is set to“Committed”.
  • FIG. 5 is a flowchart of an exemplary process for executing a transaction.
  • application 101 initiates a data modification operation to start a database transaction, and the transaction will modify the data in the database.
  • the application 101 obtains the next execution statement in the transaction.
  • steps 503a, 503b, and 503c it is determined whether the data operation involves inserting, updating or deleting a record respectively.
  • Database proxy service 102 intercepts the application request, parses, and determines the applicable data operation and corresponding SQL statement.
  • step 504 if the request is to insert new data, an insert operation is carried out and the corresponding table name is modified to working tables’ name.
  • step 505 a new insert statement is prepared by adding a state field or status field, to the content of the current record.
  • states “pending insert”, “pending modify”, “pending delete” and “committed”.
  • states: “pending insert”, “pending modify”, “pending delete” are“pending” states or status values.
  • the status field for new data being inserted is“pending insert”; the status field for“pending modify” after data is modified, and status field for a record being deleted in“pending delete”.
  • step 506 a record is inserted into the corresponding working table.
  • step 507 a record is updated in a working table using an SQL statement.
  • the agent intercepts the SQL, and then modifies the corresponding working table.
  • step 508 application 101 executes“select for update” on the work table
  • step 509 it is determined whether the current record in a pending state or has a pending status. A record in a pending state cannot participate in another transaction as that will create a conflict. If there is a conflict between the current transaction and other transactions without the blockchain consensus, the current transaction cannot be executed and the transaction fails (step 519).
  • step 510 the update SQL is executed.
  • the data is updated to the working table 103, and the state of the record is modified to“pending update”.
  • step 511 the request statement is a delete statement, the proxy intercepts the SQL, and the modified corresponding table name is the working table name.
  • step 512 a selection for update on the work table 103 is made to lock the related records, and get the state or status. Other transactions thus cannot modify the current record during the execution of the subject transaction.
  • step 513 it is determined whether the status field of result set has a pending status. If there is a conflict between the current transaction and other transactions without the blockchain consensus, the current transaction cannot be executed and fails (step 519).
  • step 514 if there is no conflict record, the application 101 executes the update instruction, updates the status field in working table 103, and modifies the current state of all records involved in the transaction to“pending delete”.
  • step 515 the application checks if the current SQL is successfully executed, and if not, the transaction fails.
  • step 516 the application 101 determines whether to commit the transaction. If there are other transaction commands in the transaction, application 101 returns to step 502 to continue executing the remaining transaction commands.
  • step 517 the transaction is committed and in step 518 a transaction execution log is generated at the database node or other database log synchronization node, and the node identifier is recorded.
  • step 519 a transaction failure is declared and an equivalent message is sent to the application 101.
  • step 520 the transaction execution log is formatted in a suitable format.
  • the format may be a Java Script Object Notation (JSON) record. The record is then written to the blockchain through the blockchain interface to start consensus.
  • JSON Java Script Object Notation
  • Blockchain data is synchronized using a process to recover and synchronize the database data as summarized in FIG. 6.
  • step 601 a new block is generated in the blockchain and synchronized to the current node.
  • step 602 transaction log list from the current new block is decoded.
  • step 603 a check is made as to whether there is a record of the uncompleted playback of the transaction log list.
  • the transaction log list includes of multiple transactions, and each involves one or more records. Only records that require consensus records that need to be synchronized with other nodes, the local data is not placed on the chain.
  • the transaction has a record that has not been completed, it continues to convert the record to SQL for playback in the database. If the transaction has completed execution, the commit action is performed in the database. If there are other transactions to be executed, continue to get the next transaction and execute.
  • step 604 the next transaction record is obtained from the transaction log by order.
  • step 605 the next record is obtained in the transaction.
  • the record is restored to the SQL operation in accordance with its data content.
  • the data content includes the original value, the changed new value, the type of operation and operation node information.
  • step 606 the operation type of the data record is checked. The type is one of: insert, update or delete requiring a corresponding method for data recovery.
  • step 607 an insert statement is constructed.
  • step 608 the query table 105 is inspected to see if the primary key corresponding to the insertion statement already exists. If the key already exists, it indicates that the transaction may conflict with other transactions in the same block. In other words, other applications on a certain blockchain node have already started an insert operation using the same key, which will be performed first, and will lead to a second insert operation for the same record to fail. Recall that a primary key uniquely identifies a record in a table in RDBMS.
  • step 609 an insert operation into the query table.
  • step 610 it is determined whether the generated node of the log is the current node. If it is the current node, it indicates that there is a corresponding operation record in the working tables, which needs further processing.
  • step 611 record is directly inserted into the working table. It has already been determined that the log generation node is not the current node (in step 610). If the working table has the same data as the primary key, the current node performs a conflict operation after the other node performs the recording operation, and directly discards it. Record in the working table and use the query table record to overwrite and change the status to“committed”.
  • step 612 the corresponding record in the working tables is located, and the status changed to“committed”. It is necessary to confirm that the recorded content in the working tables is consistent with the current record, and if the records are inconsistent, the current record wins and is taken as the correct value.
  • step 613 an update statement (e.g. SQL) is prepared and in step 614, it is compared whether there is a record of the same primary key in the query tables. If the primary key is not found, there may be another deletion transaction affecting the record that conflicts with current operation, the corresponding record is deleted before the current record executed, and the update record cannot continue to execute, the current transaction will fail, which leads to roll back of all associated steps.
  • an update statement e.g. SQL
  • step 615 the record of having the primary key in query tables is updated and replaced with the new value.
  • step 616 the transaction is discarded and original values in working tables are restored into query tables.
  • step 618 it is checked if the current node and the record update node are the same node. If not, in step 619, the query tables data is copied to the working tables. Otherwise, in step 620 as differing values indicating pending transactions submitted to the blockchain, the working tables are simply overwritten. The new modification of the application in the working tables is notified to be discarded. The working tables 103 state is set to Committed.
  • step 621 the deletion process is executed.
  • step 622 it is checked if there a record in the query tables that has the same primary key to be deleted and if not, in step 623 the record is deleted from the corresponding record in the query tables.
  • step 624 Is the current node the same as the data submission node? If the same description of the working tables has the same record, it is necessary to additionally determine whether the pending transaction in the working tables 103 conflicts with the current transaction. If the original value is different, the conflict is confirmed, and the pending transaction in the working tables 103 is discarded and the application is notified.
  • step 625 the record corresponding to the primary key is deleted from the working tables 103.
  • step 626 the transaction is rolled back. If working tables 103 related transaction operation involves other local data operations, the working tables 103 local transaction log needs to be queried.
  • step 627 a reverse operation is built according to the transaction log, to roll back other operations in the same transaction in other data tables.
  • step 628 it is checked if the current transaction ends, and if so in step 629, the transaction is committed; otherwise the process continues to acquire the next transaction in the blockchain until all operations in the block are processed.
  • the insert operation is performed on the working tables 103.
  • the worksheet or working table performs a data update operation.
  • the data format for generating synchronization to the blockchain is as follows:
  • the work table performs a data update operation, and the original data state is changed from“committed” to “pending delete”.
  • FIG. 7 depicts a flowchart summarizing the steps for migrating a database existing application into an application that interacts with both its database and a blockchain.
  • step 701 the original database structures and data are exported.
  • step 702 tables that need synchronization with the blockchain or consensus voting are selected.
  • step 703 two tables are created for each original table that needs consensus voting or synchronization with blockchain.
  • One table is a work table for which a field or column for status or state is added, in addition to the exported data structure of the original table.
  • Another table is a query table, which is structurally completely consistent with the data structure of the original table.
  • step 704 exported historical data is imported into the worksheet, and set the content of the status or state field is set to "Committed”.
  • step 705 the exported historical data is imported into the query table.
  • a transaction tracking module is configured to track transaction change information of the worksheet.
  • step 707 at the application layer, the transaction consensus event processing code is written.
  • the application layer consensus result is notified by the message mode, and the application layer determines the subsequent processing mode according to the business logic.
  • step 708 for a different new blockchain, a blockchain adaptation code is written.
  • the adaptation code mainly includes code to write the event to the new blockchain, and code to listen to the new blockchain’s data change operation and to process the change.
  • Embodiments of the present invention lead to the development or modification of applications so that the applications can seamlessly choose certain critical data to be placed on a blockchain for data consensus while keeping others part of the data private.
  • augmenting existing applications with this capability does not require substantial modification of the code.
  • data consistency can be ensured.
  • Embodiments of the present invention simplify application development and deployment for certain class of applications, and may further speed up the time to market.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • General Health & Medical Sciences (AREA)
  • Bioethics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un système permettant d'augmenter une application de base de données avec une technologie de chaîne de blocs. Le procédé consiste à enregistrer des modifications de données effectuées par une application de base de données dans une base de données correspondante ainsi que sur une chaîne de blocs en vue d'une confirmation de consensus global. Cela est effectué sans modifier l'architecture d'application existante et avec un minimum de changements de code apportés à l'application existante. Les enregistrements dans la base de données nécessitant une synchronisation avec la chaîne de blocs sont soumis à un vote par consensus, et les changements non autorisés des bases de données sont annulés, ce qui permet d'accorder des caractéristiques d'immuabilité et de non-répudiation à une application de base de données classique. Les enregistrements dans les bases de données sont ainsi rendus globalement cohérents. Une application de base de données existante peut être déployée sur une chaîne de blocs sans modifier le code de manière significative. De multiples applications peuvent synchroniser des données au moyen d'une chaîne de blocs commune, ce qui simplifie considérablement les applications de chaîne de blocs de construction.
PCT/CA2019/051700 2018-12-04 2019-11-28 Système et procédé d'augmentation d'applications de base de données avec une technologie de chaîne de blocs WO2020113314A1 (fr)

Priority Applications (7)

Application Number Priority Date Filing Date Title
CA3121919A CA3121919C (fr) 2018-12-04 2019-11-28 Systeme et procede d'augmentation d'applications de base de donnees avec une technologie de chaine de blocs
KR1020217020788A KR20210135477A (ko) 2018-12-04 2019-11-28 데이터베이스 애플리케이션들을 블록체인 기술로 증강시키기 위한 시스템 및 방법
EP19894326.8A EP3891621A4 (fr) 2018-12-04 2019-11-28 Système et procédé d'augmentation d'applications de base de données avec une technologie de chaîne de blocs
CN201980090660.4A CN113396407A (zh) 2018-12-04 2019-11-28 用于利用区块链技术扩充数据库应用的系统和方法
US17/299,736 US20220019575A1 (en) 2018-12-04 2019-11-28 System And Method For Augmenting Database Applications With Blockchain Technology
JP2021532074A JP2022511084A (ja) 2018-12-04 2019-11-28 ブロックチェーン技術を用いてデータベースアプリケーションを増強するためのシステムおよび方法
IL283696A IL283696A (en) 2018-12-04 2021-06-03 System and method for increasing application databases with blockchain technology

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862775201P 2018-12-04 2018-12-04
US62/775,201 2018-12-04

Publications (1)

Publication Number Publication Date
WO2020113314A1 true WO2020113314A1 (fr) 2020-06-11

Family

ID=70974118

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2019/051700 WO2020113314A1 (fr) 2018-12-04 2019-11-28 Système et procédé d'augmentation d'applications de base de données avec une technologie de chaîne de blocs

Country Status (8)

Country Link
US (1) US20220019575A1 (fr)
EP (1) EP3891621A4 (fr)
JP (1) JP2022511084A (fr)
KR (1) KR20210135477A (fr)
CN (1) CN113396407A (fr)
CA (1) CA3121919C (fr)
IL (1) IL283696A (fr)
WO (1) WO2020113314A1 (fr)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112200573A (zh) * 2020-10-14 2021-01-08 北京天德科技有限公司 一种可回滚的区块链交易设计方法
CN112667641A (zh) * 2021-01-05 2021-04-16 中钞信用卡产业发展有限公司 一种可记录增删改操作的数据库系统及实现方法
US20210365440A1 (en) * 2020-05-22 2021-11-25 Couchbase, Inc. Distributed transaction execution management in distributed databases
CN115796874A (zh) * 2023-01-09 2023-03-14 杭州安节科技有限公司 一种操作级别的区块链交易并发执行方法
US11681687B2 (en) 2020-08-31 2023-06-20 Couchbase, Inc. Executing transactions on distributed databases
US11709831B2 (en) 2019-08-09 2023-07-25 Couchbase, Inc. Cost-based query optimization for array fields in database systems
US11741093B1 (en) 2021-07-21 2023-08-29 T-Mobile Usa, Inc. Intermediate communication layer to translate a request between a user of a database and the database
EP4120096A4 (fr) * 2020-09-15 2023-11-01 Tencent Technology (Shenzhen) Company Limited Procédé et dispositif de récupération de données, dispositif électronique et support de stockage
US12007985B2 (en) 2023-05-05 2024-06-11 Couchbase, Inc. Executing transactions on distributed databases

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11604890B2 (en) * 2017-10-20 2023-03-14 Hewlett Packard Enterprise Development Lp Accessing information based on privileges
WO2019078879A1 (fr) 2017-10-20 2019-04-25 Hewlett Packard Enterprise Development Lp Permissions provenant d'entités et donnant accès à des informations
US10810183B1 (en) * 2019-02-19 2020-10-20 Mythical, Inc. Systems and methods for synchronizing database operations with a distributed blockchain
US11611560B2 (en) * 2020-01-31 2023-03-21 Salesforce.Com, Inc. Systems, methods, and apparatuses for implementing consensus on read via a consensus on write smart contract trigger for a distributed ledger technology (DLT) platform
CN113590595A (zh) * 2021-09-30 2021-11-02 苏州浪潮智能科技有限公司 一种数据库多写方法、装置及相关设备
CN113886502B (zh) * 2021-12-03 2022-04-22 支付宝(杭州)信息技术有限公司 一种数据库和区块链同步的数据处理方法及系统
US11954074B2 (en) * 2022-04-28 2024-04-09 Micro Focus Llc Method and apparatus for efficient file/folder synchronization
CN114780642B (zh) * 2022-05-20 2022-08-26 北京链探科技有限公司 一种区块链数据处理方法、装置及电子设备
CN115129738B (zh) * 2022-08-30 2022-12-13 太极计算机股份有限公司 一种数据跨库写入方法、装置和设备
CN117874145B (zh) * 2024-03-13 2024-05-28 连连（杭州）信息技术有限公司 一种主从数据库的强一致方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120304A1 (en) * 2006-11-21 2008-05-22 Calio Robert J Method and system for providing high performance data modification of relational database tables
US20170295023A1 (en) * 2016-04-08 2017-10-12 Chicago Mercantile Exchange Inc. Bilateral assertion model and ledger implementation thereof
US20180039667A1 (en) * 2016-08-05 2018-02-08 Chicago Mercantile Exchange Inc. Systems and methods for blockchain rule synchronization
US20180096163A1 (en) * 2016-09-30 2018-04-05 Amazon Technologies, Inc. Immutable cryptographically secured ledger-backed databases
US20180253452A1 (en) * 2017-03-05 2018-09-06 Jonathan Sean Callan System and method for enforcing the structure and content of databases synchronized over a distributed ledger

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070079119A1 (en) * 2000-11-16 2007-04-05 Ulf Mattsson Encryption key rotation
US7774565B2 (en) * 2005-12-21 2010-08-10 Emc Israel Development Center, Ltd. Methods and apparatus for point in time data access and recovery
CN101187888A (zh) * 2007-12-11 2008-05-28 浪潮电子信息产业股份有限公司 一种异构环境中复制数据库数据的方法
CN101369283A (zh) * 2008-09-25 2009-02-18 中兴通讯股份有限公司 一种内存数据库与物理数据库间的数据同步方法及系统
US8788458B2 (en) * 2009-12-30 2014-07-22 Sybase, Inc. Data caching for mobile applications
US10671641B1 (en) * 2016-04-25 2020-06-02 Gravic, Inc. Method and computer program product for efficiently loading and synchronizing column-oriented databases
US10701054B2 (en) * 2018-01-31 2020-06-30 Salesforce.Com, Inc. Systems, methods, and apparatuses for implementing super community and community sidechains with consent management for distributed ledger technologies in a cloud based computing environment
CN108804112B (zh) * 2018-05-22 2022-02-11 上海分布信息科技有限公司 一种区块链落账处理方法及系统
US11323530B2 (en) * 2018-06-06 2022-05-03 International Business Machines Corporation Proxy agents and proxy ledgers on a blockchain

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080120304A1 (en) * 2006-11-21 2008-05-22 Calio Robert J Method and system for providing high performance data modification of relational database tables
US20170295023A1 (en) * 2016-04-08 2017-10-12 Chicago Mercantile Exchange Inc. Bilateral assertion model and ledger implementation thereof
US20180039667A1 (en) * 2016-08-05 2018-02-08 Chicago Mercantile Exchange Inc. Systems and methods for blockchain rule synchronization
US20180096163A1 (en) * 2016-09-30 2018-04-05 Amazon Technologies, Inc. Immutable cryptographically secured ledger-backed databases
US20180253452A1 (en) * 2017-03-05 2018-09-06 Jonathan Sean Callan System and method for enforcing the structure and content of databases synchronized over a distributed ledger

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3891621A4 *

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11709831B2 (en) 2019-08-09 2023-07-25 Couchbase, Inc. Cost-based query optimization for array fields in database systems
US20210365440A1 (en) * 2020-05-22 2021-11-25 Couchbase, Inc. Distributed transaction execution management in distributed databases
US20210365439A1 (en) * 2020-05-22 2021-11-25 Couchbase, Inc. Distributed transaction execution in distributed databases
US11681687B2 (en) 2020-08-31 2023-06-20 Couchbase, Inc. Executing transactions on distributed databases
EP4120096A4 (fr) * 2020-09-15 2023-11-01 Tencent Technology (Shenzhen) Company Limited Procédé et dispositif de récupération de données, dispositif électronique et support de stockage
CN112200573A (zh) * 2020-10-14 2021-01-08 北京天德科技有限公司 一种可回滚的区块链交易设计方法
CN112200573B (zh) * 2020-10-14 2021-08-17 北京天德科技有限公司 一种可回滚的区块链交易设计方法
CN112667641A (zh) * 2021-01-05 2021-04-16 中钞信用卡产业发展有限公司 一种可记录增删改操作的数据库系统及实现方法
US11741093B1 (en) 2021-07-21 2023-08-29 T-Mobile Usa, Inc. Intermediate communication layer to translate a request between a user of a database and the database
CN115796874A (zh) * 2023-01-09 2023-03-14 杭州安节科技有限公司 一种操作级别的区块链交易并发执行方法
US12007985B2 (en) 2023-05-05 2024-06-11 Couchbase, Inc. Executing transactions on distributed databases

Also Published As

Publication number Publication date
CA3121919C (fr) 2023-01-24
CN113396407A (zh) 2021-09-14
EP3891621A4 (fr) 2022-08-24
JP2022511084A (ja) 2022-01-28
EP3891621A1 (fr) 2021-10-13
IL283696A (en) 2021-07-29
KR20210135477A (ko) 2021-11-15
CA3121919A1 (fr) 2020-06-11
US20220019575A1 (en) 2022-01-20

Similar Documents

Publication Publication Date Title
CA3121919C (fr) Systeme et procede d'augmentation d'applications de base de donnees avec une technologie de chaine de blocs
CN111143389B (zh) 事务执行方法、装置、计算机设备及存储介质
CN111338766B (zh) 事务处理方法、装置、计算机设备及存储介质
US9690679B2 (en) Transaction commitment and replication in a storage system
EP4254183A1 (fr) Procédé et appareil de traitement de transaction, dispositif informatique et support de stockage
US9009116B2 (en) Systems and methods for synchronizing data in a cache and database
US11132350B2 (en) Replicable differential store data structure
US8117153B2 (en) Systems and methods for a distributed cache
US8892509B2 (en) Systems and methods for a distributed in-memory database
US10754854B2 (en) Consistent query of local indexes
CN111597015B (zh) 事务处理方法、装置、计算机设备及存储介质
US20130110781A1 (en) Server replication and transaction commitment
US9576038B1 (en) Consistent query of local indexes
US9652346B2 (en) Data consistency control method and software for a distributed replicated database system
US20070239661A1 (en) Systems and methods for a distributed in-memory database and distributed cache
US20090210428A1 (en) System and method for writing data dependent upon multiple reads in a distributed database database
EP4170509A1 (fr) Procédé de lecture d'un journal sur un noeud de données, noeud de données et système
EP4189914B1 (fr) Utilisation de multiples chaînes de blocs pour appliquer des transactions à un ensemble d'objets de données persistants dans des systèmes de stockage persistants
US8131700B2 (en) Transitioning clone data maps and synchronizing with a data query
US20230315713A1 (en) Operation request processing method, apparatus, device, readable storage medium, and system
WO2023111910A1 (fr) Rétractation de transaction de base de données
CN116089359A (zh) 数据库快照的生成方法、装置、电子设备和介质
CA2618938C (fr) Methode et logiciel de controle de la coherence des donnees d'un systeme de bases de donnees reparties et dupliquees
WO2023193495A1 (fr) Procédé de traitement de demande de lecture, base de données distribuée et serveur
US20240126747A1 (en) Control method, and adapter

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19894326

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
ENP Entry into the national phase

Ref document number: 3121919

Country of ref document: CA

ENP Entry into the national phase

Ref document number: 2021532074

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2019894326

Country of ref document: EP

Effective date: 20210705