CN112597241A - Block chain-based distributed database storage method and system - Google Patents
Block chain-based distributed database storage method and system Download PDFInfo
- Publication number
- CN112597241A CN112597241A CN202011458013.5A CN202011458013A CN112597241A CN 112597241 A CN112597241 A CN 112597241A CN 202011458013 A CN202011458013 A CN 202011458013A CN 112597241 A CN112597241 A CN 112597241A
- Authority
- CN
- China
- Prior art keywords
- database
- data
- node
- instruction
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 230000003993 interaction Effects 0.000 claims abstract description 8
- 238000012795 verification Methods 0.000 claims description 38
- 238000012545 processing Methods 0.000 claims description 9
- 241000282346 Meles meles Species 0.000 claims description 8
- 239000003999 initiator Substances 0.000 claims description 2
- 238000013500 data storage Methods 0.000 abstract description 9
- 230000006870 function Effects 0.000 abstract description 9
- 239000010410 layer Substances 0.000 description 30
- 230000008569 process Effects 0.000 description 13
- 238000013461 design Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 2
- 230000000977 initiatory effect Effects 0.000 description 2
- 238000004806 packaging method and process Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012792 core layer Substances 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000004366 reverse phase liquid chromatography Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/545—Interprogram communication where tasks reside in different layers, e.g. user- and kernel-space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
Abstract
The invention discloses a distributed database storage method and system based on a block chain, and belongs to the technical field of block chains. Each database node comprises a terminal module, a local database module and a database service module, and the database service module consists of a control layer, a storage layer and a service layer. The control layer is responsible for interaction between the client and the database in an RPC (remote procedure call) mode and an HTTP (hyper text transport protocol) mode; the storage layer is responsible for storing the service data; the service layer provides core functions of server read-write, authority control, block chain information access and the like. The database node of the invention stores the data updating request in two parts respectively. One part is that the operation instruction is stored in the blockchain, and the other part is that the operation instruction is executed by a local database, and data is actually stored. The method has the advantages of unlimited data storage structure, rich data query function and high data storage speed while realizing decentralized and safe data storage.
Description
Technical Field
The invention relates to the technical field of block chains, in particular to a distributed database storage method and system based on a block chain.
Background
The traditional database cannot meet the requirements in a specific decentralized scene, namely how to solve the credibility problem of a data storage layer in a multi-party cooperation scene. A trusted, decentralized database system is essential for both the current development of blockchain applications and for meeting social needs. In a multi-party collaboration scenario in which organizations such as companies, departments, or governments commonly use the same database system or maintain the same data, there are generally two common architectural design schemes.
One is to rely on the database to a third party and to give the database to a third party organization for management. Such as deploying the database on a cloud server of a cloud service provider.
In another design, the database is centrally deployed on a server provided by one participant or the database system is deployed in a distributed cluster mode by depending on one member of the participants, and the main node of the database is still on the server of one member.
However, neither of these approaches can satisfy the trust requirements in the multi-party collaboration process. When the main node fails or the data is maliciously tampered by the node, the whole database system can be unavailable. The invention provides a database architecture based on a block chain, which has the advantages of unlimited data storage structure, rich data query function and high data storage speed while realizing decentralized and safe storage of data.
Disclosure of Invention
In order to solve the credibility problem of a data storage layer under a multi-party cooperation scene, the invention provides a block chain-based distributed database storage method and a block chain-based distributed database storage system, so as to improve the safety of stored data and realize decentralized, efficient and safe data storage.
Each database node in the invention comprises a terminal module, a local database module and a database service module, wherein the database service module consists of a control layer, a storage layer and a service layer. The control layer is responsible for interaction between the client and the database in an RPC (remote procedure call) mode and an HTTP (hyper text transport protocol) mode; the storage layer is responsible for storing the service data; the service layer provides core functions of server read-write, authority control, block chain information access and the like. The database node of the invention stores the data updating request in two parts respectively. One part is that the operation instruction is stored in the blockchain, and the other part is that the operation instruction is executed by a local database, and data is actually stored.
In order to achieve the purpose, the invention adopts the following technical scheme:
one of the objectives of the present invention is to provide a distributed database storage method based on a blockchain, where the blockchain includes blockchain nodes, and each blockchain node is connected to one database node, and the distributed database storage method includes:
step 1: a user sends an instruction operation request through HTTP or RPC;
step 2: judging the type of the instruction operation request:
if the instruction operation request is a read instruction, directly reading data from the local block chain node and returning a result;
if the instruction operation request is a write instruction, starting consensus operation, and during the consensus operation, checking the validity of the instruction by each block chain node; after the consensus passes, each block chain node executes a write instruction on a local database, updates local data, returns an execution result to a user client, and stores the write instruction and the return result into the block chain node as an instruction execution log for recovering data and new node synchronization data after the node is down.
Preferably, the local database includes a main storage database and an auxiliary storage database, the main storage database is used for storing user data, the auxiliary storage database is used for storing verification node information and an instruction execution log, and the verification node information is stored in a key value pair < PublicKey, Power > form.
Preferably, the primary storage database adopts Redis, and the secondary storage database adopts Badger.
Preferably, the method further comprises a step of verifying a data type corresponding to the instruction operation request after the step 1, wherein the data type is divided into a private type and a public type; the private type data only has the read-write permission of the creation node to the data, and the other nodes only have the read permission to the data;
if the instruction operation request corresponds to private type data, after the validity is checked in step 2, whether the private data belongs to a node sending the instruction operation request is also needed to be judged, if so, a Redis client is called to execute a specific operation instruction, and if not, the operation instruction is not executed;
and if the instruction operation request corresponds to the data of the public type, directly calling the Redis client to execute a specific operation instruction.
As a preferred embodiment of the present invention, when each blockchain node executes a write instruction on a local database, there are three selectable write modes, which are:
commit write mode: correspondingly, writing the public type data, and writing the data into the database after common identification;
async write mode: correspondingly, writing the public type data, and writing the data into the database after common identification;
private write mode: the corresponding is writing of the private type data, and the data is written into the database and then is recognized.
Preferably, in the consensus operation, the voting right identified as the byzantine node in the previous consensus is dynamically adjusted.
Another objective of the present invention is to provide a distributed database system of a block chain based on the above storage method, where the block chain includes block chain nodes, and each block chain node is connected to one database node; each database node includes:
the terminal module belongs to a block chain node in a block chain and is used for providing block chain service;
the local database module is used for storing local information and consists of a main storage database and an auxiliary storage database;
the database service module is used for providing database storage service and comprises a control layer, a service layer and a storage layer; the control layer is used for realizing interaction between the user client and the local database module, and the interaction mode comprises RPC service and HTTP service; the storage layer is used for realizing the storage of the service data and storing the service data into the local database module; the service layer is used for bearing a control layer and a storage layer and is used for database reading and writing, authority control, block chain information access and user plug-in.
Further, the terminal module interacts through the ABCI interface, and includes:
initcain interface: the system comprises a local storage database, a local Redis service and a global identity module, wherein the local storage database is used for detecting whether a local Redis service normally runs, reading all verification node information in a consensus operation and storing the verification node information to a local variable set, waiting for subsequent synchronization to a block chain node, and storing the verification node information to an auxiliary storage database in a key value pair mode;
CheckTx interface: the system is used for detecting whether the node signature is legal or not, for the write request of private data, before the checkTx is returned, a Redis client is called to execute a specific operation instruction, the executed result is returned in a character string mode, and the result is returned to a user client after being analyzed;
BeginBlock interface: the system comprises a database, a block chain node, a checking point information database, a local variable set and a block chain node, wherein the database is used for storing the checking point information of the Byzantine node, which appears in the consensus operation, and storing the updated checking point information into the auxiliary storage database;
DeliverTx interface: the system comprises a data processing unit, a data processing unit and a data processing unit, wherein the data processing unit is used for executing instruction operation requests, including write requests of public data types, write requests of private data types and requests for updating verification nodes;
EndLock interface: for synchronizing verification node information to local blockchain nodes.
Further, the DeliverTx interface firstly resolves the address of the update operation initiating node, the public key of the update target verification node and the voting right for the request of updating the verification node, stores the request in the local variable set, and if the same request is collected from nodes more than two thirds of the total verification nodes in the set, the update operation is regarded as a valid update.
Furthermore, the database service module comprises three writing interfaces, three reading interfaces and an interface for acquiring log information;
the reading interface comprises:
a Query interface: for querying the common data type;
QueryPravateData: the private data type of the local node is inquired;
QueryPravatedDataWithAddress interface: a private data type for querying a specified node;
the three reading interfaces carry out validity check on the instruction before the Redis service is called, wherein the validity check comprises whether the instruction is a Redis valid reading instruction, whether the data type is legal and whether the address is legal, and the query result is returned to the control layer;
the writing interface comprises:
execution interface: write for common type data, corresponding to Commit write mode;
exenuteasync interface: for writing of common type data, corresponding to Async write mode;
ExecuteWithPrivateKey interface: for writing of Private-type data, corresponding to the Private write mode;
the interface for obtaining the log information is used for obtaining the update log, the input parameter is a Key to be inquired, and the return type is an OperationKeyLog type.
The invention has the technical effects that:
1) compared with the existing block chain database architecture, the distributed database architecture based on the block chain can realize the effects that the data storage structure is not limited, the complete SQL query function is supported, and the data reading and writing speed is high.
2) The invention sets two data types and three writing modes, wherein the setting of the private data type ensures that each node reserves the modification authority of partial data, and the flexibility and the use scene of the system can be increased. In most scenes of multi-party cooperation, specific data is newly added or updated by one party, the situation that the other party reads the data is more, and the situation that the multiple parties repeatedly modify the same data is less, so that the two situations are independent into two data types, and the optimization of execution performance is facilitated. After two different data types are divided, data can be written into the block chain in different modes aiming at the different data types, so that the effect of optimizing performance is achieved.
3) The invention utilizes the interface provided by the terminermint, the voting right which is determined as the Byzantine node in the consensus operation is dynamically adjusted, the probability that the Byzantine node becomes a promiser in the future can be reduced, and the consensus process is more stable.
Drawings
FIG. 1 is a block chain-based distributed database node architecture according to the present invention;
FIG. 2 is a block chain-based distributed database service routine architecture according to the present invention;
FIG. 3 is a schematic diagram of a Terdermint consensus mechanism in a block chain-based distributed database according to the present invention;
FIG. 4 is a call timing diagram of an Execute interface according to an embodiment of the present invention;
FIG. 5 is a flowchart illustrating an execution of the ExecuteWithPrivateKey method according to an embodiment of the present invention;
FIG. 6 is an example of a data structure for the OperationKeyLog type in an embodiment of the present invention;
Detailed Description
The invention will be further illustrated with reference to the following figures and examples:
as shown in fig. 1, each database node includes three main components, namely a terminal module responsible for providing block chain related services, a local database module, and a database service module providing database services.
The database service module is a part of the main implementation of the system, and the detailed architecture of the database service module is shown in fig. 2. The method mainly comprises three layers, namely a control layer, a service layer and a storage layer.
A control layer: the method is mainly responsible for interaction between the user client and the local database module. For the design of the control layer, mainly combining the applicable scenes of the system, the interactive mode is designed into two modes, one is an interactive mode mainly based on RPC, the other is an interactive mode in the form of HTTP service, the system must provide an HTTP interface to respond to non-data operation requests, such as node state query, block chain information access and the like, and the RPC service only provides interfaces related to data operation.
A storage layer: the storage layer is mainly responsible for storing the business data and stores the business data into the local database module. The system comprises two parts, one is a local database part, a Redis main storage database is mainly used for executing specific user instructions, and a Badger auxiliary storage database is mainly used for storing auxiliary information irrelevant to user service data. In addition, part of the data may be stored in the form of files, such as RPCs, HTTP service logs, user profiles, and so on. The other part is the memory content associated with the blockchain.
And (3) a service layer: the service layer, which carries the control layer and the storage layer, is the core layer of the whole system, and consists of several main core functions: database reading and writing, authority control, block chain information access and a user plug-in.
The following describes the portion of the blockchain used in the present invention and its terminology.
Each Transaction in each block in the block chain directly corresponds to a write operation of a node in the database system. The block structure of the invention is modified based on the basic structure provided by the terminal frame, and is mainly characterized in that a database instruction log and return information are merged into a block body. The structure of one block in terminal mainly comprises a block head and a block body.
The block header information mainly includes meta information of the block. The most important information in the block body is a list of all transactions contained in one block, and precisely, a Hash list (Hash) of the transactions is stored in the block body, wherein the core information is a Data field in the transactions, the Data field is defined by terminrminertin, and the following information is stored in the Data field:
and (3) operating instructions: an executable Redis instruction. It should be noted that all the operation instructions stored in transactions are write instructions. During the block consensus phase, each verification node (Validator) checks the operation instructions, including the instruction validity (whether invalid or disabled instructions are included).
Instruction sequence number: the 128-bit instruction sequence number generated by the UUIDv1 algorithm is generated based on the MAC address and the time stamp of the physical machine, and the main function is to provide a global ID to mark each instruction and ensure uniqueness of transactions. For example, a node in the cluster sends two Transaction requests with the same operation instruction to the cluster for consensus, and if the Transaction does not have any identifier, the cluster cannot distinguish whether the operation is a retry request caused by network congestion or two identical data operation requests.
And (3) node address: the address token request sender, in block consensus operation, also verifies whether the address is valid by each verification node.
Node signing: and after the JSON character string consisting of the operation instruction, the instruction serial number and the node address information is converted into a byte array, the byte array is signed and generated by a verification node private key through an Ed25519 algorithm.
A consensus algorithm: the consensus algorithm of the block chain realized by the invention is realized based on the Terdermint consensus algorithm, and partial improvement is made on the basis of the algorithm to improve the stability of the system.
And dynamically adjusting the voting Power (Power) of the verification node every time the new block consensus process is started. In Terdermint, there is a Height value (block Height) for each block, and in the whole block chain, the block Height is just like an index in a linked list, and uniquely corresponds to one block. In the enddermint consensus algorithm, a block chain is at a new height, and a new block needs to be packed and consensus, which is described in fig. 3.
Enddermint will start the consensus process for the next height block at a fixed time or after receiving a certain number of transactions. The consensus of the block needs to go through four phases, namely, Propose, Prevote, Precommit and Commit, wherein the three phases of Propose, Prevote and Precommit are collectively called a Round. Multiple rounds can agree on a new block, mainly because not every proposal or vote is feasible. The main process of the Terdermint BFT consensus algorithm is as follows: the system starts a new Height consensus process according to a specific condition or a time interval, then calculates a propeser according to the current Round state, packs the blocks by the propeser, broadcasts, votes for two rounds of Prevotes and Precommit, adds one to the Round value if the voting process is abnormal, returns all the node states to the propese stage, packs by the calculated new propeser, and votes again. And if all voting flows pass smoothly, the node Commit new block completes consensus.
The following is a detailed description of each stage in the consensus process:
after the nodes enter a position stage, each node calculates a position node corresponding to the current Height and Round, namely a block node, in Terdermint, a default position node calculation method is actually only related to the Round and the voting right of the node, in the system, a verification node which is identified as a Byzantine node in the previous Round of consensus is dynamically adjusted by using an interface provided by Terdermint.
Data reading and writing:
the present invention defines two data types. The local storage uses a Key value pair database Redis, data written into the database by a user is divided by a Key, and the following two data types are defined:
the private type: the private type data only has the creation node with the read-write authority to the data, and the other nodes only have the read authority to the type data. The Key of this type of data is named in the form of an Address _ Key, where Address is the Validator Address of Peer. When the type data is modified, whether the node initiating the request is a corresponding node needs to be verified. The purpose of setting the private type is mainly two, one is that the modification authority for reserving partial data for each node can increase the flexibility and the use scene of the system. On the other hand, in most of the multi-party cooperative scenarios, a certain party adds or updates specific data, while the other party reads data mostly, and the situation that multiple parties modify the same data repeatedly is a few, so that the two situations are independent into two data types, which is beneficial to optimizing the execution performance, and the following three data writing modes will be described in detail.
Common types: a public type Key cannot start with the Validator Address of any Peer in the cluster, and this condition is checked when block chains and blocks are in common.
There are three different write modes for database writes. Different write modes are designed primarily to optimize database write performance. After two different data types are divided, data can be written into the block chain in different modes aiming at the different data types, so that the effect of optimizing performance is achieved. The three writing modes are Commit mode, Async mode and Private mode:
commit write mode: the writing of the common type data is corresponded, and the writing mode is the least efficient one. This is actually a consensus-then-write database model, and user requests may be blocked until block chain consensus is completed and database execution is completed.
Async write mode: correspondingly, the data of the common type is written, the storage process is also to write the data base after the data base is identified, but the difference is that the mode returns immediately after the Transaction is accepted by the terminal instance, and the subsequent identification and the data base writing are carried out asynchronously. This mode is user-friendly in that the results of the database execution are not actually available.
Private write mode: corresponding is the writing of private type data. This mode is in fact a mode that is commonly recognized after writing a database, in contrast to the execution logic of the two modes. The return timing of the Private write mode is returned when the Transaction confirms that the memory pool (memory) written into the terminal passes the user verification logic. This is referred to as a write-first-then-store-last-consensus pattern because the execution of the instruction by the database is accomplished in user check logic, which allows the user to obtain the instruction to the result of the execution.
The invention realizes the terminal ABCI interface:
the realization of the alliance chain in the system is realized by relying on a Terdermint framework. The main content that the terminal frame needs to be implemented by the user is an ABCI (application area chain interface, hereinafter abbreviated as ABCI interface), which is defined by terminal and given to a plurality of function interfaces implemented by the user. The interaction between the terminal instance and the ACBI interface can influence part of links of block packaging and consensus, and the block chain is customized in such a way. There are many ABCI interfaces provided by terminertin, and the present invention will only be described in detail for the implementation of a few core interfaces.
InitChain interface: the interface is called by the terminertin instance after the blockchain is initialized. In the system, whether the local Redis service normally runs or not is mainly detected, and if the Redis service is found to be unavailable, a local operating system instruction is tried to be called to start the local Redis service until the Redis normally runs. On the other hand, the method is to read all verification node information in the consensus network, read and store the verification node set information of the system initial in a local variable valaddrtoppubkeymap, and store the voting right information of the verification nodes in a form of key value pairs < PublicKey, Power > in the Badger auxiliary database. The purpose of storing the verification node information is mainly to vote and use by the auxiliary node when a new node is added subsequently.
CheckTx interface: the calling time of the interface is when the Transaction enters the local terminal memory pool, and the return value of the interface determines whether the Transaction is confirmed or deleted by the memory pool. The system mainly detects whether the instruction and the signature in the Transaction are legal or not. Second, for a write request in private mode, before the transaction check CheckTx method returns, the Redis client is invoked to perform specific operation instructions. The method for determining whether the write command is in the private mode is to determine whether the write command to be executed starts with the valid Address of the local node. The result of Redis execution will be stored in the Data field of the CheckTx interface return value ResponseCheckTx in the form of character string, and then returned to the client after being resolved in the server.
A BeginBlock interface: the BeginBlock interface is the first ABCI interface called after endrmint completes consensus on a block and writes in a block chain. The system is mainly used for recording Byzantine nodes of the block in the consensus process. The ByzantineValidators field recorded in the RequestBeginBlock is the Byzantine node in the block consensus process. And after the recorded Byzantine nodes are taken out, the voting right is reduced by one, the updating of the voting right is stored in a local Badger auxiliary database, the updated information is stored in a valUpdates local variable, and the subsequent synchronization to a Terdermint network is waited. In fact, in the improvement of the terminertin consensus algorithm introduced in the design section, the dynamic adjustment of the voting right of the verification node is completed in the interface.
DeliverTx interface: one block may contain one or more transactions, and the Transaction execution DeliverTx interface is the processing logic for a single Transaction. Termindermint calls the DeliverTx interface once in sequence for each Transaction in the block separately. The main parameter in the entry parameter of the DeliverTx interface is Transaction data, which is entered in the form of byte arrays. There are three types of transactions received, which are: a write request of a public data type, a write request of a private data type, a request to update Validator. For the three requests, they will be processed separately in DeliverTx.
For the write request of the public data type, the Operation field is analyzed from the Transaction byte array, and then the Redis client is called to execute the write request. And the execution result is assigned to the Data field of the interface return type ResponseDeliverTx, and the Data field is analyzed by the server and then returned to the client.
For the request of the private data type, after validity check, whether the private data belongs to the node is judged, if the private data belongs to the node, the request is already executed at checkTx, and if the private data belongs to the private data of other nodes, the Redis client is called to execute the request. If the Transaction is a request for updating the verification node information, the address of the update operation initiator is analyzed, and the public key and the voting right of the target verification node are updated. The request is saved in a valaddrnote local set variable, if the same request is found to have been collected from more than two thirds of the total verification nodes in the set, the update operation is regarded as a valid update, and the latest voting right of the updated target is saved in the Badger auxiliary database. In addition, the request is saved to the valUpdates variable, synchronized to terminrmindermint in the endlock interface.
EndLock interface: after teldermint completes the DelivertTx interface for each Transaction within the block, the endlock interface will be called, which is the interface that Tendermint provides to the user to implement the post-logic. The interface in the system is mainly used for synchronizing verification node information to local block link nodes. In the whole ABCI calling process, changes related to a verification node set are all stored in a local variable valUpdates, information of the valUpdates variable is assigned to a ValidatorUpdates field of an EndBlock return type ResponseEndBlock, and after a Terdermint calls an EndBlock interface, if the ValidatorUpdates field in a return value is not empty, node information is updated, and information synchronization is performed with other nodes in a network.
The core read-write interface of the database system mainly comprises three write interfaces and three read interfaces.
Introducing a reading interface, wherein a Query interface is a common Query interface and queries a public data type; the QueryPravatedData interface corresponds to an interface for inquiring the private data type of the node; the QueryPravatedDataWithAddress interface corresponds to an interface for inquiring the private data type of the specified node. For a reading interface, the implementation logic is simpler, and the data is read by directly calling Redis service to execute specific operation because the change of the database data is not involved. The three read interfaces carry out validity check on the instruction before calling Redis service, wherein the validity check comprises whether the read instruction is a Redis valid read instruction, whether a Key is valid (a private type and a public type are mainly checked here), and whether an Address is valid (mainly aiming at a QueryPrivatedDataWithAddress interface). The query result is returned to the control layer for subsequent packaging.
The three write interfaces are an Execute interface, an Execute async interface and an Execute withprivitekey interface respectively. Fig. 4 to 5 are a call timing chart of an Execute interface and a call timing chart of an Execute withprivatekey interface, respectively, where the Execute interface corresponds to a Commit write mode, and a common data type is written; the ExecuteAsync interface corresponds to an Async write mode and also serves a public data type; the ExecuteWithPrivateKey corresponds to a Private write mode and serves Private data types. The following table is an example of the returned results of the Execute interface:
the invention also realizes an interface for acquiring the Key log. The GetKeyLog is an interface for acquiring Key log information, the interface aims to acquire all update logs created by a certain Key so far, and for a system for storing certificates through a trusted log, the query function of the update log of the certain Key is naturally essential. The entry of the GetKeyLog interface is to-be-queried Key, the return type is OperationKeyLog type, and the class diagram is shown in fig. 6.
The GetKeyLog interface is realized through a Badger auxiliary database and an event listener of WebSocket. The log information is stored in the Badger database. For example, an instruction log with "item" Key is requested, and an example of the return result is shown in the following table.
The distributed database storage process mainly comprises the following steps:
a user sends an instruction operation request through HTTP or RPC;
judging the type of the instruction operation request:
if the instruction operation request is a read instruction, directly reading data from the local block chain node and returning a result;
if the instruction operation request is a write instruction, starting consensus operation, and during the consensus operation, checking the validity of the instruction by each block chain node; after the consensus passes, each block chain node executes a write instruction on a local database, updates local data, returns an execution result to a user client, and stores the write instruction and the return result into the block chain node as an instruction execution log for recovering data and new node synchronization data after the node is down.
The local database comprises a main storage database and an auxiliary storage database, wherein the main storage database is used for storing user data, the auxiliary storage database is used for storing verification node information and an instruction execution log, and the verification node information is stored in a key value pair < publicKey, Power > form.
In a specific implementation of the present invention, the method further comprises a step of verifying a data type corresponding to the instruction operation request, wherein the data type is divided into a private type and a public type; the private type data only has the read-write permission of the creation node to the data, and the other nodes only have the read permission to the data;
if the instruction operation request corresponds to private type data, after the validity is verified, whether the private data belongs to a node for sending the instruction operation request is also required to be judged, if the private data belongs to the node, a Redis client is called to execute a specific operation instruction, and if the private data does not belong to the node, the Redis client is not executed;
and if the instruction operation request corresponds to the data of the public type, directly calling the Redis client to execute a specific operation instruction.
The foregoing lists merely illustrate specific embodiments of the invention. It is obvious that the invention is not limited to the above embodiments, but that many variations are possible. All modifications which can be derived or suggested by a person skilled in the art from the disclosure of the present invention are to be considered within the scope of the invention.
Claims (10)
1. A distributed database storage method based on a blockchain, wherein the blockchain comprises blockchain nodes, and each blockchain node is connected with one database node, and the distributed database storage method comprises the following steps:
step 1: a user sends an instruction operation request through HTTP or RPC;
step 2: judging the type of the instruction operation request:
if the instruction operation request is a read instruction, directly reading data from the local block chain node and returning a result;
if the instruction operation request is a write instruction, starting consensus operation, and during the consensus operation, checking the validity of the instruction by each block chain node; after the consensus passes, each block chain node executes a write instruction on a local database, updates local data, returns an execution result to a user client, and stores the write instruction and the return result into the block chain node as an instruction execution log for recovering data and new node synchronization data after the node is down.
2. The method of claim 1, wherein the local database comprises a primary storage database and a secondary storage database, the primary storage database is used for storing user data, the secondary storage database is used for storing verification node information and instruction execution logs, and the verification node information is stored in a form of key value pair < PublicKey, Power >.
3. The method according to claim 2, wherein the primary storage database is a Redis database, and the secondary storage database is a Badger database.
4. The method for storing the distributed database based on the blockchain according to claim 1 or 2, further comprising a step of verifying a data type corresponding to the instruction operation request after the step 1, wherein the data type is classified into a private type and a public type; the private type data only has the read-write permission of the creation node to the data, and the other nodes only have the read permission to the data;
if the instruction operation request corresponds to private type data, after the validity is checked in step 2, whether the private data belongs to a node sending the instruction operation request is also needed to be judged, if so, a Redis client is called to execute a specific operation instruction, and if not, the operation instruction is not executed;
and if the instruction operation request corresponds to the data of the public type, directly calling the Redis client to execute a specific operation instruction.
5. The blockchain-based distributed database storage method according to claim 1, wherein when each blockchain node executes a write command on the local database, there are three selectable write modes, which are:
commit write mode: correspondingly, writing the public type data, and writing the data into the database after common identification;
async write mode: correspondingly, writing the public type data, and writing the data into the database after common identification;
private write mode: the corresponding is writing of the private type data, and the data is written into the database and then is recognized.
6. The blockchain-based distributed database storage method according to claim 1, wherein in the consensus operation, the voting right identified as a byzantine node in a previous round of consensus is dynamically adjusted.
7. A distributed database system based on a blockchain of the storage method of any one of claims 1 to 6, the blockchain comprising blockchain nodes, each blockchain node being connected to a database node; wherein each database node comprises:
the terminal module belongs to a block chain node in a block chain and is used for providing block chain service;
the local database module is used for storing local information and consists of a main storage database and an auxiliary storage database;
the database service module is used for providing database storage service and comprises a control layer, a service layer and a storage layer; the control layer is used for realizing interaction between the user client and the local database module, and the interaction mode comprises RPC service and HTTP service; the storage layer is used for realizing the storage of the service data and storing the service data into the local database module; the service layer is used for bearing a control layer and a storage layer and is used for database reading and writing, authority control, block chain information access and user plug-in.
8. The distributed database system of blockchains according to claim 7, wherein the terminertin modules interact through an ABCI interface, comprising:
initcain interface: the system comprises a local storage database, a local Redis service and a global identity module, wherein the local storage database is used for detecting whether a local Redis service normally runs, reading all verification node information in a consensus operation and storing the verification node information to a local variable set, waiting for subsequent synchronization to a block chain node, and storing the verification node information to an auxiliary storage database in a key value pair mode;
CheckTx interface: the system is used for detecting whether the node signature is legal or not, for the write request of private data, before the checkTx is returned, a Redis client is called to execute a specific operation instruction, the executed result is returned in a character string mode, and the result is returned to a user client after being analyzed;
BeginBlock interface: the system comprises a database, a block chain node, a checking point information database, a local variable set and a block chain node, wherein the database is used for storing the checking point information of the Byzantine node, which appears in the consensus operation, and storing the updated checking point information into the auxiliary storage database;
DeliverTx interface: the system comprises a data processing unit, a data processing unit and a data processing unit, wherein the data processing unit is used for executing instruction operation requests, including write requests of public data types, write requests of private data types and requests for updating verification nodes;
EndLock interface: for synchronizing verification node information to local blockchain nodes.
9. The distributed database system of block chains according to claim 8, wherein the DeliverTx interface first resolves the address of the update operation initiator node, the public key of the update target verification node, and the voting right for the request of updating the verification node, stores the request in the local variable set, and considers the update operation as a valid update if it is found that the same request is initiated by nodes collected from more than two thirds of the total verification nodes in the set.
10. The distributed database system of block chains according to claim 7, wherein the database service module comprises three write interfaces, three read interfaces, and an interface for obtaining log information;
the reading interface comprises:
a Query interface: for querying the common data type;
QueryPravateData: the private data type of the local node is inquired;
QueryPravatedDataWithAddress interface: a private data type for querying a specified node;
the three reading interfaces carry out validity check on the instruction before the Redis service is called, wherein the validity check comprises whether the instruction is a Redis valid reading instruction, whether the data type is legal and whether the address is legal, and the query result is returned to the control layer;
the writing interface comprises:
execution interface: write for common type data, corresponding to Commit write mode;
exenuteasync interface: for writing of common type data, corresponding to Async write mode;
ExecuteWithPrivateKey interface: for writing of Private-type data, corresponding to the Private write mode;
the interface for obtaining the log information is used for obtaining the update log, the input parameter is a Key to be inquired, and the return type is an OperationKeyLog type.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011458013.5A CN112597241A (en) | 2020-12-10 | 2020-12-10 | Block chain-based distributed database storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011458013.5A CN112597241A (en) | 2020-12-10 | 2020-12-10 | Block chain-based distributed database storage method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112597241A true CN112597241A (en) | 2021-04-02 |
Family
ID=75192967
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011458013.5A Pending CN112597241A (en) | 2020-12-10 | 2020-12-10 | Block chain-based distributed database storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112597241A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112988667A (en) * | 2021-05-12 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Data storage method and device based on block chain network |
| CN113254540A (en) * | 2021-06-30 | 2021-08-13 | 中元众诚(北京)科技有限公司 | Database operating system and scheme based on block chain |
| CN113610525A (en) * | 2021-08-24 | 2021-11-05 | 上海点融信息科技有限责任公司 | Financial data processing method, device, equipment and medium based on block chain |
| CN115129790A (en) * | 2022-07-28 | 2022-09-30 | 北京航空航天大学 | Consistency decentralized storage system based on block chain |
| CN116737810A (en) * | 2023-05-06 | 2023-09-12 | 清华大学 | Consensus service interface for distributed time sequence database |
| WO2023206142A1 (en) * | 2022-04-27 | 2023-11-02 | 西门子股份公司 | Data synchronization method and apparatus, computer device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292181A (en) * | 2017-06-20 | 2017-10-24 | 无锡井通网络科技有限公司 | Database Systems based on block chain and the application method using the system |
| CN109241181A (en) * | 2018-08-08 | 2019-01-18 | 北京百度网讯科技有限公司 | Database operation method and device |
| CN109347804A (en) * | 2018-09-19 | 2019-02-15 | 电子科技大学 | A kind of Byzantine failure tolerance common recognition optimization method for block chain |
| CN109934005A (en) * | 2019-03-15 | 2019-06-25 | 北京物资学院 | A kind of degree confirmation system and authentication method based on block chain |
| US20200117825A1 (en) * | 2018-10-16 | 2020-04-16 | Microsoft Technology Licensing, Llc | Database management |
| CN111475576A (en) * | 2020-04-15 | 2020-07-31 | 中国工商银行股份有限公司 | Block chain based distributed database storage method and system |
-
2020
- 2020-12-10 CN CN202011458013.5A patent/CN112597241A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107292181A (en) * | 2017-06-20 | 2017-10-24 | 无锡井通网络科技有限公司 | Database Systems based on block chain and the application method using the system |
| CN109241181A (en) * | 2018-08-08 | 2019-01-18 | 北京百度网讯科技有限公司 | Database operation method and device |
| CN109347804A (en) * | 2018-09-19 | 2019-02-15 | 电子科技大学 | A kind of Byzantine failure tolerance common recognition optimization method for block chain |
| US20200117825A1 (en) * | 2018-10-16 | 2020-04-16 | Microsoft Technology Licensing, Llc | Database management |
| CN109934005A (en) * | 2019-03-15 | 2019-06-25 | 北京物资学院 | A kind of degree confirmation system and authentication method based on block chain |
| CN111475576A (en) * | 2020-04-15 | 2020-07-31 | 中国工商银行股份有限公司 | Block chain based distributed database storage method and system |
Non-Patent Citations (4)
| Title |
|---|
| 尤晋元: "《高等学校教材 UNIX操作系统教程》", 30 June 1985, 西安电子科技大学出版社 * |
| 甘宏等: "《Linux系统管理与配置教程》", 31 July 2019, 北京希望电子出版社 * |
| 粒子区块链: "Cosmos整体流程", 《HTTPS://ZHUANLAN.ZHIHU.COM/P/259575199》 * |
| 美图技术团队: "深度解析Tendermint，快速融入Cosmos生态", 《HTTPS://ZHUANLAN.ZHIHU.COM/P/38252058》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112988667A (en) * | 2021-05-12 | 2021-06-18 | 腾讯科技（深圳）有限公司 | Data storage method and device based on block chain network |
| CN112988667B (en) * | 2021-05-12 | 2021-09-07 | 腾讯科技（深圳）有限公司 | Data storage method and device based on block chain network |
| CN113254540A (en) * | 2021-06-30 | 2021-08-13 | 中元众诚(北京)科技有限公司 | Database operating system and scheme based on block chain |
| CN113610525A (en) * | 2021-08-24 | 2021-11-05 | 上海点融信息科技有限责任公司 | Financial data processing method, device, equipment and medium based on block chain |
| CN113610525B (en) * | 2021-08-24 | 2024-01-19 | 上海点融信息科技有限责任公司 | Processing method, device, equipment and medium of financial data based on blockchain |
| WO2023206142A1 (en) * | 2022-04-27 | 2023-11-02 | 西门子股份公司 | Data synchronization method and apparatus, computer device and storage medium |
| CN115129790A (en) * | 2022-07-28 | 2022-09-30 | 北京航空航天大学 | Consistency decentralized storage system based on block chain |
| CN115129790B (en) * | 2022-07-28 | 2024-04-19 | 北京航空航天大学 | Consistent decentralization storage system based on block chain |
| CN116737810A (en) * | 2023-05-06 | 2023-09-12 | 清华大学 | Consensus service interface for distributed time sequence database |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112597241A (en) | Block chain-based distributed database storage method and system | |
| CN110727731B (en) | Method for adding node in block chain network and block chain system | |
| US10103893B2 (en) | Blockchain-supported, fail-safe synchronization in a data authentication infrastructure | |
| US20230316273A1 (en) | Data processing method and apparatus, computer device, and storage medium | |
| CN112287034B (en) | Data synchronization method, equipment and computer readable storage medium | |
| US10924281B2 (en) | Method and apparatus for inter-blockchain transmission of authenticable message | |
| EP3698518A1 (en) | Primary and secondary blockchain device | |
| CN109493052B (en) | Cross-chain contract system based on main chain and parallel multiple sub-chains | |
| CN101771537A (en) | Processing method and certificating method for distribution type certificating system and certificates of certification thereof | |
| CN112883117B (en) | Data synchronization method, equipment and computer readable storage medium | |
| CN111163148B (en) | Synchronization method and related equipment for consensus state of block chain system | |
| CN113269546B (en) | User identity card system and method based on block chain | |
| CN111342971A (en) | Byzantine consensus method and system | |
| US20230289782A1 (en) | Smart contract-based data processing | |
| WO2023045617A1 (en) | Transaction data processing method and apparatus, device and medium | |
| CN111444204B (en) | Synchronous processing method, device, equipment and medium | |
| CN111798234B (en) | Lightweight block chain system and construction method | |
| US20230370285A1 (en) | Block-chain-based data processing method, computer device, computer-readable storage medium | |
| JP2024515022A (en) | Blockchain-based data processing method, device, equipment, and computer program | |
| CN116827957B (en) | Information processing method, device, equipment and medium based on multi-block chain | |
| CN111414417A (en) | Video copyright management method based on block chain | |
| CN116614519A (en) | Video and related information lightweight trusted uplink method based on optimization consensus algorithm | |
| CN114499874A (en) | Byzantine fault-tolerant consensus optimization method applied to industrial internet | |
| CN116760632B (en) | Data processing method, device, equipment and readable storage medium | |
| US20240163118A1 (en) | Blockchain-based data processing method, device, and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210402 |