CN112035466B - External index development framework for block chain query - Google Patents
External index development framework for block chain query Download PDFInfo
- Publication number
- CN112035466B CN112035466B CN202010746552.2A CN202010746552A CN112035466B CN 112035466 B CN112035466 B CN 112035466B CN 202010746552 A CN202010746552 A CN 202010746552A CN 112035466 B CN112035466 B CN 112035466B
- Authority
- CN
- China
- Prior art keywords
- data
- index
- query
- logic
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000011161 development Methods 0.000 title abstract description 10
- 230000001360 synchronised effect Effects 0.000 claims description 31
- 230000006978 adaptation Effects 0.000 claims description 20
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000013500 data storage Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 abstract description 15
- 238000000034 method Methods 0.000 description 48
- 238000010586 diagram Methods 0.000 description 14
- 230000008569 process Effects 0.000 description 12
- 238000013515 script Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention provides an external index development framework for block chain query, which aims to improve the query efficiency of an application platform layer to a block chain infrastructure layer in a block chain and enable the block chain query functions to be more various. The frame is applied to a blockchain, the frame comprising: the contract generation module establishes contracts according to the index logic and the query logic defined by the index module; modifying SDK calling codes and query logic in the contracts according to the query requirements to complete contract definition; the data adapting module adapts a data structure on the blockchain to a frame preset data structure; the data synchronization module selects a corresponding data synchronization mode to synchronize the data on the block chain into the frame; after data synchronization, index logic in the contract is called to establish an index, and index data is stored in a corresponding preset database; according to the established index, query logic in the contract is called to complete the data query task, and according to the query requirement, the database returning query data can be defined.
Description
Technical Field
The invention relates to the field of blockchains, in particular to an external index development framework for blockchain inquiry.
Background
The blockchain is a distributed shared ledger and database, is agreed through a specific consensus algorithm, performs distributed accounting on the whole network, and has the characteristics of decentralization, incapability of tampering, whole-course trace, traceability, collective maintenance, openness, transparency and the like. Initially, the application of the blockchain is that the two parties of the transaction generate untrustworthy transactions under the condition of mutual distrust in the bitcoin public chain project, but the pursuit of the value of the blockchain by researchers is more than that, with the continuous maturity of the blockchain technology and the landing of the application, the blockchain finally enters into a high-speed development maturity stage, and based on the data volume and the application volume of the blockchain, the quality leaps, and how to provide a rich inquiry function for massive applications becomes one of increasingly important research directions.
However, since the blockchain infrastructure layer is often focused on improvement of writing performance, the interface requirement of the application platform layer is less, so that only a small number of standard query interfaces can be provided, and the application platform layer is difficult to define uniform query requirements to enable the infrastructure layer to be realized due to changeable requirements, so that the interfaces provided by the infrastructure layer cannot be directly used by the application platform layer, and further, the query function of the application platform layer on the blockchain infrastructure layer is single and the query efficiency is low.
Disclosure of Invention
In view of the above problems, an embodiment of the present invention provides an external index development framework for blockchain query, which aims to improve the query efficiency of an application platform layer to a blockchain infrastructure layer in a blockchain, so that the blockchain query functions are more diverse.
In order to solve the above problems, an embodiment of the present invention provides a frame for developing an external index for a blockchain query, where the frame is applied to a blockchain, and the frame includes a contract generating module, an index module, a data adapting module, and a data synchronizing module;
The index module defines index logic and query logic, wherein the index logic comprises self-defined index logic and built-in index logic, and the built-in index logic comprises a time index and an account index;
the contract generation module establishes a contract script according to the index logic defined by the index module;
Modifying the SDK calling code and the query logic in the contract into the SDK calling code and the query logic of the block chain to be queried according to the query requirement, and completing contract definition;
According to the difference of the data structures of the block chains, the data adaptation module adapts the data structures on the block chains to the data structures predefined by the framework;
According to different block chain types, the data synchronization module selects a data synchronization mode corresponding to the block chain type and synchronizes the data on the block chain into the frame;
After the data synchronization module synchronizes data, the contract calls an index logic defined by the index module to establish an index, and the index data is stored in a preset database;
According to the established index, the contract calls the query logic defined by the index module to complete index data query conforming to the query condition and return a query result;
and defining a database returning query according to the query requirement, and adding the SDK query code into query logic to finish the query of acquiring the data content through the hash value.
Optionally, the contract generating module establishes a contract according to the index logic defined by the index module, including:
When the defined index logic is built-in index logic, the contract generation module completes filling of codes according to the built-in index logic, selects a preset database type for index data storage, and establishes the contract;
When the defined index logic is the self-defined index logic, the contract generation module selects a preset database type stored by index data according to self-defined code filling of the self-defined index logic, and establishes the contract; wherein the custom index logic is stored locally.
Optionally, the data adapting module adapts the data structure on the blockchain to a frame predefined data structure according to a blockchain data structure difference, including:
The data adaptation module adapts the blockchain data structure to a frame predefined data structure;
The data adaptation module maps data on a blockchain into a predefined data structure in a frame, and inputs the field names and field types of the blockchain data into the data adapter, and the data adaptation module automatically adapts the blockchain data into the predefined data structure of the frame.
Optionally, according to different blockchain types, the data synchronization module selects a data synchronization mode corresponding to the blockchain type, and synchronizes data on the blockchain into the frame, including:
The data synchronization module selects a data synchronization mode corresponding to the block chain according to different block chain types;
synchronizing data on the blockchain into the frame according to the selected data synchronization mode;
the data synchronization mode comprises block height data synchronization, traversal database data synchronization and synchronous hash data synchronization;
The data synchronization module sets a timer, stores the currently synchronized block hash and/or block height in a cache queue, and when synchronization is performed again after synchronization is interrupted, performs breakpoint continuous transmission on the currently synchronized block hash and/or block height recorded by the timer.
Optionally, the data synchronization module further includes:
And the data synchronization module records the newly generated block hash into a memory queue, and after the data synchronization module completes the synchronization of the currently traversed block data, performs data synchronization on the newly generated block data in the queue, and continues to synchronize the data on the block chain after completing the synchronization of the newly generated block data in the queue.
Optionally, after the data synchronization module synchronizes data, the contract calls an index logic defined by the index module to establish an index, and the index data is stored in a preset database, including:
The data synchronization module completes data synchronization;
the contract calls an index logic defined by the index module to establish an index, and index data are stored in a preset database;
The preset database is provided with a plurality of types, and the preset database storage index data corresponding to the query requirements are selected according to the query requirements applied to different scenes;
The database adapter in the index module adapts the preset database to be of a database type with unified frames;
the database adapter decouples various preset databases with heterogeneous interfaces, encapsulates a unified frame preset interface, and the preset databases share the frame preset interface to operate the preset databases.
Optionally, the preset database further includes:
when the preset database does not meet the query requirement, a new database type is accessed in a self-defined mode;
the database adapter adapts the new database type of the custom access to a database type with unified framework.
The invention has the following advantages:
In the invention, a contract generating module establishes a contract according to index logic and query logic defined by an index module, modifies SDK calling codes and query logic in the contract according to query requirements to complete contract definition, a data adapting module adapts data structures on different blockchains to preset data structures for frameworks, a data synchronizing module selects a corresponding data synchronizing mode to synchronize data on the blockchains into the frameworks, after the data is synchronized, the index logic in the contract is called to establish an index, index data is stored in a corresponding preset database, the query logic in the contract is called according to the established index to complete data query tasks, and database-returning query data can be defined according to query requirements. The method and the system realize the synchronization of data on different block chains into the frame, establish indexes of the synchronous data, finish data query through the established indexes, improve the query efficiency of an application platform layer in the block chain to a block chain infrastructure layer, and enable the block chain query functions to be more various.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a blockchain query external index development framework in accordance with an embodiment of the present invention;
FIG. 2 is a class diagram of the database layer package of the present invention;
FIG. 3 is a contract creation interface diagram of the present invention;
FIG. 4 is a class diagram of a data adaptation module of the present invention;
fig. 5 is a class diagram of a data synchronization module of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
The inventive concept of the embodiment of the invention is as follows: the contract generation module establishes a contract script according to the index logic and the query logic defined by the index module, modifies SDK calling codes and the query logic in contracts for different blockchains to complete definition of the contracts, synchronizes data on the blockchain into a frame after the data adaptation module adapts the data structure on the blockchain into a data structure preset by the frame, calls the index logic to establish an index, stores index data into a preset database, invokes the query logic after completing data index, completes data query meeting query conditions in the index data, and can define whether to return to the blockchain main database to perform data query according to query requirements. Therefore, the query interfaces of the application platform layer and the blockchain infrastructure layer can be decoupled, and data on different blockchains are synchronized into the frame to perform data query tasks, so that the query efficiency of the application platform layer to the blockchain infrastructure layer in the blockchain is effectively improved, and the blockchain query functions are more various.
Referring to fig. 1, fig. 1 is a flowchart of a blockchain query external index development framework according to an embodiment of the present invention. The framework is applied to a blockchain and comprises a contract generation module, an index module, a data adaptation module and a data synchronization module.
In the invention, the block hash value refers to that a block chain encrypts transaction information in a transaction block through a hash algorithm and compresses the information into a hash character string consisting of a string of numbers and letters, and the hash value uniquely and accurately identifies the block in the block chain.
In the invention, the block height refers to the number of blocks connected to the main chain, namely the number of blocks connected to the block chain, and different blocks in the current block chain correspond to different block heights.
In the present invention, a contract refers to an intelligent contract, which is a section of computer program deployed on a blockchain and capable of automatically executing terms.
In the present invention, a contract engine refers to an underlying architecture in which a contract is run, and the contract is run in the contract engine.
In the present invention, the infrastructure layer refers to blockchain items that provide read-write interfaces, such as bitcoin, ethernet, etc.
In the invention, the application platform layer refers to a blockchain service such as electronic invoice, data storage certificate, digital asset and the like which is completed through various read-write interfaces provided by a blockchain infrastructure layer.
As shown in fig. 1, the blockchain query external index development framework includes:
S101, the index module defines index logic and query logic, wherein the index logic comprises self-defined index logic and built-in index logic, and the built-in index logic comprises a time index and an account index.
Illustratively, the indexing module is comprised of an indexing controller, a data processing algorithm portion, and an indexing algorithm portion. The index controller is called by the data synchronization module, when the latest index request on the chain reaches the index controller, the controller distributes the index request to different index logic scripts for execution according to index configuration, after the index logic scripts call built-in index logic or OnBlock () functions in index logic provided by a user to establish data indexes, whether a specific algorithm is required to be called to construct a data structure is detected, for example, transaction data are processed into a format required by base statistics, after the data are processed into a required format, the index data are stored in corresponding databases, and the index data are stored by using the corresponding databases according to query requirements applied to different scenes. After the index data is stored in the corresponding database, a OnQuery () method in the query logic is called, and transaction data conforming to the query condition is queried according to the index data.
The index module is divided into two parts, one is to build an index part, which comprises built-in index logic and custom index logic, and can define a plurality of index logics at the same time, and after acquiring data on a block chain, all indexes required by one-time construction are built; the other is a conditional query part, wherein the conditional query refers to querying the data on the chain according to index data, the condition can be one condition or a plurality of conditions, and meanwhile, the query logic can be customized, and flexible query tasks are supported when the query requirement changes.
In the invention, the index controller refers to a controller running in a contract engine and monitoring an index logic call request, and the index logic call request is called to the URL corresponding to the index logic to realize the call of the index logic, so that the index is constructed.
The index controller data structure is as in table 1:
TABLE 1
UrlList is the call path of the index, and the assignment is performed after the contract generation module generates the index script. Blockqueue is a queue for storing hashes of blocks to be synchronized, and the hash value is transferred by the data synchronization module. The Dispatch method establishes an index by continuously acquiring the block hash in blockqueue and transmitting the block hash as a parameter to a corresponding index logic according to the index Url configured by the user.
After the index logic establishes the index, a preset database corresponding to the query requirement is selected to store index data according to the query requirement applied to different scenes, for example, a full-text index database is used for enabling a large amount of text data to have searching and analyzing capabilities, a relational database is suitable for a scene with more and less writing, a K-V database is used for a scene with quicker single-value query, and the insertion efficiency is highest compared with the former two databases.
The interfaces of different database examples are heterogeneous, the call interfaces and operation commands of the databases are required to be continuously switched when different databases are used for storing index data, and the query efficiency is reduced when the database is frequently switched to be called and operated. For this reason, in the embodiment of the present invention, the database adapter in the index module adapts the preset database to a frame unified database type, and the database adapter decouples multiple preset databases with heterogeneous interfaces, encapsulates a unified frame preset interface, and the preset databases share the frame preset interface to operate the preset databases. Specifically, after decoupling multiple preset databases with heterogeneous interfaces by the database adapter, unifying the database layers with heterogeneous interfaces, adapting different types of database instances to be framework unified database instances, and when operating different database instances, designating the used database instances, namely operating the database instances by using interfaces provided by an index framework, for example, operating acquiring, storing and deleting certain data of the database, and operating creating and acquiring the database instances of database management. The database adapter performs interface encapsulation on common operations, encapsulates the common operations into a custom_repositisource interface, and unifies the operations of various database instances through one interface class.
The custom_restore interface is as in table 2:
TABLE 2
Get is to obtain the data in the database, in the course of inquiring, obtain the data in the database meeting the inquiry condition; put is to store data into a database, and after finishing index establishment, store index data into the database; delete is deleting data in the database; the creating_DB is used for creating a new database instance type, when the preset database does not meet the query requirement, the new database type is accessed in a self-defined mode, and the database adapter is used for adapting the self-defined accessed new database type to be a database type with unified frames; open DB is an Open database instance.
After encapsulating the operations of the various database instances into a custom_restore interface, the database adapter encapsulates the database layer.
Referring to fig. 2, fig. 2 is a class diagram of the database layer package of the present invention. As shown in fig. 2, DBUtils is responsible for managing the instance of customjrep, which is obtained by GETINSTANCE (). CreateDB is used for creating a database instance file, type=0 represents a database instance type, type is defined as a database instance constant, and a corresponding database instance is acquired in GETINSTANCE () of the core through a constant value of the type instance type, so that the call of the corresponding database instance is realized. In practice, when the existing database type does not meet the index requirement, the database instance type may be added at DBUtils.
When the method is specifically implemented, after the index logic is established by a user, according to the encapsulated custom_repositisource interface and the encapsulated database layer, database instance types meeting the query requirements of different application scenes can be called through appointed database instance constants. For example, the full-text index database instance constant is 0, the relational database instance constant is 1, and the K-V database instance constant is 2. If the constant of the database instance is designated as 1, the index data is stored in the relational database, and the framework directly performs various operations on the database through the custom_repositive interface.
S102: the contract generation module establishes a contract according to the index logic defined by the index module; according to the query requirement, the SDK calling code and the query logic in the contract are respectively modified into the SDK calling code and the query logic which need to query the blockchain, and the contract definition is completed.
Illustratively, the contract generation module completes the creation of the contract through the index logic and the query logic defined by the index module, and the main framework of the contract generation module is as shown in table 3, which defines the creation flow of the contract.
OnCreate, when executing the contract, judging whether the database exists, if not, creating the database, and if so, acquiring the database instance and storing the database instance into the global variable. The OnBlock method is used by the data synchronization module to pass the chunk hash or the entire chunk into the contract method when a new chunk is acquired. OnTransaction is a refinement of the OnBlock method, where the user adapts the transactions in the block and then indexes them. The OnQuery method is used for query condition construction and result return, and needs to be filled in by a user according to requirements. The database storage part can provide a built-in database for users, and under the condition that the existing database can not meet the requirements of the users, the users can expand the storage layer through the interface of the custom_database to create the Custom database instance type. The index frame can add the user-rewritten code according to the fixed code part to generate a corresponding contract file.
TABLE 3 Table 3
The built-in index logic templates are shown as table 4, and variables such as $ { IndexName } and the like generate a contract script file in running by acquiring key value pairs set in a contract generation module.
TABLE 4 Table 4
For ease of understanding, referring to FIG. 3, FIG. 3 is a contract creation interface diagram of the present invention. When the defined index logic is built-in index logic, the contract generation module completes filling of codes according to the built-in index logic, selects a preset database type for storing index data and establishes contracts; when the defined index logic is the self-defined index logic, the contract generation module fills the self-defined codes according to the self-defined index logic, selects the preset database type stored by the index data, establishes a contract, and stores the self-defined index logic to the local. Specifically, as shown in fig. 3, when the defined indexing logic is an indexing logic built in the framework, such as a time index is used for querying the transaction amount in a period of time, an account index is used for querying transaction information generated by the account, and the framework automatically fills OnBlock a method, onTrans a method and OnQuery a method code to generate required contracts by checking required indexes. When the built-in index logic does not meet the requirements of users, the index logic can be customized, in the process of creating contracts of the custom index logic, a contract generating module generates a basically complete contract, users complete the custom index logic and/or the query logic by changing partial codes in the OnBlock method and the OnQuery method, and for the same blockchain, the required contracts can be generated only by selecting fields needing to be indexed and database instance types. The custom index logic is saved to the local for the next time the same custom index is built, the index logic saved to the local is directly invoked. When creating the contract, the path of the SDK package is fully called, and when the framework generates codes, the SDK calling part is replaced by the provided SDK object.
And S103, according to the difference of the data structures of the block chains, the data adaptation module adapts the data structures on the block chains to be data structures predefined by the framework.
Illustratively, the step of the data adaptation module adapting the data structure on the blockchain to the data structure predefined by the frame is that the data adaptation module maps the data on the blockchain to the data structure predefined by the frame, the field names and the field types of the blockchain data are input into the data adapter, and the data adaptation module automatically adapts the blockchain data to the data structure predefined by the frame. Specifically, by constructing the adapter, the predefined data is stored under a folder, when the data structure of the corresponding chain needs to be acquired, only the predefined basic data structure in the frame needs to be operated, the function of the data structure on the operation chain is realized, the mapping of the data on the block chain to the predefined data structure of the frame is completed, and the specific adapting process is shown in table 5.
TABLE 5
Referring to fig. 4, fig. 4 is a class diagram of the data adaptation module of the present invention. As shown in fig. 4, transactions represent predefined primitives in the indexing framework, chaintrans meaning on-chain transactions, which are primitives that create the index. The two realize chaintrans assignment through TRANSADAPTER, input the data field in the blockchain into chaintrans, and perform type conversion and processing on the corresponding field, and convert the corresponding field into the data type which can be directly used by the index framework.
After the adaptation is completed, the data on the blockchain is called through the operation of the predefined data in the index frame by adopting a predefined data structure calling mode as shown in table 6. In an intelligent contract engine IDE, a function of calling jar package functions is provided, an adapter method is called in a contract in a path reference mode, and the calling of data on an adapted blockchain is realized through the calling of a predefined data structure. newtrans is the adapted data structure.
TABLE 6
S104: and according to different block chain types, the data synchronization module selects a data synchronization mode corresponding to the block chain type and synchronizes the data on the block chain into the frame.
In the invention, three data synchronization modes are provided according to different block chain structures. The first is block height data synchronization; secondly, traversing the block chain database directly by contract, continuously calling Iterator methods in the database to synchronize data in the block chain, adding partial parameters when the data in the block chain is stored, and deserializing key data read out from the block chain database when the data is synchronized; thirdly, synchronizing the synchronous hash data, wherein a request code is added into a main chain of a block chain to obtain a block hash, and the newly generated block hash is synchronized into a framework.
For example, after the data structures on different blockchains are adapted to the definition of the preset data structures in the frame, defining a data synchronization mode, and according to the differences of the blockchain types, selecting the data synchronization mode corresponding to the blockchain type by the data synchronization module to synchronize the data on the blockchain into the frame.
Referring to fig. 5, fig. 5 is a class diagram of the data synchronization module of the present invention. As shown in FIG. 5, syncData is a data synchronization class for storing data synchronization progress while sending newly generated chunk hashes to a queue. CKPTHEIGHT records the currently synchronized block height, hashQueue stores the currently pending block hash. SyncByIterator denotes a method of synchronizing by traversing a database; syncByHeight, a method for traversing the blockchain by height, stopping the synchronization when the height is 0, and simultaneously, syncByHeight will regularly call the SDK to acquire the height of the block to synchronize the latest block; syncByHash is a method of passively synchronizing chunk hashes that is invoked to synchronize the latest chunk hashes into the index frame when a new chunk is generated by the blockchain.
DButils is a database instance called in SyncData for persisting storage progress.
DBInstance is a database instance, GETINSTANCE controls the retrieval of specified database instances for specified database types.
IndexDispatch the index controller is a class that the data synchronization module invokes the index controller to distribute index requests to different contracts.
SDKCLIENT are SDK clients of different blockchains, through SDKCLIENT can obtain the latest height of the current blockchain and obtain the transaction through hashing or obtain the block through a block hash.
CustomRep is an interface after encapsulating a preset database instance operation, and when a new database is accessed, the method in CustomRep can implement switching of database types through automatically generated codes in the framework, without modifying existing script codes.
According to the different block chain structures, the data synchronization is performed by adopting a specified synchronization mode, for example, the data synchronization is performed by adopting a single chain structure by adopting a block height, and the data synchronization is performed by adopting a traversing database or a synchronous hash data synchronization by adopting a pattern structure. After the data synchronization mode is designated, the task of data synchronization is designated to SyncData classes for completion, syncData classes acquire a database instance through DBUtils tool classes, acquire the current synchronization progress through the database instance, judge whether the on-chain synchronization progress exists or not, if the on-chain synchronization progress does not exist, the data synchronization mode is the first data synchronization, and the synchronization module can newly establish a block chain synchronization progress preservation variable. If the hash value is synchronized by the block, the height is stored in the database, and if the hash value is synchronized by the block, the received hash value to be synchronized is stored in the queue structure.
When data synchronization is performed by adopting block height traversal, setting a timer by SyncData types after obtaining the latest height of a block chain, and obtaining blocks on the block chain through a GetBlock () method in SDKCLIENT every preset time period; when a traversing database synchronization mode is adopted, in the data synchronization process, a data deserialization method of a database in a block chain is obtained, and data in the database is deserialized and then is synchronized; when synchronous hash data is adopted for synchronization, a request code is added to a main chain of a blockchain, a blockhash value is obtained, and the newly generated blockhash is synchronized into a framework.
And for the synchronization of the traversing database and the synchronization hash data in the data synchronization module, generating new blocks on the block chain simultaneously when the data of the currently traversed blocks are synchronized, recording the newly generated block hashes into a memory queue by the data synchronization module, and after the data synchronization module completes the synchronization of the currently traversed block data, performing the data synchronization on the newly generated block data in the queue, and continuing to synchronize the data on the block chain after the synchronization of the newly generated block data in the queue is completed. Specifically, in the data synchronization process, the data is traversed from a time point of the database, then the traversed block is subjected to data synchronization, a new block is generated in the block chain in the synchronization process, when the traversed block is subjected to data synchronization, the traversed block cannot be synchronized to the new generated block, in order to synchronize to the new generated block in the synchronization process, the data synchronization module records the new generated block into the memory queue, when the traversed block is subjected to data synchronization, the new generated block in the queue is synchronized, and after the new generated block in the queue is synchronized, the new round of data synchronization on the block chain is performed.
When the data synchronization is interrupted in the synchronization process, the data synchronization module sets a timer, the currently synchronized block hash and/or block height is stored in a cache queue, and when synchronization is carried out again after the interruption of the synchronization, breakpoint continuous transmission is carried out on the currently synchronized block hash and/or block height recorded by the timer. Specifically, a timer is set in the data synchronization module SyncData, the current synchronization block hash or block height is recorded every preset time period, when synchronization is resumed after synchronization is interrupted, breakpoint continuous transmission is performed from the block height or block hash recorded in the timer by reading the block height or block hash, and data synchronization is performed again at the block height or block hash position from which synchronization is interrupted.
S105: after the data synchronization module synchronizes data, the contract calls the index logic defined by the index module to establish an index, and the index data is stored in a preset database.
For example, each time the SyncData classes obtain the latest synchronization block, information of the index request is sent to the index request controller, and the request link of the index requester is called in a contract calling mode. After receiving the index request, the index script distributes the index request to different index contracts according to the index type configured by the user, acquires complete blocks through an SDK method before sending the index request and transmits the block information to the contracts in the form of parameters. After the contract calls OnBlock () method, the block is traversed, and the OnTransaction () method in the contract is executed for each transaction in the block to build the contract. After the OnTransaction method is called, an Adapter method is called to perform data adaptation, and the data fields on the chain are adapted to the data types in the index frame. The GETINSTANCE method in DBUtils is called by the index contract script to acquire a database instance, the index is built through the Put method storage data, and the index data is stored into the acquired database instance, so that a complete index building process is completed.
S106: according to the established index, the contract calls the query logic defined by the index module to complete index data query conforming to the query condition and return a query result; and defining a database returning query according to the query requirement, and adding the SDK query code into query logic to finish the query of acquiring the data content through the hash value.
For example, after the blockchain data index is established, the contract calls the query logic OnQuery () method defined by the index module to query the transaction meeting the query condition, for example, the number of transactions on the blockchain, the transaction details of a certain account and the query content corresponding to a certain keyword in a certain time period, and the query result is returned after the query is completed. According to the query requirement, whether to query the database is defined, for example, when the synchronous block data is incomplete or the synchronous data is the index main key hash value, the database query logic can be defined, the SDK query code is added into the query logic, and the database chain main database can acquire the specific transaction content data of the corresponding block of the corresponding hash value.
The embodiment of the invention has the following advantages: in the invention, a contract generating module establishes a contract according to index logic and query logic defined by an index module, modifies SDK calling codes and query logic in the contract according to query requirements to complete contract definition, a data adapting module adapts data structures on different blockchains to preset data structures for frameworks, a data synchronizing module selects a corresponding data synchronizing mode to synchronize data on the blockchains into the frameworks, after the data is synchronized, the index logic in the contract is called to establish an index, index data is stored in a corresponding preset database, the query logic in the contract is called according to the established index to complete data query tasks, and database-returning query data can be defined according to query requirements. The method and the system realize the synchronization of data on different block chains into the frame, establish indexes of the synchronous data, finish data query through the established indexes, improve the query efficiency of an application platform layer in the block chain to a block chain infrastructure layer, and enable the block chain query functions to be more various.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or terminal device that comprises the element.
The above description of the external index development framework for blockchain query provided by the invention applies specific examples to illustrate the principle and implementation of the invention, and the above examples are only used for helping to understand the method and core ideas of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (7)
1. The system is applied to a block chain and is characterized by comprising a contract generation module, an index module, a data adaptation module and a data synchronization module;
The index module defines index logic and query logic, wherein the index logic comprises self-defined index logic and built-in index logic, and the built-in index logic comprises a time index and an account index;
The contract generation module establishes a contract according to the index logic defined by the index module;
Modifying the SDK calling code and the query logic in the contract into the SDK calling code and the query logic of the block chain to be queried according to the query requirement, and completing contract definition;
According to the difference of the data structures of the block chains, the data adaptation module adapts the data structures on the block chains to the predefined data structures of the system;
according to different block chain types, the data synchronization module selects a data synchronization mode corresponding to the block chain type and synchronizes the data on the block chain into the system; the data synchronization module sets a timer, stores the currently synchronized block hash and/or block height in a cache queue, and performs breakpoint continuous transmission from the currently synchronized block hash and/or block height recorded by the timer when synchronization is performed again after synchronization is interrupted;
After the data synchronization module synchronizes data, the contract calls an index logic defined by the index module to establish an index, and the index data is stored in a preset database;
According to the established index, the contract calls the query logic defined by the index module to complete index data query conforming to the query condition and return a query result;
and defining a database returning query according to the query requirement, and adding the SDK query code into query logic to finish the query of acquiring the data content through the hash value.
2. The system of claim 1, wherein the contract generation module establishes a contract according to the indexing logic defined by the indexing module, comprising:
when the defined index logic is built-in index logic, the contract generation module completes filling of codes according to the built-in index logic, selects a preset database type for index data storage, and establishes the contract;
When the defined index logic is the self-defined index logic, the contract generation module selects a preset database type stored by index data according to self-defined code filling of the self-defined index logic, and establishes the contract; wherein the custom index logic is stored locally.
3. The system of claim 1, wherein the data adaptation module adapts the data structure on the blockchain to a system predefined data structure based on blockchain data structures, comprising:
the data adaptation module adapts the blockchain data structure to a system predefined data structure;
The data adaptation module maps data on a blockchain into a predefined data structure in a system, and inputs the field names and field types of the blockchain data into the data adapter, and the data adaptation module automatically adapts the blockchain data into the predefined data structure of the system.
4. The system of claim 1, wherein the data synchronization module selects a data synchronization pattern corresponding to a blockchain type based on the blockchain type, and synchronizes data on the blockchain into the system, comprising:
The data synchronization module selects a data synchronization mode corresponding to the block chain according to different block chain types;
synchronizing data on the blockchain into the system according to the selected data synchronization mode;
the data synchronization mode comprises block height data synchronization, traversal database data synchronization and synchronous hash data synchronization.
5. The system of claim 4, wherein the data synchronization module further comprises:
And the data synchronization module performs data synchronization on the block data of the new block in the queue after finishing the synchronization of the block data currently traversed, and continues to synchronize the data on the block chain after finishing the synchronization of the block data newly generated in the queue.
6. The system of claim 1, wherein the contract invokes the indexing logic defined by the indexing module to create an index after the data synchronization module synchronizes the data, the index data stored in a predetermined database, comprising:
The data synchronization module completes data synchronization;
the contract calls an index logic defined by the index module to establish an index, and index data are stored in a preset database;
The preset database is provided with a plurality of types, and the preset database storage index data corresponding to the query requirements are selected according to the query requirements applied to different scenes;
The database adapter in the index module adapts the preset database to be a unified database type of the system;
The database adapter decouples various preset databases with heterogeneous interfaces, encapsulates a unified system preset interface, and the preset databases share the system preset interface to operate the preset databases.
7. The system of claim 6, wherein the pre-set database further comprises:
when the preset database does not meet the query requirement, a new database type is accessed in a self-defined mode;
the database adapter is used for adapting the new database type accessed by the user to be a unified database type of the system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010746552.2A CN112035466B (en) | 2020-07-29 | 2020-07-29 | External index development framework for block chain query |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010746552.2A CN112035466B (en) | 2020-07-29 | 2020-07-29 | External index development framework for block chain query |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112035466A CN112035466A (en) | 2020-12-04 |
| CN112035466B true CN112035466B (en) | 2024-04-19 |
Family
ID=73583513
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010746552.2A Active CN112035466B (en) | 2020-07-29 | 2020-07-29 | External index development framework for block chain query |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112035466B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113704248B (en) * | 2021-07-12 | 2024-01-09 | 北京邮电大学 | Block chain query optimization method based on external index |
| CN114168685B (en) * | 2021-12-15 | 2023-07-18 | 北京天德科技有限公司 | Novel database architecture based on blockchain system and operation method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874393A (en) * | 2017-01-13 | 2017-06-20 | 北京众享比特科技有限公司 | Synchronizing traffic data method and system, Database Systems based on block chain |
| CN109086325A (en) * | 2018-06-29 | 2018-12-25 | 阿里巴巴集团控股有限公司 | Data processing method and device based on block chain |
| CN109376146A (en) * | 2018-10-26 | 2019-02-22 | 陕西医链区块链集团有限公司 | A kind of search engine system and method based on block chain |
| CN109558411A (en) * | 2017-09-26 | 2019-04-02 | 浙江华信区块链科技服务有限公司 | A kind of lower chain synchronous method and device based on block chain data |
| CN109947855A (en) * | 2019-04-03 | 2019-06-28 | 上海中商网络股份有限公司 | Data synchronizing processing method, device, server and storage medium |
| CN111339106A (en) * | 2020-05-18 | 2020-06-26 | 杭州趣链科技有限公司 | Block chain data indexing method |
-
2020
- 2020-07-29 CN CN202010746552.2A patent/CN112035466B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874393A (en) * | 2017-01-13 | 2017-06-20 | 北京众享比特科技有限公司 | Synchronizing traffic data method and system, Database Systems based on block chain |
| CN109558411A (en) * | 2017-09-26 | 2019-04-02 | 浙江华信区块链科技服务有限公司 | A kind of lower chain synchronous method and device based on block chain data |
| CN109086325A (en) * | 2018-06-29 | 2018-12-25 | 阿里巴巴集团控股有限公司 | Data processing method and device based on block chain |
| CN109376146A (en) * | 2018-10-26 | 2019-02-22 | 陕西医链区块链集团有限公司 | A kind of search engine system and method based on block chain |
| CN109947855A (en) * | 2019-04-03 | 2019-06-28 | 上海中商网络股份有限公司 | Data synchronizing processing method, device, server and storage medium |
| CN111339106A (en) * | 2020-05-18 | 2020-06-26 | 杭州趣链科技有限公司 | Block chain data indexing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112035466A (en) | 2020-12-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111767143B (en) | Transaction data processing method, device, equipment and system | |
| CN105843182B (en) | A kind of power scheduling accident prediction system and method based on OMS | |
| CN111400326B (en) | Smart city data management system and method thereof | |
| CN105824957A (en) | Query engine system and query method of distributive memory column-oriented database | |
| CN103336813A (en) | Data integrated management scheme for Internet of Things based on middleware framework | |
| CN109241159B (en) | Partition query method and system for data cube and terminal equipment | |
| CN112035466B (en) | External index development framework for block chain query | |
| US20230342663A1 (en) | Machine learning application method, device, electronic apparatus, and storage medium | |
| CN104657149A (en) | Software framework implementation method of management module of storage system | |
| CN114564852B (en) | Operation method of FMEA (failure mode and effects analysis) data node and electronic equipment | |
| CN107977446A (en) | A kind of memory grid data load method based on data partition | |
| CN108446327A (en) | Operation system dynamic creation method, device, computer equipment and storage medium | |
| CN104598299A (en) | System and method for performing aggregation process for each piece of received data | |
| CN111949831A (en) | Graphic database establishing method and device and readable storage medium | |
| CN103390018A (en) | Web service data modeling and searching method based on SDD (service data description) | |
| CN115392501A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| CN110532058B (en) | Management method, device and equipment of container cluster service and readable storage medium | |
| CN105868170A (en) | Method for generating industrial data report in server | |
| CN114372064A (en) | Data processing apparatus, method, computer readable medium and processor | |
| CN112199443B (en) | Data synchronization method and device, computer equipment and storage medium | |
| CN115878860A (en) | Menu generation method, device, server equipment and medium | |
| US9779177B1 (en) | Service generation based on profiled data objects | |
| CN115203172B (en) | Model construction and model data subscription method and device, electronic equipment and medium | |
| CN116560626A (en) | Data processing method, system, equipment and storage medium based on custom rules | |
| CN107463618B (en) | Index creating method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |