CN113704248B - Block chain query optimization method based on external index - Google Patents
Block chain query optimization method based on external index Download PDFInfo
- Publication number
- CN113704248B CN113704248B CN202110784835.0A CN202110784835A CN113704248B CN 113704248 B CN113704248 B CN 113704248B CN 202110784835 A CN202110784835 A CN 202110784835A CN 113704248 B CN113704248 B CN 113704248B
- Authority
- CN
- China
- Prior art keywords
- block
- index
- query
- attribute
- connection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000004590 computer program Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 description 14
- 238000010586 diagram Methods 0.000 description 10
- 238000012360 testing method Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000009826 distribution Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000012163 sequencing technique Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000005192 partition Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2246—Trees, e.g. B+trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24558—Binary matching operations
- G06F16/2456—Join operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2471—Distributed queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a block chain query optimization method based on external indexes, which comprises the following steps: creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values; based on the hierarchical index, a blockchain transaction query is conducted. The block chain query optimization method based on the external index can accelerate the range query efficiency on the specific attribute, thereby optimizing the query performance of the block chain.
Description
Technical Field
The invention relates to the technical field of blockchains, in particular to a blockchain query optimization method based on external indexes.
Background
With the development of blockchains, the blockchains are applied to more and more fields, so that the requirements on the inquiry capability of the blockchains are higher and higher.
However, most of current blockchain systems use a K-V (Key-Value) model for query, and the limited expressive power of the K-V model results in insufficient semantic description of transaction data, which makes it difficult to support complex queries, so that the query semantic richness and query efficiency supported by the current blockchain are limited.
Disclosure of Invention
Aiming at the problems existing in the prior art, the invention provides a block chain query optimization method based on external indexes.
In a first aspect, the present invention provides a blockchain query optimization method based on external indexes, including:
creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of the appointed transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
And carrying out blockchain transaction inquiry based on the hierarchical index.
Optionally, the index item is generated based on the attribute values in the block and the isocenter histogram of the corresponding attribute.
Optionally, the index tree is B + And (5) a tree.
Optionally, the method further comprises:
if the leaf nodes of the index tree are determined to be full, a new leaf node is directly generated to serve as a new added attribute value in the rightmost leaf node storage block and a pointer pointing to a transaction storage position corresponding to the new added attribute value.
Optionally, the blockchain transaction query is a traceback query;
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a query attribute and an attribute value corresponding to the query attribute for querying the target transaction;
determining a set of blocks that meet the time window condition;
determining a target block set containing the target transaction from the block sets meeting the time window condition based on a first layer index corresponding to the query attribute and an attribute value corresponding to the query attribute;
and traversing and querying each block in the target block set based on the second layer index corresponding to the query attribute and the attribute value corresponding to the query attribute to obtain a transaction pointer set pointing to a query result.
Optionally, the blockchain transaction query is an on-chain connection query;
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection inquiry;
determining a set of blocks that meet the time window condition;
based on the first layer index corresponding to the connection attribute, respectively determining a first block set containing the transaction in the first connection table and a second block set containing the transaction in the second connection table from the block sets meeting the time window condition;
if it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the first block and the attribute value range corresponding to the connection attribute in the second block, sorting, merging and connecting are performed between the first block and the second block based on a second layer index corresponding to the connection attribute, so that a query result is obtained;
wherein the first block belongs to the first block set and the second block belongs to the second block set.
Optionally, the blockchain transaction query is a joint query;
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
Acquiring a time window condition, a connection attribute, an on-chain connection table and a third block set containing transactions in an off-chain connection table for joint query;
determining a set of blocks that meet the time window condition;
determining a fourth block set containing transactions in the link-on-link table from the block sets meeting the time window condition based on a first layer index corresponding to the link attribute;
if it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the third block and the attribute value range corresponding to the connection attribute in the fourth block, performing sorting, merging and connection between the third block and the fourth block based on a second layer index corresponding to the connection attribute, so as to obtain a query result;
wherein the third block belongs to the third set of blocks and the fourth block belongs to the fourth set of blocks.
In a second aspect, the present invention further provides a blockchain query optimization device based on external index, including:
a creation module for creating a hierarchical index based on specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of the appointed transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
And the query module is used for carrying out blockchain transaction query based on the hierarchical index.
In a third aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the blockchain query optimization method based on external index as described in the first aspect when the processor executes the program.
In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the blockchain query optimization method based on external index as described in the first aspect.
According to the blockchain query optimization method based on the external index, the specific hierarchical index combined with the bitmap index and the tree index structure is created on the appointed transaction attribute, so that the efficiency of range query on the specific attribute can be improved, and the query performance of the blockchain is optimized.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a blockchain query optimization method based on external indexes provided by the invention;
FIG. 2 is a schematic diagram of an index structure according to the present invention;
FIG. 3 is a schematic flow chart of a trace back query provided by the present invention;
FIG. 4 is a schematic flow diagram of an on-link connection query provided by the present invention;
FIG. 5 is a flow diagram of a federated query provided by the present invention;
FIG. 6 is a schematic diagram of a database design provided by the present invention;
FIG. 7 is a graph comparing Q1 query performance at different data set scales provided by the present invention;
FIG. 8 is a graph comparing Q1 query performance at different result set scales provided by the present invention;
FIG. 9 is a graph comparing Q3 query performance at different results set scales provided by the present invention;
FIG. 10 is a graph comparing Q4 query performance at different data set sizes provided by the present invention;
FIG. 11 is a schematic diagram of a block chain query optimization device based on external index according to the present invention;
fig. 12 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The semantic richness and the query efficiency of the current blockchain query are very limited, and the problems in terms of data query are as follows: firstly, the semantics of data are not abundant enough, and a block chain system mostly uses a K-V model with limited expression capacity to cause insufficient transaction data semantic description, so that complex inquiry is difficult to support; secondly, the operation of the data is insufficient, the existing system generally stores block data in a K-V database and only supports a simple access mode, so that the existing system cannot support relational operation on transaction data although the transaction is structured; some systems import block data into the under-chain database for inquiry, but the data migration brings extra system overhead, and the data copy storage brings extra storage cost; finally, data integration is difficult. In order to reduce storage, network overhead or protect privacy, existing schemes store large-scale data or private data in an under-chain database and store data summaries on a blockchain, but this way information related to the same entity is stored on and under the chain at the same time, resulting in difficult data integration. Because of data consistency and data privacy issues, users cannot access the under-chain data using smart contracts, and importing blockchain data into the under-chain database for querying introduces additional storage overhead.
Aiming at the problem that the current blockchain system has insufficient relation semantic description and limited supported query types, the invention introduces the relation semantic description into the blockchain system, models the block data through a relation model to support complex query consisting of basic relation operators such as selection, projection, connection and the like. The system only stores one data, and does not bring extra storage space while improving the query processing capacity of the blockchain system.
In addition, aiming at the problem that the relation query rate of the blockchain is low, the invention designs tree type, bitmap index and hierarchical index to improve data access. Based on the index structure, the query of the relation description on the blockchain can be optimized, for example, the retrospective query and the connection query are performed on the blockchain. In addition, a link-up-link-down operator can be implemented to support data-integrated queries.
Fig. 1 is a flow chart of a block chain query optimization method based on external index, as shown in fig. 1, the method includes the following steps:
step 100, creating a hierarchical index based on specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
Step 101, based on the hierarchical index, conducting blockchain transaction inquiry.
Specifically, the lack of data semantics makes it difficult for the blockchain system to support complex queries, and in order to support rich queries on the blockdata, data semantics are first added. In effect, a transaction is structured, and a typical transaction contains two attributes: system attributes and custom attributes. The system attributes include transaction hash, transaction sender, etc., and the user-defined attributes include smart contract parameters, etc. Transactions that invoke the same kind of operations have the same structure, the same type of transactions are described by a unified relationship schema, the transaction structure is determined by declaring a relationship table schema, and the attribute type may be a character, a number, or other types. Table attributes fall into two categories: system attributes and application attributes. System attributes are automatically added when creating a table schema, such as transaction type, transaction sender, etc.; the application attributes are explicitly specified by the user, typically parameters related to the application. The user may send a special transaction to create a table pattern, synchronizing the table pattern information between nodes by a consensus algorithm.
In practice, data is typically stored in both the blockchain system and the database. Data is stored in a blockchain system, also referred to as on-chain data, organized in a blockwise structure. Large-scale data is then managed by a local database, also known as under-chain data, and stored in an under-chain relational table. Accordingly, on-chain queries, i.e., querying data stored in the blockchain, and on-chain and off-chain joint queries, i.e., querying on-chain and off-chain data simultaneously.
In order to improve the query performance of block-oriented storage, the invention designs an index structure to accelerate data access, and mainly faces to three basic operations: (1) A block is obtained given a block number, transaction number, or timestamp. (2) All transaction data belonging to a certain data table is acquired. (3) And acquiring the transaction meeting the condition under the specific query condition given a certain attribute. The design of an index structure in a blockchain system faces many challenges, firstly, the influence of index update on the writing performance of the system is caused, and a single block may cause a large number of index updates to influence the writing and reading performance of the system. Second, it is desirable to support time window based queries. The blockchain stores all historical transactions, and the user may query for transactions within a particular time. The present invention therefore proposes a write efficient index structure: tree, bitmap index, and hierarchical index that integrates the two.
Fig. 2 is a schematic design diagram of an index structure provided in the present invention, and as shown in fig. 2, the index structure includes a block index (block index), a table level bitmap index (table bitmap index), and a level index (level index).
The block index is a tree index containing block numbers, transaction numbers and time stamp information, and all block information on a block chain can be recorded by using the block index.
The block index may be a B + A tree, represented as a triplet (bid, tid, ts), where bid is the block number, tid is the first transaction number in the block, which can uniquely identify a transaction, where Ts represents the time stamp of the block using increasing integer identification, as shown in fig. 2 (n, j, T sn ) The leaf nodes store pointers to the block storage locations, p as shown in FIG. 1 n . For any two blocks b i And b j If block b i Packing time ratio b of (2) j Early then block b i The block number, transaction number and time stamp are all earlier than the block b j 。
The generation of a new block triggers the updating of the block index, and since the bid, tid and Ts of the new block are increased along with the expansion of the block, the search code value corresponding to the new block is always directly additionally stored in the rightmost leaf node. Unlike conventional B + In the tree leaf splitting mode, if the leaf node is full, the block index can directly generate a new leaf node as the rightmost leaf node to store the search code value, so that the storage resource waste caused by the leaf node splitting is avoided, and the updating efficiency performance is higher.
A bitmap is a practical and less expensive data structure. Bitmaps can represent rich meaning and occupy little space. And a mark of the bitmap is associated with data distribution in the blockchain, and different meanings are given to the bitmap, so that useless blocks can be filtered, and the query efficiency is improved.
For example, a bitmap is given the meaning of a relational table (table), then the table forms a corresponding bitmap on the blockchain, such as the table-level bitmap index shown in FIG. 2. If there is relational table data in the kth block, the kth bit of the bitmap is 1, and vice versa is 0.
Bitmaps may also be built on system attributes for querying. In addition, the method can also be established on a common attribute value.
The generation of new tiles triggers the updating of the bitmap index. Assuming that the block is the ith block in the system and exactly contains a transaction of a certain relationship table, the ith bit of the bitmap corresponding to the relationship table is set to 1, otherwise, set to 0. Thus, the update overhead of the bitmap index is only one set operation. For the query request on the appointed relation table, the block containing the transaction of the table can be found by looking up the bit 1 in the bitmap corresponding to the table, so that scanning of all blocks is avoided.
The hierarchical index may be built on top of specified transaction attributes, and the index may be built based on attribute values, thereby speeding up the rate of range queries on specific attributes.
The hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index composed of block numbers and index items corresponding to each block number. Alternatively, the index entries may be generated based on the equal depth histograms of the attribute values and corresponding attributes in the block, each bucket (bucket) in the discrete attribute equal depth histogram representing one attribute value, the equal depth histograms of consecutive attributes being generated from the attribute value distribution, the buckets representing a range of attribute values, such as the bucket shown in FIG. 2 1 The range of attribute values represented is 0-300. Each block corresponds to an index item, the index item is represented by using a bitmap, and records the data range in the block, and 1 and 0 in the bitmap respectively represent transactions containing and not containing the corresponding attribute value range in the block, such as the block shown in fig. 2 1 The corresponding bitmap is 1001001101, and bit 1 in the bitmap is 1 to indicate block 1 Comprises a socket 1 Transactions with a range of attribute values (i.e., 0-300), bit 2 of the bitmap being 0 indicates block 1 Does not include socket 2 A transaction for a range of attribute values (i.e., 300-600).
The second layer of the hierarchical index is a tree-type index formed by index trees corresponding to each block, each block corresponds to an index tree built on an index column, leaf nodes of the index tree store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values, and the purpose of creating the second layer index is to avoid scanning the whole block. Alternatively, the index tree may be a B + And (5) a tree.
Optionally, in the embodiment of the present invention, the second-layer index of the hierarchical index may adopt the same leaf node splitting manner as the block index, and if it is determined that the leaf node of the index tree is full, a new leaf node is directly generated as the newly added attribute value in the rightmost leaf node storage block and a pointer pointing to the transaction storage location corresponding to the newly added attribute value.
In the above hierarchical index, the depth of the iso-depth histogram may be established according to a specific scene, and the iso-depth histogram and the hierarchical index may be established according to a specific relationship table and data. The index is typically stored in memory, but when the index occupies too much memory, the early index structure may be stored on disk, with the later index structure cached in memory.
According to the blockchain query optimization method based on the external index, the specific hierarchical index combined with the bitmap index and the tree index structure is created on the appointed transaction attribute, so that the efficiency of range query on the specific attribute can be improved, and the query performance of the blockchain is optimized.
Optionally, the blockchain transaction query is a traceback query;
the block chain transaction inquiry is performed based on the hierarchical index, which comprises the following steps:
acquiring a time window condition, a query attribute and an attribute value corresponding to the query attribute for querying the target transaction;
determining a set of blocks meeting a time window condition;
determining a target block set containing target transactions from the block sets meeting the time window conditions based on the first layer index corresponding to the query attribute and the attribute value corresponding to the query attribute;
Traversing each block in the query target block set based on the second layer index corresponding to the query attribute and the attribute value corresponding to the query attribute to obtain a transaction pointer set pointing to the query result.
In particular, retrospective querying can be achieved by scanning all the tiles and checking all transactions in the tile, but this way querying is inefficient. The query performance can be optimized through the bitmap index, for example, all transactions sent by a certain sender are traced back, all transactions of the sender are traced back, corresponding blocks are obtained through the bitmap, and then the blocks are scanned to obtain the query result, so that scanning of all blocks is avoided. In the embodiment of the invention, the query efficiency can be higher through the hierarchical index.
Specifically, in the embodiment of the invention, the retrospective query is performed based on the hierarchical index, and firstly, a time window condition, a query attribute and an attribute value corresponding to the query attribute for querying the target transaction can be obtained according to a query statement or a query condition of the retrospective query, wherein the query attribute can be one or more; then, a set of blocks meeting the time window condition can be obtained through the time query index structure, for example, the set of blocks containing the transaction of the specified time window can be rapidly determined based on the block index; based on the block set, inquiring a first layer index corresponding to the inquiry attribute based on an attribute value corresponding to the inquiry attribute, so that a target block set containing target transaction can be obtained, and if a plurality of inquiry attributes are provided, solving an intersection of the block sets obtained by each inquiry attribute based on the first layer index, so as to obtain the target block set containing target transaction; and finally, inquiring the second-layer index corresponding to the inquiry attribute based on the attribute value corresponding to the inquiry attribute, traversing each block in the inquiry target block set, and obtaining a transaction pointer set pointing to the inquiry result, wherein the transaction pointer set comprises pointers pointing to the inquiry result, and reading the transaction from the disk and outputting the inquiry result according to the pointers.
Fig. 3 is a schematic flow chart of a traceback query provided by the present invention, as shown in fig. 3, a process for performing traceback query q= < Trace, (SenID, o), (Tname, p) and [ c, e ] (where Trace represents traceback query, senID represents query attribute as transaction sender, o is corresponding attribute value, tname represents query attribute as transaction type, p is corresponding attribute value, and [ c, e ] is time window) based on a hierarchical index includes the following steps:
step 300, starting;
specifically, a hierarchical index I is created on the system fields SenID and Tname d And I n Which covers all transactions within the blockchain system.
Step 301, obtaining a set B through time inquiry block indexes;
specifically, the Search function queries the block index according to the time window and obtains a set B, where B contains i if the block i is in the time window.
Step 302, search I d And I n To obtain sets B 'and B';
specifically, the FirstLevelBitmap function search I d And I n Sets B 'and B "are obtained, j being in B' if chunk j contains transactions sent by o and j is in B, and j being in B" if chunk j contains transactions of transaction type p and j is in B.
Step 303, B 'and B' are intersected to obtain a satisfied attribute set;
specifically, intersection operations on B' and B "may result in a block within the specified time window that contains transactions with a SenID attribute value of o and a Tname attribute value of p.
Step 304, searching the second layer index to obtain a query result;
specifically, the second level function obtains a query result by searching the second level index, and for each block, queries I with "sendid=o" and "tname=p" as query conditions, respectively d And I n Respectively obtaining transaction pointer sets.
Step 305, obtaining a transaction set output result;
specifically, the intersection of the transaction pointer set includes pointers to the query results, and the Read function reads the transaction from the disk and outputs the query results according to the pointers.
Step 306, end.
Optionally, the blockchain transaction query is an on-chain connection query;
the block chain transaction inquiry is performed based on the hierarchical index, which comprises the following steps:
acquiring a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection inquiry;
determining a set of blocks meeting a time window condition;
based on the first layer index corresponding to the connection attribute, respectively determining a first block set containing the transaction in the first connection table and a second block set containing the transaction in the second connection table from the block sets meeting the time window condition;
If the intersection exists between the attribute value range of the corresponding connection attribute in the first block and the attribute value range of the corresponding connection attribute in the second block, sequencing, combining and connecting are performed between the first block and the second block based on the second layer index corresponding to the connection attribute, so that a query result is obtained;
wherein the first block belongs to a first block set and the second block belongs to a second block set.
In particular, conventional join algorithms, such as nested loop join, cannot be directly applied to blockchains due to the huge query overhead caused by data scatter storage. The embodiment of the invention can realize a one-pass scanning hash connection algorithm based on the hierarchical index, and for connection inquiry on the table r and the table s, firstly, all blocks are scanned, a hash function is used for constructing transaction partitions, a hash index is established for each partition, and for each transaction t in the partition, the hash index is probed to obtain all the transactions t meeting the connection condition. By bitmap indexing, the performance of connection queries can be improved, since only the blocks containing r or s transactions need to be read. And query performance can be made higher by hierarchical indexing.
Specifically, in the embodiment of the invention, on-link connection query is performed based on the hierarchical index, and firstly, a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection query can be obtained according to a query statement or a query condition; then, a set of blocks meeting the time window condition can be obtained through the time query index structure, for example, the set of blocks containing the transaction of the specified time window can be rapidly determined based on the block index; based on the block set and the first layer index corresponding to the connection attribute, a first block set containing transactions in a first connection table and a second block set containing transactions in a second connection table can be obtained; if the first block belongs to the first block set, the second block belongs to the second block set, and an intersection exists between an attribute value range of a corresponding connection attribute in the first block and an attribute value range of a corresponding connection attribute in the second block, the first block and the second block may generate a connection query result, so that a sorting, combining and connecting can be performed between the first block and the second block based on a second layer index corresponding to the connection attribute, and the query result is obtained.
Fig. 4 is a schematic flow chart of an On-link query provided by the present invention, as shown in fig. 4, a process of performing an On-link query q= < On, r, s, r.attr=s.attr, [ c, e ] > (where On represents an On-link query, r and s represent a link table, =represent equivalent link conditions, [ c, e ] represents a time window, and attr represents a link attribute) based On a hierarchical index includes the following steps:
step 400, starting;
specifically, I is created above attr of tables r and s r And I s And (5) hierarchical index.
Step 401, inquiring a block index to obtain a set B;
specifically, the Search function queries the block index and gets a set B, if block i is in the time window, then i is contained in B.
Step 402, search I r And I s Obtaining blocks containing tables r and s, which are sets B 'and B', in a time window;
specifically, the FirstLevelBitmap function queries I r And I s Can be obtained within the time window and contains transaction blocks belonging to table r or table s, recorded in B' and B ", respectively.
Step 403, judging (k.u < m.l > k.l > m.u);
specifically, for each element i in B' and each element j in B ", let e ri And e sj Respectively represent I r Index item and I of middle block I s The index entries, k and m, of middle bin j represent buckets in the iso-deep histogram, with upper and lower bounds (l, u). If the Intersect function returns TRUE, i.em∈e sj So that (k.u)<m.l∨k.l>m.u) is true, then blocks i and j may produce a join query result. Where k.u denotes the value of the lower bound u of bucket k, m.l denotes the value of the upper bound l of bucket m, k.l denotes the value of the upper bound l of bucket k, m.u denotes the value of the lower bound u of bucket m.
Step 404, performing sorting, merging and connecting between blocks i and j based on the second layer index, and outputting a result;
specifically, the sort merge connection on blocks i and j is performed based on the second layer index by the SortMergeJoin function, and finally the query result is output.
Step 405, end.
Optionally, the blockchain transaction query is a joint query;
the block chain transaction inquiry is performed based on the hierarchical index, which comprises the following steps:
acquiring a time window condition, a connection attribute, an on-chain connection table and a third block set containing transactions in an off-chain connection table for joint query;
determining a set of blocks meeting a time window condition;
determining a fourth block set containing transactions in the link-on-link table from the block sets meeting the time window condition based on the first layer index corresponding to the link attribute;
If the intersection exists between the attribute value range of the corresponding connection attribute in the third block and the attribute value range of the corresponding connection attribute in the fourth block, sequencing, combining and connecting are performed between the third block and the fourth block based on the second layer index corresponding to the connection attribute, so that a query result is obtained;
wherein the third block belongs to a third block set and the fourth block belongs to a fourth block set.
Specifically, in the embodiment of the invention, the link-up-link-down joint query can also realize a one-pass scanning hash connection algorithm based on the hierarchical index. Firstly, acquiring data of a link-down relation table s from a link-down database, then acquiring data in a link-up relation table r by a scanning block, and finally acquiring a query result by a hash connection algorithm. Using the hierarchical index in the query may optimize query performance, only the blocks containing transactions belonging to the relationship table r will be read.
Specifically, in the embodiment of the invention, joint query is performed based on a hierarchical index, and firstly, a time window condition, a connection attribute and an on-chain connection table for joint query can be obtained according to a query statement or a query condition, and a third block set containing transactions in the under-chain connection table is obtained through an under-chain database; then, a set of blocks meeting the time window condition can be obtained through the time query index structure, for example, the set of blocks containing the transaction of the specified time window can be rapidly determined based on the block index; based on the block set, a fourth block set containing transactions in the link-on-link table can be obtained based on the first layer index corresponding to the link attribute; if the third block belongs to the third block set, the fourth block belongs to the fourth block set, and there is an intersection between the attribute value range of the corresponding connection attribute in the third block and the attribute value range of the corresponding connection attribute in the fourth block, the third block and the fourth block may generate a connection query result, so that the sorting, merging and connection can be performed between the third block and the fourth block based on the second layer index corresponding to the connection attribute, to obtain the query result.
Fig. 5 is a schematic flow chart of a joint query provided by the present invention, as shown in fig. 5, a process of performing joint query q= < Onoff, r, s, r.attr=s.attr, [ c, e ] based on a hierarchical index (where Onoff represents the joint query, r represents an on-chain connection table, s represents an under-chain connection table, =represents an equivalent connection condition, [ c, e ] represents a time window, attr represents a connection attribute) includes the following steps:
step 500, starting;
specifically, build a hierarchical index I on an attribute r.attr r 。
Step 501, obtaining a set B by time query tree indexes;
specifically, the Search function queries the block index according to the time window and gets the set B, if the block i is in the time window, i is contained in B.
Step 502, ordering the data under the chain according to the s.attr attribute;
specifically, before the data under the chain is transmitted to the blockchain, the data under the chain is ordered according to the s.attr attribute.
Step 503, according to [ S ] min ，S max ]Filtering blocks not containing connection results;
specifically, after the sort operation, the value range S is determined by the s.attr attribute of the data under the chain min ，S max ]Blocks not containing the connection result can be filtered to obtain a range of attribute values containing s.attr S min ，S max ]A set of blocks B' of the internal transaction.
Step 504, obtaining all blocks which are in the time window and contain the data of table r through the first layer index;
specifically, the FirstLevelBitmap function queries I r Can be obtained within a time window and contains transaction blocks belonging to table r, recorded in B ".
Step 505, judging (k.u)<S min ∨k.l>S max )；
Specifically, for each element d in B' and each element i in B ", let e ri Representation I r The index entry of the middle block i, k, represents a bucket in the iso-deep histogram, whose upper and lower bounds are (l, u). If the Intersect function returns TRUE, i.eSo that (k.u)<S min ∨k.l>S max ) If true, then blocks d and i may produce a join query result. Where k.u denotes the value of the lower bound u of bucket k and k.l denotes the value of the upper bound l of bucket k.
Step 506, the second layer index performs sorting, merging and connecting between the block i and the block d, and outputs a result;
specifically, if the Intersect function returns TRUE, block i will participate in the connection process. The second layer index is used to perform sort merge connection between blocks i and d through the SortMergeJoin function, so that a result can be obtained, and finally the result is output.
And step 507, ending.
The query performance provided by the invention is described below through a specific experimental environment.
FIG. 6 is a schematic diagram of a database design provided in the present invention, as shown in FIG. 6, the database includes a database storing data on a blockchain and a database storing data under the blockchain, and query efficiency of different index structures is compared and illustrated by testing performance of different queries in the same experimental environment.
The test statement is divided into four types Q1, Q2, Q3 and Q4, as follows:
wherein, Q1 and Q2 test the performance of the traceback query. Q1 is a one-dimensional traceback query, traceback all transactions issued by the charity "org1", and Q2 is a two-dimensional traceback query, traceback Transfer transactions issued by "org 1". Q3 and Q4 test the performance of queries on the chain. Q3 is a range query on the Donate. Q4 is a federated query, querying the details of the donated person. Query performance is compared by response time of the query.
Test dataset: in all experiments, the block size was set to 4MB and the transaction size was set to 300 bytes. The experiment first tested the query performance at different result set sizes and data set sizes. To test the query performance of the system under different data set sizes, the query result set size is fixed, and the number of blocks to be queried is increased from 500, with 500 blocks as steps. To test query performance at different result set scales, the number of blocks was fixed at 1000 and the result set scale was changed. Transactions follow a uniform or gaussian distribution among blocks to test query performance under different data distributions.
SU, SG, BU, BG, LU and LG are used to represent query execution under trade data uniform distribution and gaussian distribution based on block scan, table level bitmap index and hierarchical index, respectively, where SU represents query execution under trade data uniform distribution based on block scan, SG represents query execution under trade data gaussian distribution based on block scan, BU represents query execution under trade data uniform distribution based on table level bitmap index, BG represents query execution under trade data gaussian distribution based on table level bitmap index, LU represents query execution under trade data uniform distribution based on hierarchical index, LG represents query execution under trade data gaussian distribution based on hierarchical index. SIU, SIG, TIU and TIG are used to represent the performance of the traceback query when a single or two hierarchy index is used with a uniform distribution of transaction data and a gaussian distribution, where SIU represents the performance of the traceback query when a single hierarchy index is used with a uniform distribution of transaction data, SIG represents the performance of the traceback query when a single hierarchy index is used with a gaussian distribution of transaction data, TIG represents the performance of the traceback query when two hierarchy indexes are used with a uniform distribution of transaction data, TIG represents the performance of the traceback query when two hierarchy indexes are used with a gaussian distribution of transaction data.
Retrospective query: fig. 7 is a comparison graph of Q1 query performance under different data set scales provided by the present invention, and fig. 8 is a comparison graph of Q1 query performance under different result set scales provided by the present invention. In fig. 7, the result size is fixed to 10000. The query processing based on the hierarchical index is obviously superior to the other two methods under all conditions, because the method based on the hierarchical index only needs to directly read the query result from the disk, and the current result set has smaller scale. Query processing based on table level bitmap indexing requires reading all blocks for which query results exist, while query based on block scanning requires reading all blocks in a blockchain, both of which have relatively large overhead.
Next, the performance of the Q2 query was tested. The SIU and SIG use a hierarchical index created on the system SenID to perform two-dimensional traceback query with a single hierarchical index, and the TIU and TIG use two hierarchical indexes created on the SenID and Tname to perform two-dimensional traceback. The results show that query processing based on hierarchical index is in all cases significantly better than the other two approaches.
Range query: the depth of the histogram in this set of experiments was set to 100, with 10000 Donate transactions. FIG. 9 is a graph comparing Q3 query performance at different results set scales provided by the present invention. In fig. 9, the transaction result set size increases from 1000 to 10000, and as the result set size increases, the difference between the three methods becomes smaller, because the hierarchical index-based query method increases with disk I/O (Input/Output), and the performance decreases. Transactions are distributed in more tiles, so bitmap index-based query methods also degrade.
Joint query: the experiment tested a hierarchical index based joint query. FIG. 10 is a graph comparing Q4 query performance at different data set sizes provided by the present invention. As shown in FIG. 10, the hierarchical index based approach has better performance than the other two approaches because it only needs to read the blocks that intersect the data range under the chain, reading the data based on the second level index can further improve the performance of the query.
The external index-based blockchain query optimization device provided by the invention is described below, and the external index-based blockchain query optimization device described below and the external index-based blockchain query optimization method described above can be correspondingly referred to each other.
FIG. 11 is a schematic structural diagram of a block chain query optimization device based on external index according to the present invention, as shown in FIG. 11, the device includes:
a creation module 1100 for creating a hierarchical index based on specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
A query module 1110, configured to perform a blockchain transaction query based on the hierarchical index.
Optionally, the index item is generated based on the attribute values in the block and the isocenter histogram of the corresponding attribute.
Optionally, the index tree is B + And (5) a tree.
Optionally, the apparatus further comprises:
the generating module 1120 is configured to directly generate a new leaf node as the newly added attribute value in the rightmost leaf node storage block and a pointer to the transaction storage location corresponding to the newly added attribute value if it is determined that the leaf node of the index tree is full.
Optionally, the blockchain transaction query is a traceback query;
the query module 1110 is configured to: acquiring a time window condition, a query attribute and an attribute value corresponding to the query attribute for querying the target transaction; determining a set of blocks meeting a time window condition; determining a target block set containing target transactions from the block sets meeting the time window conditions based on the first layer index corresponding to the query attribute and the attribute value corresponding to the query attribute; traversing each block in the query target block set based on the second layer index corresponding to the query attribute and the attribute value corresponding to the query attribute to obtain a transaction pointer set pointing to the query result.
Optionally, the blockchain transaction query is an on-chain connection query;
the query module 1110 is configured to: acquiring a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection inquiry; determining a set of blocks meeting a time window condition; based on the first layer index corresponding to the connection attribute, respectively determining a first block set containing the transaction in the first connection table and a second block set containing the transaction in the second connection table from the block sets meeting the time window condition; if the intersection exists between the attribute value range of the corresponding connection attribute in the first block and the attribute value range of the corresponding connection attribute in the second block, sequencing, combining and connecting are performed between the first block and the second block based on the second layer index corresponding to the connection attribute, so that a query result is obtained; wherein the first block belongs to a first block set and the second block belongs to a second block set.
Optionally, the blockchain transaction query is a joint query;
the query module 1110 is configured to: acquiring a time window condition, a connection attribute, an on-chain connection table and a third block set containing transactions in an off-chain connection table for joint query; determining a set of blocks meeting a time window condition; determining a fourth block set containing transactions in the link-on-link table from the block sets meeting the time window condition based on the first layer index corresponding to the link attribute; if the intersection exists between the attribute value range of the corresponding connection attribute in the third block and the attribute value range of the corresponding connection attribute in the fourth block, sequencing, combining and connecting are performed between the third block and the fourth block based on the second layer index corresponding to the connection attribute, so that a query result is obtained; wherein the third block belongs to a third block set and the fourth block belongs to a fourth block set.
It should be noted that, the device provided by the present invention can implement all the method steps implemented by the method embodiment and achieve the same technical effects, and the parts and beneficial effects that are the same as those of the method embodiment in the present embodiment are not described in detail herein.
Fig. 12 is a schematic structural diagram of an electronic device according to the present invention, as shown in fig. 12, the electronic device may include: processor 1210, communication interface (Communications Interface), 1220, memory 1230 and communication bus 1240, wherein processor 1210, communication interface 1220 and memory 1230 communicate with each other via communication bus 1240. The processor 1210 may invoke logic instructions in the memory 1230 to perform the steps of any of the external index based blockchain query optimization methods provided in the embodiments described above, such as: creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values; based on the hierarchical index, a blockchain transaction query is conducted.
In addition, the logic instructions in the memory 1230 described above may be implemented in the form of software functional units and sold or used as a stand-alone product, stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, are capable of performing the steps of any of the external index based blockchain query optimization methods provided by the embodiments above, such as: creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values; based on the hierarchical index, a blockchain transaction query is conducted.
In yet another aspect, the present invention further provides a non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, is implemented to perform the steps of any of the external index-based blockchain query optimization methods provided in the above embodiments, for example: creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, wherein the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of specified transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values; based on the hierarchical index, a blockchain transaction query is conducted.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. The block chain query optimization method based on the external index is characterized by comprising the following steps of:
creating a hierarchical index based on the specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of the appointed transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
based on the hierarchical index, conducting blockchain transaction inquiry;
the blockchain transaction query is an on-chain connection query or a joint query;
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection inquiry;
determining a set of blocks that meet the time window condition;
Based on the first layer index corresponding to the connection attribute, respectively determining a first block set containing the transaction in the first connection table and a second block set containing the transaction in the second connection table from the block sets meeting the time window condition;
if it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the first block and the attribute value range corresponding to the connection attribute in the second block, sorting, merging and connecting are performed between the first block and the second block based on a second layer index corresponding to the connection attribute, so that a query result is obtained;
wherein the first block belongs to the first block set and the second block belongs to the second block set; or,
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a connection attribute, an on-chain connection table and a third block set containing transactions in an off-chain connection table for joint query;
determining a set of blocks that meet the time window condition;
determining a fourth block set containing transactions in the link-on-link table from the block sets meeting the time window condition based on a first layer index corresponding to the link attribute;
If it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the third block and the attribute value range corresponding to the connection attribute in the fourth block, performing sorting, merging and connection between the third block and the fourth block based on a second layer index corresponding to the connection attribute, so as to obtain a query result;
wherein the third block belongs to the third set of blocks and the fourth block belongs to the fourth set of blocks.
2. The method of claim 1, wherein the index entries are generated based on equal depth histograms of attribute values and corresponding attributes in a block.
3. The method of claim 1, wherein the index tree is B + And (5) a tree.
4. The method of claim 1, further comprising:
if the leaf nodes of the index tree are determined to be full, a new leaf node is directly generated to serve as a new added attribute value in the rightmost leaf node storage block and a pointer pointing to a transaction storage position corresponding to the new added attribute value.
5. An external index-based blockchain query optimization device, comprising:
a creation module for creating a hierarchical index based on specified transaction attributes; the hierarchical index comprises a first layer index and a second layer index, the first layer index is a bitmap index formed by block numbers and index items corresponding to each block number, the index items are represented by bitmaps, 1 and 0 in the bitmaps respectively represent transactions containing and not containing corresponding attribute value ranges in the blocks, the second layer index is a tree index formed by index trees corresponding to each block, and leaf nodes of the index trees store attribute values of the appointed transaction attributes contained in the blocks and pointers pointing to transaction storage positions corresponding to the attribute values;
the query module is used for carrying out blockchain transaction query based on the hierarchical index;
the blockchain transaction query is an on-chain connection query or a joint query;
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a first connection table, a second connection table and a connection attribute for on-link connection inquiry;
determining a set of blocks that meet the time window condition;
Based on the first layer index corresponding to the connection attribute, respectively determining a first block set containing the transaction in the first connection table and a second block set containing the transaction in the second connection table from the block sets meeting the time window condition;
if it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the first block and the attribute value range corresponding to the connection attribute in the second block, sorting, merging and connecting are performed between the first block and the second block based on a second layer index corresponding to the connection attribute, so that a query result is obtained;
wherein the first block belongs to the first block set and the second block belongs to the second block set; or,
the block chain transaction inquiry is performed based on the hierarchical index, and the block chain transaction inquiry comprises the following steps:
acquiring a time window condition, a connection attribute, an on-chain connection table and a third block set containing transactions in an off-chain connection table for joint query;
determining a set of blocks that meet the time window condition;
determining a fourth block set containing transactions in the link-on-link table from the block sets meeting the time window condition based on a first layer index corresponding to the link attribute;
If it is determined that an intersection exists between the attribute value range corresponding to the connection attribute in the third block and the attribute value range corresponding to the connection attribute in the fourth block, performing sorting, merging and connection between the third block and the fourth block based on a second layer index corresponding to the connection attribute, so as to obtain a query result;
wherein the third block belongs to the third set of blocks and the fourth block belongs to the fourth set of blocks.
6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the program, performs the steps of the external index based blockchain query optimization method of any of claims 1 to 4.
7. A non-transitory computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the external index based blockchain query optimization method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110784835.0A CN113704248B (en) | 2021-07-12 | 2021-07-12 | Block chain query optimization method based on external index |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110784835.0A CN113704248B (en) | 2021-07-12 | 2021-07-12 | Block chain query optimization method based on external index |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113704248A CN113704248A (en) | 2021-11-26 |
| CN113704248B true CN113704248B (en) | 2024-01-09 |
Family
ID=78648457
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110784835.0A Active CN113704248B (en) | 2021-07-12 | 2021-07-12 | Block chain query optimization method based on external index |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113704248B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114153851A (en) * | 2021-12-06 | 2022-03-08 | 智慧足迹数据科技有限公司 | GEOHASH indexing method, GEOHASH indexing device, computer equipment and storage medium |
| CN116628285B (en) * | 2023-07-21 | 2023-11-03 | 北京邮电大学 | Block chain transaction data query method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109471905A (en) * | 2018-11-16 | 2019-03-15 | 华东师范大学 | A kind of block chain index method for supporting time range and range of attributes compound query |
| CN109885615A (en) * | 2019-01-24 | 2019-06-14 | 华东师范大学 | A kind of range query towards the light client of block chain based on index can verify that querying method |
| KR20190134115A (en) * | 2018-05-25 | 2019-12-04 | 주식회사 티맥스데이터 | Method and apparatus for providing efficient indexing and computer program included in computer readable medium therefor |
| CN109559231B (en) * | 2018-12-29 | 2020-08-04 | 华东师范大学 | Block chain-oriented tracing query method |
| CN111917896A (en) * | 2020-06-23 | 2020-11-10 | 北京邮电大学 | Credible domain name resolution method, system, electronic equipment and storage medium |
| CN112035466A (en) * | 2020-07-29 | 2020-12-04 | 北京智融云河科技有限公司 | Block chain query external index development framework |
| CN112765181A (en) * | 2021-01-27 | 2021-05-07 | 东北大学 | Data tracing query method based on block index structure |
-
2021
- 2021-07-12 CN CN202110784835.0A patent/CN113704248B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20190134115A (en) * | 2018-05-25 | 2019-12-04 | 주식회사 티맥스데이터 | Method and apparatus for providing efficient indexing and computer program included in computer readable medium therefor |
| CN109471905A (en) * | 2018-11-16 | 2019-03-15 | 华东师范大学 | A kind of block chain index method for supporting time range and range of attributes compound query |
| CN109559231B (en) * | 2018-12-29 | 2020-08-04 | 华东师范大学 | Block chain-oriented tracing query method |
| CN109885615A (en) * | 2019-01-24 | 2019-06-14 | 华东师范大学 | A kind of range query towards the light client of block chain based on index can verify that querying method |
| CN111917896A (en) * | 2020-06-23 | 2020-11-10 | 北京邮电大学 | Credible domain name resolution method, system, electronic equipment and storage medium |
| CN112035466A (en) * | 2020-07-29 | 2020-12-04 | 北京智融云河科技有限公司 | Block chain query external index development framework |
| CN112765181A (en) * | 2021-01-27 | 2021-05-07 | 东北大学 | Data tracing query method based on block index structure |
Non-Patent Citations (3)
| Title |
|---|
| Blockchain Meets Edge Computing: A Distributed and Trusted Authentication System;Shaoyong Guo;《IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS》;第16卷(第3期);全文 * |
| Blockchain Meets Edge Computing: Stackelberg Game and Double Auction Based Task Offloading for Mobile Blockchain;Shaoyong Guo;《IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY》;第69卷(第5期);全文 * |
| 区块链技术在物联网中的应用;陈骞，郭少勇，代美玲，阮琳娜，亓峰;信息通信技术;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113704248A (en) | 2021-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111046034B (en) | Method and system for managing memory data and maintaining data in memory | |
| JP3813089B2 (en) | System for managing RDBM fragmentation | |
| CN108369587B (en) | Creating tables for exchange | |
| CN106933833B (en) | Method for quickly querying position information based on spatial index technology | |
| CN113704248B (en) | Block chain query optimization method based on external index | |
| US11741084B2 (en) | High frequency data management (HFDM) | |
| CN107807932B (en) | Hierarchical data management method and system based on path enumeration | |
| CN103678550A (en) | Mass data real-time query method based on dynamic index structure | |
| CN107977446A (en) | A kind of memory grid data load method based on data partition | |
| CN111125229A (en) | Data blood margin generation method and device and electronic equipment | |
| Song et al. | Haery: a Hadoop based query system on accumulative and high-dimensional data model for big data | |
| CN111639075A (en) | Non-relational database vector data management method based on flattened R tree | |
| de Souza Baptista et al. | NoSQL geographic databases: an overview | |
| CN107273443B (en) | Mixed indexing method based on metadata of big data model | |
| CN115918110A (en) | Spatial search using key-value store | |
| CN113779286B (en) | Method and device for managing graph data | |
| CN114416741A (en) | KV data writing and reading method and device based on multi-level index and storage medium | |
| CN114048219A (en) | Graph database updating method and device | |
| Zollmann | Nosql databases | |
| CN111831696A (en) | Asset information storage method and system based on graph theory | |
| KR20220099745A (en) | A spatial decomposition-based tree indexing and query processing methods and apparatus for geospatial blockchain data retrieval | |
| CN112667859A (en) | Data processing method and device based on memory | |
| RU2389066C2 (en) | Multidimensional database and method of managing multidimensional database | |
| Yang et al. | Workload-based ordering of multi-dimensional data | |
| Chai et al. | A document-based data warehousing approach for large scale data mining |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |