WO2021057482A1

WO2021057482A1 - Method and device for generating bloom filter in blockchain

Info

Publication number: WO2021057482A1
Application number: PCT/CN2020/114235
Authority: WO
Inventors: 王�章; 刘明臻; 李辉忠; 张开翔; 范瑞彬
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2019-09-26
Filing date: 2020-09-09
Publication date: 2021-04-01
Also published as: CN110704438A; CN110704438B

Abstract

A method and device for generating a Bloom filter in a blockchain. The method comprises: acquiring the number of topics of a block (201); according to the number of topics, determining the number of topic storage bits needing to be occupied for storing the topics of the block (202), the number of topic storage bits being positively correlated with the number of topics; generating a Bloom filter of the block according to the number of topic storage bits (203); and for each topic of the block, according to a preset mapping relationship, updating the value of the bit mapped by the topic in the Bloom filter from a first preset value to a second preset value, so as to obtain the Bloom filter for storing the topics of the block (204). When the method above is applied to Fintech, the storage space of the Bloom filter is sufficient, so that the storage of various topics of the block can be satisfied, the bits of the mapping between the topics do not conflict, and an accurate query can be achieved when it is determined subsequently whether the block contains a queried topic.

Description

Method and device for generating bloom filter in block chain

Cross-references to related applications

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on September 26, 2019, the application number is 201910918040.7, and the application name is "a method and device for generating a Bloom filter in a blockchain", all of which The content is incorporated in this application by reference.

Technical field

The present invention relates to the field of financial technology (Fintech) and the field of blockchain (Blockchain), in particular to a method and device for generating a Bloom filter in a blockchain.

Background technique

With the development of computer technology, more and more technologies (big data, distributed, Blockchain, artificial intelligence, etc.) are applied in the financial field, and the traditional financial industry is gradually changing to Fintech. At present, in the field of financial technology, financial activities are often realized through blockchain. There may be multiple transactions in a block, each transaction includes at least one event, and each event includes at least one topic. In some scenarios, it may involve searching for an event. The event can be searched by the subject of the event.

In the prior art, a Bloom filter with a fixed length of 2048 bits is used to store the subject of the block. Bloom filters can completely store themes of blocks with small transaction volume. However, for a block that can pack tens of thousands of transactions in a single block, a Bloom filter with a fixed length of 2048 bits will inevitably have bit duplication. Phenomenon, the Bloom filter will completely fail and cannot accurately store the subject of the block. In addition, the subject of a block with a small transaction volume will waste a lot of storage space, which is an urgent problem to be solved.

Summary of the invention

The embodiments of the present application provide a method and device for generating a bloom filter in a block chain, which solves the problem of the inability to accurately store the subject of the block in the prior art and the waste of a lot of storage space.

In the first aspect, an embodiment of the present application provides a method for generating a Bloom filter in a blockchain: obtaining the number of topics of a block; according to the number of topics, determining the topics that need to be occupied for storing each topic of the block The number of stored bits; the number of stored bits of the topic is positively correlated with the number of topics; the bloom filter of the block is generated according to the number of stored bits of the topic; the storage space of the bloom filter is greater than The number of bits in the topic storage; the initial value of each bit in the storage space of the Bloom filter is a first preset value; for each topic of the block, the The value of the bits mapped by the theme in the bloom filter is updated from the first preset value to a second preset value, and a bloom filter storing each theme of the block is obtained; the bloom In the storage space of the filter, the number of bits whose value is the second preset value is the number of bits for storing the topic; the Bloom filter is used to determine whether the block contains the subject of the query.

In the above method, the number of topics in the block is first obtained, and the number of topics stored in the topic is determined according to the number of topics, so as to generate the Bloom filter of the block according to the number of bits in the topic storage. The number of topics stored in the topic is positively correlated with the number of topics, and the storage space of the bloom filter is greater than the number of topics stored in the topic. Therefore, when the number of topics in a block is large, the bloom filter The storage space of the filter should also be correspondingly larger. When the number of topics in the block is small, the storage space of the bloom filter will be correspondingly small, so a bloom with a suitable storage space can be generated according to the number of topics in the block. filter. In addition, the initial values of the bits in the storage space of the bloom filter are all the first preset values. For each theme of the block, the theme can be placed in the bloom according to a preset mapping relationship. The value of the bits mapped in the filter is updated from the first preset value to the second preset value, and the Bloom filter storing each theme of the block is obtained. In summary, in the above method, the Bloom filter The storage space of the device is sufficient to meet the storage of each topic of the block, and in the storage space of the Bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the topic , Indicating that the bits of the mapping between the topics do not conflict, and the topic of the block can be accurately stored, so that the query result can be accurately provided when determining whether the block contains the subject of the query later.

In an optional implementation manner, the relationship between the number of topic storage bits and the number of topics is as follows: K=T*n; where K is the number of topic storage bits; T is the number of topics; n Is the number of bits occupied by each topic of the block in the bloom filter.

In the above method, the number of topic storage bits is proportional to the number of topics, and the number of bits occupied by each topic in the Bloom filter is the same, so that the number of topic storage bits can be conveniently calculated according to the number of topics. A method of determining the number of bits stored in a topic is proposed.

In an optional implementation manner, the ratio of the number of bits of the topic storage bit to the storage space of the bloom filter is m; m is the storage space usage rate configured by the bloom filter.

In the above method, the discreteness of the theme bits in the Bloom filter can be adjusted according to the setting of m, thereby improving the flexibility of storing the theme through the Bloom filter.

In an optional implementation manner, the obtaining the number of topics of the block includes: determining the transactions in the block and the events of each transaction; determining the number of index parameters for each event in the block; wherein , The number of index parameters of each event corresponds to a topic; the sum of the number of index parameters of each event in the block is used as the number of topics.

In the above method, the sum of the index parameters of each event in the block is used as the number of topics. Since the index parameters can be used to search for events, it can be ensured that each topic has a corresponding index. All topics can be searched.

In a second aspect, the present application provides a device for generating bloom filters in a blockchain, including: an acquisition module, configured to acquire the number of topics in a block; a processing module, configured to determine and store the number of topics based on the number of topics. The number of topic storage bits required for each topic of the block; the number of topic storage bits is positively correlated with the number of topics; the Bloom filter of the block is generated according to the number of topic storage bits; The storage space of the Bloom filter is greater than the number of bits stored in the theme; the initial value of each bit in the storage space of the Bloom filter is a first preset value; and is used to target the block For each topic of, update the bit value of the topic mapped in the Bloom filter from the first preset value to a second preset value according to a preset mapping relationship, and obtain and store the block The Bloom filter of each theme; in the storage space of the Bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the theme; the Bloom filter uses To determine whether the subject of the query is included in the block.

In an optional implementation manner, the acquisition module is specifically configured to: determine the transaction in the block and the event of each transaction; determine the number of index parameters for each event in the block; wherein each The number of index parameters of an event corresponds to a topic; the sum of the number of index parameters of each event in the block is taken as the number of topics.

For the beneficial effects of the above-mentioned second aspect and the embodiments of the second aspect, reference may be made to the beneficial effects of the above-mentioned first aspect and the embodiments of the first aspect, which will not be repeated here.

In a third aspect, an embodiment of the present application provides a computer device including a program or instruction, and when the program or instruction is executed, it is used to execute the method of the foregoing first aspect and each embodiment of the first aspect.

In a fourth aspect, an embodiment of the present application provides a storage medium including a program or instruction, and when the program or instruction is executed, it is used to execute the methods of the first aspect and the embodiments of the first aspect.

Description of the drawings

Figure 1 is a schematic diagram of the structure of the blockchain;

FIG. 2 is a schematic flowchart of steps of a method for generating a Bloom filter in a blockchain according to an embodiment of the application;

FIG. 3 is a schematic structural diagram of a device for generating a Bloom filter in a blockchain according to an embodiment of the application.

detailed description

In order to better understand the above technical solutions, the above technical solutions will be described in detail below with reference to the drawings and specific implementations of the specification. It should be understood that the embodiments of the application and the specific features in the embodiments are detailed to the technical solutions of the application. Note, rather than limiting the technical solution of the present application, the embodiments of the present application and the technical features in the embodiments can be combined with each other if there is no conflict.

The following first explains the terms appearing in the embodiments of the present application.

Block chain: As shown in Figure 1, the block chain is a chain composed of a series of blocks. In addition to recording the data of this block, each block also records the hash value of the previous block. Through this Way to form a chain. There are two core concepts of the blockchain, one is cryptography and the other is decentralization. Based on these two concepts, the historical information on the blockchain cannot be tampered with.

Node: Each participant in the network is a node, and the node participates in network formation and data exchange. In the blockchain network, a node refers to a participant with a unique identity. The node has a complete copy of the ledger and has the ability to participate in the consensus of the blockchain network and the maintenance of the ledger.

Smart contract: A smart contract is a computer protocol designed to spread, verify, or execute a contract in an information-based way. Smart contracts allow for trusted transactions without a third party, which are traceable and irreversible. The specific form of the smart contract is the code that is deployed on the blockchain to complete specific functions. For example, Solidity is a mainstream smart contract programming language. Smart contracts written in Solidity language can be called Solidity contracts. When a smart contract is deployed on the blockchain, a contract address will be generated, and users can call this smart contract through the contract address. The function defined in the smart contract is called the contract interface. The call to the smart contract is to call a contract interface in the contract through the contract address.

Transaction: A transaction is a user's request for an operation on the smart contract interface deployed on the blockchain. The transaction is initiated by the user and sent from the user's client to the blockchain node. After the blockchain node receives the transaction, it calls the corresponding smart contract according to the contract address and interface specified by the transaction. In the blockchain, any operation (deploy contract, call contract interface, etc.) is carried out by sending transactions. The transaction is initiated by the user and sent to the blockchain node through the client. After receiving the transaction, the blockchain node will package the transaction into a block and execute it.

After each transaction in the block is executed, a corresponding transaction receipt will be generated. The transaction receipt stores the execution result of the transaction. A typical transaction receipt contains important information such as the transaction hash, status bits that identify whether the transaction was successfully executed, and logs generated during execution.

Bloom filter:

Bloom filter is a data structure, a more ingenious probabilistic data structure. The core implementation is a large bit array and several hash functions. It is characterized by efficient insertion and query, which can be used to determine whether an element may exist or be certain. does not exist. Ethereum's Bloom filter greatly improves the efficiency of querying events.

Event, Log:

Event is a convenient interface provided by the virtual machine log infrastructure, which can be predefined and used to generate logs in a predetermined format. When an event (call) is sent, it will trigger parameters to be stored in the transaction log (a special data structure on the blockchain). These logs are associated with the address of the contract and recorded in the blockchain. Each transaction receipt of the blockchain will contain 0 or more log records. The logs represent the events triggered by the smart contract.

Topic:

In the blockchain, when defining an event, you need to specify the parameters of the event. Parameters are divided into two types, one is modified by indexed parameter keywords (indexed), and the other is not modified by indexed. The difference between these two parameters is that the former will be stored in the topic field of the log, while the latter will be stored in the data field of the log. The topic field of the log can be used for search, but the data field cannot be used for search.

When a financial institution (banking institution, insurance institution, or securities institution) is in the process of conducting business (such as bank loan business, deposit business, etc.), when task scheduling is performed between data center nodes, there may be sequential dependencies between tasks Relations, often through the blockchain to achieve financial activities. In some scenarios, it may involve searching for an event. A block can include multiple transactions, each transaction includes at least one event, and each event includes at least one topic.

The current method is to store the subject of the block through a fixed-length Bloom filter. In this way, for the subject of a block with a large transaction volume, there will be a phenomenon of bit duplication, and the Bloom filter will be completely invalid and cannot accurately store the subject of the block; for the subject of a block with a small transaction volume, It will also cause a lot of waste of storage space. This situation does not meet the needs of banks and other financial institutions, and cannot guarantee the efficient operation of various businesses of financial institutions.

It can be seen that the current Bloom filter cannot meet the needs of the blockchain in multiple scenarios, and the strategy for generating the Bloom filter needs to be adjusted, as shown in the schematic diagram of the process flow shown in Figure 2. This application proposes a zone The method of generating Bloom filters in the block chain.

Step 201: Obtain the subject count of the block.

Step 202: According to the number of topics, determine the number of topic storage bits needed to store each topic of the block.

The number of theme storage bits is positively correlated with the number of themes.

Step 203: Generate a Bloom filter of the block according to the number of bits stored in the theme.

The storage space of the Bloom filter is greater than the number of bits of the theme storage; the initial value of each bit in the storage space of the Bloom filter is a first preset value;

Step 204: For each topic of the block, update the bit value of the topic mapped in the Bloom filter from the first preset value to the second preset according to a preset mapping relationship Value to obtain the Bloom filter storing each topic of the block.

In the storage space of the bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the topic; the bloom filter is used to determine whether a block contains query data theme.

Step 201 occurs after the block is executed, and the number of topics of the block is obtained, which can be obtained by counting the number of topics included in all transaction receipts of the block. Specifically, the following implementation methods can be adopted:

Determine the transaction in the block and the event of each transaction; determine the number of index parameters for each event in the block; wherein, the number of index parameters for each event corresponds to a topic; divide the block The sum of the number of index parameters of each event in, is used as the number of topics.

For example, there are 100 transactions in a block, and the number of events for each transaction is 2. There are 2 index parameters modified by indexed for each event. Then the sum of the index parameters of each event in the block is 100*2*2=400, and the number of topics in the block is 400. It should be noted that the number of events for different transactions may be different, and the number of index parameters for different events is also different. Here we only illustrate how to obtain the number of topics.

In step 202, the relationship between the number of theme storage bits and the number of themes may be as follows:

K=T*n; where K is the number of stored bits for the topic; T is the number of topics; n is the number of bits occupied by each topic of the block in the Bloom filter.

Wherein, the number of bits occupied by each theme in the bloom filter can be freely set according to requirements. For example, each theme occupies 3 bits in the bloom filter.

In the process of generating the bloom filter in step 203, configuration items may be added in advance: the storage space utilization rate of the bloom filter; m is the storage space utilization rate configured by the bloom filter. Under the configuration of m, the ratio of the number of bits of the theme storage bit to the storage space of the Bloom filter is m.

This configuration item affects the query efficiency of the Bloom filter and the storage space of the Bloom filter. The smaller the usage rate, the lower the conflict rate of the Bloom filter and the faster the query speed. However, the larger the storage space is required. Seek a balance between search efficiency and storage. For example, m can be set to 10%, which is not limited here.

In step 203, the storage space size of the bloom filter of the current block can be calculated according to the number of topics of the block and the usage rate of the bloom filter bits:

Bloom filter size=the number of topics in the block*3/bloom filter space usage rate; unit: bit.

The initial value of each bit in the storage space of the Bloom filter generated in step 203 is the first preset value. In step 204, for each topic of the block, the The value of the bits mapped by the theme in the bloom filter is updated from the first preset value to a second preset value, and a bloom filter storing each theme of the block is obtained. For example, the first preset value is 0 and the second preset value is 1. It is also possible that the first preset value is 1 and the second preset value is 0.

In step 204, it is necessary to ensure that: in the storage space of the bloom filter, the number of bits whose value is the second preset value is the number of bits for storing the theme. In other words, each bit can only uniquely identify one topic, and there will be no conflicts in storage between the topics.

After step 204, each topic of the block is correspondingly stored in the Bloom filter. When you need to query a target event, you can do so by querying a target topic in the target event. If the target subject can be queried, the target event is determined to exist; otherwise, the target event does not exist.

The following describes the method of generating the Bloom filter in the above-mentioned blockchain in combination with two different specific scenarios.

scene one:

At this time, the blockchain works in a high-load, high-throughput (Transaction Per Second, TPS) mode. The parameters of block one are as follows:

The size of the Bloom filter in block one is:

The number of topics in block one*3/bit usage rate=40,000*3/10%=1200000bit≈1.2Mb.

Scene two:

In this scenario, the load of the blockchain is relatively low, and the parameters of block two are as follows:

The size of the Bloom filter in block two is:

The number of topics in block one*3/bit usage rate=60*3/10%=1800bit≈1.8Kb.

In the method from step 201 to step 204, the number of topics in the block is first obtained, and the number of topic storage bits is determined according to the number of topics, so as to generate the layout of the block according to the number of topics stored in the topic. Long filter, because the number of topics stored in the topic is positively correlated with the number of topics, and the storage space of the Bloom filter is greater than the number of topics stored in the topic, so when the number of topics in a block is large, The storage space of the bloom filter should also be correspondingly larger. When the number of topics of the block is small, the storage space of the bloom filter will be correspondingly small, so a suitable one can be generated according to the number of topics of the block. Bloom filter for storage space. In addition, the initial values of the bits in the storage space of the bloom filter are all the first preset values. For each theme of the block, the theme can be placed in the bloom according to a preset mapping relationship. The value of the bits mapped in the filter is updated from the first preset value to the second preset value, and the Bloom filter storing each theme of the block is obtained. In summary, in the above method, the Bloom filter The storage space of the device is sufficient to meet the storage of each topic of the block, and in the storage space of the Bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the topic , Indicating that the bits of the mapping between the topics do not conflict, and the topic of the block can be accurately stored, so that the query result can be accurately provided when determining whether the block contains the subject of the query later.

As shown in FIG. 3, the present application provides a device for generating Bloom filters in a blockchain, including: an acquisition module 301, configured to acquire the number of topics in a block; and a processing module 302, configured to obtain the number of topics based on the number of topics. Determine the number of theme storage bits required to store each theme of the block; the number of theme storage bits is positively correlated with the number of topics; according to the number of theme storage bits, the layout of the block is generated Long filter; the storage space of the bloom filter is greater than the number of bits stored in the theme; the initial value of each bit in the storage space of the bloom filter is a first preset value; and For each topic of the block, the value of the bit mapped by the topic in the Bloom filter is updated from the first preset value to the second preset value according to the preset mapping relationship, and the storage is obtained. The bloom filter of each theme of the block; in the storage space of the bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the theme; Bloom filter is used to determine whether the subject of the query is included in the block.

In an optional implementation manner, the obtaining module 301 is specifically configured to: determine the transaction in the block and the event of each transaction; determine the number of index parameters for each event in the block; wherein, each The number of index parameters of each event corresponds to one topic; the sum of the number of index parameters of each event in the block is used as the number of topics.

An embodiment of the application provides a computer device, including a program or instruction, when the program or instruction is executed, it is used to execute a method for generating a Bloom filter in a blockchain provided in an embodiment of the application and any one Optional method.

The embodiment of the application provides a storage medium, including a program or instruction, when the program or instruction is executed, it is used to execute a method for generating a bloom filter in a blockchain provided in the embodiment of the application and any one Optional method.

Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, this application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program codes.

This application is described with reference to the flowcharts and/or block diagrams of the methods, equipment (systems), and computer program products according to the application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the scope of the application. In this way, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

A method for generating a Bloom filter in a blockchain, which is characterized in that it includes:

Get the number of topics in the block;

According to the number of topics, determine the number of topic storage bits required to store each topic of the block; the number of topic storage bits is positively correlated with the number of topics;

According to the number of bits stored in the theme, the Bloom filter of the block is generated; the storage space of the Bloom filter is greater than the number of bits stored in the theme; each bit in the storage space of the Bloom filter The initial values of the bits are all the first preset values;

For each topic of the block, the value of the bit mapped by the topic in the Bloom filter is updated from the first preset value to a second preset value according to a preset mapping relationship to obtain Store the Bloom filter of each theme of the block; in the storage space of the Bloom filter, the number of bits whose value is the second preset value is the number of bits stored in the theme; The bloom filter is used to determine whether the subject of the query is included in the block.
The method according to claim 1, wherein the relationship between the number of bits in the topic storage and the number of topics is as follows:

K=T*n; where K is the number of stored bits for the topic; T is the number of topics; n is the number of bits occupied by each topic of the block in the Bloom filter.
The method according to claim 1, wherein the ratio of the number of bits of the theme storage bit to the storage space of the Bloom filter is m; and m is the storage space usage rate configured by the Bloom filter.
The method according to any one of claims 1-3, wherein said obtaining the number of topics of the block comprises:

Determine the transactions in the block and the events of each transaction;

Determine the number of index parameters for each event in the block; wherein, the number of index parameters for each event corresponds to a topic;

The sum of the number of index parameters of each event in the block is used as the number of topics.
A device for generating Bloom filters in a blockchain, which is characterized in that it comprises:

Get module, used to get the number of topics in the block;

The processing module is configured to determine, according to the number of topics, the number of topic storage bits needed to store each topic of the block; the number of topic storage bits is positively correlated with the number of topics; storage according to the topic The number of bits, the bloom filter that generates the block; the storage space of the bloom filter is greater than the number of bits stored in the theme; the initial value of each bit in the storage space of the bloom filter is equal Is a first preset value; and for each topic of the block, according to a preset mapping relationship, the bit value of the topic mapped in the bloom filter is set by the first preset The value is updated to the second preset value to obtain the Bloom filter storing each theme of the block; in the storage space of the Bloom filter, the value is the number of bits of the second preset value The number of bits is stored for the topic; the Bloom filter is used to determine whether the block contains the subject of the query.
7. The device of claim 5, wherein the relationship between the number of bits of the theme storage and the number of themes is as follows:

K=T*n; where K is the number of stored bits for the topic; T is the number of topics; n is the number of bits occupied by each topic of the block in the Bloom filter.
7. The device of claim 5, wherein the ratio of the number of bits of the theme storage bit to the storage space of the Bloom filter is m; and m is the storage space usage rate configured by the Bloom filter.
7. The device according to any one of claims 5-7, wherein the acquiring module is specifically configured to:

Determine the transactions in the block and the events of each transaction;

Determine the number of index parameters for each event in the block; wherein, the number of index parameters for each event corresponds to a topic;

The sum of the number of index parameters of each event in the block is used as the number of topics.
A computer device characterized by comprising a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 4 is executed.
A storage medium, characterized by comprising a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 4 is executed.