CN111224793B - Data storage method and device, computer equipment and readable storage medium - Google Patents

Data storage method and device, computer equipment and readable storage medium Download PDF

Info

Publication number
CN111224793B
CN111224793B CN201811427813.3A CN201811427813A CN111224793B CN 111224793 B CN111224793 B CN 111224793B CN 201811427813 A CN201811427813 A CN 201811427813A CN 111224793 B CN111224793 B CN 111224793B
Authority
CN
China
Prior art keywords
block
transactions
transaction
data
node device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811427813.3A
Other languages
Chinese (zh)
Other versions
CN111224793A (en
Inventor
江山
曹建农
林钦亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201811427813.3A priority Critical patent/CN111224793B/en
Priority to PCT/CN2019/120688 priority patent/WO2020108441A1/en
Publication of CN111224793A publication Critical patent/CN111224793A/en
Application granted granted Critical
Publication of CN111224793B publication Critical patent/CN111224793B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/14—Details of searching files based on file metadata
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/901—Indexing; Data structures therefor; Storage structures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/185—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast with management of multicast group membership
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/51—Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

The application discloses a data storage method, a data storage device, computer equipment and a readable storage medium, and belongs to the technical field of data storage. The method comprises the following steps: the node equipment packages transactions with similarity to generate a first block, and stores a block characteristic value representing all transaction characteristics in the block head of the first block, so that any node equipment can know the characteristics of the transactions stored in the first block based on the block characteristic value of the first block, and then the node equipment can screen out at least one target block meeting conditions by comparing the characteristic value of target index information with the block characteristic value in the block head of the block on a block chain configured by the node equipment, further can query the at least one target block, and acquire a plurality of target transactions from the at least one target block, so that all blocks on the local block chain do not need to be queried, the purpose of block skipping query is achieved, the query time is shortened, and the query efficiency is improved.

Description

Data storage method and device, computer equipment and readable storage medium
Technical Field
The present application relates to the field of data storage technologies, and in particular, to a data storage method and apparatus, a computer device, and a readable storage medium.
Background
With the advancement of the technology, the big data sharing technology is becoming mature, wherein the blockchain system can provide a platform for big data sharing and transaction due to the characteristics of decentralization, non-tampering, traceability and the like, and the platform is used for storing data and inquiring data.
The process of querying data by the blockchain system may be as follows: after receiving a transaction query request of a user, the node equipment acquires a semantic hash vector corresponding to the transaction according to the transaction query request; on a block chain configured by the node equipment, searching each block one by one transaction according to the sequence; when any one block is searched, a semantic hash vector corresponding to each transaction in the block needs to be obtained through calculation, so that the node device can search a target transaction from the block, and the similarity between the semantic hash vector of the target transaction and the semantic hash vector of a query transaction is greater than a preset threshold; and when all the transactions on the block chain are searched, sending the searched target transaction to the user node equipment.
In the process of implementing the invention, the inventor finds that the related art has at least the following problems: when a node device searches for transactions on a blockchain, because the transaction-by-transaction searching is performed on the whole blockchain block by block, the query time is too long, and the query efficiency is low.
Disclosure of Invention
The embodiment of the application provides a data storage method and device, computer equipment and a readable storage medium, which can solve the problem of low query efficiency when a block chain system queries data. The technical scheme is as follows:
in one aspect, a data storage method is provided, where the data storage method is applied to a user node device in a blockchain system, and the method includes:
converting a data description in metadata of the original data into a data representation;
generating a transaction based on the data representation in the metadata;
sending the transaction to the blockchain system;
receiving a first block, wherein a block characteristic value is stored in a block head of the first block, and the block characteristic value represents characteristics of all transactions in the first block;
when the block chain system passes the first block consensus, the first block is stored on the block chain configured by the user node equipment.
In one possible implementation, generating a transaction based on the data representation in the metadata includes:
replacing the data description in the metadata with the data representation;
and signing the replaced metadata to obtain the transaction.
Based on the possible implementation manner, replacing the data description in the metadata with the data representation may enable the user node device to generate a transaction carrying data representation based on the replaced metadata, and further enable the transaction stored in the blockchain system to carry the data representation.
In a second aspect, a data storage method is provided, which is used for any node device in a blockchain system, and the method includes:
acquiring a plurality of transactions from a transaction pool, wherein the similarity between any two transactions in the plurality of transactions meets a preset condition;
generating a first block based on the plurality of transactions, wherein a block characteristic value is stored in a block head of the first block, and the block characteristic value represents characteristics of all transactions in the first block;
when the first block is verified, the first block is added to a block chain of the node equipment configuration, and the first block is broadcasted in the block chain system.
In one possible implementation, the similarity between any two transactions refers to the similarity between the characteristic values of the data representations of any two transactions.
In one possible implementation, the obtaining a plurality of transactions from a transaction pool includes:
selecting a plurality of transactions from a transaction pool;
dividing the head hash of the latest block on the block chain of the node equipment into a plurality of sub-head hashes;
obtaining a characteristic value of the data representation of each transaction in the plurality of transactions based on the data representation of each transaction and the plurality of sub-header hashes;
and when the similarity of the characteristic values represented by the data of any two transactions in the multiple transactions meets a preset condition, acquiring the multiple transactions.
Based on the above possible implementation manners, it may be implemented to select the transactions with similarity or higher similarity in the transaction pool, so as to store the transactions with similarity or higher similarity in one block, which is convenient for the subsequent data query.
In one possible implementation, the node device obtaining a feature value of the data representation of each of the plurality of transactions based on the data representation of each of the plurality of transactions and the plurality of sub-header hashes, including:
splicing the data representation of one transaction and one sub-header hash to obtain splicing result data;
and carrying out Hash calculation on the splicing result data to obtain a characteristic value represented by the data of the transaction.
Based on the above possible implementation manners, the feature values of the transactions may be obtained, so that when the node device generates the block, the feature values of the transactions may be stored in a centralized manner, so that when the node device queries the transactions by using the feature values of the transactions, the feature values of the transactions may not be calculated any more, and then when the blockchain system performs data query, the calculation pressure of the blockchain system may be reduced.
In one possible implementation, the method further comprises:
when the similarity between any two transactions in the multiple transactions does not accord with the preset condition, the multiple transactions are obtained in the transaction pool again;
and when the similarity of any two transactions in the plurality of retrieved transactions meets the preset condition, generating a first block based on the plurality of transactions.
Based on the above possible implementation manners, it may be ensured that transactions obtained by the node device from the transaction pool have similarity, and further, the transactions having the similarity may be stored in one block.
In one possible implementation, the method further comprises:
acquiring metadata of original data;
converting the data description in the metadata to obtain data representation;
replacing the data description in the metadata with the data representation;
and signing the replaced metadata to obtain the transaction, and storing the transaction into the transaction pool.
Based on the above possible implementation manner, the transactions acquired by the node devices all include data representations, and then the transactions can be stored by using the characteristic values of the data representations, so that similar transactions can be stored in one block.
In a third aspect, a data query method is provided, which is applied to any node device in a block chain system, and the method includes:
receiving a transaction query request, wherein the transaction query request comprises target index information;
determining at least one target block from a block chain configured by the node equipment, wherein the similarity between a block characteristic value in a block head of each target block and a characteristic value of the target index information meets a first preset condition, and the block characteristic value represents the characteristics of all transactions in the target block;
acquiring at least one target transaction from the at least one target block, wherein the similarity between each target transaction and the target index information meets a second preset condition;
outputting the at least one target transaction as a query result.
In one possible implementation, the node device determines at least one target block from a block chain configured by the node device, including:
acquiring a characteristic value of the target index information;
obtaining a block eigenvalue within a block header of at least one block on the block chain;
and when the similarity between the block characteristic value in the block head of any block in the at least one block and the characteristic value of the target index information meets the first preset condition, acquiring the block as a target block.
When the affairs are inquired based on the possible implementation mode, the target block can be determined firstly, then the affairs can be inquired in the target block, the aim of block skipping inquiry can be achieved, and the efficiency of data inquiry is greatly improved.
In one possible implementation, after the node device receives the transaction query request, the method further includes:
traversing at least one block chain configured on the node equipment, and outputting a query result as null when the height of a tail block of any block chain is zero;
when the height of the tail block of any block chain is not zero, the steps of determining a target block and acquiring a target transaction are executed on the block chain.
Based on the possible implementation manner, whether a block is stored in the block chain queried by the node device is determined, and when no block is stored in the block chain, the block chain does not need to be queried, and then other block chains configured on the ground node device can be queried, so that a query result can be obtained on other block chains.
In one possible implementation, the node device obtains at least one target transaction from the at least one target block, including:
obtaining the characteristic values of all the transactions in any target block of the at least one target block;
and when the characteristic value represented by the data of any transaction in the target block and the characteristic value of the target index information meet the second preset condition, acquiring the transaction as a target transaction.
Based on the possible implementation manner, the target transaction can be determined directly according to the comparison result of the characteristic value of the transaction and the characteristic value of the target index information, the characteristic value of the transaction in the target block does not need to be calculated, the calculation pressure of a block chain system is reduced, and the purpose of data query is improved.
In a fourth aspect, a data storage device is provided for performing the above data storage method. Specifically, the data storage apparatus includes functional modules for executing the data storage method provided in the first aspect or any optional manner of the first aspect, where the aspect is an aspect corresponding to a data storage method applied to a node device in a blockchain system.
In a fifth aspect, a data storage device is provided for performing the above data storage method. Specifically, the data storage apparatus includes functional modules for executing the data storage method provided in the second aspect or any optional manner of the second aspect, where the above aspect is an aspect corresponding to a data storage method applied to any node device in a blockchain system.
In a sixth aspect, a data query apparatus is provided for executing the data query method. Specifically, the data query apparatus includes a functional module configured to execute the data query method provided in the third aspect or any optional manner of the third aspect, where the above aspect is an aspect corresponding to a data query method applied to any node device in a blockchain system.
In a seventh aspect, a computer device is provided, which includes a processor and a memory, where at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the operation performed by the data storage method.
In an eighth aspect, a computer device is provided, which includes a processor and a memory, where at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the operations performed by the data query method.
In a ninth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, the instruction being loaded and executed by a processor to implement the operations performed by the data storage method as described above.
In a tenth aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the operations performed by the data query method as described above.
The technical scheme provided by the embodiment of the application has the following beneficial effects:
the method comprises the steps that transactions with similarity are packaged through node equipment to generate a first block, and a block characteristic value representing the characteristics of all the transactions is stored in the head of the first block, so that any node equipment can know the characteristics of the transactions stored in the first block based on the block characteristic value of the first block, and then the node equipment can screen out at least one target block meeting conditions by comparing the characteristic value of target index information with the block characteristic value in the head of the block on a block chain configured by the node equipment, further can query the at least one target block, and obtains a plurality of target transactions from the at least one target block, so that all the blocks on the local block chain do not need to be queried, the purpose of block skipping query is achieved, the query time is shortened, and the query efficiency is improved.
Drawings
Fig. 1 is a schematic structural diagram of a block chain system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a node device according to an embodiment of the present invention;
fig. 3 is a flowchart of a method for generating a transaction by a node device according to an embodiment of the present invention;
FIG. 4 is a block chain system for storing data according to an embodiment of the present invention;
FIG. 5 is a flow chart of a data storage method according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating a node device packaging transaction according to an embodiment of the present invention;
FIG. 7 is a flowchart of a data query method provided by an embodiment of the present invention;
FIG. 8 is a block diagram of a data storage device according to an embodiment of the present invention;
FIG. 9 is a block diagram of a data storage device according to an embodiment of the present invention;
fig. 10 is a block diagram of a data query apparatus according to an embodiment of the present invention.
Detailed Description
Embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a blockchain system according to an embodiment of the present invention, and referring to fig. 1, the system includes a plurality of node devices. The node devices may be configured with the same blockchain, that is, the node devices may form a blockchain system, and the node devices may be multiple devices of the same organization, multiple devices belonging to different organizations, or devices of a common user.
Fig. 2 is a schematic structural diagram of a node device according to an embodiment of the present invention, where the node device 200 includes a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 201 and one or more memories 202, where the memory 202 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 201 to implement the methods provided by the method embodiments described below. Certainly, the node device 200 may further have components such as a wired or wireless network interface, a keyboard, and an input/output interface, so as to perform input and output, and the node device 200 may further include other components for implementing device functions, which are not described herein again.
In an exemplary embodiment, there is also provided a computer readable storage medium, such as a memory including instructions executable by a processor in a terminal to perform a data storage method or a data query method in the following embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a Random Access Memory (RAM), a compact disc-read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.
Fig. 3 is a flowchart of a method for generating a transaction by a node device according to an embodiment of the present invention, where the embodiment of the present invention is an example constructed by starting from any node device in the block chain system shown in fig. 1, and referring to fig. 3, the method according to the embodiment of the present invention specifically includes:
301. the user node device U1 obtains metadata for the original data.
The user node device U1 is a node device of a user in the blockchain system, and the raw data may be transaction data, photo data, video data, medical data, etc. uploaded by the user, or raw data stored on the user node device U1, or raw data stored on a device associated with the user node device U1.
After the user uploads the raw data to the user node device U1, the user may input the owner, creation time, data format, data description, etc. of the raw data on the user node device U1, where the creation time may be the time of creating metadata or the time of uploading the raw data by the user, the data format may be text, document, video, etc., and the data description is used to describe the characteristics of the stored data, such as weather, fruit, or air temperature, etc., and when the user has input the above information, the user node device U1 may generate a piece of metadata corresponding to the raw data, which may include a data source, a data type, and a data description, where the data source may be the owner of the raw data input by the user, and the data type may be the data format of the raw data input by the user, the data description may be a data description of the raw data entered by the user. Of course, the user may also enter the above on a device associated with the user node device U1.
Fig. 4 is a schematic diagram of a block chain system storage data provided in an embodiment of the present invention, and referring to fig. 4, the system mainly includes three parts, namely, a user node device U1 generation transaction, a node device C1 generation block, and a node device C2 storage block, where a description of the part of the node device C1 generation block refers to steps 501 to 506, and a description of the part of the node device C2 storage block refers to steps 509 to 511. The part of the user node device U1 that generates the transaction may be described as follows:
after the user uploads the original data, the user node device U1 inputs owner "zhangsan", creation time "2018.01.01", data format "text file" and data description "weather" and the like, and the user node device U1 may generate a piece of metadata m based on the information input by the user and the original data uploaded by the userjThe metadata mjIncluding the owner "zhang san" creation time "2018.01.01, the data format" text file "and the data description" weather ".
302. The user node device U1 converts the data description in the metadata into a data representation.
The data representation may be a fixed-length real-valued vector that may be transformed from the data description by a predefined transformation algorithm, which is actually a semantic representation of the data description, e.g. the transformed data representation of the data description "weather" may be 00110101.
303. The user node device U1 replaces the data description in the metadata with the data representation.
The user node device U1 may replace the contents of a field representing a data representation with the data representation. The replaced metadata may include data source, data type, and data representation.
304. The user node device U1 signs the replaced metadata to get the transaction.
After joining the blockchain system, the user node device U1 is provided with a public and private key pair, that is, each node device may have a corresponding pair of public and private keys, and when the user node device U1 signs the replaced metadata, the user node device U1 signs the metadata with the private key.
It should be noted that, in the related art, the transaction obtained by the user node device U1 does not include a data representation, but the transaction in the embodiment of the present invention includes a data representation, which may represent characteristics of the transaction, and when querying data in the blockchain system, the data may be queried according to the similarity between the characteristics of the transactions on the blockchain in the blockchain system, so as to improve the querying efficiency.
305. The user node device U1 sends the transaction to the blockchain system.
In the embodiment of the present invention, a transaction is generated based on the metadata acquired by the user node device U1, and then, the user node device U1 sends the transaction to the blockchain system, so that other node devices except the user node device U1 can receive the transaction.
306. The user node device U1 stores the transaction in a transaction pool.
The transaction pool is a temporary memory pool of the node device in the blockchain system, and is used for storing the transaction generated by the node device and the transaction passing the verification, and the step of verifying the transaction is shown in step 307.
It should be noted that the steps 305 and 306 have no specific execution sequence, that is, the user node device U1 may also execute the step 306 first and then execute the step 305, and the execution sequence of the steps 305 and 306 is not specifically limited in the embodiment of the present invention.
307. After acquiring the transaction, the node device C1 validates the transaction.
The node device C1 is any node device except the user node device U1, and may be a consensus node device with a consensus function, where the node device C1 obtains the transaction, may directly obtain the transaction sent by the user node device U1, or obtain a transaction broadcasted by another node device based on the user node device U1.
The node device C1 validating the transaction may include a validation process of at least one of:
1. when the node device C1 detects that the transaction carries a data representation, the transaction is verified to be passed, for example, the data in the transaction is represented as 00110101, when the node device C1 detects that the transaction carries a data representation 00110101, the transaction is verified to be passed, and when the node device C1 detects that the transaction does not carry a data representation 00110101, the transaction is not verified to be passed.
2. When the node device C1 detects that the format of the transaction is legal, the transaction is verified to be passed, for example, when the node device C1 detects that the owner, the creation time, the data format, and the data representation of the transaction all meet the format requirements, the verification is passed, and when the node device C1 detects that any one of the owner, the creation time, the data format, the data description, and the data representation of the transaction does not meet the format requirements, the verification is not passed.
3. When the node device C1 detects that the signature of the transaction is legal, the transaction is verified, for example, when the node device C1 detects that the signature of the transaction is the signature generated by the owner of the public key in the transaction, the verification is passed, and when the node device C1 detects that the signature of the transaction is not the signature of the owner of the public key in the transaction, the verification is not passed.
Of course, based on the verification requirements of the blockchain system, any one of the above verification processes may be set in the blockchain system, but in some embodiments, a combination of the above verification processes may also be set to verify the transaction, and the transaction is considered to be verified only when the transaction passes the verification of the above verification processes, and the transaction is considered to be verified not when any one of the verification processes fails.
308. When the transaction is validated, the node device C1 stores the transaction in a transaction pool.
309. The node device C1 broadcasts the transaction in the blockchain system.
The node device C1 may broadcast the transaction so that other node devices than the node device C1 can receive the transaction.
It should be noted that, each node device in the block system maintains a transaction hash list, the transaction hash list includes all the stored transaction hash values of the node device, when the node device C1 detects that there is the transaction hash value of the transaction on the transaction hash list, it indicates that the transaction has been stored by the node device C1, the node device C1 need not store and broadcast the transaction again, when the node device C1 detects that there is no hash value for the transaction on the transaction hash list, indicating that the transaction has not been stored by the node device C1, the node device C1 needs to store and broadcast the transaction, which, when packed into a chunk, the node device C1 adds the hash value of the transaction to a transaction hash list, to indicate that the transaction has been stored, therefore, the node equipment in the blockchain system can be prevented from storing and broadcasting the same transaction for multiple times.
The above process is a process of generating a transaction for a node device in the blockchain system and storing the transaction to a transaction pool, and the blockchain system needs to store the transaction in the transaction pool to a blockchain in the blockchain system to achieve the purpose of data storage. Then, in order to further embody the process of storing data in the blockchain system, a data storage method provided by the embodiment of the present invention is described herein by taking the specific embodiment shown in fig. 5 as an example.
501. Node device C1 selects a plurality of transactions from the transaction pool.
This node device C1 is actually any node device in the blockchain system, and this step 501 can be executed for both the node device that generated the transaction and the node device that received the broadcast transaction.
The node device puts a plurality of transactions selected from the transaction pool into a transaction sequence, and it can be understood that the transactions in the transaction sequence conform to a certain order, and specifically, taking the process of packing 2 transactions by the node device in fig. 6 as an example, the following description is made: after the node device selects a transaction 1 and a transaction 2 from the transaction pool, the 2 transactions are put into a transaction sequence in sequence, wherein the transaction 1 is the 1 st transaction, and the transaction 2 is the 2 nd transaction.
Taking 2 transactions selected from the transaction pool by the node device C1 as an example, referring to the part of the block generated by the node device C1 in fig. 4, the node device C1 selects k-2 Transactions (TX) from the current transaction pool2And TXj) Wherein, TX2Data in (1) is represented as 00110001, TXjIs represented as 00110101; node device C1 will TX2And TXjPacked into a transaction sequence, wherein TX2For the 1 st transaction in the transaction sequence, TXjIs the 2 nd transaction in the transaction sequence.
It should be noted that, when the number of transactions in the transaction pool of the node device C1 reaches the first preset number, the node device C1 may perform this step 501, and the embodiment of the present invention is not limited to the first preset number of transactions in the transaction pool.
502. The node apparatus C1 splits the head hash of the newest chunk on the blockchain allocated by the node apparatus C1 into a plurality of sub-head hashes.
The latest block is the last block in a blockchain allocated to the node device C1, or the block with the largest block height in a blockchain allocated to the node device C1.
The node device C1 may equally divide the head hash of the newest chunk on the chunk chain allocated by the node device C1 into a plurality of sub-head hashes, wherein the number of the sub-head hashes is the same as the number of the selected transactions of the node device C1.
Specifically, based on the selected example in step 501, referring to the part of generating the tile by node device C1 in fig. 4, the following example is performed: node device C1 splits the head hash value H of the newest chunk on its configured chunk chain into sub-head hashes H1And h2Wherein the header hash value H is 128-bit data, and is divided into sub-header hashes H1And h2The first 64 and the last 64 of H, respectively, i.e. H1Is h 0:63]，h2Is h [64:127 ]]。
503. The node apparatus C1 obtains a characteristic value of the data representation of each of the plurality of transactions based on the data representation of each of the plurality of transactions and the plurality of sub-header hashes.
This step 503 can be completed by the following procedure from step 503A to step 503B:
in step 503A, the node device C1 concatenates the data representation of a transaction and a sub header hash to obtain concatenation result data.
The node device C1 may allocate the multiple sub-header hashes to each transaction in the order of the transaction sequence, and concatenate the data representation of one transaction and the sub-header hash of the allocated one transaction, specifically, based on the example in step 502, see the part of the node device C1 generating the chunk in fig. 4, which is described as follows by way of example: node device C1 sends h1Is allocated to TX2H is to be2Is allocated to TXjNode device C1 will TX2Data representation and TXjRespectively splicing the assigned h1And h2，00110001+h1And 00110101+ h2Namely splicing result data.
In step 503B, the node device C1 performs hash calculation on the splicing result data to obtain a feature value represented by the data of the transaction.
Specifically, based on the example in step 503A, referring to the part of the node device C1 in fig. 4 for generating the tile, the following example is performed: the node device C1 performs a semantic hash function SH () calculation on the concatenation result data to obtain a semantic hash vector, which is a feature value represented by the data of the transaction, wherein TX2The semantic hash vector of (2) is SH (00110001+ h)1)＝100011100，TXjThe semantic hash vector of (2) is SH (00110101+ h)2)＝100011000。
When the node apparatus C1 performs steps 503A and 503B on the selected multiple transactions, the node apparatus C1 acquires the characteristic value represented by the data of each transaction.
504. The node apparatus C1 acquires the similarity of any two transactions of the plurality of transactions.
The similarity between any two transactions may be the similarity between feature values of data representations of the any two transactions, and the feature values may be semantic hash vectors of the transactions.
The similarity may be determined by any similarity calculation method, for example, the node device C1 may perform a hamming distance calculation on the semantic hash vectors of the data representations of the two transactions to obtain the similarity of the two transactions, and specifically, based on the example in step 503B, see the part of generating the block by the node device C1 in fig. 4, as described below: node device C1 calculates TX2And TXjThe hamming distance of the semantic hash vectors can obtain that the similarity s of the two semantic hash vectors is 0.89.
505. When the similarity of the characteristic values represented by the data of any two transactions in the plurality of transactions meets a preset condition, the node device C1 acquires the plurality of transactions.
The preset condition may be that the similarity of the characteristic values represented by the data of any two transactions in the multiple transactions is greater than a third preset threshold, and the embodiment of the present invention does not specifically limit a specific numerical value of the third preset threshold.
Specifically, based on the example in step 504, referring to the part of the node device C1 in fig. 4 for generating the tile, the following description is made: the third preset threshold th of node device C1 is 0.8, and TX2Semantic hash vector sum TXjHas a similarity of 0.89, so TX2Semantic hash vector sum TXjThe similarity of the semantic hash vectors is greater than a third preset threshold, namely TX2And TXjThe two transactions meet the preset condition, and the node device C1 acquires the transaction TX2And TXj。
When the similarity between any two transactions in the plurality of transactions acquired by the node device C1 does not meet the preset condition, the plurality of transactions that do not meet the preset condition may be put back into the transaction pool, and then the node device C1 acquires the plurality of transactions again in the transaction pool, and when the similarity between any two transactions in the plurality of transactions that are acquired again meets the preset condition, a first chunk is generated based on the plurality of transactions that are acquired again.
As can be seen from the above description, the semantic hash vectors of any two transactions of the multiple transactions selected by the node device C1 are similar, that is, the semantic hash vectors of the any two transactions are similar, and when the third preset threshold is higher, the similarity of the multiple transactions selected by the node device C1 is higher. Then, if the transactions with higher similarity are stored in one block, see step 506, when querying data, only a plurality of target blocks can be queried according to the similarity between the transactions, so that the querying efficiency can be improved, see steps 705 to 706.
It should be noted that the terms "first", "second", "third", and the like in the embodiment of the present invention are not used to distinguish different functions of the preset threshold, but are used to distinguish different values of the preset threshold, and it should be understood that the "first preset threshold", "second preset threshold", and "third preset threshold" appearing in the embodiment of the present invention may also be the same.
It should be noted that steps 501 to 505 are procedures for acquiring a plurality of transaction correspondences from the transaction pool by the node device C1.
506. The node apparatus C1 generates a first chunk based on the plurality of transactions, the first chunk having a chunk characteristic value stored in a chunk header, the chunk characteristic value including characteristics indicative of all transactions in the first chunk.
The block syndrome may be stored in a signature field, which may be located in the block header of the first block, and the signature field may be a new field in the block header of the first block or may be a utilization of a blank field in the block header of the first block.
When the eigenvalues of the data representation of the transaction are semantic hash vectors and node device C1 is generating the first chunk, the semantic hash vectors of all transactions in the first chunk may be sequentially added to a fixed-length bloom filter at the head of the chunk, so that the bloom filter may generate a chunk eigenvalue, which may represent the characteristics of all transactions in the first chunk, based on the input semantic hash vectors, e.g., node device C1 inputs the eigenvalue of the data representation of transaction 1 into the bloom filter, which generates a chunk eigenvalue 1, and then node device C1 inputs the eigenvalue of the data representation of transaction 2 into the bloom filter, so that the chunk eigenvalue 1 is updated to the chunk eigenvalue 2, when node device C1 generates only transaction 1 and transaction 2 into the first chunk, the syndrome 2 may be used as the syndrome of the first block.
Since the plurality of transactions have higher similarity, the characteristic of the first block can be reflected from the block characteristic value, and then, when a transaction is queried, it can be determined whether the first block stores a transaction to be queried according to the block characteristic value, see step 704.
Specifically, based on the example in step 504, taking the node device C1 as an example to generate two transactions into a chunk, see the part of the node device C1 in fig. 4 for the following description:
node device C1 compares the findings s>th (th is 0.8), so the two transactions are TX2And TXjGenerating a new block BjNode ofDevice C1 will TX2And TXjSemantic hash vector of (a) is added to block BjIn the distance sensitive bloom filter of the header, the distance sensitive bloom filter can be used as a feature field, and all semantic hash vectors in the feature field can be used as a block BjThe block syndrome of (2).
When generating a block, it is necessary to add not only information in the feature field in the block header but also a block height, the number of transactions, a block size, and the like.
507. The node apparatus C1 adds the first chunk to the chain of chunks allocated by the node apparatus C1.
It should be noted that step 506 and step 507 are not in a strict execution order, and the user node device U1 may also execute step 506 first and then execute step 507, and the execution order of steps 506 and 507 is not specifically limited in the embodiment of the present invention.
508. The node apparatus C1 broadcasts the first tile in the system of tile chains.
Node device C1 may broadcast the first tile so that other node devices than node device C1 can receive the first tile.
509. When node device C2 receives the first block, it verifies the first block.
The node device C2 is any node device except the node device C1, and the node device C2 acquires the first chunk, which may be the first chunk directly acquired by the node device C1 broadcast, or the first chunk acquired by another node device based on the node device C1 broadcast.
In this step 509, the node device in the blockchain system starts to agree on the first block, and then the node device C2 may agree on the first block, where the agreement includes the following processes:
any node device in the blockchain system generates a block according to the steps of 501 and 505; after any node equipment in the block chain receives the block, verifying the block; when the block passes the verification, the block is added to a block chain configured by the node equipment, and the block is broadcasted in a block chain system so that other node equipment can verify and store the block; when all the node devices in the blockchain system verify the block and store the block, the node devices in the blockchain system agree on the block, and the block is stored on the local blockchain in the blockchain system as the latest block on the local blockchain, that is, the block is the tail block on the local blockchain.
When the verified blocks all meet the following 5 verification conditions, namely verification is passed, the blocks can be added to the block chain of the node equipment configuration, otherwise, verification is not passed, and the blocks cannot be added to the block chain of the node equipment configuration.
The 5 validation conditions were:
1. the number of transactions within the block is greater than or equal to a second predetermined number.
2. The characteristic values of the data representations of any two transactions in the block meet a preset condition.
3. The block carries the signature of the generator.
4. The block height of the block is 1 greater than the height of the latest block of the local blockchain.
5. The last chunk header hash of the chunk is equal to the hash value of the chunk header of the latest chunk of the local blockchain.
In some embodiments, when the block satisfies at least one of the last 3 verification conditions of the above 5 verification conditions, in addition to the first 2 conditions of the above 5 verification conditions, the block may also be considered to pass the verification, it may be understood that the last 3 verification conditions of the above 5 verification conditions may also be other conditions, and the content of the verification conditions, the number of the verification conditions, and the second preset number are not specifically limited in the embodiments of the present invention.
It should be noted that the node device C2 may verify a block only when the block height of the block is greater than the height of the latest block in the local blockchain, otherwise, the node device C2 may not verify the block, so that multiple verifications and storages of the same block by the node device may be avoided.
510. When the first block is verified, the node apparatus C2 adds the first block to the block chain of the node apparatus allocation.
511. The node apparatus C2 broadcasts the first tile in the tile chain.
This step 511 enables any device in the blockchain system except the node device to receive, verify and store the first block, it is understood that any node device in the blockchain system broadcasts a block that passes verification so that all node devices in the blockchain system can verify and store the block that passes verification, and then, when each node device in the blockchain system stores the block that passes verification, that is, each node device in the blockchain system agrees with the block, that is, the block is added to the blockchain in the blockchain system.
According to the method provided by the embodiment of the invention, the plurality of transactions with higher similarity are screened from the transaction pool by the node equipment, and are packed and generated into the block, so that the transactions stored in the blocks on the block chain in the block chain system have similarity, the node equipment in the block chain system can conveniently inquire data according to the similarity of the transactions, and the inquiry efficiency can be greatly improved. In addition, the characteristic field in the block header of the block stores the characteristic value represented by the data of each transaction to characterize the block characteristic value of the block, and further, the characteristics of all transactions in the block can be known from the block characteristic value to perform transaction query.
Based on the above data storage method, taking an example of querying data by any node device in a block system, the data querying method provided by the present invention is described as follows, fig. 7 is a flowchart of a data querying method provided by an embodiment of the present invention, and referring to fig. 7, the method provided by the embodiment of the present invention includes:
701. any node device in the blockchain system receives a transaction query request, wherein the transaction query request comprises target index information.
The transaction query request is used for indicating the node device to query a transaction, and the transaction query request carries target index information, for example, after a user inputs a keyword "weather" in a search box of a blockchain system interface of the node device, the user clicks a virtual search button in the blockchain system interface, the node device can receive the transaction query request about "weather", and the "weather" can be used as the target index information.
702. And the node equipment acquires the characteristic value of the target index information according to the received transaction query request.
The node device may obtain a feature value of the target index information according to the target index information in the transaction query request, where the feature value may be a semantic hash vector. The algorithm of the feature value used for obtaining the feature value in the query process is the same as the algorithm of the feature value obtained in step 503, and is not described herein again.
For example, when the feature value is a semantic hash vector and the target index information is "weather", the node device may obtain the data representation of the target index information by using the algorithm used in calculating the transaction feature value in fig. 4, where the data representation of "weather" is 00110101, and the node device may directly perform hash calculation on the data representation without splicing the data representation to obtain the semantic hash vector of the data representation, so that the semantic hash vector of "weather" is 00110100.
703. When the block height of the tail block of one block chain configured by the node equipment is zero, the node equipment outputs a query result as null.
Since the node device only inquires the tail block on the blockchain at this time, but the block height of the tail block is zero, that is, no transaction is stored on the blockchain, the inquiry result at this time is empty. Of course, when the block height of the tail block of one block chain is zero, the node device may continue to query whether the tail blocks of other block chains configured by the node device are zero.
It should be noted that step 702 and step 703 do not have a strict order, and the node device may also perform step 703 first and then perform step 702, and the execution order of steps 702 and 703 is not specifically limited in the embodiment of the present invention.
704. When the block height of the tail block of a block chain configured by the node equipment is not zero, the block eigenvalue in the block head of at least one block on the block chain is obtained.
The node device may obtain a block characteristic value from a characteristic field within a block header of the at least one block. When obtaining the block eigenvalue, the block eigenvalue may be obtained according to the block height of the block on the block chain. For example, the node device may obtain the block characteristic value in the block header of at least one block in the block chain configured by the node device according to the descending order of the block heights, and may include the following steps: the highest block height on the block chain allocated by the node equipment is H, the node equipment acquires the block with the block height of H, and acquires the block characteristic value in the block head of the block. And taking H-1, and then obtaining the block characteristic value of the block with the block height of H-1 until the block characteristic value of the block with the block height of 1 is obtained, namely the node equipment obtains the block characteristic values of all blocks on the block chain configured by the node equipment.
Of course, the node device may also start acquiring from a block with a block height of 1, or randomly acquire a block eigenvalue in a block header of at least one block on the block chain, and the embodiment of the present invention does not specifically limit the order of acquiring the block eigenvalues in the block header of the blocks and the number of the block eigenvalues in the block header of the blocks.
705. When the similarity between the block eigenvalue in the block header of any one of the at least one block and the eigenvalue of the target index information meets a first preset condition, the node device acquires the block as a target block.
The first preset condition is that the similarity between the block eigenvalue in the block header of the target block and the eigenvalue of the target index information is greater than a first preset threshold, and the embodiment of the invention specifically defines the specific numerical value of the first preset threshold.
By comparing the similarity, it can be known that the similarity between the transaction in the target block and the target index is higher, so that the transaction can be queried only from the at least one target block, and the query is not continued to be performed on other blocks, so as to achieve the purpose of block-skipping query, reduce the query range, and improve the query efficiency, see step 707.
It should be noted that, steps 702 to 705 are steps of determining a target block, that is, the node device determines at least one target block from a block chain configured by the node device, where a similarity between a block characteristic value in a block header of each target block and a characteristic value of the target index information meets a first preset condition, and the block characteristic value represents characteristics of all transactions in the target block.
706. And the node equipment acquires at least one target transaction from the at least one target block, wherein the similarity between each target transaction and the characteristic value of the target index information meets a second preset condition.
The second preset condition may be that the similarity between the feature value represented by the data of each target transaction and the feature value of the target index information is greater than a second preset threshold, that is, a transaction with a higher similarity to the target index information is queried from the at least one target block, and then at least one target transaction is screened out.
The node device obtains at least one target transaction from the at least one target block, which may be implemented by the following procedure shown in steps 706A to 706B:
in step 706A, the node device obtains the characteristic values of all transactions in any target block of the at least one target block.
Step 706B, when the eigenvalue of the data representation of any transaction in the target block and the eigenvalue of the target index information meet the second preset condition, the node device acquires the transaction as a target transaction.
In this step 706B, for each target block, the transaction sequence of each target block may be obtained first, then the semantic hash vector of each transaction in the feature field of the target block is obtained according to the sequence of each transaction sequence, each semantic hash vector is compared with the semantic hash vector of the target index information, and when the similarity between the semantic hash vector of any transaction and the semantic hash vector of the target index information is greater than a second preset threshold, the transaction is taken as the target transaction.
It should be noted that, when the node device obtains the target transaction, the node device may execute step 706B every time a target block is determined, or execute step 706B when all target blocks on the block chain are determined, or execute step 706B when a third preset number of target blocks are determined, and the embodiment of the present invention does not have a limitation on the third preset number.
It should be noted that, because the characteristic fields in the target block store the characteristic values represented by the data of all transactions in the block, when the node device queries the target block each time, the node device can avoid calculating the characteristic value represented by the data of each transaction, which reduces the amount of calculation of the block chain system, and at the same time, reduces the query time, thereby improving the query efficiency.
707. The node device outputs the at least one target transaction as a query result.
When the node device outputs the at least one target transaction, the node device may output the acquired at least one target transaction each time the node device acquires the at least one target transaction from one target block, may output the acquired at least one target transaction each time the node device acquires the at least one target transaction from a partial block of all target blocks, or may output the acquired at least one target transaction each time the node device acquires the at least one target transaction from all target blocks, of course, it may also be that each time the node device acquires a target transaction, the node device outputs the target transaction, or the node device outputs a plurality of target transactions every time when the node device queries the plurality of target transactions, and the embodiment of the present invention does not specifically limit the time and the number of outputting at least one target transaction.
According to the method provided by the embodiment of the invention, the node equipment compares the characteristic value of the target index information with the block characteristic value in the block head of the block on the block chain configured by the node equipment, so that at least one target block meeting the condition can be screened, the at least one target block can be further inquired, and a plurality of target transactions are obtained from the at least one target block, therefore, all blocks on the local block chain do not need to be inquired, the purpose of block jump inquiry is realized, the inquiry time is further shortened, and the inquiry efficiency is improved. And when the target transaction is screened from the target block, the characteristic value of the target index information can be directly compared with the characteristic value represented by the data of each transaction stored in the block head of the target block, so that the target transaction meeting the condition is screened out, the characteristic values represented by the data of all the transactions of the target block do not need to be calculated again, the calculation load of the block chain system is further reduced, the query time is further shortened, and the query efficiency is further improved.
Fig. 8 is a block diagram of a data storage device according to an embodiment of the present invention, and referring to fig. 8, the data storage device is applied to a user node equipment in a block chain system, and the data storage device includes:
a conversion module 801 for converting data descriptions in metadata of raw data into data representations;
a generating module 802 for generating a transaction based on the data representation in the metadata;
a sending module 803, configured to send the transaction to the blockchain system;
a receiving module 804, configured to receive a first block, where a block syndrome is stored in a block header of the first block, and the block syndrome indicates characteristics of all transactions in the first block;
a storage module 805, configured to store the first block on the user node equipment configured blockchain when the blockchain system passes the consensus for the first block.
Optionally, the generating module 802 is configured to replace the data description in the metadata with the data representation;
the device also includes:
and the signature module is used for signing the replaced metadata to obtain the transaction.
According to the device provided by the embodiment of the invention, the data description in the metadata is replaced by the data representation through the user node equipment, the transaction is generated, and the generated transaction is sent to the block chain system, so that other node equipment can generate the received transaction into the first block. In addition, the block characteristic value of the first block is stored in the block head of the first block to characterize the characteristics of each transaction stored in the first block, and further the characteristics of all transactions in the block can be known from the block characteristic value to perform transaction query.
Fig. 9 is a structural device diagram of a data storage device according to an embodiment of the present invention, and referring to fig. 9, the structural device is applied to any node device in a block chain system, and the device includes: acquisition module 901, generation module 902 and addition module 903
An obtaining module 901, configured to obtain multiple transactions from a transaction pool, where a similarity between any two transactions in the multiple transactions meets a preset condition;
a generating module 902, configured to generate a first block based on the plurality of transactions, where a block syndrome value is stored in a block header of the first block, and the block syndrome value represents characteristics of all transactions in the first block;
an adding module 903, configured to add the first block to a block chain of the node apparatus when the first block is verified, and broadcast the first block in the block chain system.
Optionally, the similarity between any two transactions refers to the similarity between the characteristic values of the data representations of the any two transactions.
Optionally, the obtaining module 901 includes:
a selecting unit, configured to select a plurality of transactions from a transaction pool;
a dividing unit, configured to divide a head hash of a latest block on a block chain of the node device into a plurality of sub-head hashes;
an obtaining unit, configured to obtain a feature value of a data representation of each of the plurality of transactions based on the data representation of each of the plurality of transactions and the plurality of sub-header hashes;
the obtaining unit is further configured to obtain the multiple transactions when the similarity of the feature values represented by the data of any two transactions in the multiple transactions meets a preset condition.
Optionally, the obtaining unit is configured to:
splicing the data representation of one transaction and one sub-header hash to obtain splicing result data;
and carrying out Hash calculation on the splicing result data to obtain a characteristic value represented by the data of the transaction.
Optionally, the obtaining module is further configured to obtain the multiple transactions again in the transaction pool when the similarity between any two transactions in the multiple transactions does not meet the preset condition;
the generating module is further configured to generate a first block based on the multiple transactions when the similarity of any two of the multiple transactions obtained again meets the preset condition.
Optionally, the apparatus further comprises:
the obtaining module 901 is further configured to obtain metadata of the original data;
the conversion module is used for converting the data description in the metadata to obtain data representation;
a replacement module to replace a data description in the metadata with the data representation;
and the signature module is used for signing the replaced metadata to obtain the transaction and storing the transaction into the transaction pool.
According to the device provided by the embodiment of the invention, the plurality of transactions with higher similarity are screened from the transaction pool by the node equipment, and are packed and generated into the block, so that the transactions stored in the blocks on the block chain in the block chain system have similarity, the node equipment in the block chain system can conveniently inquire data according to the similarity of the transactions, and the inquiry efficiency can be greatly improved. In addition, the characteristic field in the block header of the block stores the characteristic value represented by the data of each transaction to characterize the block characteristic value of the block, and further, the characteristics of all transactions in the block can be known from the block characteristic value to perform transaction query.
Fig. 10 is a structural device diagram of a data query apparatus provided by an embodiment of the present invention, referring to fig. 10, applied to any node device in a blockchain system, the apparatus includes a receiving module 1001, a determining module 1002, an obtaining module 1003, and an outputting module 1004;
a receiving module 1001, configured to receive a transaction query request, where the transaction query request includes target index information;
a determining module 1002, configured to determine at least one target block from a block chain configured in the node device, where a similarity between a block feature value in a block header of each target block and a feature value of the target index information meets a first preset condition, and the block feature value represents features of all transactions in the target block;
an obtaining module 1003, configured to obtain at least one target transaction from the at least one target block, where a similarity between each target transaction and the target index information meets a second preset condition;
an output module 1004 for outputting the at least one target transaction as a query result.
Optionally, the determining module 1002 is configured to:
acquiring a characteristic value of the target index information;
obtaining a block eigenvalue within a block header of at least one block on the block chain;
and when the similarity between the block characteristic value in the block head of any block in the at least one block and the characteristic value of the target index information meets the first preset condition, acquiring the block as a target block.
Optionally, the output module is further configured to traverse at least one block chain configured on the node device, and when the height of the tail block of any block chain is zero, output the query result as null;
the device also includes:
and the execution module is used for executing the contents executed by the determination module and the acquisition module on the block chain when the height of the tail block of any block chain is not zero.
Optionally, the obtaining module 1003 is configured to:
obtaining the characteristic values of all the transactions in any target block of the at least one target block;
and when the characteristic value represented by the data of any transaction in the target block and the characteristic value of the target index information meet the second preset condition, acquiring the transaction as a target transaction.
According to the device provided by the embodiment of the invention, the node equipment compares the characteristic value of the target index information with the block characteristic value in the block head of the block on the block chain configured by the node equipment, so that at least one target block meeting the condition can be screened, the at least one target block can be further inquired, and a plurality of target transactions are obtained from the at least one target block, therefore, all blocks on the local block chain do not need to be inquired, the purpose of block jump inquiry is realized, the inquiry time is further shortened, and the inquiry efficiency is improved. And when the target transaction is screened from the target block, the characteristic value of the target index information can be directly compared with the characteristic value represented by the data of each transaction stored in the block head of the target block, so that the target transaction meeting the condition is screened out, the characteristic values represented by the data of all the transactions of the target block do not need to be calculated again, the calculation load of the block chain system is further reduced, the query time is further shortened, and the query efficiency is further improved.
All the above optional technical solutions may be combined arbitrarily to form the optional embodiments of the present disclosure, and are not described herein again.
It should be noted that: in the data query and storage device provided in the above embodiment, when querying or storing data, only the division of the functional modules is illustrated, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the data query and storage device provided by the above embodiment and the data query and storage method embodiment belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment and is not described herein again.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (18)

1. A data storage method applied to a user node device in a blockchain system, the method comprising:
converting a data description in metadata of the original data into a data representation;
generating a transaction based on a data representation in the metadata;
sending the transaction to the blockchain system;
receiving a first block, wherein a block characteristic value is stored in a block head of the first block, and the block characteristic value represents characteristics of all transactions in the first block;
when the block chain system passes the consensus of the first block, the first block is stored on the block chain configured by the user node equipment.
2. The method of claim 1, wherein generating a transaction based on the data representation in the metadata comprises:
replacing a data description in the metadata with the data representation;
and signing the replaced metadata to obtain the transaction.
3. A data storage method is applied to any node device in a block chain system, and the method comprises the following steps:
acquiring a plurality of transactions from a transaction pool, wherein the similarity between any two transactions in the plurality of transactions meets a preset condition;
generating a first block based on the plurality of transactions, wherein a block characteristic value is stored in a block head of the first block, and the block characteristic value represents characteristics of all transactions in the first block;
when the first block is verified, adding the first block to a block chain configured by the node equipment, and broadcasting the first block in the block chain system.
4. The method of claim 3, wherein the similarity between any two transactions is a similarity between characteristic values of data representations of any two transactions.
5. The method of claim 3, wherein obtaining the plurality of transactions from the transaction pool comprises:
selecting a plurality of transactions from a transaction pool;
dividing the head hash of the latest block on the block chain of the node equipment configuration into a plurality of sub-head hashes;
obtaining a characteristic value of the data representation of each transaction in the plurality of transactions based on the data representation of each transaction and the plurality of sub-header hashes;
and when the similarity of the characteristic values represented by the data of any two transactions in the multiple transactions meets a preset condition, acquiring the multiple transactions.
6. The method of claim 5, wherein obtaining the characteristic value of the data representation of each of the plurality of transactions based on the data representation of each of the plurality of transactions and the plurality of sub-header hashes comprises:
splicing the data representation of one transaction and one sub-header hash to obtain splicing result data;
and carrying out Hash calculation on the splicing result data to obtain a characteristic value represented by the data of the transaction.
7. The method of claim 3, further comprising:
when the similarity between any two transactions in the multiple transactions does not meet the preset condition, obtaining the multiple transactions in the transaction pool again;
and when the similarity of any two transactions in the plurality of retrieved transactions meets the preset condition, generating a first block based on the plurality of transactions.
8. The method of claim 3, further comprising:
acquiring metadata of original data;
converting the data description in the metadata to obtain data representation;
replacing a data description in the metadata with the data representation;
and signing the replaced metadata to obtain a transaction, and storing the transaction into the transaction pool.
9. A data storage apparatus, for use in a user node device in a block chain system, the apparatus comprising:
a conversion module for converting a data description in metadata of the raw data into a data representation;
a generating module for generating a transaction based on a data representation in the metadata;
a sending module, configured to send the transaction to the blockchain system;
a receiving module, configured to receive a first block, where a block characteristic value is stored in a block header of the first block, and the block characteristic value represents characteristics of all transactions in the first block;
a storage module, configured to store the first block on a block chain configured by the user node device when the block chain system passes the consensus on the first block.
10. The apparatus of claim 9, wherein the generating module is configured to replace a data description in the metadata with the data representation;
the device further comprises:
and the signature module is used for signing the replaced metadata to obtain the transaction.
11. A data storage apparatus, applied to any node device in a block chain system, the apparatus comprising:
the system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring a plurality of transactions from a transaction pool, and the similarity between any two transactions in the plurality of transactions meets a preset condition;
a generating module, configured to generate a first block based on the plurality of transactions, where a block syndrome value is stored in a block header of the first block, and the block syndrome value represents characteristics of all transactions in the first block;
and adding the first block to a block chain configured by the node equipment when the first block is verified, and broadcasting the first block in the block chain system.
12. The apparatus of claim 11, wherein the similarity between any two transactions refers to a similarity between eigenvalues of data representations of the any two transactions.
13. The apparatus of claim 11, wherein the obtaining module comprises:
a selecting unit, configured to select a plurality of transactions from a transaction pool;
a dividing unit configured to divide a head hash of a latest block on a blockchain of the node device arrangement into a plurality of sub-head hashes;
an obtaining unit, configured to obtain a feature value of a data representation of each of the plurality of transactions based on the data representation of each of the plurality of transactions and the plurality of sub-header hashes;
the obtaining unit is further configured to obtain the multiple transactions when the similarity of the feature values represented by the data of any two of the multiple transactions meets a preset condition.
14. The apparatus of claim 13, wherein the obtaining unit is configured to:
splicing the data representation of one transaction and one sub-header hash to obtain splicing result data;
and carrying out Hash calculation on the splicing result data to obtain a characteristic value represented by the data of the transaction.
15. The apparatus according to claim 11, wherein the obtaining module is further configured to obtain a plurality of transactions again in the transaction pool when the similarity between any two transactions in the plurality of transactions does not meet the preset condition;
the generating module is further configured to generate a first block based on the multiple transactions when the similarity of any two of the multiple transactions obtained again meets the preset condition.
16. The apparatus according to claim 11, wherein the obtaining module is further configured to obtain metadata of the original data;
the device further comprises:
the conversion module is used for converting the data description in the metadata to obtain data representation;
a replacement module to replace a data description in the metadata with the data representation;
and the signature module is used for signing the replaced metadata to obtain the transaction, and storing the transaction into the transaction pool.
17. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to perform operations performed by the data storage method of any one of claims 1 to 8.
18. A computer-readable storage medium having stored therein at least one instruction which is loaded and executed by a processor to perform operations performed by a data storage method according to any one of claims 1 to 8.
CN201811427813.3A 2018-11-27 2018-11-27 Data storage method and device, computer equipment and readable storage medium Active CN111224793B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201811427813.3A CN111224793B (en) 2018-11-27 2018-11-27 Data storage method and device, computer equipment and readable storage medium
PCT/CN2019/120688 WO2020108441A1 (en) 2018-11-27 2019-11-25 Data storage method, device, computer apparatus, and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811427813.3A CN111224793B (en) 2018-11-27 2018-11-27 Data storage method and device, computer equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN111224793A CN111224793A (en) 2020-06-02
CN111224793B true CN111224793B (en) 2021-06-01

Family

ID=70827429

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811427813.3A Active CN111224793B (en) 2018-11-27 2018-11-27 Data storage method and device, computer equipment and readable storage medium

Country Status (2)

Country Link
CN (1) CN111224793B (en)
WO (1) WO2020108441A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111858520B (en) * 2020-07-21 2024-03-22 杭州溪塔科技有限公司 Method and device for separately storing block chain node data
CN111917859B (en) * 2020-07-28 2022-08-12 腾讯科技（深圳）有限公司 Data transmission method and device, computer equipment and storage medium
CN112738213B (en) * 2020-12-25 2023-06-30 深圳Tcl新技术有限公司 Task demand response method, device and system based on block chain and storage medium
CN112905554A (en) * 2021-02-07 2021-06-04 全链通有限公司 Data sharing method and device based on block chain
CN113064898A (en) * 2021-04-06 2021-07-02 北京瑞卓喜投科技发展有限公司 Retrieval method and device based on miniature index of contract on chain and electronic equipment
CN113064901A (en) * 2021-04-06 2021-07-02 北京瑞卓喜投科技发展有限公司 Method and device for forming data micro-index in contract on chain and electronic equipment
CN112988667B (en) * 2021-05-12 2021-09-07 腾讯科技（深圳）有限公司 Data storage method and device based on block chain network
CN113836332A (en) * 2021-09-08 2021-12-24 北京灵汐科技有限公司 Storage device
CN113821536B (en) * 2021-11-23 2022-03-18 腾讯科技（深圳）有限公司 Data processing method, device and equipment based on block chain and readable storage medium
CN115208880B (en) * 2022-03-29 2023-11-10 深圳大学 Expected block generation method, expected block generation device, computer equipment and storage medium

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108848119A (en) * 2018-04-03 2018-11-20 阿里巴巴集团控股有限公司 The exchange method and device, system, electronic equipment of transregional piece of chain

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9710511B2 (en) * 2015-05-14 2017-07-18 Walleye Software, LLC Dynamic table index mapping
CN105740440B (en) * 2016-02-01 2019-03-26 上海凭安网络科技有限公司 A kind of indexing means for accelerating block chain query
US10277561B2 (en) * 2016-07-22 2019-04-30 International Business Machines Corporation Database management system shared ledger support
CN107239479B (en) * 2017-03-28 2020-03-13 创新先进技术有限公司 Block chain based data storage and query method and device
CN113766035B (en) * 2017-03-28 2023-05-23 创新先进技术有限公司 Service acceptance and consensus method and device
CN107203344A (en) * 2017-05-31 2017-09-26 郑州云海信息技术有限公司 A kind of date storage method and data-storage system
CN107291862A (en) * 2017-06-12 2017-10-24 腾讯科技（深圳）有限公司 Business datum storage method, device, storage medium and electronic equipment
CN110348242B (en) * 2017-06-12 2021-01-15 腾讯科技（深圳）有限公司 Service request processing method and device

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108848119A (en) * 2018-04-03 2018-11-20 阿里巴巴集团控股有限公司 The exchange method and device, system, electronic equipment of transregional piece of chain

Also Published As

Publication number Publication date
WO2020108441A1 (en) 2020-06-04
CN111224793A (en) 2020-06-02

Similar Documents

Publication Publication Date Title
CN111224793B (en) Data storage method and device, computer equipment and readable storage medium
CN106407311B (en) Method and device for obtaining search result
JP6028567B2 (en) Data storage program, data search program, data storage device, data search device, data storage method, and data search method
CN102782643B (en) Use the indexed search of Bloom filter
US8634947B1 (en) System and method for identifying digital files
WO2015096582A1 (en) Index creation method, querying method, apparatus and device for spatial-temporal data
US8126997B2 (en) Hot data management method based on hit counter
CN108197226A (en) MPTC account status tree and MPTC block chain method for quickly retrieving
US20120109990A1 (en) Information search system, information management device, information search method, information management method, and recording medium
US9811321B1 (en) Script compilation
KR20130020050A (en) Apparatus and method for managing bucket range of locality sensitivie hash
US11494403B2 (en) Method and apparatus for storing off-chain data
US8812492B2 (en) Automatic and dynamic design of cache groups
EP2742446A1 (en) A system and method to store video fingerprints on distributed nodes in cloud systems
CN106874348A (en) File is stored and the method for indexing means, device and reading file
CN104679830A (en) File processing method and device
WO2021174836A1 (en) Differential package generation method and apparatus, computer device, and storage medium
US20180210907A1 (en) Data management system, data management method, and computer program product
US20210191640A1 (en) Systems and methods for data segment processing
CN113779025B (en) Optimization method, system and application of classified data retrieval efficiency in block chain
JP2017139733A (en) Generation and authentication of packet in process chain
US10223529B2 (en) Indexing apparatus and method for search of security monitoring data
JP5983333B2 (en) Search processing method, data generation method, and information processing apparatus
CN111597379B (en) Audio searching method and device, computer equipment and computer-readable storage medium
CN114691610A (en) Directory processing method and device, storage medium and processor

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant