CN111262876A - Data processing method, device and equipment based on block chain and storage medium - Google Patents

Data processing method, device and equipment based on block chain and storage medium Download PDF

Info

Publication number
CN111262876A
CN111262876A CN202010074110.8A CN202010074110A CN111262876A CN 111262876 A CN111262876 A CN 111262876A CN 202010074110 A CN202010074110 A CN 202010074110A CN 111262876 A CN111262876 A CN 111262876A
Authority
CN
China
Prior art keywords
transaction data
block
transaction
compressed
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010074110.8A
Other languages
Chinese (zh)
Other versions
CN111262876B (en
Inventor
杨常青
周开班
时一防
朱耿良
刘攀
刘区城
王宗友
黄焕坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010074110.8A priority Critical patent/CN111262876B/en
Publication of CN111262876A publication Critical patent/CN111262876A/en
Application granted granted Critical
Publication of CN111262876B publication Critical patent/CN111262876B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L63/00—Network architectures or network communication protocols for network security
    • H04L63/12—Applying verification of the received information
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
    • H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
    • H03M7/40—Conversion to or from variable length codes, e.g. Shannon-Fano code, Huffman code, Morse code
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/04—Protocols for data compression, e.g. ROHC
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate

Abstract

The embodiment of the application discloses a data processing method, a device, equipment and a storage medium based on a block chain, wherein the method comprises the following steps: the transaction submitting node acquires target transaction data from the transaction pool and compresses the target transaction data to obtain compressed transaction data; the transaction submitting node broadcasts the compressed transaction data in a block chain; the transaction submitting node determines the transaction data hash of the target transaction data and packs the transaction data hash of the target transaction data into a block; and the transaction submitting node compresses the block to obtain a compressed block and sends the compressed block to a consensus node so that the consensus node verifies the block and adds the block to the block chain after the block passes the verification. By adopting the embodiment of the application, the transmission rate of the transaction data can be improved, the consumption of transmission resources is reduced, and the applicability is high.

Description

Data processing method, device and equipment based on block chain and storage medium
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, and storage medium based on a block chain.
Background
The blockchain is also called a distributed ledger technology, and is an internet database technology. The method is characterized by decentralization, openness and transparency, and each user can participate in maintaining database records. The blockchain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism and an encryption algorithm. As the application of the block chain technology in various industries is more and more extensive, the data volume of the transaction data submitted by the block chain nodes is also more and more large, so that the transaction data transmission rate is slower, and more transmission resources are occupied.
Therefore, how to increase the transmission rate of the transaction data in the blockchain becomes an urgent problem to be solved.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, equipment and a storage medium based on a block chain, which can improve the transaction data transmission rate and improve the transaction data processing efficiency.
In a first aspect, an embodiment of the present application provides a data processing method based on a block chain, where the method includes:
the transaction submitting node acquires target transaction data from the transaction pool and compresses the target transaction data to obtain compressed transaction data;
the transaction submitting node broadcasts the compressed transaction data in a block chain, so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
the transaction submitting node determines the transaction data hash of the target transaction data and packs the transaction data hash of the target transaction data into a block;
and the transaction submitting node compresses the block to obtain a compressed block and sends the compressed block to a consensus node so that the consensus node verifies the block and adds the block to the block chain after the block passes the verification.
With reference to the first aspect, in a possible implementation manner, the compressing the target transaction data to obtain compressed transaction data includes:
the transaction submitting node determines whether the current character to be encoded is matched with the encoded character in a preset sliding window, wherein the current character to be encoded is the first character to be encoded of the target transaction data;
if the characters to be coded are matched with each other, the transaction submitting node continues to search the longest matching character string after the first character to be coded and outputs a ternary symbol group (off, len, c), wherein the off represents the offset of the matching character string relative to the left boundary of a window of a sliding window, the len represents the length of the matching character string, and the c is the next character to be coded adjacent to the matching character string;
if not, the transaction submitting node outputs an off symbol set (len, d), wherein d represents the current character to be coded;
the transaction submitting node moves the preset sliding window backwards by len +1 characters, and continues to execute the step of determining whether the current character to be coded is matched with the coded character in the preset sliding window or not until a plurality of ternary symbol groups corresponding to the target transaction data are obtained;
and the transaction submitting node encodes the ternary symbol groups based on a Huffman encoding algorithm to obtain compressed transaction data.
With reference to the first aspect, in a possible implementation manner, the obtaining, by the transaction submitting node, the target transaction data from the transaction pool includes:
the transaction submitting node determines the transaction data quantity in the transaction pool at intervals of preset time;
when the quantity of the transaction data in the transaction pool exceeds a preset quantity threshold value, the transaction submitting node acquires the transaction data equal to the preset quantity threshold value from the transaction pool, and determines the transaction data equal to the preset quantity threshold value as target transaction data.
In a second aspect, an embodiment of the present application provides a data processing method based on a block chain, where the method includes:
the consensus node receives compressed transaction data sent by a transaction submitting node, and the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
the consensus node decompresses the compressed transaction data to obtain the target transaction data;
when a compressed block sent by the transaction submitting node is received, the consensus node decompresses the compressed block to obtain a block;
the common identification node determines whether transaction data hash of the target transaction data exists in the block;
and if the transaction data hash of the target transaction data exists in the block, the consensus node verifies the block and adds the block to a block chain after the verification is passed.
With reference to the second aspect, in a possible implementation manner, the compressed transaction data carries a compression manner identifier; the decompressing the compressed transaction data by the consensus node to obtain target transaction data comprises:
the consensus node determines a compression mode corresponding to the compressed transaction data according to the compression mode identifier;
and the transaction node decompresses the compressed transaction data according to the decompression mode corresponding to the compression mode to obtain the target transaction data.
With reference to the second aspect, in a possible implementation manner, the compressed transaction data carries data verification information; the method further comprises the following steps:
the consensus node verifies whether the target transaction data is complete transaction data or not according to the data verification information;
if the target transaction data is complete transaction data, the consensus node stores the target transaction data;
if the target transaction data is incomplete transaction data, the consensus node sends first retransmission information to the transaction submitting node so that the transaction submitting node sends first compressed transaction data.
With reference to the second aspect, in one possible implementation, the method further includes:
the consensus node determines whether the compressed transaction data carries a compression identifier, wherein the compression identifier is generated when the transaction submitting node compresses the target transaction data;
if the compressed transaction data is determined to carry the compression identification, the consensus node performs the step of decompressing the compression block to obtain a block, and if the compressed transaction data is determined not to carry the compression identification, the consensus node sends second retransmission information to the transaction submitting node so that the transaction submitting node sends second compressed transaction data.
In a third aspect, an embodiment of the present application provides a data processing apparatus based on a block chain, where the apparatus includes:
the first processing module is used for acquiring target transaction data from the transaction pool and compressing the target transaction data to obtain compressed transaction data;
a first sending module, configured to broadcast the compressed transaction data in a block chain, so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
the second processing module is used for determining the transaction data hash of the target transaction data and packaging the transaction data hash of the target transaction data into a block;
and the second sending module is used for compressing the block to obtain a compressed block and sending the compressed block to a common node so that the common node verifies the block and adds the block to the block chain after the block passes the verification.
With reference to the third aspect, in one possible implementation, the first processing module includes:
the first determining unit is used for determining whether the current character to be coded is matched with the coded character in a preset sliding window, wherein the current character to be coded is the first character to be coded of the target transaction data;
a first search unit, configured to continue searching for a longest matching character string after the first character to be encoded during matching, and output a ternary symbol group (off, len, c), where off represents an offset of the matching character string with respect to a left window boundary of a sliding window, len represents a length of the matching character string, and c is a next character to be encoded adjacent to the matching character string;
an output unit, for outputting ternary symbol group (off, len, d) when not matched, wherein d represents the above-mentioned current character to be coded;
a second searching unit, configured to move the preset sliding window backwards by len +1 characters, and continue to perform the step of determining whether the current character to be encoded matches an encoded character in the preset sliding window, until a plurality of ternary symbol groups corresponding to the target transaction data are obtained;
and the first processing unit is used for coding the ternary symbol groups based on a Huffman coding algorithm to obtain compressed transaction data.
With reference to the third aspect, in one possible implementation, the first processing module includes:
the second determining unit is used for determining the transaction data quantity in the transaction pool at intervals of preset time;
and the acquisition unit is used for acquiring the transaction data equal to the preset quantity threshold value from the transaction pool when the quantity of the transaction data in the transaction pool exceeds the preset quantity threshold value, and determining the transaction data equal to the preset quantity threshold value as the target transaction data.
In a fourth aspect, an embodiment of the present application provides a data processing apparatus based on a block chain, where the apparatus includes:
the receiving module is used for receiving compressed transaction data sent by a transaction submitting node, and the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
the third processing module is used for decompressing the compressed transaction data to obtain the target transaction data;
the fourth processing module is used for decompressing the compressed block to obtain a block when receiving the compressed block sent by the transaction submitting node;
a first determining module, configured to determine whether a transaction data hash of the target transaction data exists in the block;
and the fifth processing module is used for verifying the block if the transaction data hash of the target transaction data exists in the block, and adding the block to the block chain after the verification is passed.
With reference to the fourth aspect, in a possible implementation manner, the compressed transaction data carries a compression manner identifier; the third processing module includes:
a third determining unit, configured to determine, according to the compression mode identifier, a compression mode corresponding to the compressed transaction data;
and the second processing unit is used for decompressing the compressed transaction data according to a decompression mode corresponding to the compression mode to obtain the target transaction data.
With reference to the fourth aspect, in a possible implementation manner, the compressed transaction data carries data verification information; the above-mentioned device includes:
the verification module is also used for verifying whether the target transaction data is complete transaction data according to the data verification information;
the storage module is also used for storing the target transaction data if the target transaction data is complete transaction data;
the third sending module is further configured to send first retransmission information to the transaction submitting node if the target transaction data is incomplete transaction data, so that the transaction submitting node sends first compressed transaction data.
With reference to the fourth aspect, in a possible implementation manner, the apparatus further includes:
the second determining module is further configured to determine whether the compressed transaction data carries a compression identifier, where the compression identifier is generated when the transaction submitting node compresses the target transaction data;
the sixth processing module is further configured to perform the step of decompressing the compressed block to obtain a block if it is determined that the compressed transaction data carries the compression identifier, and send second retransmission information to the transaction submitting node to enable the transaction submitting node to send second compressed transaction data if it is determined that the compressed transaction data does not carry the compression identifier.
In a fifth aspect, an embodiment of the present application provides an apparatus, which includes a processor and a memory, where the processor and the memory are connected to each other. The memory is configured to store a computer program that supports the terminal device to execute the method provided by the first aspect and/or any one of the possible implementation manners of the first aspect, where the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method provided by the first aspect and/or any one of the possible implementation manners of the first aspect.
In a sixth aspect, an embodiment of the present application provides an apparatus, which includes a processor and a memory, where the processor and the memory are connected to each other. The memory is used for storing a computer program supporting the terminal device to execute the method provided by the second aspect and/or any one of the possible implementation manners of the second aspect, and the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method provided by the second aspect and/or any one of the possible implementation manners of the second aspect.
In a seventh aspect, the present application provides a computer-readable storage medium, where a computer program is stored, and the computer program is executed by a processor to implement the method provided in any one of the possible implementation manners of the first aspect and/or the second aspect.
In the embodiment of the application, the transaction submitting node can reduce the data volume of the transaction data by compressing and transmitting the transaction data, so that the transmission rate of the transaction data is improved. On the other hand, when the transaction submitting node packs the block, only the transaction data of the transaction data is subjected to hash packing to the block, so that the data capacity of the block can be reduced, and the transmission efficiency of the block can be further improved by compressing the block. Meanwhile, the consensus node verifies whether the transaction data hash in the block corresponds to the transaction data stored locally, the local transaction data can be directly stored and the block is added to the block chain under the condition of consistency, and the applicability is high.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic diagram illustrating a block chain based data processing method according to an embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a data processing method based on a blockchain according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a scenario for compressing target transaction data according to an embodiment of the present application;
FIG. 4 is a schematic diagram illustrating a scenario of decompressing compressed transaction data according to an embodiment of the present application;
FIG. 5 is a schematic diagram illustrating a scenario of an encoding format of compressed transaction data according to an embodiment of the present application;
FIG. 6 is a block chain-based data processing apparatus according to an embodiment of the present disclosure;
fig. 7 is another schematic structural diagram of a data processing apparatus based on a blockchain according to an embodiment of the present application;
FIG. 8 is a schematic structural diagram of an apparatus provided by an embodiment of the present application;
fig. 9 is another schematic structural diagram of the apparatus provided in the embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, fig. 1 is a schematic diagram illustrating a block chain-based data processing method according to an embodiment of the present disclosure. In fig. 1, node 10a, node 10b, node 10c, and node 10d are each part of nodes in a block chain 20, where any node may broadcast transaction data to other nodes. Taking fig. 1 as an example, node 10a is a transaction commit node in the blockchain 20, and node 10d is a consensus node in the blockchain 20. After the node 10a selects the target transaction data 40 from the corresponding transaction pool 30, the target transaction data 40 may be compressed to obtain compressed transaction data 50, and further the compressed transaction data 50 is broadcast over the network in the blockchain 20, so that other nodes in the blockchain 20 except the node 10a may determine that the target transaction data 40 (or the compressed transaction data 50) exists in the blockchain 20 in this time zone, and meanwhile, other nodes in the blockchain 20 except the node 10a (such as the node 10d) may decompress the compressed transaction data 50 to obtain the target transaction data 40 and store the target transaction data 40 locally. The node 10a broadcasts the compressed transaction data 50 over the entire network, so that the transmission resource consumption can be reduced, and the broadcasting efficiency can be improved.
Further, node 10a may determine the transaction data hash for target transaction data 40 after broadcasting compressed transaction data 50 over the network, and may then hash the transaction data to block 60 and broadcast the block over the network to enable node 10d to verify block 60. Optionally, the node 10a may compress the block 60 to obtain a compressed block 70, and then broadcast the compressed block 70 over the network to further reduce the consumption of transmission resources and improve the transmission efficiency. When node 10d stores target transaction data 40 and receives block 60 sent by node 10a (or receives compressed block 70 and decompresses compressed block 70 to result in block 60), node 10d may determine whether the transaction data hash for its locally stored target transaction data 40 is included in block 60. That is, the node 10d may determine the transaction data hash of the target transaction data 40 stored locally to compare with the transaction data hash stored in the block 60, and if the two are identical, it indicates that the target transaction data 40 is contained in the block 60, and at this time, the node 10d may perform validity verification on the block 60 and add the block 60 to the main chain 80 in the blockchain 20 after the verification is passed. In addition, anyone, any enterprise, can set up a server to join the blockchain network into one node. Therefore, a blockchain can be understood as a database that can process and share data in a blockchain composed of a plurality of sites, different geographic locations or a plurality of organizations.
Referring to fig. 2, fig. 2 is a schematic flowchart of a data processing method based on a blockchain according to an embodiment of the present application. The block chain-based data processing method shown in fig. 2 may include the following steps S201 to S207.
S201, target transaction data are obtained from the transaction pool, and the target transaction data are compressed to obtain compressed transaction data.
In some possible embodiments, the transaction submitting node may obtain any piece of transaction data from the transaction pool as the target transaction data, and compress the transaction issue. Optionally, the transaction submitting node may determine the transaction data amount in the transaction pool at preset time intervals, and when the transaction data amount in the transaction pool exceeds a preset amount threshold, the transaction node may obtain, as the target transaction data, the transaction data equal to the preset amount threshold from the transaction pool. For example, the transaction submitting node determines the amount of the transaction data in the transaction pool every 20 seconds, and assuming that the preset amount threshold is 20, the transaction submitting node may obtain 20 pieces of transaction data from the transaction pool as the target transaction data when the amount of the transaction data in the transaction pool exceeds 20 pieces.
Optionally, each block in the block chain has a certain data capacity limit, so that the transaction submitting node may determine a total data volume of all transaction data in the transaction pool at each preset time interval, and when the total data volume of all transaction data in the transaction pool exceeds a preset data volume threshold and is smaller than the block data capacity, the transaction submitting node may obtain all transaction data in the transaction pool and use the obtained transaction data as the target transaction data. The preset data amount threshold is any data amount threshold which is far greater than 0 and slightly smaller than the block data capacity, and may be determined according to an actual application scenario, which is not limited herein. For example, assuming that the block capacity is 2M, the preset data amount threshold may be determined to be 1.7M, and when the total data amount of the transaction data in the transaction pool exceeds 1.7M and does not exceed 2M yet, the transaction submitting node may acquire all the transaction data in the transaction pool as the target transaction data.
In some feasible embodiments, after the transaction submitting node acquires the target transaction data, in order to improve the transmission rate of the target transaction data and reduce the consumption of transmission resources, the transaction submitting node may compress the target transaction data to reduce the data volume of the target transaction data. The compression modes adopted when the transaction submitting node compresses the target transaction data include, but are not limited to, an LZW compression mode, a huffman coding mode, a run-length coding mode, an LZ77 compression mode, a data compression tool and the like, or the multiple compression modes can be combined, and the compression modes can be determined according to actual application scenarios, and are not limited herein.
Taking LZ77 compression as an example, when compressing target transaction data, the transaction submitting node may determine whether a current character to be encoded matches an encoded character within a preset sliding window, where the current character to be encoded is a first character to be encoded of the target transaction data. If the current character to be coded is matched with the coded character in the preset sliding window, the transaction submitting node continuously searches the longest matching character string in the forward buffer area after the first character to be coded and outputs a ternary symbol group (off, len, c), wherein off represents the offset of the matching character string relative to the left boundary of the window of the sliding window, len represents the length of the matching character string, and c is the next character to be coded adjacent to the matching character string. And if the current coding system does not match the coded characters in the preset sliding window, the transaction submitting node outputs a ternary symbol group (off, len, d), wherein d represents the current character to be coded. After outputting the ternary symbol groups (off, len, c) or (off, len, d), the transaction submitting node may move the preset sliding window backwards by len +1 characters, and continue to perform the step of determining whether the current character to be encoded matches the encoded character in the preset sliding window, until there is no character to be encoded in the target transaction data, thereby obtaining a plurality of ternary symbol groups corresponding to the target transaction data. At the moment, the transaction submitting node completes the compression of the target transaction data to obtain compressed transaction data. The size of the predetermined sliding window (e.g. 4KB) and the size of the forward buffer (e.g. 32B) may be determined according to an actual application scenario, which is not limited herein.
As shown in fig. 3, fig. 3 is a schematic view of a scenario for compressing target transaction data according to an embodiment of the present application. In fig. 3, assuming that the character string "abacbababcad" is the target transaction data, the unencoded characters are sequentially examined in the forward buffer starting with the first character to be encoded of the target transaction data. If the sliding window is preset in the first step for a string in the forward buffer "ABAB" for which no match is found, a ternary symbol set (0, 0, a) is output, where 0 may be omitted. I.e. a is the first character that does not match the preset sliding window. At this time, the preset sliding window is slid backwards by 1 character, and then the longest matching character string is continuously searched in the forward buffer area. If no matching character string is found in the forward buffer "BABC" in the preset sliding window shade in the second step, a ternary symbol group (0, 0, B) is output, wherein 0 can be omitted. Further, the preset sliding window is slid backward by 2 characters, at this time, "AB" is found at a position where the offset of the preset sliding window is 6, at this time, a ternary symbol group (6,2, C) may be output, where C is the next character of the matched character string "AB" in the forward buffer. By analogy, in the fifth step, "BC" can be found at the position with the preset sliding window offset of 2, and the ternary symbol group (2,2, a) is output. According to the above implementation, the target transaction data (string "abacbababcad") may be compressed into compressed transaction data.
Optionally, after compressing the target transaction data in an LZ77 compression manner, the transaction submitting node may encode the obtained multiple ternary symbol groups again based on a huffman coding algorithm to obtain compressed transaction data, so as to further reduce the data size of the target transaction data. It should be particularly noted that the above compression manner is only an example, and the compression manner of the target transaction data may be determined according to an actual application scenario, which is not limited herein.
S202, broadcasting the compressed transaction data in the block chain.
In some possible embodiments, the transaction submitting node may broadcast the compressed transaction data in the blockchain so that other nodes in the blockchain know that the blockchain is processing the target transaction data. For other transaction submitting nodes in the blockchain, after receiving the compressed transaction data, the other transaction submitting nodes can decompress the compressed transaction data to obtain target transaction data, and compare the target transaction data with the transaction data in the corresponding transaction pool. If the transaction pool of one transaction submitting node has the transaction data same as the target transaction data, the target transaction data in the transaction pool can be deleted by the transaction submitting node so as to prevent the target transaction data from being repeatedly processed by other transaction submitting nodes.
Optionally, the main purpose of compressing the target transaction data is to reduce transmission resource consumption and improve a transaction data transmission rate, but when the data volume of the target transaction data is small, compressing the target transaction data may prolong the transmission time of the target transaction data due to the compression time. Therefore, the transaction submitting node can directly broadcast the target transaction data with the data volume smaller than the compressed data volume threshold in the block chain, compress the target transaction data with the data volume larger than or equal to the pre-reduced data volume threshold, and broadcast the compressed transaction data obtained after compression in the block chain to reasonably utilize transmission resources and improve the transmission rate of the target transaction data to the maximum extent.
S203, decompressing the compressed transaction data to obtain target transaction data.
In some possible embodiments, when the consensus node receives the compressed transaction data sent by the transaction submitting node, since the transaction submitting node may also send uncompressed transaction data to the consensus node, the consensus node may determine whether the data sent by the transaction submitting node carries the compressed identifier. If the data bit sent by the transaction submitting node does not carry the compression identifier, the transaction submitting node may not successfully compress the target transaction data, and at this time, the consensus node may not successfully decompress the compressed transaction data. At this time, the consensus node may send retransmission information to the transaction submitting node to cause the transaction submitting node to re-compress the target transaction data and to re-send new compressed transaction data. The compression identifier may be any information for marking the compressed transaction data, such as a certain flag bit of a data packet of the compressed transaction data, or other information broadcasted along with the compressed transaction data, and may be specifically determined according to an actual application scenario, which is not limited herein.
In some feasible embodiments, when the consensus node decompresses the compressed transaction data, the consensus node may obtain a compression mode identifier carried by the compressed transaction data to determine, according to the compression mode identifier, a compression mode adopted by the service submitting node when compressing the target transaction data. The compression mode identifier may be any information for marking the compression mode, such as any flag bit in a data packet of compressed transaction data, or other information sent by the transaction submitting node along with the compressed transaction data, and may be specifically determined according to an actual application scenario, which is not limited herein. Further, the consensus node may decompress the compressed transaction data in a decompression manner corresponding to the compression manner adopted by the transaction submitting node, thereby obtaining the target transaction data. It should be particularly noted that, when decompressing the compressed transaction data, a decompression tool may also be used, which may be determined according to an actual application scenario, and is not limited herein.
As shown in fig. 4, fig. 4 is a schematic view of a scenario for decompressing compressed transaction data according to an embodiment of the present application. In fig. 4, assuming that the compressed transaction data obtained after the transaction committing node compresses the target transaction data (using the LZ77 compression method) is AB (6,2, C) (4,3, a) (2,2, a) D, the consensus node may decode the ternary symbol sets in the compressed transaction data in the reverse order of the LZ77 compression method. Decoding the ternary symbol groups a and B in turn results in the string "AB". Further, the character string "ABABC" can be obtained after decoding (6,2, C), the character string "ABABCABA" can be obtained after decoding (4,3, A), and so on until the final character string "ABABCABABCD" is obtained after decoding D. The final character string is the target transaction data after only decompressing the compressed transaction data.
In some possible embodiments, after the consensus node decompresses the compressed transaction data to obtain the target transaction data, the consensus node may verify whether the obtained target transaction data is complete transaction data according to data verification information carried by the compressed transaction data. The data check information may be a checksum in a data packet of compressed transaction data, or may be a transaction hash of the target transaction information, which may be determined specifically according to an actual application scenario, and is not limited herein. For example, when the data verification information is the transaction hash of the target transaction information, the consensus node may perform hash calculation on the target transaction data obtained through decompression to obtain the transaction hash of the target transaction data obtained through decompression, and further compare the transaction hash with the data verification information, if the two are consistent, it may be indicated that the compressed transaction data is not tampered in the broadcasting process, or data is not lost in the process of compressing the target transaction data by the transaction submitting node, and at this time, the consensus node may store the target transaction data obtained through decompression. If the target transaction data obtained after decompression is determined to be incomplete transaction data according to the data verification information, the consensus node can send retransmission information to the transaction submitting node so that the transaction submitting node can compress the target transaction data again and send new compressed transaction data.
In some possible embodiments, after the target transaction data is obtained by decompressing the compressed transaction data, the target transaction data may be verified respectively to determine whether the target transaction data is legal transaction data, and a verification result is generated after the verification passes. When the verification result of each consensus node meets a preset consensus strategy, the target transaction data can be determined to be legal transaction data, and at the moment, a confirmation message can be sent to the transaction submitting node so that the transaction submitting node can package the target transaction data into the block.
In some feasible embodiments, the compressed transaction data may simultaneously carry a compression identifier, a compression mode identifier, data verification information, and the like, and specifically, the compressed transaction data may be presented in a compression encoding format when the target transaction data is compressed. Referring to fig. 5, fig. 5 is a schematic view of a scenario of an encoding format of compressed transaction data according to an embodiment of the present application. In fig. 5, the first flag bit may be used to indicate a compression flag, the 2 nd to 4 th flag bits may be used to indicate a compression mode flag, the 5 th to 8 th flag bits may be used to indicate the amount of the target transaction data, the last 4 flag bits may be used to indicate data verification information of the compressed transaction data, and other positions may be used to fill the compressed transaction data of the target transaction data. The content represented by any flag bit, the compression identifier, the compression mode identifier, the number of target transaction data, the number of flag bits corresponding to the target transaction data and the data verification information may be determined according to an actual application scenario, and is not limited herein.
S204, determining the transaction data hash of the target transaction data, packaging the transaction data hash into a block, and compressing the block to obtain a compressed block.
In some possible embodiments, when the transaction submitting node packs the target transaction data into the block, in order to reduce the amount of the block data, the transaction submitting node may determine a transaction data hash of the target transaction data, and pack the transaction data hash into the block to send the block containing the transaction data hash to the consensus node. Optionally, in order to further increase the transmission rate, the transaction submitting node may further compress the block, and the specific compression manner refers to the implementation manner shown in step S201, which is not described herein again. Optionally, when the block contains transaction data with a large data amount, the transaction submitting node may send the block in segments so that each segment of block data may be quickly sent to the consensus node. The specific sending mode of the block containing the transaction data hash may be determined according to an actual application scenario, and is not limited herein.
S205, sending the compressed block.
S206, decompressing the compressed block to obtain a block, and determining that the block has the transaction data hash of the target transaction data.
In some possible embodiments, after receiving the segmented block data sent by the transaction submitting node, the consensus node may restore the segmented block data to the completed block according to the receiving order. Optionally, after receiving the compressed block sent by the transaction submitting node, the consensus node may decompress the compressed block to obtain a block including the transaction hash, where a specific implementation manner for decompressing the compressed block may refer to the implementation manner shown in step S203, and details thereof are not described here. Further, after the consensus node obtains the block containing the transaction data hash, since the block does not contain specific transaction data, the consensus node needs to determine whether the block contains the transaction hash of the target transaction data stored by the consensus node. That is, the consensus node needs to determine whether the transaction data hash contained in the block is the transaction data hash of the target transaction data stored by the consensus node. Specifically, the consensus node may perform hash calculation on the target transaction number stored therein to obtain a transaction data hash of the target transaction data, and compare the transaction data hash with the transaction data hash included in the block. If the two are consistent, it can be said that no data error occurs in the transaction submitting node in the block packaging process, or the transaction submitting node does not select the transaction hash of the erroneous transaction data for packaging, and at this time, the consensus node can determine that the transaction data hash of the target transaction data exists in the block. If the two are not consistent, the transaction submitting node sends a data error in the block packing process, or the transaction submitting node selects the transaction hash of the erroneous transaction data to pack, and at the moment, the consensus node can send retransmission information to the transaction submitting node so that the transaction submitting node packs the transaction hash of the target transaction data again and sends a new block.
And S207, verifying the block, and adding the block to the block chain after the block passes the verification.
In some possible embodiments, the consensus node may verify the validity of the block to determine whether to add the block to the blockchain. Specifically, the consensus node may determine a merkel root according to the transaction hash of the target transaction data, compare the merkel root with a merkel root in a block header of the block, and determine that the block is a legal block if the merkel root and the merkel root are consistent. Optionally, the consensus node verifies the target transaction data to determine whether the target transaction data generates double flowers, the target transaction data is updated, or an account status of an account corresponding to the target transaction data is updated. If the target transaction data does not generate double flowers, the target transaction data is not updated, and the account state of the account corresponding to the target transaction data is not updated, the block can be determined to be a legal block. Optionally, the consensus node may verify whether the difficulty value of the block is correct, and if the difficulty value of the block is correct, the block may be determined to be a legal block. Optionally, the consensus node may obtain the intelligent contract in the block and re-execute the intelligent contract, to determine whether the block matches the target transaction data according to the execution result, and if the execution result matches the target transaction data, the block may be determined to be a legal block.
It should be particularly noted that the above-mentioned multiple verification methods may be performed simultaneously or only any one or more of them may be performed, for example, after the block passes all the verification, the consensus node may determine that the block is a legal block, and the specific verification method may be determined according to an actual application scenario, which is not limited herein.
In some possible embodiments, before the consensus node adds the block to the blockchain, the consensus node further performs consensus verification on the block, and adds the block to the blockchain when the consensus result satisfies a predetermined consensus policy. The preset consensus policy includes, but is not limited to, a workload certification mechanism (Proof of Work, PoW), a rights and interests certification mechanism (PoS), a rights and rights certification mechanism (freed Proof of stamp, DPoS) Practical Byzantine mechanism (PBFT), a Ripple consensus algorithm, and the like, and may be specifically determined based on an actual application scenario, which is not limited herein.
In the embodiment of the application, the transaction submitting node can reduce the data volume of the transaction data by compressing and transmitting the transaction data, so that the transmission rate of the transaction data is improved. On the other hand, when the transaction submitting node packs the block, only the transaction data of the transaction data is subjected to hash packing to the block, so that the data capacity of the block can be reduced, and the transmission efficiency of the block can be further improved by compressing the block. Meanwhile, the consensus node verifies whether the transaction data hash in the block corresponds to the transaction data stored locally, the local transaction data can be directly stored and the block is added to the block chain under the condition of consistency, and the applicability is high.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a data processing apparatus based on a block chain according to an embodiment of the present application. The device 1 provided by the embodiment of the application comprises:
the first processing module 11 is configured to obtain target transaction data from a transaction pool, and compress the target transaction data to obtain compressed transaction data;
a first sending module 12, configured to broadcast the compressed transaction data in a block chain, so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
a second processing module 13, configured to determine a transaction data hash of the target transaction data, and pack the transaction data hash of the target transaction data into a block;
a second sending module 14, configured to compress the block to obtain a compressed block, and send the compressed block to a common node, so that the common node verifies the block and adds the block to the block chain after the block passes verification.
In some possible embodiments, the first processing module 11 includes:
a first determining unit 111, configured to determine whether a current character to be encoded matches an encoded character in a preset sliding window, where the current character to be encoded is a first character to be encoded in the target transaction data;
a first search unit 112, configured to continue searching for a longest matching character string after the first character to be encoded during matching, and output a ternary symbol group (off, len, c), where off represents an offset of the matching character string with respect to a left window boundary of a sliding window, len represents a length of the matching character string, and c is a next character to be encoded adjacent to the matching character string;
an output unit 113, configured to output a ternary symbol set (off, len, d) when the characters are not matched, where d represents the current character to be encoded;
a second searching unit 114, configured to move the preset sliding window backwards by len +1 characters, and continue to perform the step of determining whether the current character to be encoded matches an encoded character in the preset sliding window, until multiple ternary symbol groups corresponding to the target transaction data are obtained;
the first processing unit 115 is configured to encode the multiple ternary symbol sets based on a huffman coding algorithm to obtain compressed transaction data.
In some possible embodiments, the first processing module 11 includes:
a second determining unit 116, configured to determine the transaction data amount in the transaction pool at preset time intervals;
an obtaining unit 117, configured to, when the amount of the transaction data in the transaction pool exceeds a preset amount threshold, obtain transaction data equal to the preset amount threshold from the transaction pool, and determine the transaction data equal to the preset amount threshold as target transaction data.
In a specific implementation, the apparatus 1 may execute the implementation manners provided in the steps in fig. 2 through the built-in functional modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.
Referring to fig. 7, fig. 7 is a schematic structural diagram of a data processing apparatus based on a blockchain according to an embodiment of the present application. The device 2 provided by the embodiment of the application comprises:
the receiving module 20 is configured to receive compressed transaction data sent by a transaction submitting node, where the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
a third processing module 21, configured to decompress the compressed transaction data to obtain the target transaction data;
a fourth processing module 22, configured to decompress the compressed block sent by the transaction submitting node to obtain a block;
a first determining module 23, configured to determine whether a transaction data hash of the target transaction data exists in the block;
a fifth processing module 24, configured to verify the block if the transaction data hash of the target transaction data exists in the block, and add the block to the block chain after the verification passes.
In some possible embodiments, the compressed transaction data carries a compression mode identifier; the third processing module 21 includes:
a third determining unit 211, configured to determine, according to the compression method identifier, a compression method corresponding to the compressed transaction data;
the second processing unit 212 is configured to decompress the compressed transaction data according to a decompression method corresponding to the compression method to obtain the target transaction data.
In some possible embodiments, the compressed transaction data carries data verification information; the above-mentioned device 2 comprises:
the verification module 25 is further configured to verify whether the target transaction data is complete transaction data according to the data verification information;
the storage module 26 is further configured to store the target transaction data if the target transaction data is complete transaction data;
the third sending module 27 is further configured to send first retransmission information to the transaction submitting node if the target transaction data is incomplete transaction data, so that the transaction submitting node sends the first compressed transaction data.
In some possible embodiments, the apparatus 2 further comprises:
a second determining module 28, configured to determine whether the compressed transaction data carries a compression identifier, where the compression identifier is generated when the transaction submitting node compresses the target transaction data;
the sixth processing module 29 is further configured to perform the step of decompressing the compressed block to obtain a block if it is determined that the compressed transaction data carries the compression identifier, and send second retransmission information to the transaction submitting node to enable the transaction submitting node to send second compressed transaction data if it is determined that the compressed transaction data does not carry the compression identifier.
In a specific implementation, the apparatus 2 may execute the implementation manners provided in the steps in fig. 2 through the built-in functional modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.
In the embodiment of the application, the transaction submitting node can reduce the data volume of the transaction data by compressing and transmitting the transaction data, so that the transmission rate of the transaction data is improved. On the other hand, when the transaction submitting node packs the block, only the transaction data of the transaction data is subjected to hash packing to the block, so that the data capacity of the block can be reduced, and the transmission efficiency of the block can be further improved by compressing the block. Meanwhile, the consensus node verifies whether the transaction data hash in the block corresponds to the transaction data stored locally, the local transaction data can be directly stored and the block is added to the block chain under the condition of consistency, and the applicability is high.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an apparatus provided in an embodiment of the present application. As shown in fig. 8, the apparatus 1000 in the present embodiment may include: the processor 1001, the network interface 1004, and the memory 1005, and the apparatus 1000 may further include: a user interface 1003, and at least one communication bus 1002. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a standard wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1004 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 8, a memory 1005, which is a kind of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.
In the device 1000 shown in fig. 8, the network interface 1004 may provide network communication functions; the user interface 1003 is an interface for providing a user with input; and the processor 1001 may be used to invoke a device control application stored in the memory 1005 to implement:
acquiring target transaction data from a transaction pool, and compressing the target transaction data to obtain compressed transaction data;
broadcasting the compressed transaction data in a block chain so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
determining the transaction data hash of the target transaction data, and packaging the transaction data hash of the target transaction data into a block;
and compressing the block to obtain a compressed block, and sending the compressed block to a common node so that the common node verifies the block and adds the block to the block chain after the block passes the verification.
In some possible embodiments, the processor 1001 is configured to:
determining whether the current character to be coded is matched with a coded character in a preset sliding window, wherein the current character to be coded is a first character to be coded of the target transaction data;
if the characters to be coded are matched, continuously searching the longest matching character string after the first character to be coded, and outputting a ternary symbol group (off, len, c), wherein the off represents the offset of the matching character string relative to the left boundary of a window of a sliding window, the len represents the length of the matching character string, and the c is the next character to be coded adjacent to the matching character string;
if not, outputting a ternary symbol group (off, len, d), wherein d represents the current character to be coded;
moving the preset sliding window back by len +1 characters, and continuing to execute the step of determining whether the current character to be coded is matched with the coded character in the preset sliding window until a plurality of ternary symbol groups corresponding to the target transaction data are obtained;
and coding the plurality of ternary symbol groups based on a Huffman coding algorithm to obtain compressed transaction data.
In some possible embodiments, the processor 1001 is configured to:
determining the transaction data quantity in the transaction pool at preset time intervals;
and when the quantity of the transaction data in the transaction pool exceeds a preset quantity threshold value, acquiring the transaction data equal to the preset quantity threshold value from the transaction pool, and determining the transaction data equal to the preset quantity threshold value as target transaction data.
It should be understood that in some possible embodiments, the processor 1001 may be a Central Processing Unit (CPU), and the processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The portion of memory may also include non-volatile random access memory. For example, the memory may also store device type information.
In a specific implementation, the device 1000 may execute the implementation manners provided in the steps in fig. 2 through the built-in functional modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.
Referring to fig. 9, fig. 9 is another schematic structural diagram of the apparatus provided in the embodiment of the present application. As shown in fig. 9, the apparatus 1100 in the present embodiment may include: the processor 1101, the network interface 1104 and the memory 1105, the apparatus 1100 may further include: a user interface 1103, and at least one communication bus 1102. Wherein a communication bus 1102 is used to enable connective communication between these components. The user interface 1103 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 1103 may also include a standard wired interface and a standard wireless interface. The network interface 1104 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1104 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). The memory 1105 may alternatively be at least one storage device located remotely from the processor 1101. As shown in fig. 9, the memory 1105, which is a type of computer-readable storage medium, may include therein an operating system, a network communication module, a user interface module, and a device control application program.
In the device 1100 shown in fig. 9, the network interface 1104 may provide network communication functions; while user interface 1103 is primarily used to provide an interface for user input; and the processor 1101 may be configured to invoke a device control application stored in the memory 1105 to implement:
receiving compressed transaction data sent by a transaction submitting node, wherein the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
decompressing the compressed transaction data to obtain the target transaction data;
when a compressed block sent by the transaction submitting node is received, decompressing the compressed block to obtain a block;
determining whether a transaction data hash of the target transaction data exists in the block;
and if the transaction data hash of the target transaction data exists in the block, verifying the block, and adding the block to a block chain after the verification is passed.
In some possible embodiments, the compressed transaction data carries a compression mode identifier; the processor 1101 is configured to:
determining a compression mode corresponding to the compressed transaction data according to the compression mode identifier;
and decompressing the compressed transaction data according to the decompression mode corresponding to the compression mode to obtain the target transaction data.
In some possible embodiments, the compressed transaction data carries data verification information; the processor 1101 is further configured to:
verifying whether the target transaction data is complete transaction data according to the data verification information;
if the target transaction data is complete transaction data, storing the target transaction data;
and if the target transaction data is incomplete transaction data, sending first retransmission information to the transaction submitting node so that the transaction submitting node sends first compressed transaction data.
In some possible embodiments, the processor 1101 is further configured to:
determining whether the compressed transaction data carries a compression identifier, wherein the compression identifier is generated when the transaction submitting node compresses the target transaction data;
if the compressed transaction data is determined to carry the compression identification, the step of decompressing the compressed block to obtain a block is executed, and if the compressed transaction data is determined not to carry the compression identification, second retransmission information is sent to the transaction submitting node so that the transaction submitting node sends second compressed transaction data.
It should be appreciated that in some possible implementations, the processor 1101 may be a Central Processing Unit (CPU), and the processor may be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The memory may include both read-only memory and random access memory, and provides instructions and data to the processor. The portion of memory may also include non-volatile random access memory. For example, the memory may also store device type information.
In a specific implementation, the device 1100 may execute the implementation manners provided in the steps in fig. 2 through the built-in functional modules, which may specifically refer to the implementation manners provided in the steps, and are not described herein again.
In the embodiment of the application, the transaction submitting node can reduce the data volume of the transaction data by compressing and transmitting the transaction data, so that the transmission rate of the transaction data is improved. On the other hand, when the transaction submitting node packs the block, only the transaction data of the transaction data is subjected to hash packing to the block, so that the data capacity of the block can be reduced, and the transmission efficiency of the block can be further improved by compressing the block. Meanwhile, the consensus node verifies whether the transaction data hash in the block corresponds to the transaction data stored locally, the local transaction data can be directly stored and the block is added to the block chain under the condition of consistency, and the applicability is high.
An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and is executed by a processor to implement the method provided in each step in fig. 2, which may specifically refer to the implementation manner provided in each step, and is not described herein again.
The computer readable storage medium may be an internal storage unit of the task processing device provided in any of the foregoing embodiments, for example, a hard disk or a memory of an electronic device. The computer readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Memory Card (SMC), a Secure Digital (SD) card, a flash card (flash card), and the like, which are provided on the electronic device. The computer readable storage medium may further include a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), and the like. Further, the computer readable storage medium may also include both an internal storage unit and an external storage device of the electronic device. The computer-readable storage medium is used for storing the computer program and other programs and data required by the electronic device. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
The terms "first", "second", and the like in the claims and in the description and drawings of the present application are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus. Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.

Claims (10)

1. A method for processing data based on a blockchain, the method comprising:
the transaction submitting node acquires target transaction data from a transaction pool and compresses the target transaction data to obtain compressed transaction data;
the transaction submitting node broadcasts the compressed transaction data in a block chain, so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
the transaction submitting node determines transaction data hash of the target transaction data and packs the transaction data hash of the target transaction data into a block;
and the transaction submitting node compresses the block to obtain a compressed block, and sends the compressed block to a consensus node, so that the consensus node verifies the block and adds the block to the block chain after the block passes the verification.
2. The method of claim 1, wherein compressing the target transaction data into compressed transaction data comprises:
the transaction submitting node determines whether the current character to be encoded is matched with the encoded character in a preset sliding window, wherein the current character to be encoded is the first character to be encoded of the target transaction data;
if the characters to be coded are matched with each other, the transaction submitting node continues to search the longest matching character string after the first character to be coded and outputs a ternary symbol group (off, len, c), wherein the off represents the offset of the matching character string relative to the left boundary of a window of a sliding window, the len represents the length of the matching character string, and the c is the next character to be coded adjacent to the matching character string;
if not, the transaction submitting node outputs a ternary symbol group (off, len, d), wherein d represents the current character to be coded;
the transaction submitting node moves the preset sliding window backwards by len +1 characters, and continues to execute the step of determining whether the current character to be coded is matched with the coded character in the preset sliding window or not until a plurality of ternary symbol groups corresponding to the target transaction data are obtained;
and the transaction submitting node encodes the ternary symbol groups based on a Huffman encoding algorithm to obtain compressed transaction data.
3. The method of claim 1 or 2, wherein the transaction submission node obtaining target transaction data from a transaction pool comprises:
the transaction submitting node determines the transaction data quantity in the transaction pool at intervals of preset time;
when the quantity of the transaction data in the transaction pool exceeds a preset quantity threshold value, the transaction submitting node acquires the transaction data equal to the preset quantity threshold value from the transaction pool, and determines the transaction data equal to the preset quantity threshold value as target transaction data.
4. A method for processing data based on a blockchain, the method comprising:
the consensus node receives compressed transaction data sent by a transaction submitting node, and the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
the consensus node decompresses the compressed transaction data to obtain the target transaction data;
when a compressed block sent by the transaction submitting node is received, the consensus node decompresses the compressed block to obtain a block;
the consensus node determines whether a transaction data hash of the target transaction data exists in the block;
if the transaction data hash of the target transaction data exists in the block, the consensus node verifies the block, and the block is added to a block chain after the verification is passed.
5. The method of claim 4, wherein the compressed transaction data carries a compression mode identification; the decompressing the compressed transaction data by the consensus node to obtain target transaction data comprises:
the consensus node determines a compression mode corresponding to the compressed transaction data according to the compression mode identification;
and the transaction node decompresses the compressed transaction data according to the decompression mode corresponding to the compression mode to obtain the target transaction data.
6. The method of claim 4 or 5, wherein the compressed transaction data carries data verification information; the method further comprises the following steps:
the consensus node verifies whether the target transaction data is complete transaction data or not according to the data verification information;
if the target transaction data is complete transaction data, the consensus node stores the target transaction data;
and if the target transaction data is incomplete transaction data, the consensus node sends first retransmission information to the transaction submitting node so that the transaction submitting node sends first compressed transaction data.
7. The method of claim 4, further comprising:
the consensus node determines whether the compressed transaction data carries a compression identifier, wherein the compression identifier is generated when the transaction submitting node compresses the target transaction data;
if the compressed transaction data is determined to carry the compression identification, the consensus node performs the step of decompressing the compression block to obtain a block, and if the compressed transaction data is determined not to carry the compression identification, the consensus node sends second retransmission information to the transaction submitting node so that the transaction submitting node sends second compressed transaction data.
8. An apparatus for data processing based on a blockchain, the apparatus comprising:
the first processing module is used for acquiring target transaction data from a transaction pool and compressing the target transaction data to obtain compressed transaction data;
the first sending module is used for broadcasting the compressed transaction data in a block chain so that other nodes in the block chain decompress the compressed transaction data to obtain the target transaction data;
the second processing module is used for determining the transaction data hash of the target transaction data and packaging the transaction data hash of the target transaction data into a block;
and the second sending module is used for compressing the block to obtain a compressed block and sending the compressed block to a consensus node so that the consensus node verifies the block and adds the block to the block chain after the block passes the verification.
9. An apparatus for data processing based on a blockchain, the apparatus comprising:
the receiving module is used for receiving compressed transaction data sent by a transaction submitting node, and the compressed transaction data is obtained by compressing target transaction data by the transaction submitting node;
the third processing module is used for decompressing the compressed transaction data to obtain the target transaction data;
the fourth processing module is used for decompressing the compressed block to obtain a block when the compressed block sent by the transaction submitting node is received;
a first determination module to determine whether a transaction data hash of the target transaction data exists in the chunk;
and the fifth processing module is used for verifying the block if the transaction data hash of the target transaction data exists in the block, and adding the block to the block chain after the block passes the verification.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method of any one of claims 1 to 7.
CN202010074110.8A 2020-01-22 2020-01-22 Data processing method, device and equipment based on block chain and storage medium Active CN111262876B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010074110.8A CN111262876B (en) 2020-01-22 2020-01-22 Data processing method, device and equipment based on block chain and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010074110.8A CN111262876B (en) 2020-01-22 2020-01-22 Data processing method, device and equipment based on block chain and storage medium

Publications (2)

Publication Number Publication Date
CN111262876A true CN111262876A (en) 2020-06-09
CN111262876B CN111262876B (en) 2022-05-27

Family

ID=70947126

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010074110.8A Active CN111262876B (en) 2020-01-22 2020-01-22 Data processing method, device and equipment based on block chain and storage medium

Country Status (1)

Country Link
CN (1) CN111262876B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111865326A (en) * 2020-07-14 2020-10-30 北京灵汐科技有限公司 Data compression method, device, equipment and storage medium
WO2020259400A1 (en) * 2019-06-25 2020-12-30 比亚迪股份有限公司 Encoding method and apparatus, storage medium, and computer device
CN112163855A (en) * 2020-09-30 2021-01-01 深圳前海微众银行股份有限公司 Data processing method, device, system and storage medium
CN112988667A (en) * 2021-05-12 2021-06-18 腾讯科技（深圳）有限公司 Data storage method and device based on block chain network
CN113242044A (en) * 2021-06-02 2021-08-10 湖北央中巨石信息技术有限公司 Block chain data storage compression method for reducing memory occupation
CN116723239A (en) * 2023-08-09 2023-09-08 腾讯科技（深圳）有限公司 Block chain data transmission method and device, electronic equipment and readable medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8156241B1 (en) * 2007-05-17 2012-04-10 Netapp, Inc. System and method for compressing data transferred over a network for storage purposes
CN107094155A (en) * 2017-06-14 2017-08-25 广东工业大学 A kind of secure storage method of data and device based on alliance's block chain
CN108415668A (en) * 2018-02-06 2018-08-17 珠海市杰理科技股份有限公司 Chip motivational techniques, device, system, computer equipment and storage medium
CN108959416A (en) * 2018-06-08 2018-12-07 浙江数秦科技有限公司 A kind of web data automatic evidence-collecting based on block chain and deposit card method
CN109617964A (en) * 2018-12-12 2019-04-12 成都四方伟业软件股份有限公司 Big data storage method and device based on block chain
CN110163755A (en) * 2019-04-30 2019-08-23 阿里巴巴集团控股有限公司 Data compression, querying method and device and electronic equipment based on block chain
CN110288477A (en) * 2019-06-26 2019-09-27 深圳市元征科技股份有限公司 A kind of block chain transaction data processing method and relevant device
CN110597839A (en) * 2019-09-20 2019-12-20 腾讯科技（深圳）有限公司 Transaction data processing method, device, equipment and storage medium

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8156241B1 (en) * 2007-05-17 2012-04-10 Netapp, Inc. System and method for compressing data transferred over a network for storage purposes
CN107094155A (en) * 2017-06-14 2017-08-25 广东工业大学 A kind of secure storage method of data and device based on alliance's block chain
CN108415668A (en) * 2018-02-06 2018-08-17 珠海市杰理科技股份有限公司 Chip motivational techniques, device, system, computer equipment and storage medium
CN108959416A (en) * 2018-06-08 2018-12-07 浙江数秦科技有限公司 A kind of web data automatic evidence-collecting based on block chain and deposit card method
CN109617964A (en) * 2018-12-12 2019-04-12 成都四方伟业软件股份有限公司 Big data storage method and device based on block chain
CN110163755A (en) * 2019-04-30 2019-08-23 阿里巴巴集团控股有限公司 Data compression, querying method and device and electronic equipment based on block chain
CN110288477A (en) * 2019-06-26 2019-09-27 深圳市元征科技股份有限公司 A kind of block chain transaction data processing method and relevant device
CN110597839A (en) * 2019-09-20 2019-12-20 腾讯科技（深圳）有限公司 Transaction data processing method, device, equipment and storage medium

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020259400A1 (en) * 2019-06-25 2020-12-30 比亚迪股份有限公司 Encoding method and apparatus, storage medium, and computer device
US11750211B2 (en) 2019-06-25 2023-09-05 Byd Company Limited Encoding method and apparatus, storage medium, and computer device
CN111865326A (en) * 2020-07-14 2020-10-30 北京灵汐科技有限公司 Data compression method, device, equipment and storage medium
CN112163855A (en) * 2020-09-30 2021-01-01 深圳前海微众银行股份有限公司 Data processing method, device, system and storage medium
CN112988667A (en) * 2021-05-12 2021-06-18 腾讯科技（深圳）有限公司 Data storage method and device based on block chain network
CN113242044A (en) * 2021-06-02 2021-08-10 湖北央中巨石信息技术有限公司 Block chain data storage compression method for reducing memory occupation
CN116723239A (en) * 2023-08-09 2023-09-08 腾讯科技（深圳）有限公司 Block chain data transmission method and device, electronic equipment and readable medium
CN116723239B (en) * 2023-08-09 2023-10-31 腾讯科技（深圳）有限公司 Block chain data transmission method and device, electronic equipment and readable medium

Also Published As

Publication number Publication date
CN111262876B (en) 2022-05-27

Similar Documents

Publication Publication Date Title
CN111262876B (en) Data processing method, device and equipment based on block chain and storage medium
CN107395209B (en) Data compression method, data decompression method and equipment thereof
CN110442377B (en) Patch package generation method, application updating method, device and electronic equipment
CN112165331A (en) Data compression method and device, data decompression method and device, storage medium and electronic equipment
CN104486434A (en) Mobile terminal and file upload and download methods of mobile terminal
US20210089492A1 (en) Rdma data sending and receiving methods, electronic device, and readable storage medium
RU2008122944A (en) METHODS AND DEVICE FOR FRAGMENTATION OF SYSTEM INFORMATION MESSAGES IN RADIO NETWORKS
US11955992B2 (en) Rate matching method and apparatus for polar code
CN112399479B (en) Method, electronic device and storage medium for data transmission
EP3584971A1 (en) Encoding method, decoding method, apparatus and device
CN112839003A (en) Data verification method and system
US9350592B2 (en) Decompressing method and apparatus
US10565182B2 (en) Hardware LZMA compressor
CN112995199B (en) Data encoding and decoding method, device, transmission system, terminal equipment and storage medium
CN113630125A (en) Data compression method, data encoding method, data decompression method, data encoding device, data decompression device, electronic equipment and storage medium
CN115409507A (en) Block processing method, block processing device, computer equipment and storage medium
CN113055455A (en) File uploading method and equipment
CN106293542B (en) Method and device for decompressing file
CN114337678A (en) Data compression method, device, equipment and storage medium
CN111865557B (en) Verification code generation method and device
CN114095037B (en) Application program updating method, updating data compression method, device and equipment
CN113162628B (en) Data encoding method, data decoding method, terminal and storage medium
CN113014551B (en) Data decompression method, data transmission method based on data decompression method, computer device and readable storage medium
WO2021036189A1 (en) Rdma data sending and receiving methods, electronic device and readable storage medium
WO2020259704A1 (en) Data compression and data decompression methods for electronic device, and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40024693

Country of ref document: HK

SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant