CN107786642B - Block chain construction method and device for data circulation, storage medium and server - Google Patents
Block chain construction method and device for data circulation, storage medium and server Download PDFInfo
- Publication number
- CN107786642B CN107786642B CN201710922682.5A CN201710922682A CN107786642B CN 107786642 B CN107786642 B CN 107786642B CN 201710922682 A CN201710922682 A CN 201710922682A CN 107786642 B CN107786642 B CN 107786642B
- Authority
- CN
- China
- Prior art keywords
- data
- summarized
- node
- summarizing
- summarized data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q40/00—Finance; Insurance; Tax strategies; Processing of corporate or income taxes
- G06Q40/04—Trading; Exchange, e.g. stocks, commodities, derivatives or currency exchange
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L9/00—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
- H04L9/32—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
- H04L9/3247—Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
Abstract
A block chain construction method and device, a storage medium and a server for data circulation are provided, wherein the block chain construction method comprises the following steps: receiving account book information in a data circulation process; dividing nodes in a block chain to create partitions; merging and summarizing the account book information to obtain summarized data; and writing the summarized data into a partition corresponding to the summarized data in the block chain, wherein the partition corresponding to the summarized data comprises a supplier node, an demander node, a trading center node and a clearing center node which are associated with the summarized data. The technical scheme provided by the invention can provide a block chain which is used for entering a chain quickly and saving network overhead and storage space in the field of data circulation so as to meet the application requirements of high concurrency and real-time mass data distribution.
Description
Technical Field
The invention relates to the field of data circulation, in particular to a block chain construction method and device for data circulation, a storage medium and a server.
Background
In the field of data circulation, based on the particularity of data assets, mutual trust problems of data providers and data demanders in the data transaction process widely exist, for example, mutual trust problems may exist between data suppliers and data demanders in the process of recording and storing the ledger information. In order to solve the mutual trust problem in the data circulation process, a block chain technology with the characteristics of high safety, non-repudiation, non-deletability and the like can be adopted. However, if the conventional blockchain technique is directly introduced into the data circulation field, the conventional blockchain technique has the following problems during the real-time data distribution process:
(1) in the real-time data distribution process, mass data transaction information exists, and if a traditional block chain is adopted and the time consumption of a consensus mechanism is too long, the requirement on the link entering speed of data cannot be met, and the application requirement on the real-time data distribution cannot be met;
(2) the traditional block chain technology needs to broadcast data packets in the data transaction process to realize data consensus, which greatly increases network overhead;
(3) data of the traditional block chain technology is stored on each block chain node in the network through broadcasting, so that the data volume is huge and the storage space consumption is large.
Therefore, in the field of data circulation, a technical scheme of a block chain which is capable of entering a chain quickly and saving network overhead and storage space is urgently needed to meet the application requirement of high concurrency and real-time distribution of mass data.
Disclosure of Invention
The invention solves the technical problem of how to design a block chain which can enter the chain quickly and save the storage space so as to meet the application requirements of high concurrency and real-time distribution of mass data in the field of data circulation.
To solve the foregoing technical problem, an embodiment of the present invention provides a block chain construction method for data circulation, where the block chain construction method includes: receiving account book information in a data circulation process; dividing nodes in a block chain to create partitions; merging and summarizing the account book information to obtain summarized data; and writing the summarized data into a partition corresponding to the summarized data in the block chain, wherein the partition corresponding to the summarized data comprises a supplier node, an demander node, a trading center node and a clearing center node which are associated with the summarized data.
Optionally, the ledger information includes a summary key, and the summary key is used to identify a combination of a supplier node, an acquirer node, and a transaction price.
Optionally, the step of merging and summarizing the ledger information by the block chain construction method to obtain summarized data includes: and merging and summarizing the account book information according to the summarizing main key to obtain the summarized data, wherein the account book information with the same summarizing main key is summarized to the same summarized data.
Optionally, the step of merging and summarizing the ledger information according to a summarizing key by the block chain constructing method to obtain the summarized data includes: merging and summarizing the account book information according to the summarizing main key every a first time period to obtain primary summarized data, wherein the account book information with the same summarizing main key is summarized to the same primary summarized data; and summarizing the data after the preliminary summarization every a second time period according to the summarization main key to obtain the summarized data, wherein the data after the preliminary summarization with the same summarization main key are summarized to the same summarized data, and the second time period is longer than the first time period.
Optionally, the step of performing summarization on the data after the preliminary summarization every second time period according to the summarization primary key by the block chain construction method includes: respectively writing the data after the preliminary summary into a queue and a redo log file; clearing the redo log file and the primarily summarized data which are already summarized in the queue at intervals of the second time period; and taking out the data which are not finished with the summarization after the primary summarization from the queue for summarization every the second time period.
Optionally, the step of performing summarization on the data after the preliminary summarization every second time period according to the summarization primary key by the block chain construction method further includes: when the redo log file is restarted, restoring the queue according to the data after the initial summary in the redo log file; emptying the redo log file and the primarily summarized data which are already summarized in the queue; and taking out the data which are not finished with the summarization after the primary summarization from the queue for summarization every the second time period.
Optionally, the ledger information further includes one or more of the following items: the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Optionally, the summarized data includes one or more of the following: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Optionally, the step of dividing the nodes in the blockchain to create partitions by the blockchain construction method includes: based on a Linux namespace, dividing a single node in a block chain to create an isolated partition of the single node; and creating a partition corresponding to the summarized data based on a discovery algorithm and a consensus algorithm.
To solve the foregoing technical problem, an embodiment of the present invention further provides a block chain constructing apparatus for data circulation, including: the receiving module is suitable for receiving the account book information in the data circulation process; the partitioning module is suitable for partitioning nodes in the block chain to create partitions; the summarizing module is suitable for merging and summarizing the book information to obtain summarized data; and the link entering module is suitable for writing the summarized data into a partition corresponding to the summarized data in the block link, and the partition corresponding to the summarized data comprises a supplier node, an demander node, a transaction center node and a clearing center node which are associated with the summarized data.
Optionally, the ledger information includes a summary key, and the summary key is used to identify a combination of a supplier node, an acquirer node, and a transaction price.
Optionally, the summarizing module includes: and the summarizing submodule is suitable for merging and summarizing the account book information according to the summarizing main key so as to obtain the summarized data, wherein the account book information with the same summarizing main key is summarized to the same summarized data.
Optionally, the summarization sub-module includes: the first summarizing unit is suitable for merging and summarizing the account book information every other first time period according to the summarizing main key to obtain primary summarized data, wherein the account book information with the same summarizing main key is summarized to the same primary summarized data; and the second summarizing unit is suitable for summarizing the primarily summarized data every other second time period according to the summarizing main key to obtain the summarized data, wherein the primarily summarized data with the same summarizing main key are summarized to the same summarized data, and the second time period is greater than the first time period.
Optionally, the second summarizing unit includes: the writing subunit is suitable for respectively writing the data after the preliminary summary into a queue and a redo log file; the first emptying subunit is used for counting the redo log files every other second time period, and the first emptying unit is suitable for emptying the redo log files and the primarily summarized data which are already summarized in the queue; and the first taking-out subunit is suitable for taking out the data which are not finished to be summarized again after the primary summary from the queue for summarization every the second time period.
Optionally, the second summarizing unit further includes: a recovery subunit, adapted to recover the queue according to the preliminary summarized data written in the redo log file when restarting; the second emptying subunit is suitable for emptying the redo log file and the primarily summarized data which are already summarized in the queue; and the second taking-out subunit is suitable for taking out the data which are not finished to be summarized again after the primary summary from the queue for summarization every the second time period.
Optionally, the ledger information further includes one or more of the following items: the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Optionally, the summarized data includes one or more of the following: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Optionally, the partitioning module includes: the first partition submodule is suitable for partitioning a single node in a block chain based on a Linux namespace so as to create a single node isolation partition; and the second partition submodule is used for creating a partition corresponding to the summarized data based on a discovery algorithm and a consensus algorithm.
In order to solve the above technical problem, an embodiment of the present invention further provides a storage medium, where computer instructions are stored, and when the computer instructions are executed, the steps of the block chain construction method are executed.
In order to solve the above technical problem, an embodiment of the present invention further provides a server, including a memory and a processor, where the memory stores computer instructions executable on the processor, and the processor executes the steps of the block chain construction method when executing the computer instructions.
Compared with the prior art, the technical scheme of the embodiment of the invention has the following beneficial effects:
the technical scheme provided by the embodiment of the invention firstly receives the ledger information from the data transaction process and can determine the supplier node, the demander node, the transaction center node and the clearing center node which are associated with the ledger information; then, partitions can be created for the supplier node, the acquirer node, the transaction center node and the clearing center node based on block chains, the partitions only comprise block chain link points associated with the book information, the book information is only stored on the associated block chain nodes, and by improving a bottom layer mechanism of a traditional block chain, the nodes in the traditional block chain are prevented from all participating in data consensus, network overhead in a data circulation process and storage space of the book information are greatly saved, and the method is particularly suitable for the field of data circulation with high confidentiality requirements and aiming at the participation of a small number of nodes; secondly, the account book information can be merged and summarized in the subareas, so that summarized data are obtained, the data storage capacity is further reduced, and quick link entry can be realized due to the fact that the data volume is reduced after the data are summarized; and then writing the summarized data into the supplier node, the demander node, the transaction center node and the clearing center node, wherein each block link node can achieve consensus only by storing data related to data transaction of the block link node, so that the efficiency of achieving consensus of each node in the partition can be improved, and the data link entering speed is further accelerated.
Further, the ledger information includes a summary key, which is used to identify a combination of supplier nodes, acquirer nodes, and transaction prices. And the supplier node and the demander node in data circulation can be consistent according to the summary key, and the ledger information with consistent transaction price is merged and summarized, so that the consumption of storage space can be reduced.
Further, the block chain construction method can write the data after the preliminary summary into a queue and a redo log file respectively, and the queue and the redo log file respectively store the data after the preliminary summary, so that the condition that the data after the preliminary summary is lost due to the crash of a server can be avoided, and the correctness and the integrity of the data can be ensured.
Drawings
Fig. 1 is a flowchart of a block chain construction method for data circulation according to a first embodiment of the present invention;
FIG. 2 is a flowchart of one embodiment of step S103 of FIG. 1;
FIG. 3 is a flowchart of one embodiment of step S1032 of FIG. 2;
FIG. 4 is a flow diagram of yet another embodiment of step S1032 of FIG. 2;
fig. 5 is a schematic structural diagram of a block chain constructing apparatus for data circulation according to a second embodiment of the present invention.
Detailed Description
As will be understood by those skilled in the art, as the background art, the consensus mechanism of the conventional block chain technology is time-consuming, network overhead is large, and storage space is large, so that it is difficult to meet the requirement of the data circulation field on the link entry speed of data, and the application requirement of real-time data distribution cannot be met.
The technical scheme provided by the embodiment of the invention firstly receives the ledger information from the data transaction process and can determine the supplier node, the demander node, the transaction center node and the clearing center node which are associated with the ledger information; then, partitions can be created for the supplier node, the acquirer node, the transaction center node and the clearing center node based on block chains, the partitions only comprise block chain link points associated with the book information, the book information is only stored on the associated block chain nodes, and by improving a bottom layer mechanism of a traditional block chain, the nodes in the traditional block chain are prevented from all participating in data consensus, network overhead in a data circulation process and storage space of the book information are greatly saved, and the method is particularly suitable for the field of data circulation with high confidentiality requirements and aiming at the participation of a small number of nodes; secondly, the account book information can be merged and summarized in the subareas, so that summarized data are obtained, the data storage capacity is further reduced, and quick link entry can be realized due to the fact that the data volume is reduced after the data are summarized; and then writing the summarized data into the supplier node, the demander node, the transaction center node and the clearing center node, wherein each block link node can achieve consensus only by storing data related to data transaction of the block link node, so that the efficiency of achieving consensus of each node in the partition can be improved, and the data link entering speed is further accelerated.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the embodiments of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.
Fig. 1 is a flowchart of a block chain construction method for data circulation according to a first embodiment of the present invention. The block chain construction method shown in fig. 1 may include the following steps:
step S101, receiving account book information in a data circulation process;
step S102, dividing nodes in a block chain to create partitions;
step S103, merging and summarizing the book information to obtain summarized data; and writing the summarized data into a partition corresponding to the summarized data in the block chain, wherein the partition corresponding to the summarized data comprises a supplier node, an demander node, a trading center node and a clearing center node which are associated with the summarized data.
Specifically, in step S101, account book information of both the data supply and demand parties is received from the data distribution process. The ledger information may be generated when the data supply and demand parties perform data transaction, and the ledger information may be used as a basis for settlement of transaction. The ledger information may include transaction price information when the data supply and demand parties perform transactions. Further, the demander node may send a data request to the supplier node, and the demander node may record transaction price information corresponding to the data request; then, the supplier node feeds back data corresponding to the data request to the demander node; finally, the acquirer node receives the data. Ledger information generated during this transaction may include the transaction price information.
In the above transaction process, the ledger information may include signature information and a timestamp of its own node, and may also include signature information and a timestamp of an opposite node, that is, a signature of the supplier node, and a signature and a timestamp of the acquirer node. The signature of the supplier node and the signature of the acquirer node may be identifiers of servers used by the supplier node and the acquirer node, for example, Media Access Control (MAC) addresses of the servers, or identifiers of servers approved by both parties of the data transaction.
Further, considering that time stamps between each blockchain node may be inconsistent or there is an error gap in time due to network delay, the time stamp in the ledger information may be subject to a time agreed by both parties of the data transaction, for example, subject to the time stamp of the supplier, and a time error gap is allowed. In addition, errors in the transaction price information and the transaction data amount may also be allowed. The ledger information can be counted in the clearing range as long as it is within the error range allowed by both parties to the transaction.
Further, the ledger information may further include a summary key, and the summary key may be used to identify a combination of a supplier node, an acquirer node, and a transaction price for data transaction, that is, different summary keys correspond to different combinations, where different combinations refer to at least one of the supplier node, the acquirer node, and the transaction price being different. Based on the summary key (such as a data distribution task number), the ledger information with large data volume can be easily merged and summarized, and the obtained summarized data can reduce the data storage space.
In step S102, the block link points in the conventional block chain are divided to create partitions. Upon receiving the ledger information, the blockchain may determine blockchain nodes included in the partition. The server may partition supplier nodes, acquirer nodes, transaction center nodes, and clearing center nodes associated with the ledger information into the same partition. The subareas do not comprise other subarea chain nodes irrelevant to data transaction, and the book information is only stored on a supplier node, an acquirer node, the transaction center node and the clearing center node of the data transaction so as to manage and maintain the data transaction process and assist both data transaction parties in completing the data transaction. By partitioning the block chain nodes, the storage space of the transaction data can be greatly saved.
Further, in step S102, a sequestered partition of a single node may be created based on the Linux Namespace (Namespace). The Namespace is the characteristic that the Linux kernel isolates and virtualizes system resources. The Namespace can perform resource isolation, and the resources can include process Identifiers (IDs), host names, user IDs, network access, interprocess communication, file systems, and the like. In the Namespace, each process is bound in a specific Namespace, and only the resource bound in the Namespace is allowed to be viewed and operated, so that a single node in a block chain can be partitioned based on the Namespace, the isolation of the account book information generated by different data transactions is realized, and the problem of mutual trust is avoided.
Specifically, during the data transaction process, whether to create a partition using Namespace for a single node in the block chain may be determined according to whether to associate with ledger information. Furthermore, a single node in the block chain may include one server or a server cluster composed of a plurality of servers, and only stores the account book information, the preliminarily summarized data and the summarized data related to the single node, thereby greatly saving the network overhead in the data circulation process and the storage space of the account book information. Under the condition of hardware configuration and port number permission, the partition of a single node can support dynamic expansion into a plurality of Namespaces, and different Namespaces in the same node can be distinguished by using the port numbers or can be distinguished by using marks in the same port. For example, if a supplier node A3 is newly added in the data flow, and the supplier node A3 supplies data to the demander node B3 and the demander node B4 at the same time, two namespaces may be newly added in the supplier node A3, where one Namespace partition is used to store the transaction data of the supplier node A3 and the demander node B3 (e.g., the ledger information or the preliminary summarized data or the summarized data of the supplier node A3 and the demander node B3), and the other Namespace partition is used to store the transaction data of the supplier node A3 and the demander node B4. At the same time Namespace is automatically generated at the acquirer node B3 (for saving transaction data for the acquirer node B3 and the donor node A3), and Namespace is automatically generated at the acquirer node B4 (for saving transaction data for the acquirer node B4 and the donor node A3). The two Namespace resources are isolated, data sharing is not carried out, the confidentiality is high, and the data storage space is reduced. According to the technical scheme, the bottom layer mechanism of the traditional block chain is improved, and the situation that all nodes in the traditional block chain participate in data consensus is avoided.
Further, partitioning of the supplier node, the demander node, the transaction center node and the clearing center node of the data transaction associated with the ledger information in the blockchain can be achieved according to a discovery algorithm and a consensus algorithm.
In step S103, the book information may be merged and summarized to obtain summarized data; and writing the summarized data into the partition corresponding to the summarized data in the block chain, namely respectively writing into an acquirer node, a supplier node, a transaction center node and a clearing center node. The consolidated summary may be consolidated and summarized on the basis of the summary primary key, for example, the summary information of which the supply node and the demand node in data circulation are consistent and the transaction prices are consistent is consolidated and summarized according to the summary primary key, so that the consumption of storage space may be reduced.
Further, after the supplier node and/or the demander node obtain the summarized data, the data consensus of the supplier node, the demander node, the trading center node and the clearing center node is realized through a present algorithm and a consensus algorithm, and the data entering chain is completed.
In practical applications, those skilled in the art may change more embodiments according to actual needs, for example, the data consensus may also be implemented in a customized manner. The customized mode can select a client trusted node from all nodes contained in a block chain participating in data transaction based on client requirements, and data consensus is realized by adopting a data discovery algorithm and a data consensus algorithm.
Further, the data transaction may also be implemented by establishing a blockchain sub-chain, but since each blockchain sub-chain is independent from each other, each node is not in the same blockchain, and when the data transaction center node and the clearing center node count data information (e.g., book information), the count complexity is undoubtedly increased.
Further, the summarized data is consistent with the ledger information, obviously, the summarized data may include one or more of the following items: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node. For specific meanings of each content included in the summarized data, reference may be made to the related description in the above-mentioned ledger information, and details are not described here.
Since the basic unit of the ledger information generated in step S101 is a single piece of data, the data amount of the ledger information is considerably large. If the account book information is directly written into the block chain, a large amount of storage space is consumed, so that the account book information is not suitable for being directly used as a data record of the account book block chain, but the account book information can be summarized at regular time (for example, minute level) to reduce the storage space. As a non-limiting example, referring to fig. 2, the step S103 may include the steps of:
step S1031, merging and summarizing the account book information according to the summarizing main key every other first time period to obtain primary summarized data, wherein the account book information with the same summarizing main key is summarized to the same primary summarized data;
and S1032, re-summarizing the primarily summarized data every other second time period according to the summarizing main key to obtain the summarized data, wherein the primarily summarized data with the same summarizing main key are summarized to the same summarized data, and the second time period is greater than the first time period.
Specifically, in step S1031, the ledger information may be periodically summarized based on the summarization primary key, that is, at intervals (for example, a first time period), data entries sent by the supplier and data entries received by the acquirer node are counted based on the summarization primary key, and the ledger information is periodically summarized to obtain the preliminary summarized data. For example, the ledger information in the same partition on the same server may be merged and summarized based on the summarizing master key, or the ledger information on a server cluster composed of different servers may be merged and summarized based on the summarizing master key, so as to obtain the preliminary summarized data.
In step S1032, in order to further summarize the data, the primarily summarized data may be re-summarized still based on the summarizing main key, for example, the primarily summarized data with the same summarizing main key may be merged and summarized to the same summarized data every second time period (the time period is greater than the first time period). The first time period and the second time period are both dynamically configurable. After the summarized data is generated, consensus verification can be performed only on the participants of the data transaction (the supplier node and the demander node), the transaction center node and the clearing center node, because a complete data transaction can be completed only by the participation of the four nodes, and thus the consensus verification can be completed only in the four nodes. Therefore, the transaction center node and the clearing center node need to keep the summarized data generated by each data transaction, but the participants of the data transaction only need to store the self-related book information, and the storage space is greatly saved.
Further, re-summarizing the preliminarily summarized data may combine and summarize the preliminarily summarized data in the same partition on the same node (for example, any one of a supplier node, an demander node, a data transaction center node, and a data clearing center node, each of which may be a server or a server cluster composed of multiple servers) based on the summarizing key to obtain the summarized data.
Furthermore, the ledger information is merged and summarized to obtain summarized data before entering a link, and then common identification and link entry are performed on the summarized data in a supplier node, a demander node, a data transaction center node and a data clearing center node through a discovery algorithm and a common identification algorithm, so that the integrity of the summarized data needs to be ensured before entering the link, and the problem that the ledger information is lost due to the crash of a server in the summarizing process is solved.
Referring to fig. 3, as a non-limiting example, the step S1032 may include the following steps to solve the ledger information loss problem:
step S10321, writing the data after the preliminary summary into a queue and a redo log file respectively;
step S10322, every other second time period, emptying the redo log file and the primarily summarized data which are already summarized in the queue;
step S10323, every second time period, taking out the data after the preliminary summarization that is not completed and summarized from the queue for summarization.
Further, in step S10321, the server writes the data after the preliminary summarization into a queue and a redo log file, and the redo log file may further record an execution step of the preliminary summarized data in a re-summarization process. And the queue and the redo log file are used for storing the data after the initial summary so as to avoid the problem of data loss caused by the downtime of a server before the data after the initial summary enters the chain. The queues may be placed independently in other servers to reduce the risk of data loss due to a server downtime.
Further, the server may re-summarize the preliminary summarized data at intervals (e.g., at intervals of a second time period). At this time, the summarized data before the current time period may be emptied through step S10322, that is, the redo log file and the preliminary summarized data in the queue that has been completed with re-summarization may be emptied. This is because the summarized data that is re-summarized before the current time period is written into the blockchain node, and the problem of data loss due to the downtime of the server does not occur, so that the data can be emptied to reduce the storage space of the useless data.
Further, in step S10323, the server may perform summarization on the preliminary summarized data every other second time period to obtain the summarized data, and write the summarized data into a block link node. The step of re-summarizing the preliminary summarized data may comprise: firstly, based on a timestamp carried by account book information in the primarily summarized data, primarily summarized data which is subjected to re-summarization in the second time period can be calculated; then, sequentially taking out the data after the preliminary summary from the queue and writing the data into a hash table; and finally, in the second time period, re-summarizing the preliminarily summarized data in the hash table according to the summarizing main key to obtain the summarized data.
As another non-limiting embodiment, referring to fig. 4, after the server is down and restarted, the data after the preliminary summary may be recovered according to the queue and the redo log file, and specifically, the data after the preliminary summary may be recovered by the following steps:
step S10324, when restarting, restoring the queue according to the data after the preliminary summary in the redo log file;
step S10325, emptying the redo log file and the primarily summarized data which is already summarized in the queue;
step S10326, taking out the data after the preliminary summarization that is not completed and summarized from the queue for summarization every other time period.
Specifically, in step S10324, when the server is down and restarted, the queue may be restored according to the redo log file, that is, the data after the preliminary summary in the redo log file may be written into the queue.
Further, in step S10325, the data after the preliminary summary, which has been completed by the re-summary but is not emptied in time before the server is down, is emptied from the redo log file and the queue, so as to reduce the storage space of the useless data.
Further, similar to step S10323, in step S10326, the data after the initial summarization that is not completed by the summarization is taken out from the queue for summarization every second time period. The difference is that because the hash table is lost due to the downtime of the server, a new hash table can be established and re-summarized before the queue is written into the hash table, and the specific steps can be as follows: firstly, obtaining the primarily summarized data which can be subjected to re-summarization in the second time period based on the timestamp carried by the account book information in the primarily summarized data; then establishing a hash table, and writing the primarily summarized data which can be summarized again in the second time period into the hash table from the queue; finally, in the second time period according to step S10323, the data after the preliminary summarization in the hash table may be summarized again according to the summarization key, so as to obtain the summarized data.
Those skilled in the art understand that the preliminary aggregation and re-aggregation of the ledger information can be implemented by using a hash table, but in practical application, the invention is not limited to this.
Fig. 5 is a schematic structural diagram of a block chain constructing apparatus for data circulation according to a second embodiment of the present invention. Those skilled in the art understand that the block chain constructing apparatus 5 of the present embodiment may be used to implement the technical solution of the block chain constructing method in the embodiments shown in fig. 1 to fig. 4.
Referring to fig. 5, the blockchain constructing apparatus 5 may include: a receiving module 51, a partitioning module 52, a summarizing module 53 and an in-chain module 54.
The receiving module 51 is adapted to receive ledger information in a data circulation process; the partitioning module 52 is adapted to partition the nodes in the block chain to create partitions; the summarizing module 53 is adapted to merge and summarize the ledger information to obtain summarized data; the chaining module 54 is adapted to write the summarized data into the partition corresponding to the summarized data in the block chain, where the partition corresponding to the summarized data includes a supplier node, an demander node, a trading center node, and a clearing center node associated with the summarized data.
Preferably, the ledger information may include a summary primary key, which may be used to identify a combination of supplier nodes, acquirer nodes, and transaction prices.
Further, the ledger information may further include one or more of the following items: the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Further, the summarized data includes one or more of: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
Preferably, the partition module 52 may include: a first partition submodule 521, adapted to partition a single node in a block chain based on a Linux namespace to create a single node isolation partition; and the second partition submodule 522 creates a partition corresponding to the summarized data based on a discovery algorithm and a consensus algorithm.
Preferably, the summarizing module 53 may include: and the summarizing submodule 531 is adapted to combine and summarize the ledger information according to summarizing major keys to obtain the summarized data, wherein the ledger information with the same summarizing major keys can be summarized to the same summarized data.
As a non-limiting example, the summarization sub-module 531 may comprise: a first totaling unit 5311 and a second totaling unit 5312.
The first summarizing unit 5311 is adapted to combine and summarize the book information every other first time period according to the summarizing key to obtain preliminary summarized data, wherein the book information with the same summarizing key is summarized to the same preliminary summarized data; the second summarizing unit 5312 is adapted to summarize the preliminary summarized data every a second time period according to the summarizing key to obtain the summarized data, wherein the preliminary summarized data with the same summarizing key are summarized to the same summarized data, and the second time period is longer than the first time period.
Preferably, the second summarizing unit 5312 may include: a write subunit 53121, a first clear subunit 53122, and a first fetch subunit 53123.
The writing subunit 53121 is adapted to write the preliminarily summarized data into a queue and a redo log file, respectively; the redo log file is counted every second time period, and the first emptying unit 53122 may be adapted to empty the redo log file and the primarily summarized data in the queue that has been completed with re-summarization; every second time period, the first dequeue subunit 53123 may be adapted to dequeue the outstanding re-summarized preliminary post-summary data from the queue for re-summarization.
As a variation, the second summarizing unit 5312 may further include: a restore sub-unit 53124, a second empty sub-unit 53125, and a second fetch sub-unit 53126.
Upon a reboot, the recovery subunit 53124 may be adapted to recover the queue from preliminary summary data written to the redo log file; the second emptying subunit 53125 is adapted to empty the redo log file and the re-summarized preliminary summarized data in the queue; every second time period, the second dequeue subunit 53126 may be adapted to dequeue the outstanding re-summarized preliminary summarized data from the queue for re-summarization.
For more details of the working principle and the working manner of the block chain constructing device 5, reference may be made to the above description of the embodiments shown in fig. 1 to fig. 4, and details are not repeated here.
Further, the embodiment of the present invention also discloses a storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the block chain construction method according to the embodiment shown in fig. 1 to fig. 4 are executed. Preferably, the storage medium may include a computer-readable storage medium. Preferably, the storage medium may include ROM, RAM, magnetic or optical disks, or the like.
Further, the embodiment of the present invention also discloses a server, which includes a memory and a processor, where the memory stores computer instructions capable of being executed on the processor, and the processor executes the computer instructions to execute the steps of the block chain construction method according to the embodiment shown in fig. 1 to 4.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (20)
1. A method for building a block chain for data circulation, comprising:
receiving account book information in a data circulation process;
dividing nodes in a block chain to create partitions;
merging and summarizing the account book information to obtain summarized data;
writing the summarized data into a partition corresponding to the summarized data in the block chain, wherein the partition corresponding to the summarized data comprises a supplier node, an demander node, a trading center node and a clearing center node which are associated with the summarized data;
wherein the partition corresponding to the summarized data does not include other blockchain nodes unrelated to data transactions.
2. The blockchain construction method of claim 1, wherein the ledger information includes a summary primary key that identifies a combination of supplier nodes, acquirer nodes, and transaction prices.
3. The method according to claim 2, wherein the step of merging and summarizing the ledger information to obtain summarized data comprises: and merging and summarizing the account book information according to the summarizing main key to obtain the summarized data, wherein the account book information with the same summarizing main key is summarized to the same summarized data.
4. The blockchain construction method according to claim 3, wherein the step of merging and summarizing the ledger information according to a summarization primary key to obtain the summarized data includes:
merging and summarizing the account book information according to the summarizing main key every a first time period to obtain primary summarized data, wherein the account book information with the same summarizing main key is summarized to the same primary summarized data;
and summarizing the data after the preliminary summarization every a second time period according to the summarization main key to obtain the summarized data, wherein the data after the preliminary summarization with the same summarization main key are summarized to the same summarized data, and the second time period is longer than the first time period.
5. The blockchain construction method according to claim 4, wherein the step of reintegrating the preliminarily summarized data every second time period according to the summarization key includes:
respectively writing the data after the preliminary summary into a queue and a redo log file;
clearing the redo log file and the primarily summarized data which are already summarized in the queue at intervals of the second time period;
and taking out the data which are not finished with the summarization after the primary summarization from the queue for summarization every the second time period.
6. The blockchain construction method according to claim 5, wherein the step of reintegrating the preliminarily summarized data every second time period according to the summarization key further comprises: when the redo log file is restarted, restoring the queue according to the data after the initial summary in the redo log file;
emptying the redo log file and the primarily summarized data which are already summarized in the queue;
and taking out the data which are not finished with the summarization after the primary summarization from the queue for summarization every the second time period.
7. The blockchain construction method according to claim 2, wherein the ledger information further includes one or more of the following items: the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
8. The blockchain construction method of claim 2, wherein the summarized data includes one or more of: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
9. The blockchain construction method according to any one of claims 1 to 8, wherein the step of dividing the nodes in the blockchain to create partitions includes:
based on a Linux namespace, dividing the single nodes in the block chain to create isolated partitions of the single nodes;
and creating a partition corresponding to the summarized data based on a discovery algorithm and a consensus algorithm.
10. A block chain construction apparatus for data circulation, comprising:
the receiving module is suitable for receiving the account book information in the data circulation process;
the partitioning module is suitable for partitioning nodes in the block chain to create partitions;
the summarizing module is suitable for merging and summarizing the book information to obtain summarized data;
the link entering module is suitable for writing the summarized data into a partition corresponding to the summarized data in the block chain, and the partition corresponding to the summarized data comprises a supplier node, an demander node, a transaction center node and a clearing center node which are associated with the summarized data;
wherein the partition corresponding to the summarized data does not include other blockchain nodes unrelated to data transactions.
11. The blockchain construction device of claim 10 wherein the ledger information includes a summary primary key that identifies a combination of supplier nodes, acquirer nodes and transaction prices.
12. The blockchain construction device of claim 11, wherein the aggregation module comprises: and the summarizing submodule is suitable for merging and summarizing the account book information according to the summarizing main key so as to obtain the summarized data, wherein the account book information with the same summarizing main key is summarized to the same summarized data.
13. The blockchain construction device of claim 12, wherein the aggregation sub-module includes:
the first summarizing unit is suitable for merging and summarizing the account book information every other first time period according to the summarizing main key to obtain primary summarized data, wherein the account book information with the same summarizing main key is summarized to the same primary summarized data;
and the second summarizing unit is suitable for summarizing the primarily summarized data every other second time period according to the summarizing main key so as to obtain the summarized data, wherein the primarily summarized data with the same summarizing main key are summarized to the same summarized data, and the second time period is greater than the first time period.
14. The blockchain building apparatus of claim 13, wherein the second aggregation unit includes:
the writing subunit is suitable for respectively writing the data after the preliminary summary into a queue and a redo log file;
the first emptying subunit is used for counting the redo log files every other second time period, and the first emptying unit is suitable for emptying the redo log files and the primarily summarized data which are already summarized in the queue;
and the first taking-out subunit is suitable for taking out the data which are not finished to be summarized again after the primary summary from the queue for summarization every the second time period.
15. The blockchain construction device of claim 14, wherein the second aggregation unit further comprises:
a recovery subunit, adapted to recover the queue according to the preliminary summarized data written in the redo log file when restarting;
the second emptying subunit is suitable for emptying the redo log file and the primarily summarized data which are already summarized in the queue;
and the second taking-out subunit is suitable for taking out the data which are not finished to be summarized again after the primary summary from the queue for summarization every the second time period.
16. The blockchain construction device of claim 11, wherein the ledger information further includes one or more of the following: the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
17. The blockchain construction device of claim 11, wherein the summarized data includes one or more of: the summary primary key, the signature of the supplier node, the signature of the demander node and a timestamp, wherein the timestamp is subject to the time of the supplier node.
18. The blockchain building apparatus of any one of claims 10 to 17, wherein the partition module includes:
the first partition submodule is suitable for partitioning nodes in the block chain based on a Linux namespace so as to create a single node isolation partition;
and the second partition submodule is used for creating a partition corresponding to the summarized data based on a discovery algorithm and a consensus algorithm.
19. A storage medium having stored thereon computer instructions which, when executed, perform the steps of the blockchain construction method of any one of claims 1 to 9.
20. A server comprising a memory and a processor, the memory having stored thereon computer instructions executable on the processor, wherein the processor, when executing the computer instructions, performs the steps of the blockchain construction method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710922682.5A CN107786642B (en) | 2017-09-30 | 2017-09-30 | Block chain construction method and device for data circulation, storage medium and server |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710922682.5A CN107786642B (en) | 2017-09-30 | 2017-09-30 | Block chain construction method and device for data circulation, storage medium and server |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107786642A CN107786642A (en) | 2018-03-09 |
| CN107786642B true CN107786642B (en) | 2020-04-17 |
Family
ID=61434188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710922682.5A Active CN107786642B (en) | 2017-09-30 | 2017-09-30 | Block chain construction method and device for data circulation, storage medium and server |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107786642B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108399572A (en) * | 2018-03-22 | 2018-08-14 | 深圳市网心科技有限公司 | block chain transaction processing method, device and storage medium |
| CN110557416B (en) * | 2018-05-31 | 2022-02-18 | 本无链科技(深圳)有限公司 | Multi-node collaborative blocking method and system |
| CN108932433B (en) * | 2018-06-14 | 2021-11-05 | 江苏百倍云信息科技有限公司 | Industrial data sharing system and method based on block chain |
| CN110717825B (en) * | 2018-06-27 | 2023-04-07 | 复旦大学 | Distributed data transaction accounting system based on block chain |
| CN108989410B (en) * | 2018-07-04 | 2020-10-30 | 清华大学 | Method for improving throughput efficiency of block chain |
| CN108959654A (en) * | 2018-08-06 | 2018-12-07 | 安徽笛申科技有限公司 | A kind of data management system based on block chain intelligence contract node and storage |
| CN109309708B (en) * | 2018-08-16 | 2021-06-04 | 泰链（厦门）科技有限公司 | Method, medium, apparatus and system for increasing block production speed in block chain system |
| CN109146573A (en) * | 2018-09-05 | 2019-01-04 | 深圳正品创想科技有限公司 | Commodity integration method and its device, block chain node based on block chain |
| CN109767216B (en) * | 2018-12-29 | 2020-10-02 | 莆田市烛火信息技术有限公司 | Account book accounting system of block chain network with regularly converged parallel chains |
| CN111447055B (en) * | 2019-01-17 | 2022-02-25 | 华为技术有限公司 | Block chain account book storage method and device |
| CN110060058B (en) * | 2019-04-25 | 2023-10-10 | 深圳市元征科技股份有限公司 | Transaction method, device, equipment and readable storage medium |
| CN110264348B (en) * | 2019-05-07 | 2021-08-20 | 北京奇艺世纪科技有限公司 | Processing method, device and storage medium for transaction uplink |
| CN110471928B (en) * | 2019-07-08 | 2022-06-21 | 上海数据交易中心有限公司 | Data circulation system |
| CN110532270A (en) * | 2019-08-30 | 2019-12-03 | 南京荣链科技有限公司 | The associated data managing method of account book and its system |
| CN110989925A (en) * | 2019-10-30 | 2020-04-10 | 烽火通信科技股份有限公司 | Deployment method and device of distributed storage system |
| CN111221910A (en) * | 2019-12-31 | 2020-06-02 | 杭州趣链科技有限公司 | Fragment storage method for improving block chain read-write performance |
| CN111782644A (en) * | 2020-07-27 | 2020-10-16 | 江苏工程职业技术学院 | Scientific research data management control system and method based on block chain technology |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106897933A (en) * | 2017-04-18 | 2017-06-27 | 杜伯仁 | A kind of block chain network node serve device towards multi-tiling platform chain |
| CN106980649A (en) * | 2017-02-28 | 2017-07-25 | 阿里巴巴集团控股有限公司 | The method and apparatus and business subclass for writing block chain business datum determine method |
| CN106991164A (en) * | 2017-03-31 | 2017-07-28 | 北京京东金融科技控股有限公司 | Method, device and electronic equipment that finance data is handled are used for based on block chain |
| CN107038638A (en) * | 2017-02-24 | 2017-08-11 | 杭州象链网络技术有限公司 | A kind of equity registration transaction system construction method based on alliance's chain |
| CN107180350A (en) * | 2017-03-31 | 2017-09-19 | 唐晓领 | A kind of method of the multi-party shared transaction metadata based on block chain, apparatus and system |
| CN107194798A (en) * | 2017-04-28 | 2017-09-22 | 广东网金控股股份有限公司 | A kind of bank clearing method based on block chain alliance chain |
-
2017
- 2017-09-30 CN CN201710922682.5A patent/CN107786642B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107038638A (en) * | 2017-02-24 | 2017-08-11 | 杭州象链网络技术有限公司 | A kind of equity registration transaction system construction method based on alliance's chain |
| CN106980649A (en) * | 2017-02-28 | 2017-07-25 | 阿里巴巴集团控股有限公司 | The method and apparatus and business subclass for writing block chain business datum determine method |
| CN106991164A (en) * | 2017-03-31 | 2017-07-28 | 北京京东金融科技控股有限公司 | Method, device and electronic equipment that finance data is handled are used for based on block chain |
| CN107180350A (en) * | 2017-03-31 | 2017-09-19 | 唐晓领 | A kind of method of the multi-party shared transaction metadata based on block chain, apparatus and system |
| CN106897933A (en) * | 2017-04-18 | 2017-06-27 | 杜伯仁 | A kind of block chain network node serve device towards multi-tiling platform chain |
| CN107194798A (en) * | 2017-04-28 | 2017-09-22 | 广东网金控股股份有限公司 | A kind of bank clearing method based on block chain alliance chain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107786642A (en) | 2018-03-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107786642B (en) | Block chain construction method and device for data circulation, storage medium and server | |
| US20230100223A1 (en) | Transaction processing method and apparatus, computer device, and storage medium | |
| US20210256016A1 (en) | Blockchain system and method | |
| US10713654B2 (en) | Enterprise blockchains and transactional systems | |
| EP2810164B1 (en) | Managing partitions in a scalable environment | |
| US10031935B1 (en) | Customer-requested partitioning of journal-based storage systems | |
| US9063790B2 (en) | System and method for performing distributed parallel processing tasks in a spot market | |
| US10296371B2 (en) | Passive two-phase commit system for high-performance distributed transaction execution | |
| US8694647B2 (en) | Read-only operations processing in a paxos replication system | |
| CN111338766A (en) | Transaction processing method and device, computer equipment and storage medium | |
| CN103092903A (en) | Database Log Parallelization | |
| US9798639B2 (en) | Failover system and method replicating client message to backup server from primary server | |
| US9372879B1 (en) | Balanced append tree data structure | |
| CN112559637B (en) | Data processing method, device, equipment and medium based on distributed storage | |
| CN110188110A (en) | A kind of method and device constructing distributed lock | |
| CN101834904A (en) | Method and equipment for database backup | |
| US20230275771A1 (en) | Pre-execution of block chain transaction in parallel during block consensus | |
| CN105373563B (en) | Database switching method and device | |
| CN112600914B (en) | Data processing method and device, computer readable medium and electronic equipment | |
| CA2948404A1 (en) | System and method for running application processes | |
| US7260611B2 (en) | Multi-leader distributed system | |
| US11522966B2 (en) | Methods, devices and systems for non-disruptive upgrades to a replicated state machine in a distributed computing environment | |
| CN116974983A (en) | Data processing method, device, computer readable medium and electronic equipment | |
| US8775505B2 (en) | Optimized event routing in distributed data management | |
| US20230409535A1 (en) | Techniques for resource utilization in replication pipeline processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |