CN115481417A - Method for data snapshot, filing and complete recovery of uplink scheme based on chain - Google Patents
Method for data snapshot, filing and complete recovery of uplink scheme based on chain Download PDFInfo
- Publication number
- CN115481417A CN115481417A CN202211068751.8A CN202211068751A CN115481417A CN 115481417 A CN115481417 A CN 115481417A CN 202211068751 A CN202211068751 A CN 202211068751A CN 115481417 A CN115481417 A CN 115481417A
- Authority
- CN
- China
- Prior art keywords
- data
- snapshot
- node
- chain
- archiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/602—Providing cryptographic facilities or services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of block chains, in particular to a method for data snapshot, archiving and complete recovery of uplink scheme based on a chain, wherein a client initiates a data snapshot request, creates voting (vote) transaction of the data snapshot of an appointed chain, acquires information such as a snapshot list and the like, each node can initiate a data archiving request of the appointed node for the height of the snapshot according to the storage capacity condition of the node, after the node is archived, the chain does not have complete data, and if the uplink scheme of the archived data needs to be recovered; the beneficial effects are that: the scheme of the method for snapshotting, archiving and completely recovering the uplink scheme based on the chain has stronger integrity and usability. The scheme comprises three main processes of data snapshot, data archiving and data recovery uplink, and the data snapshot operation ensures regular multiple snapshots of the consistency of the distributed data.
Description
Technical Field
The invention relates to the technical field of block chains, in particular to a method for data snapshot, archiving and complete recovery of an uplink scheme based on a chain.
Background
The block chain is a novel decentralized and non-falsifiable distributed account book, the nodes are equal, all the nodes store full data, although the storage mechanism ensures the credibility of the data to a certain degree, the information is public and transparent, and the trust cost of each participant of the system is reduced.
In the prior art, as the running time of a block chain increases, chain data continuously increases, and node data of the whole network sharply increases, which all cause the block data to be greatly expanded, and bring huge pressure to system storage and operation. And therefore need to archive data on the chain in a timely manner.
However, the industry does not have a complete, comprehensive and easy-to-use scheme mechanism for archiving data, many of the schemes are simple and violent and the data of a single node member is migrated and stored, and the scheme is inflexible and imperfect.
Disclosure of Invention
The present invention provides a method for chain-based data snapshot, archiving and complete uplink recovery, so as to solve the problems in the background art, and flexibly implement that each node in a block chain performs data snapshot at the same block height, and further performs operation mechanisms such as data archiving and recovery according to snapshot data.
In order to achieve the purpose, the invention provides the following technical scheme: the method for recovering the uplink scheme based on chain data snapshot, archiving and integrity comprises the following steps:
the client initiates a data snapshot request, creates a voting (vote) transaction of a data snapshot of an appointed chain, sends the data snapshot transaction request to an appointed consensus node in the block chain, and immediately and independently sends out blocks and submits the blocks to an account book after the consensus node successfully receives all organized voting results; when each node receives the independent data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy or not; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism;
the obtained information such as the snapshot list and the like can be combined with each node to initiate a data filing request of the appointed node for appointing the snapshot height according to the storage capacity condition of each node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting the corresponding local snapshot data and the corresponding block file data on the chain, and returning to the client for successful data filing; otherwise, reporting that the client end data archiving fails, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment;
after the node is filed, the chain has no full complete data, if the filing data uplink needs to be recovered, a filing data uplink recovery request of the node can be initiated, and cold-stored filing data uplink is recovered.
Preferably, the voting (vote) transaction comprises a link ID, a transaction type (vote type), voting policy contents;
the consensus node judges that the transaction type is a vote (vote) type, broadcasts the transaction to all organizations on the chain, votes for each organization, and returns a voting result to the consensus node;
when the voting result in the transaction is checked whether to meet the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number division; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported.
Preferably, the data archive request is sent, upon request,
for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, such as an archiving request with a snapshot height of 1000, can be initiated according to the specific requirements of the current node;
after receiving the archive request, the node 1 archives the snapshot data with the block height of 1000 or less to the off-chain cold storage, such as a cloud database, a cloud file system, or the like, which is cheaper to store.
Preferably, when the archive data is queried, the query operation is performed on the archive data stored outside the chain through the command line tool cmc or the client SDK, and because the archive data is generally cold data with relatively low frequency access, the query requirement capability supported is relatively simple.
Preferably, archiving the data restores the integrity of the uplink, tamper protection: calculating a HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with a HASH metadata value calculated by an original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or is tampered, not recovering the filing data, and reporting a client data filing recovery response error;
when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of segmented _ num of the node 1 needs to be judged, and if the block data block _ num of the synchronization start is smaller than or equal to the segmented _ num, the data is synchronized from the archived data and referred to and the process of recovering the uplink of the archived data is recovered; otherwise, the data on the link of the node 1 is synchronized, and the new node is ensured to be synchronized to complete data.
A system for data snapshot, filing and complete recovery of uplink scheme based on chain is composed of a data snapshot module, a data filing module and a data recovery module;
the data snapshot module is used for initiating a data snapshot request by a client, creating voting (vote) transaction of a data snapshot of a specified chain, sending the data snapshot transaction request to a specified consensus node in the block chain, and immediately and independently sending out a block and submitting the block to an account book after the consensus node successfully receives all organized voting results; when each node receives a single data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism;
the data archiving module is used for acquiring information such as a snapshot list, and each node can initiate a data archiving request of a specified node for specifying the snapshot height according to the storage capacity of the node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting the corresponding local snapshot data and the corresponding block file data on the chain, and returning to the client for successful data filing; otherwise, reporting the client data archiving failure, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment;
and the data recovery module is used for initiating the request of recovering the uplink of the archived data of the node and recovering the uplink of the cold-stored archived data if the uplink of the archived data needs to be recovered if the node does not have full complete data on the chain after the node is archived.
Preferably, the voting (vote) transaction of the data snapshot module includes a link ID, a transaction type (vote type), and voting policy contents;
the consensus node judges that the transaction type is a vote (vote) type, broadcasts the transaction to all organizations on the chain, votes for each organization, and returns a voting result to the consensus node;
when the voting result in the transaction is checked whether to meet the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported.
Preferably, when the data archiving module requests to archive data,
for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, such as an archiving request with a snapshot height of 1000, can be initiated according to the specific requirements of the current node;
after receiving the archive request, the node 1 archives the snapshot data with the block height of 1000 or less to the off-chain cold storage, such as a cloud database, a cloud file system, or the like, which is cheaper to store.
Preferably, when the archive data of the data archive module is queried, the query operation is performed on the archive data stored in the cold storage outside the chain through the command line tool cmc or the client SDK, and because the archive data is generally cold data accessed with a relatively low frequency, the query requirement capability supported by the query operation is relatively simple.
Preferably, the archived data of the data recovery module recovers the integrity of the uplink, and the tamper-resistant protection: calculating a HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with a HASH metadata value calculated by an original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or is tampered, not recovering the filing data, and reporting a client data filing recovery response error;
when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of occupied _ num of the node 1 needs to be judged, and if block data block _ num of the synchronization start is smaller than or equal to occupied _ num, a process of restoring an uplink of the archived data by referencing the synchronous data in the archived data is carried out; otherwise, the data on the link of the node 1 is synchronized, and the new node is synchronized to complete data.
Compared with the prior art, the invention has the beneficial effects that:
the scheme of the method for snapshotting, archiving and completely recovering the uplink scheme based on the chain has stronger integrity and usability. The scheme comprises three main flows of data snapshot, data archiving and data recovery uplink, wherein the data snapshot operation ensures the regular multiple snapshots of the consistency of the distributed data; the data archiving operation user can flexibly archive the cold storage of the designated node data according to the snapshot data, and the method is simple and easy to use; the data recovery uplink operation enables cold-stored archived data to completely recover uplink, and the integrity of the data on the link is ensured;
the scheme has a more complete distributed security trust mechanism. The scheme forms snapshot voting transactions, each organization needs to vote for the transactions, whether voting results of the organizations meet voting strategies or not is judged when the transactions are executed, if yes, the safety trust requirements are met, and the transactions are executed; otherwise, the transaction is not executed;
the scheme ensures the integrity, tamper resistance and the like of the archived data. HASH operation of data is carried out on the archived data according to rules during snapshot, the archived data is stored as metadata content, when HASH operation of the previous transmission archived data before chain restoration of the cold-stored archived data is compared with the metadata of the snapshot stored before chain restoration, and safety problems such as integrity, anti-tampering property and the like of the archived data are guaranteed.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clear and fully described, embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It is to be understood that the specific embodiments described herein are merely illustrative of some embodiments of the invention and are not limiting of the invention, and that all other embodiments obtained by those of ordinary skill in the art without the exercise of inventive faculty are within the scope of the invention.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "inner", "outer", "top", "bottom", "side", "vertical", "horizontal", and the like indicate orientations or positional relationships based on orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element referred to must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," "third," "fourth," "fifth," and "sixth" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
For the purposes of simplicity and explanation, the principles of the embodiments are described by referring mainly to examples. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments. It will be apparent, however, to one skilled in the art that the embodiments may be practiced without these specific details. In some instances, well-known methods and structures have not been described in detail so as not to unnecessarily obscure the embodiments. In addition, all embodiments may be used in combination with each other.
Referring to fig. 1, the present invention provides a technical solution: the method for recovering the uplink scheme based on chain data snapshot, archiving and integrity comprises the following steps:
the method comprises the steps that a client side initiates a data snapshot request, voting (vote) transaction of a data snapshot of an appointed chain is established, the voting (vote) transaction comprises a chain ID, a transaction type (vote type) and voting strategy contents, the data snapshot transaction request is sent to an appointed consensus node in a block chain, the consensus node judges that the transaction type is the voting (vote) type, the transaction is broadcasted to all organizations on the chain, each organization votes, a voting result is returned to the consensus node, and after the consensus node successfully receives the voting results of all the organizations, a block is immediately and independently sent out and submitted to an account book; when each node receives the independent data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy or not; when the voting result in the transaction is checked to be in accordance with the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number division; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism;
the obtained information such as the snapshot list and the like can be combined with the information to initiate a data filing request of the appointed snapshot height of the appointed node according to the storage capacity of each node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting the corresponding local snapshot data and the corresponding block file data on the chain, and returning to the client for successful data filing; otherwise, reporting that the client end data archiving fails, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment; for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, such as an archiving request with a snapshot height of 1000, can be initiated according to the specific requirements of the current node; after receiving the archive request, the node 1 archives the snapshot data with the block height of 1000 or less into the cold storage outside the chain, such as the cheaper storage of a cloud database, a cloud file system, and the like; when the archived data is queried, query operation is carried out on the archived data stored in cold outside the chain through a command line tool cmc or a client side SDK, and the archived data is generally cold data with relatively low frequency access, so that the supported query demand capacity is relatively simple;
after the node is filed, the chain has no full and complete data, if the filing data uplink needs to be recovered, a filing data uplink recovery request of the node can be initiated, and cold-stored filing data uplink recovery is carried out; and (3) recovering the integrity of the uplink by archiving data, and protecting the integrity of the uplink by tamper resistance: calculating a HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with a HASH metadata value calculated by an original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or is tampered, not recovering the filing data, and reporting a client data filing recovery response error; when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of occupied _ num of the node 1 needs to be judged, and if block data block _ num of the synchronization start is smaller than or equal to occupied _ num, a process of restoring an uplink of the archived data by referencing the synchronous data in the archived data is carried out; otherwise, the data on the link of the node 1 is synchronized, and the new node is ensured to be synchronized to complete data.
A said data based on chain snapshot, file, complete system to resume upper link scheme, the said system is made up of data snapshot module, data file module and data recovery module;
the data snapshot module is used for initiating a data snapshot request by a client, creating voting (vote) transaction of a data snapshot of an appointed chain, sending the data snapshot transaction request to an appointed consensus node in a block chain, and immediately and independently sending out a block and submitting the block to an account book after the consensus node successfully receives all organized voting results; when each node receives a single data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism; the voting (vote) transaction of the data snapshot module comprises a link ID, a transaction type (vote type), and voting strategy contents; the consensus node judges that the transaction type is a vote (vote) type, broadcasts the transaction to all organizations on the chain, votes for each organization, and returns a voting result to the consensus node; when the voting result in the transaction is checked whether to meet the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number division; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported;
the data archiving module is used for acquiring information such as a snapshot list, and each node can initiate a data archiving request of a specified node for specifying the snapshot height according to the storage capacity of the node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting corresponding local snapshot data and corresponding block file data on the chain, and returning to the client that the data is successfully filed; otherwise, reporting that the client end data archiving fails, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment; for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, for example, an archiving request with a snapshot height of 1000, may be initiated according to the specific requirements of the current node; after receiving the archive request, the node 1 archives snapshot data with a block height of 1000 or less into off-chain cold storage, such as cheaper storage of a cloud database, a cloud file system and the like; when the archived data of the data archiving module is queried, query operation is carried out on the archived data stored in cold outside the chain through a command line tool cmc or a client side SDK, and the archived data is generally cold data with lower access frequency, so that the supported query requirement capacity is relatively simple;
the data recovery module is used for initiating a filing data recovery uplink request of the node and recovering the cold-stored filing data to uplink if the node does not have full complete data on the chain after filing; and the integrity of the uplink is restored by the archived data of the data restoration module, and the tamper resistance protection is as follows: calculating a HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with a HASH metadata value calculated by an original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or is tampered, not recovering the filing data, and reporting a client data filing recovery response error; when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of occupied _ num of the node 1 needs to be judged, and if block data block _ num of the synchronization start is smaller than or equal to occupied _ num, a process of restoring an uplink of the archived data by referencing the synchronous data in the archived data is carried out; otherwise, the data on the link of the node 1 is synchronized, and the new node is ensured to be synchronized to complete data.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (10)
1. The method for chain-based data snapshot, archiving and complete recovery of the uplink scheme comprises the step of recovering the system, and is characterized in that: the method for snapshotting, archiving and completely recovering the uplink scheme based on the chain comprises the following steps:
the client initiates a data snapshot request, creates a voting (vote) transaction of a data snapshot of an appointed chain, sends the data snapshot transaction request to an appointed consensus node in the block chain, and immediately and independently sends out blocks and submits the blocks to an account book after the consensus node successfully receives all organized voting results; when each node receives a single data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism;
the obtained information such as the snapshot list and the like can be combined with the information to initiate a data filing request of the appointed snapshot height of the appointed node according to the storage capacity of each node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting the corresponding local snapshot data and the corresponding block file data on the chain, and returning to the client for successful data filing; otherwise, reporting the client data archiving failure, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment;
after the node is filed, the chain has no full complete data, if the filed data uplink needs to be recovered, the filed data recovery uplink request of the node can be initiated, and the cold-stored filed data can be recovered to uplink.
2. The method for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 1, wherein: voting (vote) transactions include a link ID, a transaction type (vote type), voting policy content;
the consensus node judges that the transaction type is a vote (vote) type, broadcasts the transaction to all organizations on the chain, votes for each organization, and returns a voting result to the consensus node;
when the voting result in the transaction is checked to be in accordance with the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported.
3. The method for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 2, wherein: when a request for data archiving is made,
for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, such as an archiving request with a snapshot height of 1000, can be initiated according to the specific requirements of the current node;
after receiving the archive request, the node 1 archives the snapshot data with the block height of 1000 or less to the off-chain cold storage, such as a cloud database, a cloud file system, or the like, which is cheaper to store.
4. The method for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 1, wherein: when the archived data is queried, query operation is performed on the archived data stored in cold outside the chain through a command line tool cmc or a client side SDK, and the archived data is generally cold data with relatively low frequency access, so that the supported query demand capability is relatively simple.
5. The method for chain-based data snapshot, archiving, and full recovery of an uplink scheme of claim 1, wherein: and (3) recovering the integrity of the uplink by archiving data, and protecting the integrity of the uplink by tamper resistance: calculating a HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with a HASH metadata value calculated by an original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or is tampered, not recovering the filing data, and reporting a client data filing recovery response error;
when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of occupied _ num of the node 1 needs to be judged, and if block data block _ num of the synchronization start is smaller than or equal to occupied _ num, a process of restoring an uplink of the archived data by referencing the synchronous data in the archived data is carried out; otherwise, the data on the link of the node 1 is synchronized, and the new node is ensured to be synchronized to complete data.
6. A system for chain-based data snapshot, archiving, and full recovery uplink scheme according to any of the preceding claims 1-5, wherein: the system consists of a data snapshot module, a data archiving module and a data recovery module;
the data snapshot module is used for initiating a data snapshot request by a client, creating voting (vote) transaction of a data snapshot of an appointed chain, sending the data snapshot transaction request to an appointed consensus node in a block chain, and immediately and independently sending out a block and submitting the block to an account book after the consensus node successfully receives all organized voting results; when each node receives a single data snapshot block at the same height, checking whether the voting result in the transaction meets the voting strategy; after the data snapshot is finished, reporting snapshot list information to a client through an asynchronous event monitoring mechanism;
the data filing module is used for acquiring information such as a snapshot list, and each node can initiate a data filing request of a designated node for designating the snapshot height according to the storage capacity of the node; after the data is successfully filed, refreshing and recording the filed block height archive _ num to the chain, deleting corresponding local snapshot data and corresponding block file data on the chain, and returning to the client that the data is successfully filed; otherwise, reporting that the client end data archiving fails, and finishing the archiving function from the data on the chain to the cold storage outside the chain at the moment;
and the data recovery module is used for initiating a filing data recovery uplink request of the node and recovering the cold-stored filing data to uplink if the node does not have full complete data on the chain after filing.
7. The system for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 6, wherein: the voting (vote) transaction of the data snapshot module comprises a link ID, a transaction type (vote type) and voting strategy contents;
the consensus node judges that the transaction type is a vote (vote) type, broadcasts the transaction to all organizations on the chain, votes for each organization, and returns a voting result to the consensus node;
when the voting result in the transaction is checked whether to meet the voting strategy, if yes, the data snapshot operation of the latest block number is executed at the same block height, and the snapshot data comprises: block file data and latest state data, and calculating a HASH value of snapshot data as metadata, and storing the data in a snapshot directory named by block number; otherwise, the data snapshot operation is not executed, and the data snapshot error of the client is reported.
8. The system for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 7, wherein: upon a data archiving request by the data archiving module,
for example, the snapshot list information is {100,300,600,1000, 1500. }, when the storage capacity pressure of the node 1 is large or an alarm is given, a data archiving request with a specified snapshot height, such as an archiving request with a snapshot height of 1000, can be initiated according to the specific requirements of the current node;
after receiving the archive request, the node 1 archives the snapshot data with the block height of 1000 or less into the off-link cold storage, such as a cloud database, a cloud file system, or the like, which is cheaper to store.
9. The system for chain-based data snapshot, archiving, and full recovery uplink scheme according to claim 8, wherein: when the archived data of the data archiving module is queried, query operation is performed on the archived data stored in cold outside the chain through a command line tool cmc or a client side SDK, and the archived data is generally cold data with relatively low frequency access, so that the supported query demand capacity is relatively simple.
10. The system for chain-based data snapshot, archiving, and full recovery uplink scheme of claim 9, wherein: the integrity of the uplink is restored by the archived data of the data restoration module, and the tamper resistance protection: calculating the HASH value of the cold-stored filing recovery data according to a safety rule, comparing the HASH value with the HASH metadata value calculated by the original data snapshot, if the HASH metadata values are the same, indicating that the data is complete and is not tampered, otherwise, indicating that the data is incomplete or tampered, not recovering the filing data, and reporting a client data filing recovery response error;
when a new node joins a chain through a node 1, data needs to be synchronized from the node 1, at this time, the value of occupied _ num of the node 1 needs to be judged, and if block data block _ num of the synchronization start is smaller than or equal to occupied _ num, a process of restoring an uplink of the archived data by referencing the synchronous data in the archived data is carried out; otherwise, the data on the link of the node 1 is synchronized, and the new node is ensured to be synchronized to complete data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211068751.8A CN115481417A (en) | 2022-09-02 | 2022-09-02 | Method for data snapshot, filing and complete recovery of uplink scheme based on chain |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211068751.8A CN115481417A (en) | 2022-09-02 | 2022-09-02 | Method for data snapshot, filing and complete recovery of uplink scheme based on chain |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115481417A true CN115481417A (en) | 2022-12-16 |
Family
ID=84422056
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211068751.8A Pending CN115481417A (en) | 2022-09-02 | 2022-09-02 | Method for data snapshot, filing and complete recovery of uplink scheme based on chain |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115481417A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116755640A (en) * | 2023-08-21 | 2023-09-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium of alliance chain |
| CN117806876A (en) * | 2024-03-01 | 2024-04-02 | 北京邮电大学 | Marine satellite Internet of things blockchain lightweight storage and recovery method and device |
-
2022
- 2022-09-02 CN CN202211068751.8A patent/CN115481417A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116755640A (en) * | 2023-08-21 | 2023-09-15 | 腾讯科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium of alliance chain |
| CN116755640B (en) * | 2023-08-21 | 2024-02-09 | 腾讯科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium of alliance chain |
| CN117806876A (en) * | 2024-03-01 | 2024-04-02 | 北京邮电大学 | Marine satellite Internet of things blockchain lightweight storage and recovery method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115481417A (en) | Method for data snapshot, filing and complete recovery of uplink scheme based on chain | |
| US10860547B2 (en) | Data mobility, accessibility, and consistency in a data storage system | |
| RU2449358C1 (en) | Distributed file system and data block consistency managing method thereof | |
| EP2521037B1 (en) | Geographically distributed clusters | |
| JP5254611B2 (en) | Metadata management for fixed content distributed data storage | |
| EP0722236B1 (en) | System and method for fault tolerant key management | |
| JP4461147B2 (en) | Cluster database using remote data mirroring | |
| US8762342B1 (en) | Method of inserting a validated time-image on the primary CDP subsystem in a continuous data protection and replication (CDP/R) subsystem | |
| US20050240564A1 (en) | System and method for state preservation in a stretch cluster | |
| WO2001077835A1 (en) | Data backup/recovery system | |
| US20070294310A1 (en) | Method and apparatus for storing and recovering fixed content | |
| WO2021226905A1 (en) | Data storage method and system, and storage medium | |
| CN102955720A (en) | Method for improving stability of EXT (extended) file system | |
| CN101809558A (en) | System and method for remote asynchronous data replication | |
| CN112835743B (en) | Distributed account book data storage optimization method and device, electronic equipment and medium | |
| CN110163009B (en) | Method and device for safety verification and repair of HDFS storage platform | |
| WO2021208400A1 (en) | Backup recovery system and method for modern application | |
| CN112380067B (en) | Metadata-based big data backup system and method in Hadoop environment | |
| CN106254161B (en) | HDFS-based node failure rapid detection and recovery method and system | |
| CN110348826A (en) | Strange land disaster recovery method, system, equipment and readable storage medium storing program for executing mostly living | |
| CN113986143A (en) | Block chain log storage-oriented high-reliability low-overhead data storage method | |
| US11042454B1 (en) | Restoration of a data source | |
| CN111404737B (en) | Disaster recovery processing method and related device | |
| EP3896571B1 (en) | Data backup method, apparatus and system | |
| JPH03256146A (en) | Parallel operation type data base management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |