CN117971834A

CN117971834A - Collaborative storage method and system based on blockchain

Info

Publication number: CN117971834A
Application number: CN202410190216.2A
Authority: CN
Inventors: 兰春嘉; 王磊; 朱名生; 宋泽浩
Original assignee: Shanghai Lingshuzhonghe Information Technology Co ltd
Current assignee: Shanghai Lingshuzhonghe Information Technology Co ltd
Priority date: 2024-02-20
Filing date: 2024-02-20
Publication date: 2024-05-03

Abstract

The disclosure provides a collaborative storage method and a collaborative storage system based on a blockchain, which relate to the technical field of blockchains, wherein the method comprises the following steps: and constructing a chain-up and chain-down storage module, establishing data operation association, then performing storage decomposition, establishing a first communication encryption protocol, transmitting a data set to the chain-up storage module, transmitting the chain-down data to the storage module through a second communication encryption protocol, then performing storage triggering monitoring and synchronous updating verification, and finally performing synchronicity early warning. According to the method and the device, the problem that the prior art cannot effectively classify and store according to the characteristics and the importance of data, so that the storage efficiency is greatly reduced, meanwhile, the storage state cannot be monitored in real time, the data loss is possibly caused, the efficiency of data processing and the integrity of the data are further affected, the problems of storage, transmission, synchronization and safety of the data in a block chain are solved, the consistency of the data under the chain upper chain is ensured, and the performance, stability and safety of the block chain are improved.

Description

Collaborative storage method and system based on blockchain

Technical Field

The disclosure relates to the technical field of blockchains, in particular to a collaborative storage method and a collaborative storage system based on blockchains.

Background

The blockchain technology has been rapidly applied and developed in various fields since birth by virtue of its characteristics of decentralization, non-falsification, transparency and high security. With the explosive growth of data volumes, traditional centralized storage approaches face increasing challenges. Security and privacy protection of data is an important issue. Such as security and privacy protection of data, increased storage costs, data redundancy and synchronization problems, and the like.

Currently, the existing centralized data storage manner is easy to be subjected to hacking and internal leakage risks due to the centralized characteristic of the centralized data storage manner, and in order to ensure the redundancy and consistency of data, a complex backup and synchronization mechanism needs to be established, which increases the cost and complexity of data storage.

In summary, the existing technology cannot effectively classify and store according to the characteristics and importance of the data, which results in reduced storage efficiency, and meanwhile, cannot monitor the storage state in real time, which may result in data loss, thereby further affecting the efficiency of data processing and data integrity.

Disclosure of Invention

The method solves the problems that the prior art cannot effectively store the data in a classified mode according to the characteristics and the importance of the data, so that the storage efficiency is reduced, and meanwhile, the storage state cannot be monitored in real time, the data is possibly lost, and the data processing efficiency and the data integrity are further affected.

In view of the above, the present disclosure provides a collaborative storage method and system based on blockchain.

In a first aspect, the present disclosure provides a blockchain-based collaborative storage, the method implemented by a blockchain-based collaborative storage system, wherein the method includes: constructing an on-chain data storage module and an off-chain data storage module of a blockchain, and establishing data operation association of the on-chain data storage module and the off-chain data storage module; based on the data operation association, carrying out storage decomposition on the data to be stored to obtain a uplink storage data set and a downlink storage data set; a data transmission port for collecting data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port; transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol, and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol; the on-chain data storage module and the off-chain data storage module are subjected to storage triggering monitoring through an event trigger to obtain an on-chain storage triggering state and an off-chain storage triggering state; performing synchronous update verification of the data on the upper chain and the lower chain based on the on-chain storage trigger state and the off-chain storage trigger state to obtain a synchronous verification result; and carrying out on-link and off-link storage synchronicity early warning based on the synchronicity checking result.

In a second aspect, the present disclosure further provides a blockchain-based collaborative storage system for performing a blockchain-based collaborative storage method according to the first aspect, wherein the system includes: the data operation association establishing module is used for establishing an on-chain data storage module and an off-chain data storage module of the block chain and establishing data operation association of the on-chain data storage module and the off-chain data storage module; the storage decomposition module is used for carrying out storage decomposition on the data to be stored based on the data operation association to obtain a uplink storage data set and a downlink storage data set; the transmission port acquisition module is used for acquiring a data transmission port of data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port; the data set transmission module is used for transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol; the trigger state acquisition module is used for carrying out storage trigger monitoring on the on-chain data storage module and the off-chain data storage module through event triggers to obtain an on-chain storage trigger state and an off-chain storage trigger state; the synchronous verification result acquisition module is used for carrying out synchronous updating verification on the data under the chain on the basis of the on-chain storage trigger state and the under-chain storage trigger state to obtain a synchronous verification result; and the synchronicity early warning module is used for carrying out on-link and off-link storage synchronicity early warning based on the synchronicity verification result.

In a third aspect, the present disclosure also provides an electronic device, including:

At least one processor;

a memory communicatively coupled to the at least one processor;

wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of the first aspects above.

In a fourth aspect, a computer readable storage medium having stored thereon a computer program which, when executed, implements the steps of the method of any of the first aspects above.

One or more technical solutions provided in the present disclosure have at least the following technical effects or advantages:

1. Constructing an on-chain data storage module and an off-chain data storage module of a blockchain, and establishing data operation association of the on-chain data storage module and the off-chain data storage module; based on the data operation association, carrying out storage decomposition on the data to be stored to obtain a uplink storage data set and a downlink storage data set; a data transmission port for collecting data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port; transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol, and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol; the on-chain data storage module and the off-chain data storage module are subjected to storage triggering monitoring through an event trigger to obtain an on-chain storage triggering state and an off-chain storage triggering state; performing synchronous update verification of the data on the upper chain and the lower chain based on the on-chain storage trigger state and the off-chain storage trigger state to obtain a synchronous verification result; and carrying out on-link and off-link storage synchronicity early warning based on the synchronicity checking result. The performance of the blockchain and the consistency of data are improved.

2. The state of the data storage modules on the chain and the data storage modules under the chain can be monitored in real time through the event trigger, so that the timely storage of the data is ensured, and the problem of data loss or delay storage is solved. According to the characteristics and importance of the data, the data are decomposed into an uplink storage data set and a downlink storage data set, and different communication encryption protocols are used for transmission, so that the efficiency and the safety of data processing are improved.

3. The collaborative storage method based on the block chain is adopted to replace the existing method, so that the problems that the prior art cannot effectively classify and store according to the characteristics and the importance of data, the storage efficiency is greatly reduced, meanwhile, the storage state cannot be monitored in real time, the data is possibly lost, the efficiency of data processing and the integrity of the data are further affected, the problems of storage, transmission, synchronization and safety of the data in the block chain are solved, the consistency of the data under the chain is ensured, and the performance, stability and safety of the block chain are improved.

The foregoing description is merely an overview of the technical solutions of the present disclosure, and may be implemented according to the content of the specification in order to make the technical means of the present disclosure more clearly understood, and in order to make the above and other objects, features and advantages of the present disclosure more clearly understood, the following specific embodiments of the present disclosure are specifically described. It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

In order to more clearly illustrate the application or the technical solutions of the prior art, the following brief description will be given of the drawings used in the description of the embodiments or the prior art, it being obvious that the drawings in the description below are only exemplary and that other drawings can be obtained from the drawings provided without the inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a collaborative storage method based on blockchain in accordance with the present disclosure;

FIG. 2 is a flow chart of feasibility determination in a blockchain-based collaborative storage according to the present disclosure;

FIG. 3 is a schematic diagram of a block chain based collaborative storage system according to the present disclosure;

Fig. 4 is a schematic structural diagram of an exemplary electronic device of the present disclosure.

Reference numerals illustrate: the system comprises a data operation association establishing module 11, a storage decomposing module 12, a transmission port collecting module 13, a data set transmitting module 14, a trigger state acquiring module 15, a synchronous check result acquiring module 16, a synchronous early warning module 17, a processor 302, a memory 304, a receiver 301, a bus interface 305, a transmitter 303 and a bus 300.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The method solves the problems that the prior art cannot effectively store the data in a classified mode according to the characteristics and the importance of the data, so that the storage efficiency is greatly reduced, meanwhile, the storage state cannot be monitored in real time, the data is possibly lost, the efficiency of data processing and the integrity of the data are further affected, the problems of storage, transmission, synchronization and safety of the data in a block chain are solved, the consistency of the data under the chain is ensured, and the performance, stability and safety of the block chain are improved.

In the following, the technical solutions in the present disclosure will be clearly and completely described with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present disclosure, but not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited by the example embodiments described herein. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments of the present disclosure without making any inventive effort, are intended to be within the scope of the present disclosure. It should be further noted that, for convenience of description, only a part, but not all, of the drawings related to the present disclosure are shown.

Example 1

Referring to fig. 1, the disclosure provides a collaborative storage method based on a blockchain, wherein the method is applied to a collaborative storage system based on a blockchain, and the method specifically includes the following steps:

Step one: constructing an on-chain data storage module and an off-chain data storage module of a blockchain, and establishing data operation association of the on-chain data storage module and the off-chain data storage module;

Specifically, firstly, the requirements and targets of the on-chain and off-chain data storage modules are explicitly built, such as the type, the size, the storage period, the security requirement and the like of data, and the structure, the size and the number of the data and the storage mode of the data such as key value storage, a NoSQL database, a relational database and the like are designed according to the data type. On-chain data storage module: the on-chain data storage module is part of the direct interaction with the blockchain. It is responsible for storing the data that is written to the blockchain. Such data is typically transaction data, metadata, or other information related to the activity on the chain. The link-down data storage module: the under-chain data storage modules refer to those portions of the data storage that do not interact directly with the blockchain. Such data may be a large number of files, pictures, video, or other large data that is not suitable for direct storage on the blockchain. The association of the data under the chain and the data on the chain is established, so that when the data needs to be verified or retrieved, the related information can be quickly and accurately found. Establishing data operation association: an index or metadata is first created for the data under the chain, which is stored on the chain. Thus, even if the data under the chain changes, the corresponding data can be quickly found through the index. Ensuring the integrity and authenticity of the data under the chain. When the under-chain data needs to be verified, it can be verified by the on-chain data. An automatic or manual synchronization mechanism is established to ensure that the on-and off-link data remain consistent. When the data under the chain changes, the corresponding data on the chain should also be updated. With the on-chain data, access control policies can be defined and enforced, ensuring that only authorized users can access or modify the corresponding under-chain data.

Step two: based on the data operation association, carrying out storage decomposition on the data to be stored to obtain a uplink storage data set and a downlink storage data set;

Specifically, the data to be stored is stored and decomposed based on the data operation association, so that a uplink storage data set and a downlink storage data set can be obtained. For example: the final result data after the transaction processing with higher complexity and frequency and all data of the transaction with lower complexity and frequency are stored on the chain, and only the data involved in the transaction processing with higher complexity and frequency are stored under the chain. The uplink stores the data set: the uplink store data sets refer to those data sets that need to be written to the blockchain. These data are typically transaction data, metadata or other information directly related to the activity on the chain. The downlink stores the data set: the downlink storage data sets refer to those data sets that are not suitable for direct writing to the blockchain, but are closely related to the on-chain data. Such data may include a large amount of files, pictures, video, or other large data. And carrying out storage decomposition on the data to be stored to obtain an uplink storage data set and a downlink storage data set, thereby being beneficial to better organizing and utilizing storage resources. The decomposition mode can ensure the integrity and the safety of the data, simultaneously meet the storage requirements of different types of data, and improve the efficiency of the whole storage system.

Step three: a data transmission port for collecting data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port;

In particular, when constructing a blockchain-based collaborative storage system, it is important to ensure the security of data transfer and storage. Data transmission port for collecting data to be stored: the data to be stored needs to be collected and its data transmission port determined. This may be an output port of a hardware device, software application, or other data source for sending data to the storage system. The data receiving ports of the on-chain data storage module and the off-chain data storage module are determined according to the design of the system. The ports are used to receive data sent from the data transmission ports. A communication encryption protocol is established from the data transmission port to a first data receiving port of the in-chain data storage module. Such encryption protocols should employ a strong encryption algorithm and ensure confidentiality, integrity and availability of the data, e.g., strong encryption algorithms AES, RSA, SHA-256, etc. may be employed to ensure security of the data during transmission, where the data is encrypted and then decrypted at the first data receiving port. Likewise, a communication encryption protocol is established from the data transmission port to a second data reception port of the in-chain data storage module. Such encryption protocols should also be strong encryption algorithms to ensure confidentiality and integrity of the data, e.g., strong encryption algorithms AES, RSA, SHA-256, etc. may be employed to ensure security of the data during transmission, where the data is encrypted and then decrypted at the second data receiving port. By establishing the first communication encryption protocol and the second communication encryption protocol, it can be ensured that the data to be stored is not stolen or tampered with during transmission. Meanwhile, the encryption mode can also provide data integrity check to ensure that the received data is consistent with the transmitted data.

Step four: transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol, and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol;

Specifically, the uplink data set to be stored is encrypted and encapsulated in a format prescribed by an encryption protocol. And sending the packaged uplink data set to the on-link data storage module through the data transmission port. And carrying out corresponding encryption processing on the downlink storage data set, and packaging the downlink storage data set in a format specified by a second communication encryption protocol. And sending the packaged downlink data set to the downlink data storage module through the data transmission port. By means of the layered encryption transmission mode, the safety and the integrity of the uplink and downlink storage data sets in the transmission process can be ensured. Meanwhile, different communication encryption protocols can be customized according to actual requirements, so that more flexible and safe storage services are provided.

Step five: the on-chain data storage module and the off-chain data storage module are subjected to storage triggering monitoring through an event trigger to obtain an on-chain storage triggering state and an off-chain storage triggering state;

In particular, to achieve efficient storage management, storage trigger monitoring may be performed on the on-chain and off-chain data storage modules by event triggers. Such a trigger monitor mechanism may automatically trigger a storage operation when the amount of data reaches a certain threshold, a change in data occurs, or other specific event occurs. An event trigger is a monitoring system that can monitor events or conditions associated with a data store. When the trigger detects that a certain event or condition meets a preset condition, the trigger performs a corresponding operation, such as triggering a storage process. The on-chain storage trigger state refers to a storage state automatically triggered by an event trigger when an on-chain data storage module meets certain conditions. These conditions may include the amount of data exceeding a threshold, the data being changed, etc. By monitoring the activity and changes of the data on the chain, the storage and backup of the data on the chain can be managed more intelligently. The under-chain storage triggering state refers to a storage state automatically triggered by an event trigger when the under-chain data storage module meets certain conditions. These conditions may include that the file size exceeds a threshold, that the file content has changed, etc. By monitoring the activity and changes of the data under the chain, the storage and backup of large amounts of unstructured data can be managed more efficiently. The on-chain data storage module and the off-chain data storage module are stored and triggered to be monitored through the event trigger, so that more intelligent and efficient storage management can be realized. The automatic triggering mechanism can reduce the need of manual intervention and improve the response speed and stability of the storage system.

Step six: performing synchronous update verification of the data on the upper chain and the lower chain based on the on-chain storage trigger state and the off-chain storage trigger state to obtain a synchronous verification result;

Specifically, the on-link storage trigger state and the off-link storage trigger state are used for synchronous update and verification of the data under the on-link, so that the integrity and consistency of the data can be ensured. Determining a triggering condition: and determining when to perform data synchronous update verification according to the on-link storage trigger state and the off-link storage trigger state. This may be based on time intervals, data volume changes, or other relevant factors. For example, the data synchronization status may be checked once an hour. If the data synchronization is not performed for more than one hour, triggering the verification update. Or data volume change: when the data volume change in the data storage module under the chain exceeds a certain threshold value, triggering the data synchronous update verification. For example, when the newly added data amount exceeds 1000 records, one data synchronization update check is performed. And if the data quantity in the on-chain data storage module exceeds a preset upper limit and the data synchronous update verification is not carried out in the last hour, triggering the data synchronous update. And if the data quantity in the under-chain data storage module exceeds a preset upper limit and the data synchronous update verification is not carried out in the last hour, triggering the data synchronous update. The version, content or metadata of the data on the chain and the data under the chain are compared to detect whether there is a discrepancy or inconsistency. Comparing version information of the data on the chain with version information of the data under the chain to check whether the version inconsistency exists. This may be done by comparing version numbers, time stamps or other identifiers. And comparing the specific contents of the data on the chain with the specific contents of the data under the chain one by one, and checking whether the data are different or inconsistent. This may involve comparing field values, data structures or data relationships, etc. Metadata information of the data on the link and the data off the link are compared, such as data source, creation time, modification time, and the like. These metadata are checked for consistency to determine the integrity and accuracy of the data. In the case of data inconsistencies, conflicts need to be resolved. This may include determining which version of the data is correct, merging different versions of the data, or performing other corresponding operations. And analyzing whether the data is inconsistent, missing or damaged according to the result of the data synchronous update verification. And executing corresponding operation according to the synchronous check result. For example, if the verification fails, it may be necessary to resynchronize the data or repair the corrupted data. If the data is inconsistent, conflicts may need to be resolved or data integration may occur. By the trigger state-based synchronous update and verification mechanism for the data under the chain upper link, the integrity and consistency of the data can be ensured. Meanwhile, the mechanism can also timely find and process the problem of inconsistent or conflict of data, and improves the reliability and stability of the whole storage system. In addition, according to actual requirements and service scenes, trigger conditions and a verification mechanism can be flexibly configured so as to meet data storage and management requirements in different scenes.

Step seven: and carrying out on-link and off-link storage synchronicity early warning based on the synchronicity checking result.

Specifically, the early warning of the storage synchronicity of the link up and down is performed based on the synchronicity checking result, so as to timely find and process the problem of inconsistent or conflict of the data, and ensure the integrity and consistency of the data. Determining early warning conditions: and determining corresponding early warning conditions according to the synchronous verification result. For example, if there is a large discrepancy between the on-chain data and the off-chain data, a data version conflict, or other inconsistency problem, an early warning may be triggered. And automatically generating early warning information according to the early warning conditions. The pre-warning information may include the type, extent, impact, and other relevant information of the data inconsistency. And sending the early warning information to related personnel or systems. This may be accomplished by email, text messaging, system notification, etc. to inform relevant personnel of the process in time. And after receiving the early warning information, the related personnel perform corresponding processing according to the early warning content. This may include checking for data consistency, resolving conflicts, resynchronizing the data, etc., to ensure data integrity and consistency. Through the on-chain and off-chain storage synchronicity early warning mechanism based on the synchronicity checking result, the problem of inconsistent or conflict of data can be timely found and processed, and data damage or loss is avoided. Meanwhile, the early warning mechanism can also improve the reliability and stability of the whole storage system and ensure the integrity and consistency of data. In addition, according to actual requirements and service scenes, the early warning conditions and response processing can be flexibly configured so as to meet the data storage and management requirements in different scenes.

Further, step one of the present disclosure includes:

Obtaining a transaction record database of the blockchain, wherein the transaction record database contains various transaction types performed in history;

extracting transaction calculation complexity and transaction frequency of transaction types in the transaction record database;

And setting a data storage rule under the uplink of the chain, carrying out data storage type analysis under the uplink of the chain by combining the transaction calculation complexity and the transaction frequency, and outputting the data operation association.

Specifically, a blockchain transaction record database is obtained: and obtaining a transaction record database on the blockchain through connection with the blockchain network. Through API interfaces, node software, or other means. And analyzing information such as various transaction types, transaction time, transaction participants, transaction data and the like from a transaction record database. Calculating transaction calculation complexity: and evaluating the computational complexity of each transaction type according to the analyzed transaction data. This may be evaluated based on the amount of data in the transaction, the intelligent contract logic involved, etc., such as the amount of data: to how much transaction data, such as the input/output number of transactions, the byte size of the transaction data, etc. Intelligent contract logic: the complexity of the logic and computing operations involved in the smart contracts involved in the transactions. For example, performing certain operations may require a significant amount of computing resources. External dependencies: whether the transaction depends on external data or events, such as market price, time stamps, etc. Analyzing transaction frequency: the transaction frequency, i.e., the number of transaction types over a period of time, for each transaction type is counted. Setting a data storage rule under a chain upper link: determining a storage rule: and setting rules of data storage under the uplink of the chain according to service requirements and data characteristics. This may include criteria such as data type, data size, data access frequency, etc. Data classification and evaluation: and classifying and evaluating the data according to the storage rules and the extracted information of transaction calculation complexity, transaction frequency and the like, and determining which data are suitable for being stored on the chain and which data are suitable for being stored under the chain. For example: for data requiring frequent access or high computational complexity, storage on the chain may be considered, and for data of infrequent access or low computational complexity, storage off the chain may be considered. Or complex calculations and high frequency transactions are transferred to under-chain, the on-chain database only storing the final results. And outputting a data operation association result according to the analysis. Through the steps, the analysis of the data storage types under the chain uplink can be performed based on the transaction record database of the blockchain, the data operation association is output, and guidance and support are provided for actual data storage management.

Further, the present disclosure further includes:

the data storage rules under the uplink chain comprise a transaction complexity threshold, a transaction frequency threshold and a preset transaction data state;

Extracting a first set of transaction types in the transaction record database based on the transaction computational complexity being greater than or equal to the transaction complexity threshold or the transaction frequency being greater than the transaction frequency threshold;

Performing transaction data processing state segmentation on the first transaction type set based on the preset transaction data state to obtain an on-chain transaction type, on-chain transaction state data and off-chain transaction state data;

the data operation association is established with the on-chain transaction type, on-chain transaction status data, and off-chain transaction status data.

Specifically, transaction complexity threshold: a transaction complexity threshold is set based on historical data and business requirements. Transactions with a complexity exceeding this threshold will be considered as high complexity transactions. Transaction frequency threshold: a threshold of transaction frequency is set for distinguishing between high frequency and low frequency transactions. Historical transaction data may preferably be collected over a period of time, including various transaction types and their computational complexity. Such data may be obtained by parsing transaction records on a blockchain. And analyzing the calculation complexity of the historical transaction data to know the distribution condition of various transaction types. A histogram may be drawn or other visualization tool may be used to show the distribution of complexity. A reasonable transaction complexity threshold range is determined based on the analysis of business needs and historical data. For example, if a transaction that is deemed to be more complex than average in business may be considered a high-complexity transaction, then the threshold may be set to the average of historical transaction complexity. Outliers and edge conditions also need to be considered when setting the threshold. These may be due to errors, malicious behavior, or other special circumstances. These outliers may be specially handled or excluded from the threshold range depending on the traffic demand. Once the transaction complexity threshold is determined, it may be applied to real-time or future transaction data. For transactions with computational complexity exceeding the threshold, corresponding measures may be taken, such as additional verification, notifying relevant personnel or rejecting the transaction, etc. Over time, business requirements and transaction data may change. Therefore, it is recommended to periodically re-evaluate and adjust the transaction complexity threshold to ensure its consistency and effectiveness with the actual situation. Presetting transaction data states: some preset transaction data states are defined, such as active, dormant, historical, etc. Extracting all transaction types meeting the following conditions from a transaction record database: the transaction computational complexity is greater than or equal to the transaction complexity threshold. The transaction frequency is greater than the transaction frequency threshold. Transaction data processing state segmentation: and further processing and classifying the extracted first transaction type set based on the preset transaction data state. Type of transaction on chain: these transactions are considered more suitable for storage on a chain due to their high complexity or high frequency. On-chain transaction status data: the final result data after processing the transaction with higher complexity and frequency and all data of the transaction with lower complexity and frequency are stored. Link transaction status data: data involved in transaction processing is relatively high in complexity and frequency. Based on the above classification, an explicit data manipulation association is established for each transaction type, on-chain transaction state data, and off-chain transaction state data. When a high complexity or high frequency transaction occurs: the under-chain system first performs preprocessing and preliminary calculation of transaction data. Once the transaction is validated and processed to some intermediate state, the relevant data (e.g., transaction hashes, partial calculations, etc.) may be transferred to chain storage as transaction state data. When the transaction is completed and eventually validated, its final state data (e.g., transaction validation state, timestamp, etc.) will also be stored on the chain. The on-chain storage ensures the non-tamper property and transparency of the transaction state, while the off-chain storage provides greater flexibility and processing capacity, and the method based on the on-chain and off-chain data storage rule can more reasonably manage and utilize the data on the blockchain to meet the requirements of different business scenes.

Further, as shown in fig. 2, the present disclosure further includes:

Acquiring a uplink data transmission channel based on the first communication encryption protocol, and acquiring a parallel data transmission quantity threshold of the uplink data transmission channel;

extracting real-time data quantity to be uplink of the uplink storage data set;

carrying out transmission priority identification on the uplink storage data set to generate a data priority sequence;

And judging the feasibility of the data synchronous transmission based on the real-time data quantity to be uplink and the parallel data transmission quantity threshold, and if the judging result is negative, carrying out data uplink storage according to the parallel data transmission quantity threshold and the data priority sequence through the first communication encryption protocol.

Specifically, a uplink data transmission channel is acquired: and establishing connection with the block chain network to acquire an uplink data transmission channel based on the first communication encryption protocol. The connection to the blockchain network may be established using an API or SDK of the blockchain network. Including configuring network connection parameters such as node address, port number, etc. Implementing a first communications encryption protocol: encryption and decryption logic for the data transfer port is implemented in accordance with the selected first communication encryption protocol. Acquiring a parallel data transmission amount threshold value: and acquiring the parallel data transmission quantity threshold value from the uplink data transmission channel through the block chain network API. The threshold is known to ensure that the endurance of the channel is not exceeded during data chaining, avoiding data loss or transmission failure. Extracting real-time data quantity to be uplink: this may extract the amount of data to be uplinked in real time from the storage system by querying the system state. The current data volume to be uplink is known for better subsequent data synchronization and transmission management. And carrying out transmission priority identification on the uplink storage data set. A priority sequence is generated for the data in the data set based on the urgency, importance, or other relevant criteria of the data. And (3) judging the feasibility of uplink storage of data synchronous transmission: and carrying out feasibility judgment of data synchronous transmission based on the real-time data quantity to be uplink and the parallel data transmission quantity threshold value. If the current data amount exceeds the transmission capacity of the channel, corresponding processing such as batch transmission, channel capacity adjustment or better waiting for transmission is needed. And (3) data uplink storage: and encrypting the data by using a first communication encryption protocol to ensure that the data is not stolen or tampered in the uplink process. Data segmentation: if the data quantity to be uplink is large, the data can be divided according to the requirement, and the uplink operation is carried out in batches. And (3) state monitoring: in the data uplink process, the state and the safety of data transmission are continuously monitored, and the integrity and the safety of data transmission are ensured. Including checking the hash value, checksum, etc. of the data. Exception handling: if an abnormal condition occurs, such as data loss, transmission interruption or verification failure, corresponding processing, such as retransmission, data repair or fault removal, is required. And (3) judging the feasibility of downlink storage of data synchronous transmission: and carrying out feasibility judgment of data synchronous transmission based on the real-time data quantity to be uplink and the parallel data transmission quantity threshold value. If the current data amount exceeds the transmission capacity of the channel, corresponding processing such as batch transmission, channel capacity adjustment or better waiting for transmission is needed. And judging the feasibility of data synchronous transmission: and carrying out feasibility judgment of data synchronous transmission based on the real-time data quantity to be uplink and the parallel data transmission quantity threshold value. If the current data amount exceeds the transmission capacity of the channel, corresponding processing such as batch transmission, channel capacity adjustment or better waiting for transmission is needed. And (3) data uplink storage: and encrypting the data by using a second communication encryption protocol to ensure that the data is not stolen or tampered in the uplink process. Data segmentation: if the data quantity to be uplink is large, the data can be divided according to the requirement, and the uplink operation is carried out in batches. And (3) state monitoring: in the data uplink process, the state and the safety of data transmission are continuously monitored, and the integrity and the safety of data transmission are ensured. Including checking the hash value, checksum, etc. of the data. Exception handling: if an abnormal condition occurs, such as data loss, transmission interruption or verification failure, corresponding processing, such as retransmission, data repair or fault removal, is required. Through the steps, uplink storage management of the data can be effectively carried out, safe transmission and storage of the data are ensured, and meanwhile, the efficiency and response speed of a storage system are improved.

Further, the present disclosure further includes:

carrying out transaction processing association analysis on the uplink storage data set, and establishing a transaction processing association data set;

Carrying out highest priority identification on the transaction processing association data set;

And generating the data priority sequence according to the highest priority identification.

Specifically, the uplink stored data set is subjected to in-depth analysis, and the transaction type, the transaction participant, the transaction data and the like related in the data set are identified. The association between the various transaction types is analyzed, for example, whether dependencies, concurrency, or other associations exist. And establishing a transaction processing association data set according to the analysis result, and defining the processing sequence and the dependency relationship between different transaction types. Highest priority identification: and carrying out deep analysis on the transaction processing association data set, and determining the priority of each transaction type according to the service requirement and the data characteristics. The transaction type of the highest priority is determined and identified. Generating a data priority sequence: and according to the highest priority identification, arranging the transaction types in the transaction processing associated data set according to the order of the priorities from high to low. And generating a data priority sequence to provide guidance for subsequent data synchronous transmission and uplink storage. Application data priority sequence: the guidance of the data priority sequence is followed when the data synchronous transmission and the uplink storage are carried out. For the transaction type with high priority, the transmission and the storage are carried out preferentially, so that timeliness and high efficiency are ensured. For low priority transaction types, transmission and storage can be performed after high priority transaction processing is completed, so that overall data processing efficiency is improved. Through the steps, transaction processing association analysis of the uplink stored data can be effectively carried out, a transaction processing association data set is established, and a data priority sequence is generated according to the priority identification. This helps to improve the efficiency and response speed of data processing, ensuring the integrity and consistency of the data.

Further, step six of the present disclosure includes:

the on-link storage trigger state and the off-link storage trigger state both comprise storage trigger time nodes;

setting a link up-link and link down-link synchronous storage response time threshold;

Calculating real-time synchronous storage response time based on the storage trigger time node;

And comparing the real-time synchronous storage response time with the synchronous storage response time threshold value, and generating the synchronous verification result based on the comparison result.

Specifically, the on-chain store trigger state and the off-chain store trigger state are acquired: and acquiring the related information of the on-chain storage triggering state and the off-chain storage triggering state from the storage system. Ensuring that the acquired state information comprises a time node for storing the trigger, and setting a link up-link and link down-link synchronous storage response time threshold value: if the data needs to be synchronized quickly to the on-chain and off-chain storage systems, the response time threshold should be relatively low. For example, for financial transactions or critical business data, any delay may cause data inconsistencies or operational failures, so the threshold should be as short as possible. System processing capabilities include hardware performance, network bandwidth, and storage I/O, among others. If the processing power of the system is strong, a relatively low threshold may be set, and if the processing power is limited, the threshold needs to be increased appropriately to ensure that the system is not overloaded. When the amount of stored data is large, the synchronization process may take longer. Therefore, it is necessary to adjust the threshold value according to the size of the data amount. For synchronization of large amounts of data, it may be necessary to set a higher threshold to avoid excessive system pressure. Some data may be more important or urgent than others, thus requiring higher real-time requirements. For high priority data, a shorter threshold may be set to ensure that it is synchronized quickly. Various abnormal conditions, such as network delay, system failure, etc., may occur during the synchronous storage process. The threshold setting should take these anomalies into account and leave a buffer time to cope with possible delays or errors. Calculating real-time synchronous storage response time: the data retrieved from the monitoring mechanism should contain time node information storing the trigger, such as a trigger time stamp. This is the starting point for calculating the real-time sync storage response time. When the data triggers the storage operation, the time node at which the data synchronization starts is recorded. The synchronization of the data from the source location to the target location is continuously monitored and any possible delays or errors are recorded. This may be accomplished by monitoring API responses, network transport status, etc. And when the data is successfully synchronized to the target position, recording a time node at which the synchronization is finished. This point in time is ensured to be accurate to avoid calculation bias due to delay or error. The real-time sync store response time is the total time required from the data trigger store to the successful synchronization of the data to the target location. It can be calculated by subtracting the trigger time node from the end time node and subtracting the sync start time node. Comparing the real-time synchronous storage response time with a synchronous storage response time threshold value: and comparing the calculated real-time synchronous storage response time with a set synchronous storage response time threshold value. And analyzing whether the real-time synchronous storage response time meets the threshold requirement or not, and obtaining a comparison result. Generating a synchronous check result: and generating a synchronous check result according to the comparison result of the real-time synchronous storage response time and the synchronous storage response time threshold. If the real-time synchronous storage response time is smaller than or equal to the threshold value, the data synchronization is considered to be successful, and the result of the synchronization check is 'pass'. If the real-time synchronous storage response time is greater than the threshold value, the data synchronization is considered to be failed, and the result of the synchronization check is 'failed'. If the synchronous check result is "pass", the data synchronous operation can be considered to be normally completed. If the synchronous check result is "failed", further investigation of the cause is required, and corresponding measures such as optimizing the data transmission path, improving the processing capacity of the system, etc. are taken to ensure the real-time performance and consistency of the data.

Further, the present disclosure further includes:

setting a calibration data calling format, and establishing a data format conversion module based on the calibration data calling format;

The data format conversion module is connected with the data transmission port, the first data receiving port and the second data receiving port and is used for carrying out format conversion before and after data transmission, and the data format conversion is stored in the on-chain data storage module and the off-chain data storage module.

Specifically, a calibration data calling format is set according to service requirements and data characteristics, and the calibration data calling format comprises information such as data structure, data type, length and the like. For example, it may be provided that the data is in JSON format, contains specific fields and data types, and length restrictions for each field, etc. A data format conversion module is developed or configured based on the calibration data call format. This module should have data parsing, format conversion, data generation, etc. to ensure proper format conversion of the data. In particular, corresponding codes or configuration files can be written to realize conversion and processing of the data format. The data format conversion module should be connected to the data transmission port, the first data receiving port and the second data receiving port. This may be accomplished by way of a programming interface, network connection, etc. In defining the format conversion rules before and after data transmission, it is necessary to specify the input/output format of data and how to perform format conversion. Before data transmission, the data format conversion module is used for converting the format of the data, so that the format of the data meets the expected standard. This can be achieved by inputting the raw data into a data format conversion module, outputting data conforming to a standard format. After the data transmission, the data format conversion module is used again to perform format conversion on the received data so as to ensure that the data format of the subsequent processing is correct. This can be achieved by inputting the received data into a data format conversion module, outputting data conforming to a standard format. The converted data may be stored in an on-chain data storage module and an off-chain data storage module. Prior to storage, it is necessary to ensure that the stored data format meets the intended standards. When stored, suitable data storage means may be employed, such as relational databases, noSQL databases, file systems, and the like. Meanwhile, a backup and recovery mechanism of data needs to be considered to ensure reliability and availability of data. Through the steps, the calibration data calling format can be effectively set, the data format conversion module is established, the module connection and function are defined, the format conversion before and after data transmission is carried out, and the converted data is stored. These steps help ensure proper parsing and generation of the data, as well as accuracy of subsequent processing and analysis.

In summary, the collaborative storage method based on the blockchain provided by the disclosure has the following technical effects:

Example two

Based on the same inventive concept as the blockchain-based collaborative storage method in the foregoing embodiment, the present disclosure further provides a blockchain-based collaborative storage system, referring to fig. 3, the system includes:

A data operation association establishing module 11, wherein the data operation association establishing module 11 is used for constructing an on-chain data storage module and an off-chain data storage module of a blockchain and establishing data operation association of the on-chain data storage module and the off-chain data storage module;

The storage decomposition module 12 is configured to perform storage decomposition on data to be stored based on the data operation association, so as to obtain a uplink storage data set and a downlink storage data set;

A transmission port acquisition module 13, where the transmission port acquisition module 13 is configured to acquire a data transmission port of data to be stored, and a first data receiving port of the on-link data storage module and a second data receiving port of the off-link data storage module, and establish a first communication encryption protocol from the data transmission port to the first data receiving port, and a second communication encryption protocol from the data transmission port to the second data receiving port;

A data set transmission module 14, where the data set transmission module 14 is configured to transmit the uplink stored data set to the on-link data storage module through the first communication encryption protocol, and transmit the downlink stored data set to the off-link data storage module through the second communication encryption protocol;

The trigger state acquisition module 15 is configured to perform storage trigger monitoring on the on-link data storage module and the off-link data storage module through an event trigger, so as to obtain an on-link storage trigger state and an off-link storage trigger state;

the synchronous verification result acquisition module 16 is used for carrying out synchronous update verification on the data under the chain on the basis of the on-chain storage trigger state and the under-chain storage trigger state to obtain a synchronous verification result;

and the synchronicity early warning module 17 is used for carrying out the on-link and off-link storage synchronicity early warning based on the synchronicity verification result.

Further, the data operation association establishing module 11 in the system is further configured to:

Obtaining a transaction record database of the blockchain, wherein the transaction record database contains various transaction types performed in history; extracting transaction calculation complexity and transaction frequency of transaction types in the transaction record database; and setting a data storage rule under the uplink of the chain, carrying out data storage type analysis under the uplink of the chain by combining the transaction calculation complexity and the transaction frequency, and outputting the data operation association.

The data storage rules under the uplink chain comprise a transaction complexity threshold, a transaction frequency threshold and a preset transaction data state; extracting a first set of transaction types in the transaction record database based on the transaction computational complexity being greater than or equal to the transaction complexity threshold or the transaction frequency being greater than the transaction frequency threshold; performing transaction data processing state segmentation on the first transaction type set based on the preset transaction data state to obtain an on-chain transaction type, on-chain transaction state data and off-chain transaction state data; the data operation association is established with the on-chain transaction type, on-chain transaction status data, and off-chain transaction status data.

Further, the system also includes a uplink storage module configured to:

Acquiring a uplink data transmission channel based on the first communication encryption protocol, and acquiring a parallel data transmission quantity threshold of the uplink data transmission channel; extracting real-time data quantity to be uplink of the uplink storage data set; carrying out transmission priority identification on the uplink storage data set to generate a data priority sequence; and judging the feasibility of the data synchronous transmission based on the real-time data quantity to be uplink and the parallel data transmission quantity threshold, and if the judging result is negative, carrying out data uplink storage according to the parallel data transmission quantity threshold and the data priority sequence through the first communication encryption protocol.

Further, the system also includes a priority sequence generation module for:

Carrying out transaction processing association analysis on the uplink storage data set, and establishing a transaction processing association data set; carrying out highest priority identification on the transaction processing association data set; and generating the data priority sequence according to the highest priority identification.

Further, the synchronization verification result obtaining module 16 in the system is further configured to:

the on-link storage trigger state and the off-link storage trigger state both comprise storage trigger time nodes; setting a link up-link and link down-link synchronous storage response time threshold; calculating real-time synchronous storage response time based on the storage trigger time node; and comparing the real-time synchronous storage response time with the synchronous storage response time threshold value, and generating the synchronous verification result based on the comparison result.

Further, the system also includes a format conversion module for:

setting a calibration data calling format, and establishing a data format conversion module based on the calibration data calling format; the data format conversion module is connected with the data transmission port, the first data receiving port and the second data receiving port and is used for carrying out format conversion before and after data transmission, and the data format conversion is stored in the on-chain data storage module and the off-chain data storage module.

Various embodiments of the present disclosure are described in a progressive manner, and each embodiment focuses on the differences from the other embodiments, so that a blockchain-based collaborative storage method and a specific example in the foregoing embodiment of fig. 1 are equally applicable to a blockchain-based collaborative storage system in the present embodiment, and by the foregoing detailed description of a blockchain-based collaborative storage method, those skilled in the art may clearly understand that a blockchain-based collaborative storage system in the present embodiment is not described in detail herein for brevity of the disclosure. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.

Example III

Based on the inventive concept of the collaborative storage method based on the blockchain in the foregoing embodiments, the present disclosure further provides an electronic device, including: at least one processor; a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of the embodiments above.

Fig. 4 is a schematic structural diagram of an exemplary electronic device of the present disclosure. In fig. 4, a bus architecture is represented by bus 300, where bus 300 may include any number of interconnected buses and bridges, with bus 300 connecting together various circuits, including one or more processors, represented by processor 302, and memory, represented by memory 304. Bus 300 may also connect together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art, and therefore, will not be described any further herein. Bus interface 305 provides an interface between bus 300 and receiver 301 and transmitter 303. The receiver 301 and the transmitter 303 may be the same element, i.e. a transceiver, providing a means for communicating with various other apparatus over a transmission medium. The processor 302 is responsible for managing the bus 300 and general processing, while the memory 304 may be used to store data used by the processor 302 in performing operations.

Example IV

Based on the same inventive concept as the blockchain-based collaborative storage method in the previous embodiment, the present disclosure further provides a computer readable storage medium having a computer program stored thereon, the computer program implementing the steps of the method in any of the previous embodiments when executed.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the spirit or scope of the disclosure. Thus, given that such modifications and variations of the disclosure are within the scope of the disclosure and its equivalents, the disclosure is also intended to include such modifications and variations.

Claims

1. A blockchain-based collaborative storage method, comprising:

constructing an on-chain data storage module and an off-chain data storage module of a blockchain, and establishing data operation association of the on-chain data storage module and the off-chain data storage module;

Based on the data operation association, carrying out storage decomposition on the data to be stored to obtain a uplink storage data set and a downlink storage data set;

A data transmission port for collecting data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port;

Transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol, and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol;

The on-chain data storage module and the off-chain data storage module are subjected to storage triggering monitoring through an event trigger to obtain an on-chain storage triggering state and an off-chain storage triggering state;

performing synchronous update verification of the data on the upper chain and the lower chain based on the on-chain storage trigger state and the off-chain storage trigger state to obtain a synchronous verification result;

and carrying out on-link and off-link storage synchronicity early warning based on the synchronicity checking result.

2. The method of claim 1, wherein the building the on-chain and off-chain data storage modules of a blockchain and establishing a data operational association of the on-chain and off-chain data storage modules comprises:

3. The method of claim 2, wherein the setting of the link-up-link-down data storage rules and the link-up-link-down data storage type analysis in combination with the transaction computational complexity and the transaction frequency, outputting the data operation association, comprises:

4. The method of claim 1, wherein the method further comprises:

extracting real-time data quantity to be uplink of the uplink storage data set;

5. The method of claim 4, wherein said performing transmission priority identification on the uplink stored data set to generate a data priority sequence comprises:

6. The method of claim 1, wherein the performing a synchronization update check of the data on the link and the link down based on the link up storage trigger state and the link down storage trigger state, to obtain a synchronization check result, comprises:

7. The method of claim 1, wherein the method further comprises:

8. A blockchain-based collaborative storage system for implementing the steps of the method of any of claims 1-7, the system comprising:

The data operation association establishing module is used for establishing an on-chain data storage module and an off-chain data storage module of the block chain and establishing data operation association of the on-chain data storage module and the off-chain data storage module;

The storage decomposition module is used for carrying out storage decomposition on the data to be stored based on the data operation association to obtain a uplink storage data set and a downlink storage data set;

The transmission port acquisition module is used for acquiring a data transmission port of data to be stored, a first data receiving port of the on-chain data storage module and a second data receiving port of the off-chain data storage module, and establishing a first communication encryption protocol from the data transmission port to the first data receiving port and a second communication encryption protocol from the data transmission port to the second data receiving port;

the data set transmission module is used for transmitting the uplink stored data set to the on-chain data storage module through the first communication encryption protocol and transmitting the downlink stored data set to the off-chain data storage module through the second communication encryption protocol;

The trigger state acquisition module is used for carrying out storage trigger monitoring on the on-chain data storage module and the off-chain data storage module through event triggers to obtain an on-chain storage trigger state and an off-chain storage trigger state;

The synchronous verification result acquisition module is used for carrying out synchronous updating verification on the data under the chain on the basis of the on-chain storage trigger state and the under-chain storage trigger state to obtain a synchronous verification result;

and the synchronicity early warning module is used for carrying out on-link and off-link storage synchronicity early warning based on the synchronicity verification result.

9. An electronic device, comprising:

At least one processor;

a memory communicatively coupled to the at least one processor;

Wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed, implements the steps of the method according to any of claims 1 to 7.