CN111767251A - Alliance chain beneficial to large file storage - Google Patents
Alliance chain beneficial to large file storage Download PDFInfo
- Publication number
- CN111767251A CN111767251A CN202010581771.XA CN202010581771A CN111767251A CN 111767251 A CN111767251 A CN 111767251A CN 202010581771 A CN202010581771 A CN 202010581771A CN 111767251 A CN111767251 A CN 111767251A
- Authority
- CN
- China
- Prior art keywords
- file
- storage
- files
- module
- storing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
- G06F16/137—Hash-based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q20/00—Payment architectures, schemes or protocols
- G06Q20/38—Payment protocols; Details thereof
- G06Q20/382—Payment protocols; Details thereof insuring higher security of transaction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Accounting & Taxation (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Finance (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a federation chain beneficial to large file storage, and belongs to the technical field of social affair management systems. The method solves the problem that the storage resources of the nodes are excessively consumed due to the fact that the nodes on the existing chain store large files. The alliance chain beneficial to large file storage comprises all consensus nodes running on a block chain platform at the bottom layer, wherein each consensus node is provided with the following modules: a data size detection module for detecting the size of a file for a storage transaction within a block of data blocks to be uplinked; and the large file evaluation module is used for determining whether the files for storing the transaction are large files or not according to the value of the size of the files for storing the transaction detected by the data size detection module. The advantages of the federation chain that facilitates large file storage are: the flexibility of alliance chain storage is improved, the size of the account book is reduced, the storage cost is reduced, and a flexible solution is provided for a user during storage.
Description
Technical Field
The invention belongs to the technical field of social affair management systems, and particularly relates to a federation chain beneficial to large file storage.
Background
The blockchain technology is characterized in that information stored on the blockchain cannot be forged and tampered, a blockchain consensus algorithm drives each node on the blockchain to participate in a transaction verification process, the transactions on the blockchain are confirmed to be credible, each node on the blockchain maintains a public account book, and any node modifies the account book maintained by the node to be unacknowledged by other nodes, so that the public account book cannot be forged and tampered.
In addition, according to the openness degree, the blockchain can be divided into a public chain, a alliance chain and a private chain, specifically, the public chain system is the most open, any person can participate in maintaining and reading the blockchain data, the application program is easy to deploy, and the whole decentralized process is not controlled by any organization; the federation chain is a block chain which needs to be accessed by registration permission, from the viewpoint of using objects, the federation chain is only limited to participation of federation members, the federation chain usually adopts a mode of computing by a designated node, and the number of accounting nodes is relatively small; the private chain is most closed and is only used in enterprises, national institutions or single individuals, and the alliance chain is open between the public chain and the private chain, has the functions of a distributed account book and is high in safety compared with the public chain, so that the alliance chain is popular among more users.
The existing alliance chain is directly stored in the account book on the large file storage, so that the account book is too large, and each node needs to be provided with more storage devices to store the account book containing the large files due to the fact that each node needs to synchronize the account book, and the storage cost of an enterprise is increased.
Disclosure of Invention
It is an object of the present invention to provide a federation chain facilitating large file storage that addresses at least part of the above problems.
In order to achieve the purpose, the invention adopts the following technical scheme: the alliance chain beneficial to large file storage comprises all consensus nodes running on a block chain platform at the bottom layer, and is characterized in that each consensus node is provided with the following modules:
a data size detection module for detecting the size of a file for a storage transaction within a block of data blocks to be uplinked;
the big file evaluation module is used for determining whether the files for storing the transaction are big files according to the value of the size of the files for storing the transaction detected by the data size detection module;
and the data storage mode specifying module is used for determining a storage mode of files for storing the transaction in the data blocks to be linked on the common node according to the evaluation result of the large file evaluation module.
In the above alliance chain beneficial to large file storage, each consensus node is further provided with a data configuration module, the data configuration module is used for storing relevant threshold data, the threshold data comprises a large file size evaluation threshold, and the large file evaluation module judges whether a file in storage transaction is a large file according to the large file size evaluation threshold.
In the above federation chain beneficial to large file storage, the storage modes in the data storage mode specifying module include a direct storage mode and a storage mode for storing a hash value of a file, that is, when a file for storing transaction is not a large file, the file is directly stored on the consensus node, otherwise, only the hash value of the file is stored on the consensus node, and the large file itself is stored on an external server connected to the federation chain.
In the above federation chain beneficial to large file storage, when the storage mode is a storage mode for storing the hash value of a file, if a storage error occurs when the large file is stored in an external server, the storage mode is changed to a direct storage mode.
In the above federation chain for facilitating storage of a large file, the hash value of the file is the hash value of the address of the file stored on the external server.
In the above-described federation chain for facilitating the storage of large files, the address at which the file is stored on an external server includes the network address of the server and the physical address located on the server.
In the above federation chain beneficial to large file storage, each consensus node is further provided with an index creation module, and the module performs different indexes according to different storage modes of files in storage transaction on the federation chain, namely, when the files are in a direct storage mode, an index is created for the files, and when the hash value storage mode of the files is used, the hash value of the files is used as the index.
In the above alliance chain which is beneficial to large file storage, each consensus node is also provided with a user query module for user query, and the module performs different queries according to different indexes created by the index creation module, namely directly queries the required file by using the index created by the file, queries the hash value of the file by using the index created by the hash value of the stored file, and finds the file stored on an external server by using the hash value of the file.
Compared with the prior art, the alliance chain beneficial to large file storage has the advantages that:
1. different from the mode that the existing alliance chain directly stores the large file in the node, the scheme adopts a method that only the hash value of the file is stored in the consensus node and the file is stored in the external server for storing the large file, so that the consumption of the large file on the limited storage resource on the node is effectively reduced;
2. whether the files to be linked in the storage transaction are large files or not is automatically confirmed through a data size detection module and a large file evaluation module, so that different storage modes can be realized according to whether the files in the storage transaction are large files or not;
3. in order to avoid the phenomenon that the large file is not stored on the consensus node when a storage error occurs when the large file is stored in an external server, the consensus node stores the large file onto the node when the storage error occurs when the external server stores the large file, so that the loss of the large file data in the storage process is prevented;
4. different indexes are established according to different storage modes of the files, and storage resources consumed by the indexes are saved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 provides a system architecture diagram of an embodiment of the invention.
Detailed Description
Before further detailed description of the embodiments of the present invention, terms and expressions mentioned in the embodiments of the present invention are explained, and the terms and expressions mentioned in the embodiments of the present invention are applied to the following explanations.
1. Transaction (Transaction), equivalent to the computer term "thing", includes three different Transaction types: deployment (deployment), invocation (Invoke) and Query (Query). The present embodiments follow this convention in view of the convention that in blockchain technology, conventions are used to install a designated chain code to nodes of a blockchain network, and transactions of the call and query type are used to invoke a chain code of a deployment number to effect operations on data related to a target account in an ledger, including operations of adding, deleting, looking up, changing, modifying data in the form of Key-Value pairs in an account, or adding a new account in an ledger, not simply referring to a transaction in a business context.
2. A Block (Block) recording a data structure of the ledger data updated by the exchange within a period of time, marked with a timestamp and a unique mark (e.g. a digital fingerprint) of a previous Block, and after the Block is subjected to consensus verification by nodes in the Block chain network, the Block is appended to the end of the Block chain to become a new Block.
3. A chain of blocks (Blockchain), a chain of data structures in which blocks are assembled in a sequentially contiguous manner, in each block referencing a hash value of a previous block or a subset thereof, thereby cryptographically securing the recorded transaction as being non-tamperable and non-forgeable.
4. A blockchain network incorporates new blocks into a set of centerless nodes of the blockchain in a consensus manner.
5. The account book (Legger) is the sum of data recorded by taking an account as a dimension in a block chain network, and comprises the elements of the account book data, the state of the account book, the state certification of the account book, a block index and the like.
6. The ledger data, the actual block data storage, i.e. the record of a series of ordered and non-falsifiable transactions recorded in the block chain, may be expressed in the form of a file system, and the update of the data in the account/account is realized when an intelligent contract called in the transaction is executed.
7. Consensus (Consensus), a process in a blockchain network, is used to agree on the transaction results among the nodes involved, and the mechanisms for achieving Consensus include Proof of workload (PoW), Proof of rights and interests (PoS, Proof of approval of stamp), Proof of share authorization (DPoS), Proof of Elapsed Time (PoET, Proof of Elapsed Time), and so on.
8. Intelligent Contracts (Smart Contracts), built Contracts, also known as chain codes (chaincodes), conditionally executed programs deployed in blockchain networks, run in a secure container to initialize and manage ledger data and ledger states.
The present invention will be described in further detail below by way of examples with reference to the accompanying drawings, which are illustrative of the present invention and are not to be construed as limiting the present invention.
Example 1
As shown in fig. 1, the federation chain beneficial to large file storage includes common nodes running on a bottom blockchain platform, where each common node is provided with the following modules: the device comprises a data size detection module, a large file evaluation module and a data storage mode specifying module.
The modules are described in detail below.
A data size detection module for detecting the size of a file for a storage transaction within a block of the data blocks to be uplinked.
The size of the file of the storage transaction is determined by detecting the overall length value of the file to provide a data source for the following large file rating module.
And the large file evaluation module is used for determining whether the files for storing the transaction are large files according to the value of the size of the files for storing the transaction detected by the data size detection module.
Whether the file is a large file or not can be judged by setting a parameter, namely a large file size evaluation threshold, for example, the value set for the parameter is 2K, namely, the file can be judged as the large file if the size of the file for storing transaction is more than or equal to 2K, and the file is not the large file if the size of the file for storing transaction is not more than 2K.
Preferably, a data configuration module is further provided at each consensus node, and the data configuration module is used for storing relevant threshold data, wherein a large file size assessment threshold value can be included, so that when the large file assessment module is called, the large file size assessment threshold value can be read from the configuration file set by the data configuration module without setting death in a program.
And the data storage mode specifying module is used for determining a storage mode of files for storing the transaction in the data blocks to be linked on the common node according to the evaluation result of the large file evaluation module.
Specifically, the storage modes in the data storage mode specifying module include a direct storage mode and a storage mode for storing a hash value of a file, that is, when the file for storing the transaction is not a large file, the file is directly stored on the consensus node, otherwise, only the hash value of the file is stored on the consensus node, and the large file itself is stored on an external server connected to the federation chain.
Therefore, the flexibility of the storage part of the alliance chain is improved, different storage types can be customized according to the requirements of users, for example, when the users develop own external storage servers, the hash values of the large files can be stored in the consensus nodes, the large files are stored in the external storage servers, and if the cost sensitivity of the users to the hard disk is not high, the users do not need to determine whether the files to be chained are the large files or not and can directly chain the files.
The hash value of the file is the hash value of the address of the file stored on the external server, and the address of the file stored on the external server comprises the network address of the server and the physical address located on the server, so that the large file located on the external server can be quickly found.
In order to avoid the situation that the large file is not stored, when the storage mode is a storage file hash value storage mode, if a storage error occurs when the large file is stored in an external server, the direct storage mode is used instead.
Example 2
The present embodiment is different from embodiment 1 in common in that:
each common node is also provided with an index creation module which performs different indexes according to different storage modes of files for storing transactions on the alliance chain, namely, when the files are in a direct storage mode, the indexes are created for the files, and when the hash value storage mode of the files is used, the hash value of the files is used as the index.
In addition, each common node is also provided with a user query module for user query, and the module carries out different queries according to different indexes created by the index creation module, namely, the index created by the file itself directly queries the required file, and the index created by the hash value of the stored file first queries the hash value of the file and then finds the file stored on the external server according to the hash value of the file.
Therefore, when a user inquires a file, a small file can be found on the common identification node directly, and for a large file, the hash value of the large file can be found on the common identification node, and then the large file is searched on an external server according to the hash value of the large file (namely the hash value of the address of the file) and then sent to the user.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.
Claims (8)
1. A federation chain beneficial to large file storage comprises all consensus nodes running on a block chain platform at the bottom layer, and is characterized in that each consensus node is provided with the following modules:
a data size detection module for detecting the size of a file for a storage transaction within a block of data blocks to be uplinked;
the big file evaluation module is used for determining whether the files for storing the transaction are big files according to the value of the size of the files for storing the transaction detected by the data size detection module;
and the data storage mode specifying module is used for determining a storage mode of files for storing the transaction in the data blocks to be linked on the common node according to the evaluation result of the large file evaluation module.
2. A federation chain for facilitating the storage of large files according to claim 1, wherein each consensus node is further provided with a data configuration module for storing associated threshold data, said threshold data comprising a large file size rating threshold, and said large file rating module is adapted to evaluate whether a file storing a transaction is a large file based on the large file size rating threshold.
3. A federation chain for facilitating the storage of large files according to claim 1, wherein the storage modes in the data storage mode specifying module include a direct storage mode and a storage mode for storing hash values of files, that is, when a file for storing transactions is not a large file, the file is directly stored on the consensus node, otherwise, only the hash value of the file is stored on the consensus node, and the large file itself is stored on an external server connected to the federation chain.
4. A federation chain for facilitating storage of a large file as claimed in claim 3, wherein when the storage mode is a storage mode for storing hash values of files, if a storage error occurs when the large file is stored in an external server, the storage mode is changed to a direct storage mode.
5. A federation chain for facilitating storage of large files according to claim 3, wherein the hash value of a file is the hash value of the address at which the file is stored on the external server.
6. A federation chain for facilitating the storage of large files as claimed in claim 5 wherein the address at which a file is stored on an external server includes the network address of the server and the physical address located at that server.
7. A federation chain for facilitating the storage of large files according to claim 5, wherein each consensus node is further provided with an index creation module, which performs different indexing according to the storage mode of the files storing transactions on the federation chain, that is, creating an index for the file itself in the direct storage mode and using the hash value of the stored file as an index in the hash value storage mode.
8. A federation chain for facilitating the storage of large files according to claim 7, wherein each consensus node is further provided with a user query module for user query, and the module performs different queries according to the difference of the indexes created by the index creation module, i.e. the index created by the file itself is used to directly query the required file, and the index created by the hash value of the stored file is used to query the hash value of the file first, and then the hash value of the file is used to find the file stored on the external server.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581771.XA CN111767251A (en) | 2020-06-23 | 2020-06-23 | Alliance chain beneficial to large file storage |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010581771.XA CN111767251A (en) | 2020-06-23 | 2020-06-23 | Alliance chain beneficial to large file storage |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111767251A true CN111767251A (en) | 2020-10-13 |
Family
ID=72722079
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010581771.XA Withdrawn CN111767251A (en) | 2020-06-23 | 2020-06-23 | Alliance chain beneficial to large file storage |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111767251A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821478A (en) * | 2021-11-24 | 2021-12-21 | 南京金宁汇科技有限公司 | Block chain-based large file storage method and system |
-
2020
- 2020-06-23 CN CN202010581771.XA patent/CN111767251A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113821478A (en) * | 2021-11-24 | 2021-12-21 | 南京金宁汇科技有限公司 | Block chain-based large file storage method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10412170B2 (en) | Retention-based data management in a network-based data store | |
| US11880494B2 (en) | Secure decentralized system utilizing smart contracts, a blockchain, and/or a distributed file system | |
| WO2020215836A1 (en) | Blockchain-based housing information attest and application method and device | |
| CN107391758B (en) | Database switching method, device and equipment | |
| US11341118B2 (en) | Atomic application of multiple updates to a hierarchical data structure | |
| CN111770149B (en) | Novel alliance chain system based on distributed storage | |
| US11546366B2 (en) | Threat information sharing based on blockchain | |
| JP2020514935A (en) | Method and system for a database | |
| CN105912428B (en) | Realize that source data is converted into the system and method for virtual machine image in real time | |
| WO2020052578A1 (en) | Method for managing and providing recruitment information | |
| Mendes et al. | Charon: A secure cloud-of-clouds system for storing and sharing big data | |
| US20200175583A1 (en) | Blockchain-based leasing | |
| US9910881B1 (en) | Maintaining versions of control plane data for a network-based service control plane | |
| TWI724570B (en) | Method, device, electronic equipment and storage medium for reading and updating data structure | |
| US11250019B1 (en) | Eventually consistent replication in a time-series database | |
| Renner et al. | Endolith: A blockchain-based framework to enhance data retention in cloud storages | |
| TW202025057A (en) | Field updating method and device and electronic equipment | |
| CN106254466A (en) | HDFS distributed file sharing method based on LAN | |
| US11409771B1 (en) | Splitting partitions across clusters in a time-series database | |
| CN111429250A (en) | Data management method and device in escort scene | |
| CN109246248B (en) | Data credible safety sharing system and method based on block chain technology | |
| CN111767251A (en) | Alliance chain beneficial to large file storage | |
| US11366598B1 (en) | Dynamic lease assignments in a time-series database | |
| WO2024021417A1 (en) | Data account creation method and apparatus | |
| US11620364B2 (en) | Layered-infrastructure blockchain-based system for software license distribution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20201013 |
|
| WW01 | Invention patent application withdrawn after publication |