CN114637472A - DHT-based block chain storage expansion method, electronic equipment and storage medium - Google Patents
DHT-based block chain storage expansion method, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114637472A CN114637472A CN202210322338.3A CN202210322338A CN114637472A CN 114637472 A CN114637472 A CN 114637472A CN 202210322338 A CN202210322338 A CN 202210322338A CN 114637472 A CN114637472 A CN 114637472A
- Authority
- CN
- China
- Prior art keywords
- node
- block
- block chain
- new
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0608—Saving storage space on storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a block chain storage expansion method based on DHT, electronic equipment and a storage medium, wherein the method comprises the following steps: when the target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block; taking the hash value of the new block as the block ID of the new block; respectively carrying out last bit minimum XOR operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster, and determining the target block chain node with the minimum XOR distance as a storage position for storing a new block; and storing the new block to the target block chain node. According to the method, the block chain nodes are divided into a plurality of clusters according to the ID values of the nodes, each node cluster stores one complete block chain datum, and when a new block is generated, one node is distributed in each cluster for storage, so that the storage pressure of the block chain nodes is reduced, the storage consumption of a block chain system is reduced, and meanwhile, the decentralized characteristic of the block chain system cannot be influenced.
Description
Technical Field
The present application relates to the field of blockchain technologies, and in particular, to a block chain storage expansion method based on DHT, an electronic device, and a storage medium.
Background
The block chain technology has wide application prospect and practical value in the fields of finance, agricultural product circulation, smart power grids, industrial internet and the like by the characteristics of anonymity, traceability, non-tampering, decentralization and the like. The decentralized nature of blockchain networks depends on the particular protocol.
Dht (distributed Hash table) technology, known as a distributed Hash table in chinese, is a technology for organizing, managing, and discovering services on distributed resources. Each node in the network adopting the DHT technology is responsible for a small part of routing and storing a part of data, and a specific search mechanism is used in a scene without a neutralized server to finally acquire a specific storage resource under the relay assistance of each node. Accordingly, DHT techniques may also store particular resources on corresponding network nodes through some mapping mechanism. DHT technology is implemented by many P2P protocols, notably CAN, Chord, fashion, Tapestry, Kademlia.
In order to achieve the purposes of decentralization and tamper resistance, the blockchain adopts a storage strategy that a complete piece of blockchain data is stored on each network node, and the storage strategy has the advantages that each node can conveniently and independently verify a transaction or a newly generated block, so that the blockchain data is prevented from being tampered by a malicious node, and the defects of the storage strategy are gradually revealed with the rapid increase of the blockchain data.
Take the bitcoin as an example: bitcoin has an average transaction of less than 3.5 per second, and even at this rate, nodes need to consume approximately 160MB of storage space per day, approximately 60 GB/year on average. By 5 months and 5 days 2021, bitcoin yielded a total of 683015 blocks, with an average block size of 1.3MB and the entire blockchain size already exceeded 300 GB. In order to achieve the highest security and trust level, the node needs to waste more than 300GB of storage space to record a large amount of unrelated transaction data, the whole blockchain system needs to use nearly 9000 PB of space to store 300GB of valid data, and as the number of blocks and nodes increases, the storage consumption will increase continuously, so that the storage problem of the blockchain is increasingly urgent.
To solve this problem, a solution is needed that acts the same as a complete node, validating and relaying transactions, reducing memory size. There are some schemes, such as lightweight nodes like SPV (simple payment verification), to reduce the storage size. However, these solutions rely on the complete node for the lightweight node, which eventually weakens the decentralized nature of the blockchain.
Disclosure of Invention
The embodiment of the application provides a block chain storage expansion method based on DHT, an electronic device and a storage medium, which reduce the storage pressure of a block chain node, reduce the storage consumption of a block chain system and simultaneously do not influence the decentralized characteristic of the block chain system.
On one hand, the present application provides a block chain storage expansion method based on DHT, which is applied to a block chain system, where the block chain system includes a plurality of block chain nodes, the plurality of block chain nodes are divided into a plurality of node clusters, and each node cluster stores one complete block chain data, and the block chain storage expansion method based on DHT includes:
when a target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block;
taking the hash value of the new block as a block ID of the new block, wherein the block ID is represented by binary number;
performing last bit minimum exclusive-or operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster respectively, and determining the target block chain node with the minimum exclusive-or distance as a storage position for storing the new block;
storing the new block to the target blockchain node.
In some embodiments of the present application, before obtaining the hash value of the new chunk from the chunk header of the new chunk when the target node cluster receives the new chunk, the method further includes:
and dividing the plurality of block chain nodes into a plurality of node clusters.
In some embodiments of the present application, the plurality of blockchain nodes are organized in a Kademlia tree, each blockchain node is provided with a corresponding unique node ID, the node ID of each blockchain node follows a Kademlia protocol, a position of each blockchain node in the Kademlia protocol tree is obtained based on the node ID of the blockchain node, the node ID of each blockchain is obtained based on preset node identification information, and the node ID of a blockchain link point is represented by a binary number.
In some embodiments of the present application, the method further comprises:
when a target node cluster receives a joining request of a new block chain node, acquiring node identification information of the new block chain node;
determining a node ID of the new blockchain node based on the node identification information;
determining the node position of the new block chain node in the target node cluster on a Kademlia protocol tree according to a Kademlia protocol;
and adding the new block chain node into the target node cluster based on the node position.
In some embodiments of the present application, after said storing the new block to the target blockchain node, the method further comprises:
determining a plurality of backup nodes of the new block before and after the target block chain node;
and respectively storing the new blocks at the plurality of backup nodes so as to backup the new blocks.
In some embodiments of the present application, the method further comprises:
detecting whether the storage pressure of each block chain node in the target node cluster reaches a preset threshold value;
and if so, merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster.
In some embodiments of the present application, the merging and recombining the target node cluster and the neighboring node cluster of the target node cluster to obtain a merged node cluster includes:
merging the target node cluster and nodes in adjacent node clusters of the target node cluster into a larger cluster to obtain a merged node cluster;
and carrying out duplicate removal on the same block data stored in each block chain link point in the merged node cluster.
On the other hand, the present application provides a DHT blockchain storage expansion device, which is applied to a blockchain system, where the blockchain system includes a plurality of blockchain nodes, the plurality of blockchain nodes are divided into a plurality of node clusters, each node cluster stores one complete piece of blockchain data, and the DHT-based blockchain storage expansion device includes:
an obtaining module, configured to obtain, when a target node cluster receives a new block, a hash value of the new block from a block header of the new block, and use the hash value of the new block as a block ID of the new block, where the block ID is represented by a binary number;
a determining module, configured to perform last minimum xor operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster, respectively, and determine a target block chain node with a minimum xor distance as a storage location where the new block is stored;
a storage module for storing the new block to the target block chain node.
In some embodiments of the present application, the apparatus further includes a cluster partitioning module, where the cluster partitioning module is specifically configured to:
when a target node cluster receives a new block, dividing the block chain nodes into a plurality of node clusters before obtaining the hash value of the new block from the block head of the new block.
In some embodiments of the present application, the plurality of blockchain nodes are organized in a Kademlia tree, each blockchain node is provided with a corresponding unique node ID, the node ID of each blockchain node follows a Kademlia protocol, a position of each blockchain node in the Kademlia protocol tree is obtained based on the node ID of the blockchain node, the node ID of each blockchain is obtained based on preset node identification information, and the node ID of a blockchain link point is represented by a binary number.
In some embodiments of the present application, the apparatus further includes a node joining module, where the node joining module is specifically configured to:
when a target node cluster receives a joining request of a new block chain node, acquiring node identification information of the new block chain node;
determining a node ID of the new blockchain node based on the node identification information;
determining the node position of the new block chain node in the target node cluster on a Kademlia protocol tree according to a Kademlia protocol;
and adding the new blockchain node into the target node cluster based on the node position.
In some embodiments of the present application, the apparatus further includes a backup module, where the backup module is specifically configured to:
determining a number of backup nodes for the new block before and after the target blockchain node after the storing of the new block to the target blockchain node;
and respectively storing the new blocks at the plurality of backup nodes so as to backup the new blocks.
In some embodiments of the present application, the apparatus further includes a cluster dynamic adjustment module, where the cluster dynamic adjustment module is specifically configured to:
detecting whether the storage pressure of each block chain node in the target node cluster reaches a preset threshold value;
and if so, merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster.
In some embodiments of the present application, the cluster dynamic adjustment module is specifically configured to:
merging the target node cluster and nodes in adjacent node clusters of the target node cluster into a larger cluster to obtain a merged node cluster;
and carrying out duplicate removal on the same block data stored in each block chain link point in the merged node cluster.
In another aspect, the present application further provides an electronic device, including:
one or more processors;
a memory; and
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the DHT-based blockchain storage expansion method of any of the first aspects.
In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is loaded by a processor to execute the steps in the DHT-based blockchain storage expansion method according to any one of the first aspects.
In the embodiment of the application, when a target node cluster receives a new block, a hash value of the new block is obtained from a block head of the new block; taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary number; respectively carrying out last bit minimum XOR operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster, and determining the target block chain node with the minimum XOR distance as a storage position for storing a new block; and storing the new block to the target block chain node. In the embodiment of the application, the block chain link points are divided into a plurality of clusters according to the ID values of the nodes, each node cluster stores a piece of complete block chain data, when a new block is generated, the last bits of the block ID values and the last bits of the node ID values in the clusters are subjected to XOR operation, and when the XOR distance is minimum (namely, the distance is closest), the node position of the new block stored in the node cluster is determined, so that the storage pressure of the block chain nodes is reduced, the storage consumption of a block chain system is reduced, and the decentralized characteristic of the block chain system is not influenced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic view of a block chain system according to an embodiment of the present disclosure;
fig. 2 is a flowchart illustrating an embodiment of a DHT-based blockchain storage expansion method provided in an embodiment of the present application;
FIG. 3 is a flowchart illustrating an embodiment of a newly added blockchain node in the node cluster based on FIG. 1 according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a scenario in which a two-cluster network is recombined into a single-node cluster in the embodiment of the present application;
FIG. 5 is a schematic diagram comparing experimental results of three storage schemes provided in the examples of the present application;
fig. 6 is a schematic structural diagram of an embodiment of a DHT-based blockchain storage expansion apparatus provided in an embodiment of the present application;
fig. 7 is a schematic structural diagram of an embodiment of an electronic device provided in the embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only some embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
In the description of the present application, it is to be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed in a particular orientation, and be operated, and thus should not be considered as limiting the present application. Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or to implicitly indicate the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more unless specifically limited otherwise.
In this application, the word "exemplary" is used to mean "serving as an example, instance, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the application. In the following description, details are set forth for the purpose of explanation. It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes are not set forth in detail in order to avoid obscuring the description of the present application with unnecessary detail. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Some basic concepts involved in the embodiments of the present application are first described below:
dht (distributed Hash table) technology, known as a distributed Hash table in chinese, is a technology for organizing, managing, and discovering services on distributed resources. Each node in the network adopting the DHT technology is responsible for a small part of routing and storing a part of data, and a specific storage resource is finally obtained under the relay assistance of each node by using a specific search mechanism in the scene without a neutralized server. Accordingly, DHT techniques may also store particular resources on corresponding network nodes through some mapping mechanism. DHT technology is implemented by many P2P protocols, notably CAN, Chord, fashion, Tapestry, Kademlia.
Blockchains are a term of art in information technology. In essence, the system is a shared database, and the data or information stored in the shared database has the characteristics of 'unforgeability', 'whole-course trace', 'traceability', 'public transparency', 'collective maintenance', and the like. Based on the characteristics, the block chain technology lays a solid 'trust' foundation, creates a reliable 'cooperation' mechanism and has wide application prospect.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. The Blockchain (Blockchain) is an important concept of the bitcoin, which is essentially a decentralized database, and is used as the underlying technology of the bitcoin, and is a series of data blocks which are generated by using a cryptographic method to be related, wherein each data block contains information of a batch of bitcoin network transactions, and the information is used for verifying the validity (anti-counterfeiting) of the information and generating the next block.
Block chain Node (Node): the nodes are network nodes in a block chain distributed system, are servers, computers, telephones and the like connected through a network, and have different modes of becoming the nodes aiming at block chains with different properties, and take bitcoin as an example, the nodes are involved in transaction or mine digging to form one node.
Intelligent contract: is a set of conventions that are defined, propagated, verified, or enforced digitally, including agreements on which contract participants may enforce these conventions. Smart contracts allow trusted transactions to be conducted without third parties, which transactions are traceable and irreversible. The intelligent contracts are used as a set of logical rules governing transactions in the blockchain, based on script code. The program will read the contract code when executing the transaction, execute and process the results. The nature of the smart contract may be limited to code specific to an application or may be used to program the terms and conditions of an agreement into the transaction flow.
Creating a new appearance block: the first block data in the block chain account book is used as the first block of the block chain.
Embodiments of the present application provide a block chain storage expansion method based on DHT, an electronic device, and a storage medium, which are described in detail below.
Referring to fig. 1, fig. 1 is a schematic view of a scenario of a blockchain system according to an embodiment of the present disclosure, where the blockchain system includes a plurality of blockchain nodes (i.e., cluster nodes in fig. 1), the plurality of blockchain nodes are divided into a plurality of node clusters, each node cluster stores a complete piece of blockchain data, a block is allocated to the blockchain nodes in each node cluster through a special block-node mapping mechanism (last-bit minimum exclusive or mapping mechanism), and each blockchain node stores complete block header information, so that storage pressure of the node is reduced and the blockchain data is prevented from being tampered by retaining the block header information.
The method in the embodiment of the application may be an electronic device corresponding to a link point of a certain block, and the electronic device is mainly used for obtaining a hash value of a new block from a block head of the new block when a target node cluster receives the new block; taking the hash value of the new block as a block ID of the new block, wherein the block ID is represented by binary number; performing last bit minimum exclusive-or operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster respectively, and determining the target block chain node with the minimum exclusive-or distance as a storage position for storing the new block; storing the new block to the target blockchain node.
In this embodiment, the block link point may be an electronic device, such as a user terminal or a server, where the server may be an independent server, or may also be a server network or a server cluster composed of servers, for example, the server described in this embodiment includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets, or a cloud server composed of a plurality of servers. Among them, the Cloud server is constituted by a large number of computers or web servers based on Cloud Computing (Cloud Computing). In the embodiment of the present application, the server and the User terminal may implement communication through any communication manner, including but not limited to mobile communication based on the third Generation Partnership Project (3 GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP/IP Protocol Suite (TCP/IP), User Datagram Protocol (UDP) Protocol, and the like.
It will be appreciated that the user terminal used in the embodiments of the present application includes both receiving and transmitting hardware devices, i.e. devices having receiving and transmitting hardware capable of performing bi-directional communication over a bi-directional communication link. Such a user terminal device may include: a cellular or other communication device having a single line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific user terminal may be a desktop terminal or a mobile terminal, and the user terminal may be one of a mobile phone, a tablet computer, a notebook computer, and the like.
Those skilled in the art will understand that the application environment shown in fig. 1 is only one application scenario related to the present application, and does not constitute a limitation on the application scenario of the present application, and that other application environments may further include more or less block link points than those shown in fig. 1, for example, only a part of block link nodes are shown in fig. 1, and it is understood that the block link system may further include more block link points, and is not limited herein.
It should be noted that the scenario diagram of the blockchain system shown in fig. 1 is merely an example, and the blockchain system and the scenario described in the embodiment of the present application are for more clearly illustrating the technical solution of the embodiment of the present application, and do not form a limitation to the technical solution provided in the embodiment of the present application, and as a person having ordinary skill in the art knows, with the evolution of the blockchain system and the occurrence of a new service scenario, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
Aiming at the problem of storage defects of a block chain system, the application provides a dynamic cluster node storage expansion scheme based on DHT, the organization mode of nodes adopts a Kademlia tree, each node is distributed with a unique node ID value according to self information such as IP addresses, Mac addresses and the like, the node ID value is represented by binary numbers, block chain network nodes are divided into a plurality of clusters according to the ID values of the nodes, each node cluster stores a complete block chain data, a block is distributed to the nodes in each cluster through a special block-node mapping mechanism, and meanwhile, each node stores the complete block head information, so that the storage pressure of the nodes is reduced, and the block chain data is prevented from being tampered by reserving the block head information.
Specifically, in an embodiment of the present application, an execution main body of the block chain storage and expansion method based on DHT is a block chain storage and expansion device based on DHT, and the block chain storage and expansion device based on DHT is applied to an electronic device, and the block chain storage and expansion method based on DHT includes: when a target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block; taking the hash value of the new block as a block ID of the new block, wherein the block ID is represented by binary number; performing last bit minimum exclusive-or operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster respectively, and determining the target block chain node with the minimum exclusive-or distance as a storage position for storing the new block; storing the new block to the target blockchain node.
As shown in fig. 2, which is a schematic flow chart of an embodiment of a block chain storage expansion method based on DHT in the embodiment of the present application, the block chain storage expansion method based on DHT includes the following steps 201 to 204:
201. when the target node cluster receives a new block, the hash value of the new block is obtained from the block head of the new block.
The target node cluster is one of a plurality of node clusters in the blockchain system, and the target node cluster can receive a new block when a transaction is received to generate the new block.
202. And taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary numbers.
203. And performing last bit minimum exclusive-or operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster respectively, and determining the target block chain node with the minimum exclusive-or distance as a storage position for storing the new block.
204. Storing the new block to the target blockchain node.
In the embodiment of the application, when a target node cluster receives a new block, a hash value of the new block is obtained from a block head of the new block; taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary number; respectively carrying out last bit minimum XOR operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster, and determining the target block chain node with the minimum XOR distance as a storage position for storing a new block; and storing the new block to the target block chain node. In the embodiment of the application, the block chain link points are divided into a plurality of clusters according to the ID values of the nodes, each node cluster stores a piece of complete block chain data, when a new block is generated, the last bits of the block ID values and the last bits of the node ID values in the clusters are subjected to XOR operation, and when the XOR distance is minimum (namely, the distance is closest), the node position of the new block stored in the node cluster is determined, so that the storage pressure of the block chain nodes is reduced, the storage consumption of a block chain system is reduced, and the decentralized characteristic of the block chain system is not influenced.
In some embodiments of the present application, before obtaining the hash value of a new chunk from a chunk header of the new chunk when the target node cluster receives the new chunk, the method further includes a step of partitioning the node cluster, that is, the method further includes: and dividing the plurality of block chain nodes into a plurality of node clusters. The node cluster division rule may be based on a preset number requirement, for example, a preset number (e.g., 30) of block link nodes are randomly selected to be divided into one node cluster.
Further, the plurality of block chain nodes are organized in a Kademlia tree manner, each block chain node is provided with a corresponding unique node ID, the node ID of each block chain node follows a Kademlia protocol, the position of each block chain node in the Kademlia protocol tree is obtained based on the node ID of the block chain node, the node ID of each block chain is obtained based on preset node identification information, and the node ID of the block chain link point is represented by binary numbers.
In the embodiment of the present application, a last-bit minimum exclusive or mapping mechanism is proposed based on the block-node allocation scheme proposed by Kaneko. Specifically, when a node cluster receives a new block, the new block hash value is used as a block ID, the block ID value is represented by binary numbers, and the last bits of the block ID value and the last bits of the node ID value in the node cluster are subjected to xor operation, so that when the xor distance is minimum (i.e., the distance is closest), the block link point at the time is the position where the block is stored.
As shown in fig. 1, fig. 1 illustrates the process of four blocks being mapped onto a Kademlia tree with a total of 8 blockchain nodes for two clusters, where x in the block ID represents the bits before the last two bits. The whole block chain link point can be seen to be divided into two node clusters, each node cluster has at most four block chain nodes, so that block allocation is only needed to be carried out according to the rear log24 of the node ID as 2 bits, the allocation principle adopts a last-bit minimum exclusive-or mapping mechanism, and finally each node cluster stores a complete block chain data.
In some embodiments of the present application, for a newly added block link point, the position of the node may be determined in a certain manner, and specifically, as shown in fig. 3, the method may further include the following steps 301:
301. and when the target node cluster receives a joining request of a new block chain node, acquiring the node identification information of the new block chain node.
302. Determining a node ID of the new blockchain node based on the node identification information.
303. And according to the Kademlia protocol, determining the node position of the new blockchain node in the target node cluster on a Kademlia protocol tree.
Specifically, the hash (such as SHA1) calculation is performed on the information of the new blockchain node, such as the information of the public network IP address, the internal network IP address, the port number occupied by the internal network, and the like, to obtain a 160-bit node ID unique to the whole network, and the Kademlia protocol tree position to which the new blockchain node belongs is allocated to the new blockchain node according to the ID of the new blockchain node, that is, the node position of the new blockchain node in the target node cluster.
304. And adding the new blockchain node into the target node cluster based on the node position.
In order to prevent the downtime of some block chain nodes from affecting the normal operation of the whole node cluster, the node cluster in the embodiment of the present application may further adopt a redundancy backup mechanism, and each newly generated block is backed up by several copies before and after the closest block link point (for example, the copy number of the Kademlia algorithm is 20, that is, 20 closest block link point backup blocks are selected).
Specifically, after the storing the new block to the target blockchain node, the method further includes: determining a plurality of backup nodes of the new block before and after the target block chain node; and respectively storing the new blocks at the plurality of backup nodes so as to backup the new blocks.
At the initial stage of the block chain, the number of block chain links in the node cluster is small due to small storage pressure; then, as the height of the block increases and the block chain adopts an additional storage mechanism, the data of the block chain only increases but not decreases, and the storage pressure of the block chain link points in the node cluster is increased, which finally causes the cluster to crash. In order to solve the problem, the dynamic cluster node reorganization mechanism is provided, and after the storage pressure of the cluster nodes reaches a specific value, two adjacent node clusters are reorganized to form a large node cluster.
Specifically, in some embodiments of the present application, the method further comprises: detecting whether the storage pressure of each block chain node in the target node cluster reaches a preset threshold value; and if so, merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster.
Further, the merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster includes: combining the target node cluster and nodes in the adjacent node clusters of the target node cluster into a larger cluster to obtain a combined node cluster; and carrying out duplicate removal on the same block data stored in each block chain link point in the merged node cluster.
As shown in fig. 4, fig. 4 shows a case of a two-cluster network regrouping into a single node cluster, where a-h are 8 different blocks, and the block ID values are x000, x001, x010, x011, x100, x101, x110, and x111, where x represents the number of bits except the former.
As can be seen from this example, when the number of nodes in a node cluster is 4, according to the last-bit minimum xor mapping mechanism, the block allocation only needs to compare the block ID with the post log24 of the node ID by an xor distance of 2 bits, and finally each block link node is allocated to two blocks, where the storage pressure of the block link points in the node cluster reaches the threshold value, and the number of the block link points in the node cluster needs to be increased, so as to reduce the storage pressure of each block link node in the node cluster, which needs to recombine the two cluster nodes, where the number of the cluster nodes after the recombination becomes 8, and the block allocation compares the post log28 of the block ID with the post log28 of the node ID by an xor distance of 3 bits. Because the last bit minimum exclusive-or mapping mechanism is adopted, when two node clusters are recombined, data exchange is not needed between block chain link points, the node clusters can be recombined only by deleting block data far away from the block chain link points, finally, the storage capacity of each block chain node is reduced to half of the original storage capacity, and the recombined large node clusters jointly store a complete block chain data.
Further, in the embodiment of the present application, a simulation experiment is performed on the DHT-based storage expansion scheme proposed in the present application in a 32-node blockchain network, and compared with a bitcoin native storage policy and a Chord-based storage expansion scheme, an experimental index is a storage scale of a blockchain network node. The parameter settings of the simulation experiment are shown in table 1, the maximum storage of each node is 256MB, the size of each block is set to 1MB, the block chain network node is set to 32 nodes, the number of cluster nodes of the Chord storage extension scheme is set to 4, and the experiment result is shown in fig. 5.
TABLE 1 simulation parameters
As shown in fig. 5, it can be seen that as the number of blocks increases, storage scales of three storage strategies increase correspondingly, where storage scales of bitcoin native nodes and nodes based on a Chord scheme and the number of blocks are in a linear relationship, and a difference is that the node growth speed based on the Chord scheme is slower than that of the bitcoin native nodes, but finally, a storage upper limit 256MB of the nodes is also broken through, and because a dynamic restructuring mechanism is adopted in the nodes based on the Kademlia storage scheme, storage potentials of the nodes are respectively increased by 700% and 100% compared with those of the bitcoin native storage scheme and the Chord scheme.
In order to better implement the block chain storage and expansion method based on DHT in the embodiment of the present application, on the basis of the block chain storage and expansion method based on DHT, an embodiment of the present application further provides a block chain storage and expansion device based on DHT, where the block chain storage and expansion device based on DHT is applied to a block chain system, the block chain system includes a plurality of block chain nodes, the plurality of block chain nodes are divided into a plurality of node clusters, and each node cluster stores a complete piece of block chain data, as shown in fig. 6, the block chain storage and expansion device 600 based on DHT includes:
an obtaining module 601, configured to obtain a hash value of a new block from a block header of the new block when a target node cluster receives the new block, where the hash value of the new block is used as a block ID of the new block, and the block ID is represented by a binary number;
a determining module 602, configured to perform last minimum xor operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster, respectively, and determine a target block chain node with a minimum xor distance as a storage location for storing the new block;
a storage module 603, configured to store the new block to the target block chain node.
In this embodiment of the present application, when a target node cluster receives a new block, the obtaining module 601 obtains a hash value of the new block from a block header of the new block; taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary number; the determining module 602 performs last minimum xor operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster, and determines the target block chain node with the minimum xor distance as a storage location for storing a new block; the storage module 603 stores the new tile to the target tile chain node. In the embodiment of the application, the block chain link points are divided into a plurality of clusters according to the ID values of the nodes, each node cluster stores a piece of complete block chain data, when a new block is generated, the last bits of the block ID values and the last bits of the node ID values in the clusters are subjected to XOR operation, and when the XOR distance is minimum (namely, the distance is closest), the node position of the new block stored in the node cluster is determined, so that the storage pressure of the block chain nodes is reduced, the storage consumption of a block chain system is reduced, and the decentralized characteristic of the block chain system is not influenced.
In some embodiments of the present application, the apparatus further includes a cluster partitioning module, where the cluster partitioning module is specifically configured to:
when a target node cluster receives a new block, dividing the block chain nodes into a plurality of node clusters before obtaining the hash value of the new block from the block head of the new block.
In some embodiments of the present application, the plurality of blockchain nodes are organized in a Kademlia tree, each blockchain node is provided with a corresponding unique node ID, the node ID of each blockchain node follows a Kademlia protocol, a position of each blockchain node in the Kademlia protocol tree is obtained based on the node ID of the blockchain node, the node ID of each blockchain is obtained based on preset node identification information, and the node ID of a blockchain link point is represented by a binary number.
In some embodiments of the present application, the apparatus further includes a node joining module, where the node joining module is specifically configured to:
when a target node cluster receives a joining request of a new block chain node, acquiring node identification information of the new block chain node;
determining a node ID of the new blockchain node based on the node identification information;
determining the node position of the new block chain node in the target node cluster on a Kademlia protocol tree according to a Kademlia protocol;
and adding the new blockchain node into the target node cluster based on the node position.
In some embodiments of the present application, the apparatus further includes a backup module, where the backup module is specifically configured to:
determining a number of backup nodes of the new block before and after the target blockchain node after the storing of the new block to the target blockchain node;
and respectively storing the new blocks at the plurality of backup nodes so as to backup the new blocks.
In some embodiments of the present application, the apparatus further includes a cluster dynamic adjustment module, where the cluster dynamic adjustment module is specifically configured to:
detecting whether the storage pressure of each block chain node in the target node cluster reaches a preset threshold value;
and if so, merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster.
In some embodiments of the present application, the cluster dynamic adjustment module is specifically configured to:
merging the target node cluster and nodes in adjacent node clusters of the target node cluster into a larger cluster to obtain a merged node cluster;
and carrying out duplicate removal on the same block data stored in each block chain link point in the merged node cluster.
An embodiment of the present application further provides an electronic device, which integrates any block chain storage expansion device based on DHT provided in the embodiment of the present application, where the electronic device includes:
one or more processors;
a memory; and
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor as steps in the DHT-based blockchain storage expansion method in any of the above embodiments of the DHT-based blockchain storage expansion method.
The embodiment of the present application further provides an electronic device, which integrates any block chain storage expansion device based on DHT provided by the embodiment of the present application. As shown in fig. 7, a schematic structural diagram of an electronic device according to an embodiment of the present application is shown, specifically:
the electronic device may include components such as a processor 701 of one or more processing cores, memory 702 of one or more computer-readable storage media, a power supply 703, and an input unit 704. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 7 does not constitute a limitation of the electronic device and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. Wherein:
the processor 701 is a control center of the electronic device, connects various parts of the whole electronic device by using various interfaces and lines, and performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 702 and calling data stored in the memory 702, thereby performing overall monitoring of the electronic device. Optionally, processor 701 may include one or more processing cores; preferably, the processor 701 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 701.
The memory 702 may be used to store software programs and modules, and the processor 701 executes various functional applications and data processing by operating the software programs and modules stored in the memory 702. The memory 702 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data created according to use of the electronic device, and the like. Further, the memory 702 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the memory 702 may also include a memory controller to provide the processor 701 with access to the memory 702.
The electronic device further includes a power source 703 for supplying power to each component, and preferably, the power source 703 is logically connected to the processor 701 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The power supply 703 may also include any component including one or more of a dc or ac power source, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The electronic device may also include an input unit 704, and the input unit 704 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the electronic device may further include a display unit and the like, which are not described in detail herein. Specifically, in this embodiment, the processor 701 in the electronic device loads an executable file corresponding to a process of one or more application programs into the memory 702 according to the following instructions, and the processor 701 runs the application programs stored in the memory 702, thereby implementing various functions as follows:
when a target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block; taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary number; respectively carrying out last bit minimum XOR operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster, and determining the target block chain node with the minimum XOR distance as a storage position for storing a new block; and storing the new block to the target block chain node.
It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like. The DHT-based blockchain storage expansion method comprises a computer program stored thereon, and the computer program is loaded by a processor to execute the steps of any DHT-based blockchain storage expansion method provided by the embodiment of the present application. For example, the computer program may be loaded by a processor to perform the steps of:
when a target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block; taking the hash value of the new block as the block ID of the new block, wherein the block ID is represented by binary number; respectively carrying out last bit minimum XOR operation on the last preset number bits of the block ID value and the last preset number bits of the node ID value of each block chain node in the target node cluster, and determining the target block chain node with the minimum XOR distance as a storage position for storing a new block; and storing the new block to the target block chain node.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and parts that are not described in detail in a certain embodiment may refer to the above detailed descriptions of other embodiments, and are not described herein again.
In a specific implementation, each unit or structure may be implemented as an independent entity, or may be combined arbitrarily to be implemented as one or several entities, and the specific implementation of each unit or structure may refer to the foregoing method embodiment, which is not described herein again.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
The block chain storage expansion method based on DHT, the electronic device, and the storage medium provided in the embodiments of the present application are described in detail above, and specific examples are applied in the present application to explain the principles and embodiments of the present application, and the description of the above embodiments is only used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A block chain storage expansion method based on DHT is characterized in that the block chain storage expansion method based on DHT is applied to a block chain system, the block chain system comprises a plurality of block chain nodes, the plurality of block chain nodes are divided into a plurality of node clusters, each node cluster stores a complete block chain data, and the block chain storage expansion method based on DHT comprises the following steps:
when a target node cluster receives a new block, obtaining a hash value of the new block from a block head of the new block;
taking the hash value of the new block as a block ID of the new block, wherein the block ID is represented by binary number;
performing last bit minimum exclusive-or operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster respectively, and determining the target block chain node with the minimum exclusive-or distance as a storage position for storing the new block;
storing the new block to the target blockchain node.
2. The DHT-based blockchain storage expansion method of claim 1, wherein before obtaining the hash value of a new chunk from the chunk header of the new chunk when the target node cluster receives the new chunk, the method further comprises:
and dividing the plurality of block chain nodes into a plurality of node clusters.
3. The DHT-based blockchain storage expansion method according to claim 2, wherein the plurality of blockchain nodes are organized in a Kademlia tree, each blockchain node is provided with a corresponding unique node ID, the node ID of each blockchain node follows a Kademlia protocol, the position of each blockchain node in the Kademlia protocol tree is obtained based on the node ID of the blockchain node, the node ID of each blockchain is obtained based on preset node identification information, and the node ID of the blockchain link point is represented by binary numbers.
4. The DHT-based blockchain storage expansion method of claim 3, wherein the method further comprises:
when a target node cluster receives a joining request of a new block chain node, acquiring node identification information of the new block chain node;
determining a node ID of the new blockchain node based on the node identification information;
determining the node position of the new block chain node in the target node cluster on a Kademlia protocol tree according to a Kademlia protocol;
and adding the new blockchain node into the target node cluster based on the node position.
5. The DHT-based blockchain storage expansion method of claim 1, wherein after said storing of the new block to the target blockchain node, the method further comprises:
determining a plurality of backup nodes of the new block before and after the target block chain node;
and respectively storing the new blocks at the plurality of backup nodes so as to backup the new blocks.
6. The DHT-based blockchain storage expansion method of claim 1, further comprising:
detecting whether the storage pressure of each block chain node in the target node cluster reaches a preset threshold value;
and if so, merging and recombining the target node cluster and the adjacent node cluster of the target node cluster to obtain a merged node cluster.
7. The DHT-based blockchain storage expansion method of claim 6, wherein the merging and recombining the target node cluster and the neighboring node cluster of the target node cluster to obtain a merged node cluster comprises:
merging the target node cluster and nodes in adjacent node clusters of the target node cluster into a larger cluster to obtain a merged node cluster;
and carrying out duplicate removal on the same block data stored in each block chain link point in the merged node cluster.
8. The block chain storage expansion device based on the DHT is applied to a block chain system, the block chain system comprises a plurality of block chain nodes, the plurality of block chain nodes are divided into a plurality of node clusters, each node cluster stores a complete block chain data, and the block chain storage expansion device based on the DHT comprises:
an obtaining module, configured to obtain, when a target node cluster receives a new block, a hash value of the new block from a block header of the new block, and use the hash value of the new block as a block ID of the new block, where the block ID is represented by a binary number;
a determining module, configured to perform last minimum xor operation on the last preset number of bits of the block ID value and the last preset number of bits of the node ID value of each block chain node in the target node cluster, respectively, and determine a target block chain node with a minimum xor distance as a storage location where the new block is stored;
a storage module for storing the new block to the target block chain node.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a memory; and
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the DHT-based blockchain storage expansion method of any one of claims 1 to 7.
10. A computer-readable storage medium, having stored thereon a computer program which is loaded by a processor to perform the steps of the DHT-based blockchain storage expansion method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210322338.3A CN114637472A (en) | 2022-03-30 | 2022-03-30 | DHT-based block chain storage expansion method, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210322338.3A CN114637472A (en) | 2022-03-30 | 2022-03-30 | DHT-based block chain storage expansion method, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114637472A true CN114637472A (en) | 2022-06-17 |
Family
ID=81951273
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210322338.3A Pending CN114637472A (en) | 2022-03-30 | 2022-03-30 | DHT-based block chain storage expansion method, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114637472A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117376352A (en) * | 2023-10-07 | 2024-01-09 | 山东山科智能科技有限公司 | Block chain-based Internet of things system |
-
2022
- 2022-03-30 CN CN202210322338.3A patent/CN114637472A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117376352A (en) * | 2023-10-07 | 2024-01-09 | 山东山科智能科技有限公司 | Block chain-based Internet of things system |
| CN117376352B (en) * | 2023-10-07 | 2024-03-12 | 山东山科智能科技有限公司 | Block chain-based Internet of things system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7362654B2 (en) | Maintenance and management of blockchain blocks in a divided blockchain network | |
| Huang et al. | Brokerchain: A cross-shard blockchain protocol for account/balance-based state sharding | |
| CN110692071B (en) | Script-based blockchain interactions | |
| US11902290B2 (en) | Systems and methods for hybrid blockchain control | |
| US11405181B2 (en) | Lightweight blockchain based on split-trust | |
| CN111630830B (en) | Distributed blockchain data storage under account model | |
| US11190525B2 (en) | Blockchain system and permission management method thereof | |
| US7974221B2 (en) | Efficient content authentication in peer-to-peer networks | |
| KR102566892B1 (en) | Blockchain consensus method, device and system | |
| CN111630507A (en) | Distributed blockchain data storage under account model | |
| CN110754061A (en) | System and method for avoiding or reducing encryption stranded resources on blockchain networks | |
| CN107193490A (en) | A kind of distributed data-storage system and method based on block chain | |
| Li et al. | Blockchain-based security architecture for distributed cloud storage | |
| CN113553375B (en) | Partitioned storage device and method for image type block chain | |
| WO2021190179A1 (en) | Synchronous processing method and related apparatus | |
| CN113238996A (en) | Block chain data archiving method based on DHT, electronic equipment and storage medium | |
| CN111444204B (en) | Synchronous processing method, device, equipment and medium | |
| CN114637472A (en) | DHT-based block chain storage expansion method, electronic equipment and storage medium | |
| Liu et al. | Secure and balanced scheme for non-local data storage in blockchain network | |
| CN113300851B (en) | DHT-based block chain message broadcasting method, electronic equipment and storage medium | |
| An et al. | Blockchain technology research and application: A literature review and future trends | |
| Fernando et al. | Breaking chains, empowering iot: A comparative study of holochain and blockchain | |
| WO2008147400A1 (en) | Authentication for operations over an outsourced file system stored by an untrusted unit | |
| CN113612810B (en) | Method for optimizing performance among cluster nodes based on DHT, electronic equipment and storage medium | |
| Li et al. | BTS-PD: a blockchain based traceability system for P2P distribution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |