CN112463753B - Block chain data storage method, system, equipment and readable storage medium - Google Patents
Block chain data storage method, system, equipment and readable storage medium Download PDFInfo
- Publication number
- CN112463753B CN112463753B CN202011229981.9A CN202011229981A CN112463753B CN 112463753 B CN112463753 B CN 112463753B CN 202011229981 A CN202011229981 A CN 202011229981A CN 112463753 B CN112463753 B CN 112463753B
- Authority
- CN
- China
- Prior art keywords
- target
- block
- directory
- target block
- area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 63
- 238000003860 storage Methods 0.000 title claims abstract description 41
- 238000013500 data storage Methods 0.000 title claims abstract description 30
- 238000013507 mapping Methods 0.000 claims abstract description 59
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004519 manufacturing process Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 10
- 238000010586 diagram Methods 0.000 description 17
- 239000007787 solid Substances 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 7
- 101000824318 Homo sapiens Protocadherin Fat 1 Proteins 0.000 description 5
- 102100022095 Protocadherin Fat 1 Human genes 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 239000004065 semiconductor Substances 0.000 description 4
- 239000010410 layer Substances 0.000 description 3
- 230000001133 acceleration Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 101100334010 Drosophila melanogaster sotv gene Proteins 0.000 description 1
- 101000824299 Homo sapiens Protocadherin Fat 2 Proteins 0.000 description 1
- 102100022093 Protocadherin Fat 2 Human genes 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
Abstract
The invention discloses a block chain data storage method, a system, a device and a readable storage medium, wherein the method comprises the following steps: the block file system acquires a target block serial number and target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses; sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block serial numbers and the target cluster addresses in a directory area; and writing the target block contents into the data area sequentially according to the target cluster address. In the method, the block file system can directly write the content of the target block into the data area in sequence after determining the target cluster address in the process of storing the block, and the directory and data writing can be performed in sequence, so that a large amount of searching time and data writing time can be saved, and the data writing speed in a block chain can be increased.
Description
Technical Field
The present invention relates to the field of blockchain technologies, and in particular, to a method, a system, a device, and a readable storage medium for storing blockchain data.
Background
The technology of blockchain storage essentially refers to that each blockchain storage system (such as the simplest desktop, middle and high-end server) is used as the minimum storage unit to manage, and it is hoped to use the characteristics of blockchain, and add distributed storage, so as to achieve security, data fault-tolerant backup, and so on at the protocol layer or software application layer.
Current computer systems, when developing blockchain storage technology, consider not only data storage but also program execution at the software level file system and storage controller, and compatibility with earlier systems. This limits the development of new techniques for memory optimization of blockchain storage. Thus, under the increasing demand of data storage and fast storage, the current block chain storage technology still has the problem of slow data storage speed.
In summary, how to effectively solve the problems of slow data storage speed in the block chain is a technical problem that needs to be solved urgently by those skilled in the art.
Disclosure of Invention
The invention aims to provide a method, a system and equipment for storing block chain data and a readable storage medium, which are used for accelerating the data storage speed of a block chain.
In order to solve the technical problems, the invention provides the following technical scheme:
a method of blockchain data storage, comprising:
the block file system acquires a target block serial number and target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses;
sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block sequence numbers and the target cluster addresses in the directory area;
and sequentially writing the target block contents into the data area according to the target cluster address.
Preferably, the method further comprises the following steps:
receiving a read request for reading the target block;
searching the target cluster address corresponding to the target block sequence number in the directory area;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
Preferably, the method further comprises the following steps:
reading the directory in the directory area, and writing the directory into a cache to obtain a cache directory;
correspondingly, in the directory area, recording a target mapping relationship between the target block sequence number and the target cluster address, including:
and recording the target mapping relation in the cache directory.
Preferably, the method further comprises the following steps:
receiving a read request for reading the target block;
searching the cache directory for the target cluster address corresponding to the target block sequence number;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
Preferably, the method further comprises the following steps:
reading a directory in the directory area, and copying the directory into a CAM (computer-aided manufacturing) to obtain a CAM directory;
correspondingly, in the directory area, recording a target mapping relationship between the target block sequence number and the target cluster address, including:
and recording the target mapping relation in the CAM directory.
Preferably, the method further comprises the following steps:
receiving a read request for reading the target block;
in the CAM directory, inquiring the target cluster address corresponding to the target block sequence number;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
Preferably, sequentially writing the target block contents into the data area comprises:
and writing the target block content into a continuous hard disk sector corresponding to the data area.
A block file system, comprising:
the to-be-stored block acquiring module is used for acquiring a target block serial number and target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses;
a mapping relation recording module, configured to sequentially allocate target cluster addresses to the target blocks, and record a target mapping relation between the target block sequence numbers and the target cluster addresses in the directory area;
and the data writing module is used for sequentially writing the target block contents into the data area according to the target cluster address.
An electronic device, comprising:
a memory for storing a computer program;
and the processor is used for realizing the steps of the block chain data storage method when executing the computer program.
A readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, realizes the steps of the above block chain data storage method.
By applying the method provided by the embodiment of the invention, the block file system acquires the target block serial number and the target block content of the target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses; sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block serial numbers and the target cluster addresses in the directory area; and writing the target block contents into the data area in sequence according to the target cluster address.
In the method, the block file system sets the size of the cluster of the data area as the block size by means of the unique serial number of each block of the block chain, namely the block serial number, and the fixed size of the block, so that the mapping relation recorded in the directory area corresponds to the mapping relation between the block serial number and the cluster address. That is, the block corresponds to the cluster, and the directory area directly uses the block number as the cluster index. The simple and clear block chain directory structure and the mapping relation with the data area, and one block corresponds to one cluster. According to the method, in the process of storing the block, after the target cluster address is determined, the content of the target block can be directly written into the data area in sequence.
Accordingly, embodiments of the present invention further provide a block file system, a device, and a readable storage medium corresponding to the block chain data storage method, which have the above technical effects and are not described herein again.
Drawings
In order to more clearly illustrate the embodiments of the present invention or technical solutions in related arts, the drawings used in the description of the embodiments or related arts will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating an embodiment of a method for storing blockchain data according to the present invention;
FIG. 2 is a block file system according to an embodiment of the present invention;
FIG. 3 is a diagram of a FAT file system format;
FIG. 4 is a diagram illustrating data access according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating data access based on a cache directory according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating a CAM directory-based data access operation according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating a CAM directory according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of another embodiment of a CAM directory-based data access;
FIG. 9 is a block file system according to an embodiment of the present invention;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the disclosure, the invention will be described in further detail with reference to the accompanying drawings and specific embodiments. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
In order to facilitate understanding of the technical solutions provided by the embodiments of the present invention, related technical terms are briefly described below:
block Chain (Block Chain) refers to a series of word records (also called blocks) that are cryptographically connected in series and protect content.
An InterPlanetary File System (IPFS) is a distributed File System that attempts to connect all computing devices to the same File System, where the IPFS protocol combines the advantages of blockchain technology and various network protocols to store unalterable data, remove duplicate files on the network, and obtain address information of storage nodes to search for a File IPFS in the network to segment the File into fixed-size blocks, where each block is stored in the distributed File System with a unique set of serial number (CID/ID) indices and block contents. The distributed file system reads all blocks based on the CID and restores the blocks to the original file.
The File System (File System), which is a module in the current operating System, is responsible for managing and storing data in the form of files in the computer System.
The File Allocation Table (FAT) is a File system invented by microsoft and owned by some patents for MS-DOS and all Windows systems.
Ext2: the most traditional disk file system in Linux is followed by Ext3/4, etc., which has the same basic principle as FAT, but can be regarded as an improvement on FAT.
Combined storage (CAM), or Content Addressable Memory, is a special type of computer storage that is applied to some very high-speed search programs. The CAM not only has a data comparator and an address encoder, but also has the characteristic of data access in a common memory, and provides high-speed data searching capability. In a conventional application, a search is performed for the contents of a specific network packet in a semiconductor integrated circuit, and the query performance can reach more than 200M/s (200 million per second) in the current semiconductor technology. The CAM memory uses static memory (SRAM) with a latency of as little as 5 ns per lookup, and if the CAM memory uses dynamic memory (DRAM) with a large memory space requirement, the latency of each lookup is about 100-150 ns.
System Call (System Call): the kernel provides a series of functions. These system calls are implemented in the kernel and are then given to the user in some way. A system call is an interface for a user program to interact with the kernel.
A Driver (Device Driver) refers to a Device Driver, and is a special program that enables a computer and a Device/apparatus to communicate with each other. The operating system can only control the operation of the hardware device/equipment through the interface corresponding to the hardware interface.
Logical Block Addresses (LBAs) are a common mechanism used on computer system data storage devices to represent where data is located.
The Physical Block Address (PBA) is a data sector of the mechanical hard disk or a data page of the solid state disk. For compatibility, the software program will request the file from the system by the logical block address when reading and writing the file, the file system and the storage device controller of the system will convert it into the physical block address, find the real address of the data and read and write. And the conversion mode from the LBA to the PBA of the data sector of the mechanical hard disk or the data paging of the solid state disk.
Sector (Sector), the minimum storage unit of a traditional mechanical hard disk, the basic unit of disk reading and writing is the Sector, and the basic unit of a solid state disk is paging.
Clusters/clusters (Cluster), i.e. a group of consecutive sectors, each Cluster/Cluster may comprise 2, 4, 8, 16, 32, 64 \8230, i.e. the nth power sector of 2, the smallest unit of a file system operating file being a Cluster.
File Directory Table (FDT).
Linked list (linked list) refers to a data structure, the linked list is a non-continuous and non-sequential storage structure on a physical storage unit, and the logical sequence of data elements is realized by the link order of pointers in the linked list. The linked list is made up of a series of nodes (each element in the linked list is called a node), which can be dynamically generated at runtime. Each node comprises two parts: one is a data field that stores the data element and the other is a pointer field that stores the address of the next node. Compared with a linear table sequential structure, the operation is complex. Since the sequential storage is not required, the linked list can be inserted with O (1) complexity much faster than another linear list sequence table, but O (n) time is required to find a node or access a node with a specific number, and O (logn) and O (1) are the time complexities corresponding to the linear list and sequence table, respectively.
Referring to fig. 1 and fig. 4, in which fig. 1 is a flowchart illustrating a method for storing blockchain data according to an embodiment of the present invention, and fig. 4 is a schematic diagram illustrating a data access according to an embodiment of the present invention.
As can be seen from fig. 4, the writing and reading processes of the blocks of the block chain need to go through the software program, the system call, and the file system, and finally go to the physical hard disk. Wherein. The file system (in conjunction with the storage device controller) plays a key role in converting the logical block addresses mapped by the software program and the operating system kernel into physical block addresses of the hard disk. Actually read and write to the hard disk to obtain the block content corresponding to the required file (one file corresponds to one or more blocks). The method can be applied to a block file system in a data access schematic architecture as shown in fig. 4, and comprises the following steps:
s101, the block file system acquires a target block serial number and a target block content of a target block.
The block file system comprises a directory area and a data area, the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses.
The target block may be any block to be stored.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating a format of a block file system according to an embodiment of the invention. It can be seen that the mapping relationship between the CID and the cluster address is stored in the directory area, and the size of each cluster in the data area is consistent with the size of the block. I.e. the block file system has the following features/characteristics:
the size of a cluster in the block file system is the block size, and block data can be searched more quickly;
the block file system simplifies the directory structure and the mapping with the data area;
each sequence number (CID) in the directory area represents each block;
each cluster address in the directory area = a cluster address of the data area.
To facilitate understanding of the features of the block file system provided by the embodiment of the present invention, reference is made to fig. 3, where fig. 3 is a schematic diagram of a format of a FAT file system. As shown in fig. 3, assuming that the hard disk space is arranged in a linear manner, the simplified FAT file system is represented from left to right as follows:
a boot sector: the method is positioned at the initial position and mainly records important information of system startup and a file system;
FAT1/FAT2: two file allocation tables, which are for system redundancy reasons, FAT indicates how clusters (clusters) are stored;
root directory area: a directory table for storing files and directory information;
a data area: the area where the data is actually stored.
Compared with the FAT file system, the block file system directory structure in the embodiment of the invention is simpler and clearer.
For example, a directory is simply a correspondence of block size to actual data read and write.
Suppose that the FILE system wants to read/write the FILE name FILE1, first search in the root directory area, and find the beginning cluster of FILE1 in the 3 rd cluster by FDT (FILE directory table), and then know the next cluster of FILE1 by the linked list in FAT1 until the last cluster is 5,7,8. That is, FILE1 requires 4 clusters as storage space. Assuming that each sector of the hard disk is 512 bytes, the FILE system plan is 8 sectors per cluster, i.e. each cluster is 4096bytes =4k bytes, and that FILE1 occupies 4 clusters in the data area for 16K bytes. It can be seen that reading and writing files in the file system actually takes time to search in the root directory area and FAT1, respectively, to access data, which includes various cooperative work between the software program, the file system and the hardware controller and the hard disk.
In the block file system in the embodiment of the present invention, the block size is used as the cluster size, one block corresponds to one cluster, and the block sequence number can be searched in the directory, so that the search process can be omitted, and particularly, the directory and the data writing sequential writing (see S102 and S103 for details) can accelerate the data writing speed.
S102, sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block sequence numbers and the target cluster addresses in the directory area.
After the target block sequence number and the target block content of the target block are obtained, target cluster addresses can be sequentially allocated to the target block.
The cluster addresses are sequentially allocated, specifically, a target serial number and target block content are obtained, and the target block serial number is sequentially written into the directory area according to the directory area and the data area cluster address (mapping each cluster address in the directory area), that is, the first received block chain block cluster address is allocated to be 0, the hard disk cluster address is written into 0, the second received block chain block cluster address is allocated to be 1, the hard disk cluster address 1 is written into, and so on, and the next available space is written into. In contrast, in a common file system, the hard disk available space does not need to be searched, and the disk searching time is saved by sequentially writing.
It should be noted that, in the present embodiment, the target cluster addresses are sequentially allocated to the target blocks, and only one cluster address needs to be allocated, that is, one cluster is convenient for one block to correspond to.
After the target cluster address is allocated, the mapping relationship between the address and the target block can be determined, that is, the mapping relationship exists between the target block serial number and the target cluster address.
After the target mapping relationship is defined, the target mapping relationship may be stored in the directory area. As can be seen, in this embodiment, the directory area only needs to store the mapping relationship between the target block number and the target cluster address, and the storage location of the block content can be directly retrieved based on the block number.
S103, writing the target block content into the data area sequentially according to the target cluster address.
According to the target cluster address, the contents of the target block can be written into the data area sequentially in a sequential writing mode.
The sequential data writing can refer to sequential writing of block sequence numbers, which is not described in detail herein.
Since one block corresponds to one cluster, the content of the target block is written into the continuous space of the disk in the process of writing the content of the target block into the data area, so that the data reading and writing speed can be improved. That is, sequentially writing the target block contents into the data area includes: and writing the content of the target block into the continuous hard disk sector corresponding to the data area.
By applying the method provided by the embodiment of the invention, the block file system acquires the target block serial number and the target block content of the target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses; sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block serial numbers and the target cluster addresses in the directory area; and writing the target block contents into the data area sequentially according to the target cluster address.
In the method, the block file system sets the size of the cluster of the data area as the block size by means of the unique serial number of each block of the block chain, namely the block serial number, and the fixed size of the block, so that the mapping relation recorded in the directory area corresponds to the mapping relation between the block serial number and the cluster address. That is, the block corresponds to the cluster, and the directory area directly uses the block number as the cluster index. The simple and clear directory structure of the block chain and the mapping relation with the data area, and one block corresponds to one cluster. According to the method, in the process of storing the block, after the target cluster address is determined, the content of the target block can be directly written into the data area in sequence.
It should be noted that, based on the above embodiments, the embodiments of the present invention also provide corresponding improvements. In the preferred/improved embodiment, the same steps as those in the above embodiment or corresponding steps may be referred to each other, and corresponding advantageous effects may also be referred to each other, which are not described in detail in the preferred/improved embodiment herein.
In an embodiment of the present invention, a processing procedure for quickly reading a target block is further provided, and a specific implementation process includes:
step one, receiving a reading request for reading a target block;
step two, inquiring a target cluster address corresponding to the target block sequence number in a directory area;
reading the target block content from the data area by using the target cluster address;
and step four, outputting the target block content.
For convenience of description, the above four steps will be described in combination.
Because the directory area records the target mapping relationship between the target block serial number and the target cluster address storing the target block content, after receiving the read request for reading the target block, the directory area can directly inquire the target cluster address corresponding to the target block serial number. After the target cluster address is obtained, the target block content can be directly read from the data area, and then the target block content is output. That is, after receiving a request for reading a target block, the contents of the block can be obtained and recovered in the hard disk data area by only searching the directory area for a target cluster address corresponding to the target block.
Compared with the current file system (such as FAT for example), the time for searching the directory and waiting for reading can be saved.
In one embodiment of the present invention, the data access speed can be increased by caching the directory. The specific implementation mode comprises the following steps: reading a directory in the directory area, and writing the directory into a cache to obtain a cache directory; correspondingly, in the directory area, recording a target mapping relationship between a target block number and a target cluster address, including: and recording the target mapping relation in the cache directory. Accordingly, the data reading process may specifically include:
step one, receiving a reading request for reading a target block;
step two, searching a target cluster address corresponding to the target block serial number in a cache directory;
reading the target block content from the data area by using the target cluster address;
and step four, outputting the target block content.
For convenience of description, the data storage and the data reading based on the cache directory will be described below in combination.
Referring to fig. 5, fig. 5 is a schematic diagram illustrating data access based on a cache directory according to an embodiment of the present invention.
Specifically, when the computer system is powered on and started, the hard disk directory table (i.e., the directory table stored in the directory area) can be completely or partially copied to the cache directory. Specifically, if the cache is large enough or the system volatile memory is used in combination, all directories can be copied to the cache directory; of course, only the hot data correspondence directory may be copied to the cache directory.
Writing to a block of the block chain: the sequence number and content of the target block are obtained, the cache directory is updated (sequential write update can be adopted) first, and then the sequence number and content of the target block are written into the hard disk directory area and the data area sequentially according to the available hard disk space. Further, when updating the cache directory, the directory of the hard disk may not be updated synchronously each time, but only when the system is powered off, the system is about to be powered off, or the system is idle (i.e. there is no request for writing/reading the blockchain block), the directory of the hard disk is updated.
Reading a blockchain block: after the serial number of the target block to be read is determined, the content of the target block can be obtained and output from the hard disk only by searching the cache directory for the target cluster address mapping the serial number of the target block. Since the cache directory is searched faster than the directory of the hard disk, the reading speed can be increased.
It can be seen that the use of cache targets can improve the overall performance of the write/read blockchain block for both data storage and reading.
In one embodiment of the present invention, a CAM directory may also be used to speed up the speed of storing and reading block chain data. The specific implementation mode comprises the following steps: reading a directory in the directory area, and copying the directory into a CAM (computer-aided manufacturing) to obtain a CAM directory; correspondingly, in the directory area, recording a target mapping relationship between a target block number and a target cluster address, including: in the CAM directory, a target mapping relationship is recorded. Accordingly, a data reading process, comprising:
step one, receiving a reading request for reading a target block;
step two, inquiring a target cluster address corresponding to the target block sequence number in the CAM directory;
reading the target block content from the data area by using the target cluster address;
and step four, outputting the target block content.
For convenience of description, data storage and data reading based on the CAM directory will be described below in combination.
Referring to fig. 6, fig. 6 is a schematic diagram illustrating data access based on a CAM directory according to an embodiment of the invention.
The CAM can match with the block file directory to quickly find the address, i.e. the cluster address, in the CAM directory by using CID as the content.
Fig. 7 may be specifically referred to as a CAM directory, and fig. 7 is a schematic diagram of a CAM directory according to an embodiment of the present invention. In the following fig. 7, the top is a schematic block diagram of the CAM directory in the present text, and the bottom is a schematic diagram of the CAM general design in the semiconductor integrated circuit. The CAM shown in FIG. 7 includes a data comparator, a memory (CELL) and an address encoder, and the example shown in FIG. 7 is the address of the query Qm345678 \8230.
On the basis of a storage device controller, the addition of the CAM directory can realize block read-write acceleration in a hardware layer. When the computer system is powered on and started, the hard disk completely copies the directory table to the CAM directory of the storage device controller.
Writing to a block of the block chain: the program receives the target block serial number and the target block content, first updates the CAM directory (by sequential writing), and then sequentially writes the target block serial number and the target block content into the hard disk directory area and the data area according to the available hard disk space. Since the memory device controller and the CAM directory are highly integrated hardware circuits, the data reading and writing speed is faster, and the effect of updating the CAM directory on the performance of the write blockchain block is less. Of course, in practical applications, when updating the CAM directory, the directory of the hard disk is not updated each time synchronously, but only updated when the system is powered off, the system is about to be powered off, or the system is idle (for example, there is no request for writing/reading the blockchain block).
It should be noted that, no matter how to update the directory, it is necessary to ensure that the CAM directory and the hard disk directory area maintain data synchronization.
Reading a blockchain block: the read target block serial number is determined, and the block content can be obtained and recovered from the hard disk only by searching the target cluster address mapping the target block serial number from the CAM directory. Because the CAM directory is faster to search than the directory of the hard disk and faster than the cache directory, the reading speed can be improved to the maximum extent.
In this embodiment, a CAM memory, a dynamic memory (DRAM) or a static memory (SRAM) may be used.
Further, referring to fig. 8, fig. 8 shows an embodiment of the present invention in which the system is configured with a block file system in the software core layer, and the memory device controller and the CAM directory can be designed and manufactured by the current semiconductor integrated circuit technology; with corresponding software, driver and storage device controller plus CAM directory, each storage device controller plus CAM directory, multiple hard disk system levels can be accessed, storage device controller plus CAM directory,
corresponding to the above method embodiments, the embodiment of the present invention further provides a block file system, and the block file system described below and the above block chain data storage method may be referred to correspondingly.
Referring to fig. 9, the system includes the following modules:
a to-be-stored block acquiring module 101, configured to acquire a target block serial number and a target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses;
a mapping relation recording module 102, configured to sequentially allocate target cluster addresses to target blocks, and record a target mapping relation between a target block sequence number and a target cluster address in a directory area;
and the data writing module 103 is configured to sequentially write the target block contents into the data area according to the target cluster address.
By applying the system provided by the embodiment of the invention, the block file system acquires the target block serial number and the target block content of the target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses; sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block serial numbers and the target cluster addresses in the directory area; and writing the target block contents into the data area sequentially according to the target cluster address.
In the system, the block file system sets the size of the cluster of the data area as the block size by means of the unique serial number, i.e. the block serial number, of each block of the block chain and the fixed size of the block, so that the mapping relation recorded in the directory area corresponds to the mapping relation between the block serial number and the cluster address. That is, the block corresponds to the cluster, and the directory area directly uses the block number as the cluster index. The simple and clear directory structure of the block chain and the mapping relation with the data area, and one block corresponds to one cluster. The system can directly write the target block content into the data area in sequence after determining the target cluster address in the process of storing the block, and the data writing can be written in sequence due to simple directory, so that a large amount of searching time and data writing time can be saved, and the data writing speed in a block chain can be accelerated.
In one embodiment of the present invention, the method further comprises: the read data module is used for receiving a read request for reading a target block; searching a target cluster address corresponding to the target block serial number in a directory area; reading the target block content from the data area by using the target cluster address; and outputting the target block content.
In an embodiment of the present invention, the method further comprises:
the cache acceleration module is used for reading the directory in the directory area and writing the directory into the cache to obtain a cache directory;
accordingly, the mapping relation recording module 102 is specifically configured to record the target mapping relation in the cache directory.
In an embodiment of the present invention, the method further comprises: the read data module is used for receiving a read request for reading a target block; searching a target cluster address corresponding to the target block serial number in a cache directory; reading the target block content from the data area by using the target cluster address; and outputting the target block content.
In one embodiment of the present invention, the method further comprises: the CAM accelerating module is used for reading the directory in the directory area and copying the directory into the CAM to obtain the CAM directory;
accordingly, the mapping relation recording module 102 is specifically configured to record the target mapping relation in the CAM directory.
In an embodiment of the present invention, the method further comprises: the read data module is used for receiving a read request for reading a target block; in the CAM directory, inquiring a target cluster address corresponding to the target block sequence number; reading the target block content from the data area by using the target cluster address; and outputting the target block content.
In an embodiment of the present invention, the data writing module 103 is specifically configured to write the content of the target block into a continuous hard disk sector corresponding to the data area.
For better understanding of the technical effects of the block chain data storage method and the block file system provided by the embodiments of the present invention, the following describes the block file system provided by the embodiments of the present invention in detail with reference to the related art.
The results of comparing the performance of the block file system and the file system based on the mechanical hard disk in the embodiment of the invention are shown in table 1:
TABLE 1
Wherein, the comparison condition only lists the time required by the file system and the hard disk.
The FAT directory and the FAT1/2 linked list are read from the hard disk;
read-write a block chain block, block size 256KB;
mechanical hard disks each sector size =4KB;
FAT uses 4KB (corresponding to 1 sector size) and 64KB (corresponding to 16 sector sizes) per cluster size.
The block file system is of size =256KB per cluster (corresponding to 64 sectors).
SAS is 15000rpm, i.e., the average access time of the hard disk = rotational lag time + head seek time = about 2+3.5=5.5ms;
the SAS is 15000rpm, i.e. the assumption of random read/write of 4KB/64KB on the hard disk is up to 12MB/s, and the assumption of read/write above 256KB or sequential read/write is up to 120MB/s.
In addition to the special situation of sector damage in the block file system, the characteristic of sequential writing can make the hard disk sector continuous, which cannot guarantee the sector continuity for the FAT file system, and the random read/write performance of 4KB/64KB is achieved.
The results of comparing the performance of the block file system and the performance of the file system based on the solid state disk in the embodiment of the invention are shown in tables 2 and 3:
TABLE 2
TABLE 3
The comparison conditions only list the time required in relation to the file system and the hard disk.
Reading both the FAT directory and the FAT1/2 linked list from the hard disk;
read-write a block chain block, block size 256KB;
hard disk size =4KB per sector;
FAT uses 4KB and 64KB per cluster size;
the size per cluster of the block file system =256KB;
the average access time (i.e. seek time) of the solid state disk is about 100us;
m.2pcie solid state disk: the read-write or sequential read-write assumption above 256K can reach more than 3000 MB/s; 4KB random read; the writing assumptions are approximately: 50MB/s;150MB/s;
64KB random read; the writing assumptions are approximately: 250MB/s:500MB/s;
the block file system has the characteristic of sequential writing except the special condition of damaged sectors, so that the hard disk sectors are also continuous, and the sectors cannot be ensured to be continuous relative to the FAT file system.
In summary, the performance of the block file system in the embodiment of the present invention is improved as follows:
based on a mechanical hard disk: the read-write efficiency of the block file system is 10.1 times to 22.95 times faster;
based on the solid state disk: the block file system reading efficiency is 6.22 times to 26.93 times faster; block file system write performance is 4.86 times to 17.85 times faster.
The performance comparison is stored in the mechanical hard disk and the solid state hard disk by a 256KB block chain block for analysis, so that quantized analysis comparison data is obtained.
Corresponding to the above method embodiment, an embodiment of the present invention further provides an electronic device, and a reference between the electronic device described below and the above block chain data storage method may be referred to correspondingly.
Referring to fig. 10, the electronic device includes:
a memory 332 for storing computer programs;
the processor 322 is configured to implement the steps of the block chain data storage method of the foregoing method embodiment when executing the computer program.
Specifically, referring to fig. 11, fig. 11 is a schematic structural diagram of an electronic device provided in this embodiment, which may generate relatively large differences due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, where the memory 332 stores one or more computer applications 342 or data 344. Memory 332 may be, among other things, transient storage or persistent storage. The program stored in memory 332 may include one or more modules (not shown), each of which may include a sequence of instructions operating on a data processing device. Still further, the central processor 322 may be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the electronic device 301.
The electronic device 301 may also include one or more power sources 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341.
The steps in the blockchain data storage method described above may be implemented by the structure of an electronic device.
Corresponding to the above method embodiment, an embodiment of the present invention further provides a readable storage medium, and a readable storage medium described below and a block chain data storage method described above may be referred to correspondingly.
A readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the block chain data storage method of the above-mentioned method embodiment.
The readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
Those of skill would further appreciate that the various illustrative components and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
Claims (9)
1. A method for storing blockchain data, comprising:
the block file system acquires a target block serial number and target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between block serial numbers and cluster addresses;
sequentially distributing target cluster addresses for the target blocks, and recording a target mapping relation between the target block serial numbers and the target cluster addresses in the directory area;
writing the target block contents into the data area in sequence according to the target cluster address;
wherein, writing the target block content into the data area sequentially comprises:
and writing the target block content into a continuous hard disk sector corresponding to the data area.
2. The blockchain data storage method of claim 1, further comprising:
receiving a read request for reading the target block;
searching the target cluster address corresponding to the target block sequence number in the directory area;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
3. The blockchain data storage method of claim 1, further comprising:
reading the directory in the directory area, and writing the directory into a cache to obtain a cache directory;
correspondingly, in the directory area, recording a target mapping relationship between the target block sequence number and the target cluster address, including:
and recording the target mapping relation in the cache directory.
4. The blockchain data storage method of claim 3, further comprising:
receiving a read request for reading the target block;
searching the cache directory for the target cluster address corresponding to the target block sequence number;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
5. The blockchain data storage method of claim 1, further comprising:
reading a directory in the directory area, and copying the directory into a CAM (computer-aided manufacturing) to obtain a CAM directory;
correspondingly, in the directory area, recording a target mapping relationship between the target block sequence number and the target cluster address, including:
and recording the target mapping relation in the CAM directory.
6. The blockchain data storage method of claim 5, further comprising:
receiving a read request for reading the target block;
in the CAM directory, inquiring the target cluster address corresponding to the target block sequence number;
reading the target block content from the data area by using the target cluster address;
and outputting the target block content.
7. A block file system, comprising:
the storage block acquisition module is used for acquiring a target block serial number and target block content of a target block; the block file system comprises a directory area and a data area, wherein the size of each cluster in the data area is the same as the block size of a block chain, and the directory area stores the mapping relation between a block sequence number and a cluster address;
a mapping relation recording module, configured to sequentially allocate target cluster addresses to the target blocks, and record a target mapping relation between the target block sequence numbers and the target cluster addresses in the directory area;
the data writing module is used for sequentially writing the target block content into the data area according to the target cluster address; wherein, sequentially writing the target block content into the data area comprises: and writing the target block content into a continuous hard disk sector corresponding to the data area.
8. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the blockchain data storage method of any one of claims 1 to 6 when executing the computer program.
9. A readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the blockchain data storage method according to any one of claims 1 to 6.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011229981.9A CN112463753B (en) | 2020-11-06 | 2020-11-06 | Block chain data storage method, system, equipment and readable storage medium |
| PCT/CN2021/089875 WO2022095346A1 (en) | 2020-11-06 | 2021-04-26 | Blockchain data storage method, system, device, and readable storage medium |
| US18/246,659 US20240045853A1 (en) | 2020-11-06 | 2021-04-26 | Blockchain data storage method, system, device, and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011229981.9A CN112463753B (en) | 2020-11-06 | 2020-11-06 | Block chain data storage method, system, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112463753A CN112463753A (en) | 2021-03-09 |
| CN112463753B true CN112463753B (en) | 2023-01-10 |
Family
ID=74825887
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011229981.9A Active CN112463753B (en) | 2020-11-06 | 2020-11-06 | Block chain data storage method, system, equipment and readable storage medium |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US20240045853A1 (en) |
| CN (1) | CN112463753B (en) |
| WO (1) | WO2022095346A1 (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112463753B (en) * | 2020-11-06 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Block chain data storage method, system, equipment and readable storage medium |
| CN113420083B (en) * | 2021-06-02 | 2024-03-19 | 湖南大学 | System with heterogeneous parallel block chain structure of extensible distributed ledger |
| CN113806803B (en) * | 2021-09-17 | 2023-06-02 | 厦门服云信息科技有限公司 | Data storage method, system, terminal equipment and storage medium |
| CN114925401A (en) * | 2022-06-14 | 2022-08-19 | 北京师范大学 | Learning condition recording system and method based on block chain and distributed storage |
| CN116893787B (en) * | 2023-09-06 | 2023-12-05 | 四川易利数字城市科技有限公司 | Disk storage method based on blockchain big data application |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794024A (en) * | 2015-04-15 | 2015-07-22 | 四川神琥科技有限公司 | Data recovery method |
| CN107644056A (en) * | 2017-08-04 | 2018-01-30 | 武汉烽火众智数字技术有限责任公司 | A kind of file memory method, apparatus and system |
| CN110286859A (en) * | 2019-06-28 | 2019-09-27 | 中国海洋大学 | Date storage method and device based on FAT file system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008206A (en) * | 2019-03-22 | 2019-07-12 | 深圳前海微众银行股份有限公司 | A kind of data processing method and device based on block catenary system |
| KR102204605B1 (en) * | 2019-04-25 | 2021-01-19 | 주식회사 웨이투빗 | Method and device for providing blockchain capable of optimizing storages of each node included therein |
| CN111782656B (en) * | 2020-06-30 | 2024-04-12 | 京东科技信息技术有限公司 | Data reading and writing method and device |
| CN112463753B (en) * | 2020-11-06 | 2023-01-10 | 苏州浪潮智能科技有限公司 | Block chain data storage method, system, equipment and readable storage medium |
-
2020
- 2020-11-06 CN CN202011229981.9A patent/CN112463753B/en active Active
-
2021
- 2021-04-26 US US18/246,659 patent/US20240045853A1/en active Pending
- 2021-04-26 WO PCT/CN2021/089875 patent/WO2022095346A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104794024A (en) * | 2015-04-15 | 2015-07-22 | 四川神琥科技有限公司 | Data recovery method |
| CN107644056A (en) * | 2017-08-04 | 2018-01-30 | 武汉烽火众智数字技术有限责任公司 | A kind of file memory method, apparatus and system |
| CN110286859A (en) * | 2019-06-28 | 2019-09-27 | 中国海洋大学 | Date storage method and device based on FAT file system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112463753A (en) | 2021-03-09 |
| US20240045853A1 (en) | 2024-02-08 |
| WO2022095346A1 (en) | 2022-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112463753B (en) | Block chain data storage method, system, equipment and readable storage medium | |
| US11636032B2 (en) | Memory system, data storage device, user device and data management method thereof | |
| KR101944876B1 (en) | File access method and apparatus and storage device | |
| US6907505B2 (en) | Immediately available, statically allocated, full-logical-unit copy with a transient, snapshot-copy-like intermediate stage | |
| JP3371044B2 (en) | Area allocation method and disk array access method for disk array | |
| US4536837A (en) | Improved disk file allocation and mapping system utilizing cylinder control blocks and file map having unbalanced tree structure | |
| US7035881B2 (en) | Organization of read-write snapshot copies in a data storage system | |
| US8966476B2 (en) | Providing object-level input/output requests between virtual machines to access a storage subsystem | |
| US8285967B1 (en) | Method for on-demand block map generation for direct mapped LUN | |
| CN100405304C (en) | Method for realizing high speed solid storage device based on storage region network | |
| US6567887B2 (en) | Buffering of partition tables, file system directory structures and individual file cluster chains in a mass storage device | |
| US8694563B1 (en) | Space recovery for thin-provisioned storage volumes | |
| WO2013162954A1 (en) | Efficient data object storage and retrieval | |
| US7987328B2 (en) | Data archive system | |
| JP4222917B2 (en) | Virtual storage system and operation method thereof | |
| US20030115408A1 (en) | Cache storage system and method | |
| US11366609B2 (en) | Technique for encoding deferred reference count increments and decrements | |
| US10013201B2 (en) | Region-integrated data deduplication | |
| JP4394467B2 (en) | Storage system, server apparatus, and preceding copy data generation method | |
| US11874795B2 (en) | Targetless snapshot virtual replication data pointer table | |
| US20180316758A1 (en) | Method and apparatus for logical mirroring to a multi-tier target node | |
| JPH06110766A (en) | Directory constitution method and computer system for sharing stored file system | |
| WO2022262381A1 (en) | Data compression method and apparatus | |
| US11847100B2 (en) | Distributed file system servicing random-access operations | |
| US11409664B2 (en) | Logical memory allocation and provisioning using timestamps |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |