WO2022001718A1 - 数据读写方法及装置 - Google Patents

数据读写方法及装置 Download PDF

Info

Publication number
WO2022001718A1
WO2022001718A1 PCT/CN2021/101193 CN2021101193W WO2022001718A1 WO 2022001718 A1 WO2022001718 A1 WO 2022001718A1 CN 2021101193 W CN2021101193 W CN 2021101193W WO 2022001718 A1 WO2022001718 A1 WO 2022001718A1
Authority
WO
WIPO (PCT)
Prior art keywords
data block
space
target data
cluster space
block
Prior art date
Application number
PCT/CN2021/101193
Other languages
English (en)
French (fr)
Inventor
唐振田
Original Assignee
京东科技信息技术有限公司
京东科技控股股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东科技信息技术有限公司, 京东科技控股股份有限公司 filed Critical 京东科技信息技术有限公司
Priority to EP21832633.8A priority Critical patent/EP4174677A1/en
Priority to JP2022581690A priority patent/JP2023532567A/ja
Priority to US18/003,845 priority patent/US20230251796A1/en
Publication of WO2022001718A1 publication Critical patent/WO2022001718A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0673—Single storage device
    • G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]

Definitions

  • the present disclosure generally relates to the technical field of data storage, and more particularly, to a method and apparatus for reading and writing data.
  • the blockchain storage uses memory block data, and the blockchain data is stored in the KV database or file.
  • the KV database is a database that is organized, indexed, and stored in the form of key-value.
  • a key-value pair is established for each block, where Key stores the hash value of the block, and Value stores the data of the block.
  • retrieve blockchain data traverse key-value pairs according to the hash value of the blockchain data to be retrieved, and find the corresponding Value, that is, blockchain data. It can be seen that in the current method, the efficiency of blockchain data retrieval is relatively low.
  • the present disclosure relates to a data reading and writing method, which is applied to an electronic device, wherein a designated physical storage space of the electronic device is divided into at least one data block cluster space, each data block cluster space includes a plurality of block nodes, Each block node is used to store a data block, and the method includes:
  • a read operation or a write operation is performed on the target data block according to the target storage address.
  • the method further comprises:
  • the space information of the data block cluster space at the designated position of the designated storage space at least includes: the cluster index of the data block cluster space, the starting address, and the space of the block node in the data block cluster space Size, the number of block nodes included in the block cluster space.
  • the determining of the target data block cluster space to which the target data block to be read and written belongs includes:
  • the data block cluster space specified by the user is determined as the target data block cluster space to which the target data block to be read and written belongs; or, when the user-specified data block cluster space does not exist, the pre-specified data block cluster space is determined as the to-be-read and written data block cluster space.
  • the target data block is determined according to the starting address of the target data block cluster space, the space size of the block node in the target data block cluster space, and the block index of the target data block
  • the corresponding target storage address including:
  • each data block cluster space includes the same number of block nodes, or each data block cluster space includes a different number of block nodes;
  • the size of the block nodes included in each data block cluster space is the same.
  • block nodes in the same data block cluster space are used to store data blocks of the same collection point.
  • the present disclosure relates to an apparatus for reading and writing data, the apparatus being applied to electronic equipment, and a designated physical storage space of the electronic equipment is divided into at least one data block cluster space, each data block cluster space including a plurality of block nodes , each block node is used to store a data block, and the device includes:
  • the first determining module is configured to determine the target data block cluster space to which the target data block to be read and written belongs;
  • a second determining module configured to determine the target storage address corresponding to the target data block according to the space information of the target data block cluster space and the block index of the target data block;
  • An operation module configured to perform a read operation or a write operation on the target data block according to the target storage address.
  • the device further comprises:
  • a recording module configured to record the space information of the data block cluster space at a designated position of the designated storage space; the space information at least includes: the cluster index of the data block cluster space, the starting address, the data block cluster space The space size of the medium block node and the number of block nodes included in the data block cluster space.
  • the first determination module is configured to:
  • the data block cluster space specified by the user is determined as the target data block cluster space to which the target data block to be read and written belongs; or, when the user-specified data block cluster space does not exist, the pre-specified data block cluster space is determined as the to-be-read and written data block cluster space.
  • the second determination module includes:
  • a first calculation submodule configured to calculate the product of the space size of the block node in the target data block cluster space and the block index of the target data block
  • the second calculation submodule is configured to calculate the sum of the starting address of the target data block cluster space and the product, and determine the calculation result as the target storage address corresponding to the target data block.
  • each data block cluster space includes the same number of block nodes, or each data block cluster space includes a different number of block nodes;
  • the size of the block nodes included in each data block cluster space is the same.
  • block nodes in the same data block cluster space are used to store data blocks of the same collection point.
  • the present disclosure relates to an electronic device comprising: a processor and a memory, the processor is configured to execute a data reading and writing program stored in the memory, so as to implement the data reading and writing method described in the present disclosure.
  • the present disclosure relates to a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the data reading and writing method described in the present disclosure .
  • each block node in the designated physical storage space of the electronic device since each block node in the designated physical storage space of the electronic device is used to store one data block, that is, the block node has a one-to-one mapping relationship with the data block, and each data block cluster space has Cluster index, each block node has a block index, based on this, when reading and writing the target data block, by determining the target data block cluster space to which the target data block belongs, according to the starting address of the target data block cluster space, target data The space size of the block node in the block cluster space and the block index of the target data block can determine the target storage address corresponding to the target data block.
  • FIG. 1 shows an example of a mapping relationship between a "block" of a physical storage medium and a data block
  • FIG. 2 shows a flowchart of a data reading and writing method provided by an embodiment of the present disclosure
  • FIG. 3 shows a block diagram of a data reading and writing device provided by an embodiment of the present disclosure
  • FIG. 4 shows a block diagram of another data reading and writing device provided by an embodiment of the present disclosure.
  • FIG. 5 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the electronic devices herein may be various types of devices, such as smart phones, tablet computers, laptop computers, desktop computers, servers, etc., which are not specifically limited in the present disclosure.
  • the file system is used to organize and allocate the physical storage space of the electronic device, and is responsible for data storage and data retrieval.
  • the file system here may be a user space file system (Filesystem in Userspace, FUSE for short), which is implemented by the kernel module FUSE support. Based on this, the data in the physical storage space of the electronic device can be managed and operated at the application layer.
  • the minimum management unit of a file system is a "block" of physical storage space, and each "block” contains a set number of sectors, such as 2, 4, 8, and 16 sectors.
  • the set number here is related to the total capacity of the physical storage medium on the electronic device, such as a magnetic disk, that is, the number of sectors contained in each "block" is related to the total capacity of the magnetic disk on the electronic device .
  • the set number here can also be set by the user according to actual business requirements, for example, the user can set according to the size of the data block to be managed and operated.
  • the size of a blockchain it is understandable that the blockchain is a data block, hereinafter referred to as a block
  • the size of a sector is 512 bytes. It means that one block of storage space requires two sectors of the physical storage medium. Therefore, the above-mentioned set number can be set to 2, that is, each "block" contains 2 sectors.
  • FIG. 1 it is an example of the mapping relationship between blocks and sectors.
  • each "block” can be used to store a data block, in other words, through this kind of processing, one-to-one mapping between the "blocks" of the physical storage medium and the data blocks can be realized.
  • a designated physical storage space of an electronic device is divided into at least one data block cluster space, and each data block cluster space includes a plurality of block nodes (that is, the above-described "piece").
  • the block nodes in the same data block cluster space are used to store the data blocks of the same collection point, thereby ensuring the continuity of the data of the same collection point on the physical storage medium.
  • block nodes in different data block cluster spaces may be used to store data blocks of different collection points, or to store data blocks of the same collection point.
  • each data block cluster space includes the same number of block nodes, or each data block cluster space includes a different number of block nodes.
  • the designated physical storage space of the electronic device is divided into 100 data block cluster spaces, and each data block cluster space includes 1 million block nodes.
  • the designated physical storage space of the electronic device is divided into 100 data block cluster spaces, of which 50 data block cluster spaces include 1 million block nodes, and the other 50 data block cluster spaces include 500,000 block nodes.
  • the size of the block nodes included in each data block cluster space is the same. That is, the number of sectors included in the block nodes in each data block cluster space is the same.
  • an index is set for each data block cluster space (hereinafter referred to as a cluster index), and an index is set for the block nodes included in each data block cluster space (hereinafter referred to as a block index).
  • the value of the above cluster index starts from 0, that is, the cluster index of the first data block cluster space is 0 (hereinafter referred to as data block cluster space 0), and the cluster indexes of other data block cluster spaces are Incrementing from 0.
  • the cluster index of the second data block cluster space is 1 (hereinafter referred to as data block cluster space 1)
  • the cluster index of the third data block cluster space is 2 (hereinafter referred to as data block cluster space 2).
  • the value of the above block index starts from 0, that is, for a specific data block cluster space, the block index of the first block node included in it is 0 (hereinafter referred to as block node 0, denoted as Block0 ), and the block indices of other block nodes increase from 0.
  • block node 1 the block index of the second block node
  • Block2 the block index of the third block node
  • Block2 the block index of the third block node
  • the spatial information of the data block cluster space may be recorded at a designated location in a designated physical storage space.
  • the designated location herein refers to the header space of multiple sectors starting from the starting sector of the designated physical storage space. For example, assuming that the starting sector of the specified physical storage space is the 10000th sector, and assuming that the specified physical storage space is divided into 100 data block cluster spaces, the space information of the data block cluster space 0 can be recorded in the 10000th sector.
  • the space information of the data block cluster space 1 is recorded in the header space of the 10001st sector, and the space information of the data block cluster space 2 is recorded in the header of the 10002th sector space, and so on, the space information of the data block cluster space 99 is recorded in the header space of the 10099th sector.
  • this also means that the starting sector of the data block cluster space 0 is the 10100th sector.
  • the space information here includes at least: the cluster index of the data block cluster space, the starting address, the space size of the block nodes in the data block cluster space, and the number of block nodes included in the data block cluster space.
  • the space information recorded in the header space of the 10000th sector is as follows:
  • ClusterIndex 0;//Indicates that the cluster index is 0;
  • ClusterBeginAddr 10100;//Indicates that the starting address is the address corresponding to the 10100th sector;
  • BlockSize 1024 bytes; //Indicates that the space size of the block node is 1024 bytes;
  • ClusterBlockNumber 1 million; //indicates including 1 million block nodes.
  • Block0 in the data block cluster space 0 is mapped to the 10100th and 10101st sectors
  • Block1 is mapped to the 10102nd and 10103rd sectors
  • Block2 is mapped to the 10104th and 10105th sectors. area, and so on, which will not be described in detail in the present disclosure.
  • mapping relationship between the block node and the sector can be established through the block index of the block node, and the mapping relationship is shown in the following formula (1):
  • Block(N) (ClusterBeginAddr+2N,ClusterBeginAddr+2N+1)
  • the start address BlockAddr(N) of each block node can also be determined by the block index of the block node, and determined by the following formula (2):
  • BlockAddr(N) ClusterBeginAddr+N*BlockSize Formula (2)
  • FIG. 2 it is a flowchart of a data reading and writing method provided by an embodiment of the present disclosure. As shown in Figure 2, the process includes:
  • the target data block to be read and written refers to the data block to be read from the designated physical storage space of the electronic device, or the data block to be written into the designated physical storage space of the electronic device.
  • the target data block includes a blockchain.
  • sending a data retrieval instruction to the electronic device may be: the user inputs retrieval information on a data retrieval page provided by the electronic device and clicks a button on the data retrieval page for triggering data retrieval. When the electronic device detects that the button is clicked, it means that the electronic device receives a data retrieval instruction.
  • the retrieval information may include a cluster index and a block index of the target data block.
  • the data block cluster space corresponding to the cluster index included in the retrieval information may be determined as the data block cluster space to which the target data block belongs (hereinafter referred to as the target data block cluster space), that is, the user-specified data block
  • the cluster space is determined as the target data block cluster space.
  • the retrieval information may include the block index of the target data block.
  • the pre-specified data block cluster space may be determined as the target data block cluster space, that is, when there is no user-specified data block cluster space, the pre-specified data block cluster space is determined as the target data block cluster space. For example, when the designated physical storage space of the electronic device includes only one data block cluster space, the user can only input the block index of the target data block, that is, the above-mentioned retrieval information only includes the block index of the target data block.
  • a data write instruction may be sent to the electronic device where a block of data is written to a designated physical storage space of the electronic device.
  • the sending of the data writing instruction to the electronic device may be: the user sends the data writing instruction to the electronic device according to actual business requirements.
  • the sending of the data writing instruction to the electronic device may be: the electronic device itself triggers the generation of the data writing instruction.
  • the generation of the data write instruction triggered by the electronic device itself may be: the electronic device periodically triggers the generation of the data write instruction, that is, the electronic device triggers the generation of the data write instruction every set time interval, So that when the electronic device receives the data writing instruction, it writes the data block acquired in the most recent set time interval into the designated physical storage space.
  • the electronic device may trigger the generation of a data write instruction when the number of acquired data blocks reaches a set number, such as 1000, so that when the electronic device receives the data write instruction, the newly acquired data The set number of data blocks are written to the specified physical storage space.
  • the above-described data write instructions may carry a cluster index.
  • the data block cluster space corresponding to the cluster index included in the data writing instruction may be determined as the data block cluster space to which the target data block to be written belongs (hereinafter referred to as the target data block cluster space).
  • the cluster index carrying the data block cluster space in the data write instruction can be set by the user, that is, the data block cluster space specified by the user is determined as the target data block cluster space.
  • the cluster index of the data block cluster space carried in the data write instruction may be set by the electronic device.
  • the electronic device may pre-store the correspondence between the collection points and the data block cluster space. Based on this, the electronic device may determine the corresponding data block cluster space according to the collection point corresponding to the target data block to be written. , and set the cluster index of the determined data block cluster space in the data write instruction.
  • the above data write instruction may also not carry the cluster index of the data block cluster space.
  • the pre-designated data block cluster space may then be determined as the target data block cluster space. For example, when the designated physical storage space of the electronic device includes only one data block cluster space, the above-mentioned data writing instruction may not need to carry the cluster index of the data block cluster space.
  • the recorded space information of the target data block cluster space can be obtained from the specified location of the above-mentioned specified physical storage space.
  • the storage address where the space information of the target data block cluster space is recorded can be determined by the following formula (3):
  • BeginAddr represents the starting sector of the above-mentioned specified physical storage space, such as the 10000th sector
  • Cluster(N) represents the starting sector of the data block cluster space N.
  • the target data block cluster space is data block cluster space 5
  • the space information of the target data block cluster space 5 can be obtained and recorded in the header space of the 10005th sector. Accordingly, the space information of the target data block cluster space 5 can be obtained from the header space of the 10005th sector.
  • the block nodes in the target data block cluster space can be calculated by calculating the Calculate the product of the space size and the block index of the target data block, and then calculate the sum of the starting address of the target data block cluster space and the product to obtain the storage address corresponding to the target data block (hereinafter referred to as the target storage address).
  • the storage address corresponding to the target data block can be obtained by retrieving one sector, which improves the addressing efficiency.
  • the electronic device calls the FUSE kernel module, and the FUSE kernel module calls the read operation function, from The data block is extracted from the physical storage medium corresponding to the target storage address, and the extracted data block is returned as FUSE.
  • FUSE calls the kernel operation again to return the extracted data block to the user process.
  • the electronic device calls the FUSE kernel module, and the FUSE kernel module calls the write operation function to write The target data block is written into the physical storage medium corresponding to the target storage address.
  • each block node in the designated physical storage space of the electronic device is used to store a data block, that is, the block node and the data block have a one-to-one mapping relationship, and each data block cluster space has a cluster. Index, each block node has a block index. Based on this, when reading and writing the target data block, by determining the target data block cluster space to which the target data block belongs, according to the starting address of the target data block cluster space, the target data block The space size of the block node in the cluster space and the block index of the target data block can determine the target storage address corresponding to the target data block.
  • the present disclosure also provides embodiments of a data reading and writing device.
  • FIG. 3 it is a block diagram of a data reading and writing device according to an embodiment of the present disclosure. As shown in Figure 3, the device includes:
  • the first determining module 31 is configured to determine the target data block cluster space to which the target data block to be read and written belongs;
  • the second determination module 32 is configured to determine the target storage address corresponding to the target data block according to the space information of the target data block cluster space and the block index of the target data block;
  • the operation module 33 is configured to perform a read operation or a write operation on the target data block according to the target storage address.
  • the device further comprises:
  • the recording module 34 is configured to record the space information of the data block cluster space at a designated position of the designated storage space; the space information at least includes: the starting address of the data block cluster space, the block in the data block cluster space The space size of the node and the number of block nodes included in the block cluster space.
  • the second determination module 32 includes:
  • a first calculation submodule 321, configured to calculate the product of the space size of the block node in the target data block cluster space and the block index of the target data block;
  • the second calculation sub-module 322 is configured to calculate the sum of the starting address of the target data block cluster space and the product, and determine the calculation result as the target storage address corresponding to the target data block.
  • the first determination module 31 is configured to:
  • the data block cluster space specified by the user is determined as the target data block cluster space to which the target data block to be read and written belongs; or, when the user-specified data block cluster space does not exist, the pre-specified data block cluster space is determined as the to-be-read and written data block cluster space.
  • each data block cluster space includes the same number of block nodes, or each data block cluster space includes a different number of block nodes;
  • the size of the block nodes included in each data block cluster space is the same.
  • block nodes in the same data block cluster space are used to store data blocks of the same collection point.
  • FIG. 5 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.
  • the electronic device 500 shown in FIG. 5 includes: at least one processor 501 , memory 502 , at least one network interface 504 and other user interfaces 503 .
  • the various components in electronic device 500 are coupled together by bus system 505 .
  • the bus system 505 is used to implement the connection communication between these components.
  • the bus system 505 also includes a power bus, a control bus and a status signal bus.
  • the various buses are labeled as bus system 505 in FIG. 5 .
  • the user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, trackball, touch pad or touch screen, etc.).
  • a pointing device eg, a mouse, trackball, touch pad or touch screen, etc.
  • memory 502 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.
  • the non-volatile memory may be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically programmable read-only memory (Erasable PROM, EPROM). Erase programmable read-only memory (Electrically EPROM, EEPROM) or flash memory.
  • Volatile memory may be Random Access Memory (RAM), which acts as an external cache.
  • RAM Static RAM
  • DRAM Dynamic RAM
  • SDRAM Synchronous DRAM
  • SDRAM double data rate synchronous dynamic random access memory
  • Double Data Rate SDRAM DDRSDRAM
  • enhanced SDRAM ESDRAM
  • synchronous link dynamic random access memory Synchlink DRAM, SLDRAM
  • Direct Rambus RAM Direct Rambus RAM
  • memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set of them: an operating system 5021 and applications 5022.
  • the operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks.
  • the application program 5022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services.
  • a program implementing the method of the present disclosure may be included in the application program 5022 .
  • the processor 501 is configured to perform the methods provided by the present disclosure by invoking the programs or instructions stored in the memory 502, which may be programs or instructions stored in the application program 5022, for example, including:
  • a read operation or a write operation is performed on the target data block according to the target storage address.
  • the methods disclosed in the above embodiments of the present disclosure may be applied to the processor 501 or implemented by the processor 501 .
  • the processor 501 may be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above-mentioned method can be completed by an integrated logic circuit of hardware in the processor 501 or an instruction in the form of software.
  • the above-mentioned processor 501 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a ready-made programmable gate array (Field Programmable Gate Array, FPGA) or other Programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor.
  • the software unit may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502, and completes the steps of the above method in combination with its hardware.
  • the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof.
  • the processing unit can be implemented in one or more Application Specific Integrated Circuits (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSP Device, DSPD), programmable Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), general purpose processor, controller, microcontroller, microprocessor, other for performing the functions described in this disclosure electronic unit or a combination thereof.
  • ASIC Application Specific Integrated Circuits
  • DSP Digital Signal Processor
  • DSP Device Digital Signal Processing Device
  • PLD programmable Programmable Logic Device
  • FPGA Field-Programmable Gate Array
  • the techniques described herein may be implemented by means of units that perform the functions described herein.
  • Software codes may be stored in memory and executed by a processor.
  • the memory can be implemented in the processor or external to the processor.
  • the electronic device provided by the present disclosure may be the electronic device shown in FIG. 5 , and can execute the data reading and writing method shown in FIG. 2 , thereby realizing the technical effect of the data reading and writing method shown in FIG. 2 , please refer to FIG. 2 Relevant descriptions are not repeated here for brevity.
  • the present disclosure also provides computer-readable storage media.
  • the storage medium here stores one or more programs.
  • the storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state hard disk; the memory may also include the above-mentioned types of memory. combination.
  • One or more programs in the storage medium can be executed by one or more processors, so as to implement the above-mentioned data reading and writing method executed on the electronic device side.
  • the processor is configured to execute the data reading and writing program stored in the memory, so as to realize the following data reading and writing method performed on the electronic device side:
  • a read operation or a write operation is performed on the target data block according to the target storage address.
  • a software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.
  • RAM random access memory
  • ROM read only memory
  • electrically programmable ROM electrically erasable programmable ROM
  • registers hard disk, removable disk, CD-ROM, or any other in the technical field. in any other known form of storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

公开了数据读写方法及装置，所述方法包括：确定待读写的目标数据块所属的目标数据块集群空间；根据所述目标数据块集群空间的起始地址、所述目标数据块集群空间中块节点的空间大小，以及所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及根据所述目标存储地址对所述目标数据块进行读操作或写操作。

Description

数据读写方法及装置
相关申请的引用
本公开要求于2020年6月30日向中华人民共和国国家知识产权局提交的申请号为202010623609.X、名称为“数据读写方法及装置”的发明专利申请的全部权益，并通过引用的方式将其全部内容并入本文。
领域
本公开大体上涉及数据存储技术领域，更具体地，涉及数据读写方法及装置。
背景
目前，区块链式存储大多采用内存块数据方式，将区块链数据存储在KV数据库或者文件中。以KV数据库为例，KV数据库是一种使用键值(Key-Value)形式进行组织、索引、存储的数据库。在进行区块链数据的存储时，为每一个区块建立一个键值对，其中Key存储该区块的哈希值，Value存储该区块的数据。在进行区块链数据的检索时，根据待检索区块链数据的哈希值遍历键值对，查找到对应的Value，即区块链数据。由此可见，在目前的方式中，区块链数据检索的效率比较低。
概述
一方面，本公开涉及数据读写方法，所述方法应用于电子设备，所述电子设备的指定物理存储空间被划分为至少一个数据块集群空间，每个数据块集群空间包括多个块节点，每个块节点用于存储一个数据块，所述方法包括：
确定待读写的目标数据块所属的目标数据块集群空间；
根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
根据所述目标存储地址对所述目标数据块进行读操作或写操作。
在某些实施方案中，所述方法还包括：
将所述数据块集群空间的空间信息记录在所述指定存储空间的指定位置处；所述空间信息至少包括：数据块集群空间的集群索引、起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。
在某些实施方案中，所述确定待读写的目标数据块所属的目标数据块集群空间，包括：
将用户指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间；或者，在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间。
在某些实施方案中，所述根据目标数据块集群空间的起始地址、所述目标数据块集群空间中块节点的空间大小，以及所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址，包括：
计算所述目标数据块集群空间中块节点的空间大小和所述目标数据块的块索引的乘积；以及
计算所述目标数据块集群空间的起始地址和所述乘积的和，将计算结果确定为所述目标数据块对应的目标存储地址。
在某些实施方案中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同；并且
各个数据块集群空间包括的块节点的空间大小相同。
在某些实施方案中，同一数据块集群空间中的块节点用于存储同一采集点的数据块。
另一方面，本公开涉及数据读写装置，所述装置应用于电子设备，所述电子设备的指定物理存储空间被划分为至少一个数据块集群空间，每个数据块集群空间包括多个块节点，每个块节点用于存储一个数据块，所述装置包括：
第一确定模块，配置为确定待读写的目标数据块所属的目标数据块集群空间；
第二确定模块，配置为根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
操作模块，配置为根据所述目标存储地址对所述目标数据块进行读操作或写操作。
在某些实施方案中，所述装置还包括：
记录模块，配置为将所述数据块集群空间的空间信息记录在所述指定存储空间的指定位置处；所述空间信息至少包括：数据块集群空间的集群索引、起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。
在某些实施方案中，所述第一确定模块配置为：
将用户指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间；或者，在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间。
在某些实施方案中，所述第二确定模块包括：
第一计算子模块，配置为计算所述目标数据块集群空间中块节点的空间大小和所述目标数据块的块索引的乘积；以及
第二计算子模块，配置为计算所述目标数据块集群空间的起始地址和所述乘积的和，将计算结果确定为所述目标数据块对应的目标存储地址。
在某些实施方案中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同；并且
各个数据块集群空间包括的块节点的空间大小相同。
在某些实施方案中，同一数据块集群空间中的块节点用于存储同一采集点的数据块。
又一方面，本公开涉及电子设备，其包括：处理器和存储器，所述处理器配置为执行所述存储器中存储的数据读写程序，以实 现本公开所述的数据读写方法。
再一方面，本公开涉及计算机可读存储介质，其存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现本公开所述的数据读写方法。
在某些实施方案中，由于电子设备的指定物理存储空间中的每个块节点用于存储一个数据块，即，块节点与数据块具有一一映射关系，并且，每个数据块集群空间具有集群索引，每个块节点具有块索引，基于此，在对目标数据块进行读写时，通过确定目标数据块所属的目标数据块集群空间，根据目标数据块集群空间的起始地址、目标数据块集群空间中块节点的空间大小，以及目标数据块的块索引则可以确定目标数据块对应的目标存储地址。这一过程中只需检索一个扇区获得目标数据块集群空间的起始地址等空间信息即可实现寻址，从而提高了寻址效率，进而可以提高数据存储和检索的效率。基于此，在区块链场景下应用本公开某些实施方案提供的数据读写方法，可以提高区块链数据的存储和检索效率。
附图的简要说明
图1示出了物理存储介质的“块”与数据块之间映射关系的一示例；
图2示出了本公开一实施例提供的数据读写方法的流程图；
图3示出了本公开一实施例提供的数据读写装置的框图；
图4示出了本公开一实施例提供的另一数据读写装置的框图；并且
图5示出了本公开一实施例提供的电子设备的结构示意图。
详述
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实 施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
本公开提供的数据读写方法可应用于电子设备。在某些实施方案中，这里的电子设备可以为多种类型的设备，比如智能手机、平板电脑、膝上型便携计算机、台式计算机、服务器等，本公开对此不做具体限定。为便于对本公开实施例的理解，下面首先对电子设备的物理存储空间的存储结构进行说明：
在本公开中，采用文件系统对电子设备的物理存储空间进行组织和分配，并负责数据存储和数据检索。在某些实施方案中，这里的文件系统可以为用户空间文件系统(Filesystem in Userspace，简称FUSE)，通过内核模块FUSE支持实现。基于此，则可以在应用层实现对电子设备的物理存储空间中的数据进行管理和操作。
目前，文件系统的最小管理单位为物理存储空间的“块”，每一“块”包含设定数量的扇区，比如包含2个、4个、8个、16个扇区等。
在某些实施方案中，这里的设定数量与电子设备上物理存储介质，比如磁盘的总容量大小有关，即，每一“块”包含的扇区数与电子设备上磁盘的总容量大小有关。
在某些实施方案中，这里的设定数量还可以由用户根据实际业务需求设置，例如，用户可根据待管理和操作的数据块的大小设置。以区块链为例，假设一个区块链(可以理解的是，区块链则为一个数据块，以下称区块)的大小为1K，并假设一个扇区的大小为512字节，这则意味着一个区块的存储空间需要两个扇区的物理存储介质。因此，可将上述设定数量设置为2，即，每一“块”包含2个扇区。如图1所示，为区块与扇区之间映射关系的一示例。
由此可见，通过该种处理，可以实现每个“块”用于存储一个数据块，换言之，通过该种处理可以实现物理存储介质的“块”与数据块进行一一映射。
在目前的一些存储结构中，比如关系型数据库、区块链KV数据库中，难以保证同一个采集点的数据在物理存储介质上的连续性。为了解决这一问题，本公开提出，在某些实施方案中，将电子设备的指定物理存储空间划分为至少一个数据块集群空间，每个数据块集群空间包括多个块节点(即上述描述的“块”)。其中，同一数据块集群空间中的块节点用于存储同一采集点的数据块，由此则可以保证同一个采集点的数据在物理存储介质上的连续性。
在某些实施方案中，不同数据块集群空间中的块节点可用于存储不同采集点的数据块，或者用于存储同一采集点的数据块。
在某些实施方案中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同。比如，将电子设备的指定物理存储空间划分为100个数据块集群空间，每个数据块集群空间包括100万个块节点。再比如，将电子设备的指定物理存储空间划分为100个数据块集群空间，其中50个数据块集群空间包括100万个块节点，另外50个数据块集群空间包括50万个块节点。
在某些实施方案中，各个数据块集群空间包括的块节点的空间大小相同。即，各个数据块集群空间中块节点包括的扇区数相同。
在某些实施方案中，为了便于寻址，为各个数据块集群空间设置索引(以下称集群索引)，并为各个数据块集群空间包括的块节点设置索引(以下称块索引)。
在某些实施方案中，上述集群索引的值从0开始，即，第一个数据块集群空间的集群索引为0(以下称为数据块集群空间0)，其它数据块集群空间的集群索引则从0开始递增。比如，第二个数据块集群空间的集群索引为1(以下称为数据块集群空间1)，第三个数据块集群空间的集群索引为2(以下称为数据块集群空间2)，以此类推，本公开对此不再一一详述。
需要说明的是，上述集群索引从0开始只是用作举例，并非用作限定。
在某些实施方案中，上述块索引的值从0开始，即，对于一个具体的数据块集群空间，其包括的第一个块节点的块索引为0(以下简称块节点0，记为Block0)，其它块节点的块索引则从0开始递增。比如，第二个块节点的块索引为1(以下简称块节点1，记为Block1)，第三个块节点的块索引为2(以下简称块节点2，记为Block2)，以此类推，本公开对此不再一一详述。
需要说明的是，上述块索引从0开始只是用作举例，并非用作限定。
在某些实施方案中，为了便于寻址，针对每个数据块集群空间，可以将该数据块集群空间的空间信息记录在指定物理存储空间的指定位置处。在某些实施方案中，这里的指定位置是指从指定物理存储空间的起始扇区开始的多个扇区的头部空间。比如，假设指定物理存储空间的起始扇区为第10000个扇区，并假设指定物理存储空间被划分为100个数据块集群空间，则可以将数据块集群空间0的空间信息记录在第10000个扇区的头部空间中，将数据块集群空间1的空间信息记录在第10001个扇区的头部空间中，将数据块集群空间2的空间信息记录在第10002个扇区的头部空间中，以此类推，将数据块集群空间99的空间信息记录在第10099个扇区的头部空间中。同时，这也意味着数据块集群空间0的起始扇区为第10100个扇区。
在某些实施方案中，这里的空间信息至少包括：数据块集群空间的集群索引、起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。比如，以数据块集群空间0为例，在第10000个扇区的头部空间中记录的空间信息如下：
ClusterIndex＝0；//表示集群索引为0；
ClusterBeginAddr＝10100；//表示起始地址为第10100个扇区对应的地址；
BlockSize＝1024字节；//表示块节点的空间大小为1024字节；
ClusterBlockNumeber＝100万；//表示包括100万个块节点。
至此，可以得知：数据块集群空间0中的Block0映射到第10100个和第10101个扇区，Block1映射到第10102个和第10103个扇区，Block2映射到第10104个和第10105个扇区，以此类推，本公开对此不再一一详述。
由此可见，对于一个具体的数据块集群空间，可以通过块节点的块索引建立起块节点与扇区之间的映射关系，该映射关系如下述公式(一)所示：
Block(N)＝(ClusterBeginAddr+2N,ClusterBeginAddr+2N+1)
                                           公式(一)
上述公式(一)表示，Block(N)映射到第(ClusterBeginAddr+2N)个和第(ClusterBeginAddr+2N+1)个扇区。
在某些实施方案中，还可以通过块节点的块索引确定每一个块节点的起始地址BlockAddr(N)，通过如下公式(二)确定：
BlockAddr(N)＝ClusterBeginAddr+N*BlockSize    公式(二)
以上对本公开实施例中电子设备的物理存储空间的存储结构进行了说明。下面将在以上描述的基础上，结合附图以具体实施例对本公开提供的数据读写方法做进一步的解释说明。需要说明的是，以下实施例并不构成对本公开实施例的限定。
参见图2，为本公开一实施例提供的数据读写方法的流程图。如图2所示，该流程包括：
201：确定待读写的目标数据块所属的目标数据块集群空间；
202：根据目标数据块集群空间的空间信息和目标数据块的块索引，确定目标数据块对应的目标存储地址；以及
203：根据目标存储地址对目标数据块进行读操作或写操作。
在本公开中，待读写的目标数据块是指：待从电子设备的指 定物理存储空间中读取的数据块，或者待写入电子设备的指定物理存储空间的数据块。
在某些实施方案中，目标数据块包括区块链。
在某些实施方案中，在从电子设备的指定物理存储空间中读取数据块的情况下，用户可以根据实际业务需求向电子设备发送数据检索指令。这里的向电子设备发送数据检索指令可为：用户在电子设备提供的数据检索页面上输入检索信息并点击数据检索页面上，用于触发进行数据检索的按钮。电子设备在检测到该按钮被点击时，则意味着电子设备接收到数据检索指令。
在某些实施方案中，上述检索信息可以包括集群索引和目标数据块的块索引。在这些实施方案中，则可以将检索信息中包括集群索引对应的数据块集群空间确定为目标数据块所属的数据块集群空间(以下称目标数据块集群空间)，即，将用户指定的数据块集群空间确定为目标数据块集群空间。
在某些实施方案中，上述检索信息可以包括目标数据块的块索引。在这些实施方案中，则可以将预先指定的数据块集群空间确定为目标数据块集群空间，即在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为目标数据块集群空间。举例来说，在电子设备的指定物理存储空间仅包括一个数据块集群空间时，用户可以仅输入目标数据块的块索引，即上述检索信息仅包括目标数据块的块索引。
在某些实施方案中，在向电子设备的指定物理存储空间中写入数据块的情况下，可以向电子设备发送数据写入指令。
在某些实施方案中，这里的向电子设备发送数据写入指令可为：用户根据实际业务需求向电子设备发送数据写入指令。
在某些实施方案中，这里的向电子设备发送数据写入指令可为：由电子设备自身触发生成数据写入指令。在某些实施方案中，由电子设备自身触发生成数据写入指令可为：电子设备周期性地触发生成数据写入指令，即，电子设备每间隔设定时间间隔则触发生成数据写入指令，以使电子设备在接收到数据写入指令时， 将最近一段设定时间间隔内获取到的数据块写入指定物理存储空间中。在某些实施方案中，电子设备可以在获取到的数据块数量达到设定数量，比如1000时，触发生成数据写入指令，以使电子设备在接收到数据写入指令时，将最新获取到的设定数量的数据块写入指定物理存储空间中。
在某些实施方案中，上述数据写入指令可以携带集群索引。在这些实施方案中，则可以将数据写入指令中包括的集群索引对应的数据块集群空间确定为待写入的目标数据块所属的数据块集群空间(以下称目标数据块集群空间)。
在某些实施方案中，数据写入指令中携带数据块集群空间的集群索引可由用户设置，即，将用户指定的数据块集群空间确定为目标数据块集群空间。
在某些实施方案中，数据写入指令中携带的数据块集群空间的集群索引可由电子设备设置。在某些实施方案中，电子设备可以预先存储有采集点和数据块集群空间的对应关系，基于此，电子设备则可以根据待写入的目标数据块对应的采集点确定对应的数据块集群空间，将确定出的数据块集群空间的集群索引设置在数据写入指令中。
在某些实施方案中，上述数据写入指令也可以不携带数据块集群空间的集群索引。在这些实施方案中，则可将预先指定的数据块集群空间确定为目标数据块集群空间。举例来说，在电子设备的指定物理存储空间仅包括一个数据块集群空间时，上述数据写入指令中可无需携带数据块集群空间的集群索引。
由以上描述可知，在某些实施方案中，已知目标数据块集群空间，则可以从上述指定物理存储空间的指定位置处获取已记录的该目标数据块集群空间的空间信息。
在某些实施方案中，可以通过以下公式(三)确定记录有目标数据块集群空间的空间信息的存储地址：
Cluster(N)＝BeginAddr+N      公式(三)
在上述公式(三)中，BeginAddr表示上述指定物理存储空间的起始扇区，比如第10000个扇区，Cluster(N)表示数据块集群空间N的起始扇区。举例来说，假设目标数据块集群空间为数据块集群空间5，基于公式(三)，则可以得到目标数据块集群空间5的空间信息记录在第10005个扇区的头部空间中。据此，则可以从第10005个扇区的头部空间中获取目标数据块集群空间5的空间信息。
在某些实施方案中，由以上公式(二)的相关描述可知，在本步骤202中，已知目标数据块所属的目标数据块集群空间，则可以通过计算目标数据块集群空间中块节点的空间大小和目标数据块的块索引的乘积，然后计算目标数据块集群空间的起始地址和所述乘积的和，得到目标数据块对应的存储地址(以下称目标存储地址)。
由此可见，通过检索一个扇区即可以得到目标数据块对应的存储地址，这则提高了寻址效率。
在某些实施方案中，在从电子设备的指定物理存储空间中读取数据块的情况下，以用户空间文件系统为例，电子设备调用FUSE内核模块，FUSE内核模块则调用read操作函数，从目标存储地址对应的物理存储介质中提取出数据块，并将提取出的数据块返回为FUSE。FUSE再次调用内核操作，将提取出的数据块返回给用户进程。
在某些实施方案中，在向电子设备的指定物理存储空间中写入数据块的情况下，以用户空间文件系统为例，电子设备调用FUSE内核模块，FUSE内核模块则调用write操作函数，将目标数据块写入目标存储地址对应的物理存储介质中。
至此，完成图2所示流程的描述。
由上述实施例可见，由于电子设备的指定物理存储空间中的每个块节点用于存储一个数据块，即，块节点与数据块具有一一映射关系，并且，每个数据块集群空间具有集群索引，每个块节点具有块索引，基于此，在对目标数据块进行读写时，通过确定 目标数据块所属的目标数据块集群空间，根据目标数据块集群空间的起始地址、目标数据块集群空间中块节点的空间大小，以及目标数据块的块索引则可以确定目标数据块对应的目标存储地址。这一过程中只需检索一个扇区获得目标数据块集群空间的起始地址等空间信息即可实现寻址，从而提高了寻址效率，进而可以提高数据存储和检索的效率。基于此，在区块链场景下应用本公开某些实施方案提供的数据读写方法，可以提高区块链数据的存储和检索效率。
与前述数据读写方法的实施例相对应，本公开还提供数据读写装置的实施例。
参见图3，为本公开一实施例提供的数据读写装置的框图。如图3所示，该装置包括：
第一确定模块31，配置为确定待读写的目标数据块所属的目标数据块集群空间；
第二确定模块32，配置为根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
操作模块33，配置为根据所述目标存储地址对所述目标数据块进行读操作或写操作。
在某些实施方案中，如图4所示，所述装置还包括：
记录模块34，配置为将所述数据块集群空间的空间信息记录在所述指定存储空间的指定位置处；所述空间信息至少包括：数据块集群空间的起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。
在某些实施方案中，如图4所示，所述第二确定模块32包括：
第一计算子模块321，配置为计算所述目标数据块集群空间中块节点的空间大小和所述目标数据块的块索引的乘积；以及
第二计算子模块322，配置为计算所述目标数据块集群空间的起始地址和所述乘积的和，将计算结果确定为所述目标数据块对应的目标存储地址。
在某些实施方案中，所述第一确定模块31配置为：
将用户指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间；或者，在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间。
在某些实施方案中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同；并且
各个数据块集群空间包括的块节点的空间大小相同。
在某些实施方案中，同一数据块集群空间中的块节点用于存储同一采集点的数据块。
图5示出了本公开一实施例提供的电子设备的结构示意图，图5所示的电子设备500包括：至少一个处理器501、存储器502、至少一个网络接口504和其他用户接口503。电子设备500中的各个组件通过总线系统505耦合在一起。总线系统505用于实现这些组件之间的连接通信。总线系统505除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图5中将各种总线都标为总线系统505。
在某些实施方案中，用户接口503可以包括显示器、键盘或者点击设备(例如，鼠标，轨迹球(trackball)、触感板或者触摸屏等。
在某些实施方案中，存储器502可以是易失性存储器或非易失性存储器，或可包括易失性和非易失性存储器两者。其中，非易失性存储器可以是只读存储器(Read-Only Memory，ROM)、可编程只读存储器(Programmable ROM，PROM)、可擦除可编程只读存储器(Erasable PROM，EPROM)、电可擦除可编程只读存储器(Electrically EPROM，EEPROM)或闪存。易失性存储器可以是随机存取存储器(Random Access Memory，RAM)，其用作外部高速缓存。通过示例性但不是限制性说明，许多形式的RAM可用，例如静态随机存取存储器(Static RAM，SRAM)、动态随机存取存储器(Dynamic RAM，DRAM)、同步动态随机存取存储器(Synchronous DRAM，SDRAM)、双倍数据速率同步动态随机存取存储器(Double  Data Rate SDRAM，DDRSDRAM)、增强型同步动态随机存取存储器(Enhanced SDRAM，ESDRAM)、同步连接动态随机存取存储器(Synchlink DRAM，SLDRAM)和直接内存总线随机存取存储器(Direct Rambus RAM，DRRAM)。本文描述的存储器502旨在包括但不限于这些和任意其它适合类型的存储器。
在某些实施方案中，存储器502存储了如下的元素，可执行单元或者数据结构，或者他们的子集，或者他们的扩展集：操作系统5021和应用程序5022。
在某些实施方案中，操作系统5021包含各种系统程序，例如框架层、核心库层、驱动层等，用于实现各种基础业务以及处理基于硬件的任务。应用程序5022，包含各种应用程序，例如媒体播放器(Media Player)、浏览器(Browser)等，用于实现各种应用业务。实现本公开方法的程序可以包含在应用程序5022中。
在某些实施方案中，通过调用存储器502存储的程序或指令，其可以是应用程序5022中存储的程序或指令，处理器501配置为执行本公开提供的方法，例如包括：
确定待读写的目标数据块所属的目标数据块集群空间；
根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
根据所述目标存储地址对所述目标数据块进行读操作或写操作。
上述本公开实施例揭示的方法可以应用于处理器501中，或者由处理器501实现。处理器501可能是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器501中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器501可以是通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本公开实施例中的 公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本公开实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件单元组合执行完成。软件单元可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器502，处理器501读取存储器502中的信息，结合其硬件完成上述方法的步骤。
可以理解的是，本文描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processor，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、微处理器、用于执行本公开所述功能的其它电子单元或其组合中。
对于软件实现，可通过执行本文所述功能的单元来实现本文所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
本公开提供的电子设备可以是如图5中所示的电子设备，可执行如图2中所示的数据读写方法，进而实现图2所示数据读写方法的技术效果，请参照图2相关描述，为简洁描述，在此不作赘述。
本公开还提供了计算机可读存储介质。这里的存储介质存储有一个或者多个程序。其中，存储介质可以包括易失性存储器，例如随机存取存储器；存储器也可以包括非易失性存储器，例如只读存储器、快闪存储器、硬盘或固态硬盘；存储器还可以包括上述种类的存储器的组合。
存储介质中一个或者多个程序可被一个或者多个处理器执行， 以实现上述在电子设备侧执行的数据读写方法。
所述处理器配置为执行存储器中存储的数据读写程序，以实现以下在电子设备侧执行的数据读写方法：
确定待读写的目标数据块所属的目标数据块集群空间；
根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
根据所述目标存储地址对所述目标数据块进行读操作或写操作。
专业人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
结合本文中所公开的实施例描述的方法或算法的步骤可以用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的存储介质中。
以上所述的具体实施方式，对本公开的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本公开的具体实施方式而已，并不用于限定本公开的保护范围，凡在本公开的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本公开的保护范围之内。

Claims (14)

  1. 数据读写方法，其应用于电子设备，所述电子设备的指定物理存储空间被划分为至少一个数据块集群空间，每个数据块集群空间包括多个块节点，每个块节点用于存储一个数据块，所述方法包括：
    确定待读写的目标数据块所属的目标数据块集群空间；
    根据所述目标数据块集群空间的空间信息和所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址，所述目标数据块包括区块链；以及
    根据所述目标存储地址对所述目标数据块进行读操作或写操作。
  2. 如权利要求1所述的方法，其还包括：
    将所述数据块集群空间的空间信息记录在所述指定存储空间的指定位置处；所述空间信息至少包括：数据块集群空间的集群索引、起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。
  3. 如权利要求1或2所述的方法，其中，所述确定待读写的目标数据块所属的目标数据块集群空间，包括：
    将用户指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间；或者，在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间。
  4. 如权利要求1至3中任一权利要求所述的方法，其中，所述根据目标数据块集群空间的空间信息，确定所述目标数据块对应的目标存储地址，包括：
    计算所述目标数据块集群空间中块节点的空间大小和所述目标数据块的块索引的乘积；以及
    计算所述目标数据块集群空间的起始地址和所述乘积的和，将计算结果确定为所述目标数据块对应的目标存储地址。
  5. 如权利要求1至4中任一权利要求所述的方法，其中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同；并且
    各个数据块集群空间包括的块节点的空间大小相同。
  6. 如权利要求1至5中任一权利要求所述的方法，其中，同一数据块集群空间中的块节点用于存储同一采集点的数据块。
  7. 数据读写装置，其应用于电子设备，所述电子设备的指定物理存储空间被划分为至少一个数据块集群空间，每个数据块集群空间包括多个块节点，每个块节点用于存储一个数据块，所述装置包括：
    第一确定模块，配置为确定待读写的目标数据块所属的目标数据块集群空间；
    第二确定模块，配置为根据所述目标数据块集群空间的起始地址、所述目标数据块集群空间中块节点的空间大小，以及所述目标数据块的块索引，确定所述目标数据块对应的目标存储地址；以及
    操作模块，配置为根据所述目标存储地址对所述目标数据块进行读操作或写操作。
  8. 如权利要求7所述的装置，其还包括：
    记录模块，配置为将所述数据块集群空间的空间信息记录在所述指定存储空间的指定位置处；所述空间信息至少包括：数据块集群空间的集群索引、起始地址、数据块集群空间中块节点的空间大小、数据块集群空间中包括的块节点数量。
  9. 如权利要求7或8所述的装置，其中，所述第一确定模块配置为：
    将用户指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间；或者，在不存在用户指定的数据块集群空间时，将预先指定的数据块集群空间确定为待读写的目标数据块所属的目标数据块集群空间。
  10. 如权利要求7至9中任一权利要求所述的装置，其中，所述第二确定模块包括：
    第一计算子模块，配置为计算所述目标数据块集群空间中块节点的空间大小和所述目标数据块的块索引的乘积；以及
    第二计算子模块，配置为计算所述目标数据块集群空间的起始地址和所述乘积的和，将计算结果确定为所述目标数据块对应的目标存储地址。
  11. 如权利要求7至10中任一权利要求所述的装置，其中，各个数据块集群空间包括的块节点数量相同，或者各个数据块集群空间包括的块节点数量不相同；并且
    各个数据块集群空间包括的块节点的空间大小相同。
  12. 如权利要求7至11中任一权利要求所述的装置，其中，同一数据块集群空间中的块节点用于存储同一采集点的数据块。
  13. 电子设备，其包括：处理器和存储器，所述处理器配置为执行所述存储器中存储的数据读写程序，以实现权利要求1至6中任一权利要求所述的数据读写方法。
  14. 计算机可读存储介质，其存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现权利要求1至6中任一权利要求所述的数据读写方法。
PCT/CN2021/101193 2020-06-30 2021-06-21 数据读写方法及装置 WO2022001718A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP21832633.8A EP4174677A1 (en) 2020-06-30 2021-06-21 Data reading and writing method and device
JP2022581690A JP2023532567A (ja) 2020-06-30 2021-06-21 データ読み書き方法及び装置
US18/003,845 US20230251796A1 (en) 2020-06-30 2021-06-21 Method and apparatus for reading and writing data

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010623609.X 2020-06-30
CN202010623609.XA CN111782656B (zh) 2020-06-30 2020-06-30 数据读写方法及装置

Publications (1)

Publication Number Publication Date
WO2022001718A1 true WO2022001718A1 (zh) 2022-01-06

Family

ID=72760129

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/101193 WO2022001718A1 (zh) 2020-06-30 2021-06-21 数据读写方法及装置

Country Status (5)

Country Link
US (1) US20230251796A1 (zh)
EP (1) EP4174677A1 (zh)
JP (1) JP2023532567A (zh)
CN (1) CN111782656B (zh)
WO (1) WO2022001718A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111782656B (zh) * 2020-06-30 2024-04-12 京东科技信息技术有限公司 数据读写方法及装置
CN112463753B (zh) * 2020-11-06 2023-01-10 苏州浪潮智能科技有限公司 一种区块链数据存储方法、系统、设备及可读存储介质
CN113377989A (zh) * 2021-06-04 2021-09-10 上海云从汇临人工智能科技有限公司 基于gpu的数据检索方法、系统、介质及装置
CN117234408A (zh) * 2022-06-06 2023-12-15 中科寒武纪科技股份有限公司 基于指令读取数据中的目标数据的方法及其设备
CN117270789B (zh) * 2023-10-25 2024-02-06 苏州元脑智能科技有限公司 一种基于分布式存储的数据控制方法、系统及电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109947703A (zh) * 2017-11-09 2019-06-28 北京京东尚科信息技术有限公司 文件系统、文件存储方法、存储装置和计算机可读介质
CN110134323A (zh) * 2018-02-02 2019-08-16 河南许继仪表有限公司 一种flash存储器
US20190306190A1 (en) * 2018-03-30 2019-10-03 Konica Minolta Laboratory U.S.A., Inc. Method for adjusting mining difficulty of a cryptocurrency blockchain system by monitoring malicious forks and implementing a miners blockchain
CN111339106A (zh) * 2020-05-18 2020-06-26 杭州趣链科技有限公司 一种区块链数据索引的方法
CN111782656A (zh) * 2020-06-30 2020-10-16 北京海益同展信息科技有限公司 数据读写方法及装置

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7487321B2 (en) * 2004-04-19 2009-02-03 Cisco Technology, Inc. Method and system for memory leak detection
US8266365B2 (en) * 2008-12-17 2012-09-11 Sandisk Il Ltd. Ruggedized memory device
US8619866B2 (en) * 2009-10-02 2013-12-31 Texas Instruments Incorporated Reducing memory bandwidth for processing digital image data
US9396106B2 (en) * 2011-05-12 2016-07-19 Avago Technologies General Ip (Singapore) Pte. Ltd. Advanced management of a non-volatile memory
US9916112B1 (en) * 2013-09-30 2018-03-13 EMC IP Holding Company LLC Efficient file copy that avoids data duplication
US10496283B2 (en) * 2016-01-22 2019-12-03 Suraj Prabhakar WAGHULDE Adaptive prefix tree based order partitioned data storage system
US10880071B2 (en) * 2018-02-23 2020-12-29 Samsung Electronics Co., Ltd. Programmable blockchain solid state drive and switch
FR3079323B1 (fr) * 2018-03-26 2020-04-17 Commissariat A L'energie Atomique Et Aux Energies Alternatives Methode et systeme d'acces a des donnees anonymisees
US20190305959A1 (en) * 2018-04-02 2019-10-03 Ca, Inc. Announcement smart contracts to announce software release
WO2019236638A1 (en) * 2018-06-06 2019-12-12 Argosoperem Llc Method and system for data storage and retrieval
WO2020041928A1 (zh) * 2018-08-27 2020-03-05 深圳市锐明技术股份有限公司 数据存储方法、系统及终端设备
CN110096542A (zh) * 2019-04-29 2019-08-06 百度在线网络技术（北京）有限公司 去中心化的数据验证处理方法、装置、系统和介质
CN110263014A (zh) * 2019-05-15 2019-09-20 广州致链科技有限公司 面向时序型数据的区块链存储系统及方法
US10860259B1 (en) * 2019-07-17 2020-12-08 Tyson York Winarski Multi-tiered storage system for blockchain
US11411743B2 (en) * 2019-10-01 2022-08-09 Tyson York Winarski Birthday attack prevention system based on multiple hash digests to avoid collisions

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109947703A (zh) * 2017-11-09 2019-06-28 北京京东尚科信息技术有限公司 文件系统、文件存储方法、存储装置和计算机可读介质
CN110134323A (zh) * 2018-02-02 2019-08-16 河南许继仪表有限公司 一种flash存储器
US20190306190A1 (en) * 2018-03-30 2019-10-03 Konica Minolta Laboratory U.S.A., Inc. Method for adjusting mining difficulty of a cryptocurrency blockchain system by monitoring malicious forks and implementing a miners blockchain
CN111339106A (zh) * 2020-05-18 2020-06-26 杭州趣链科技有限公司 一种区块链数据索引的方法
CN111782656A (zh) * 2020-06-30 2020-10-16 北京海益同展信息科技有限公司 数据读写方法及装置

Also Published As

Publication number Publication date
EP4174677A1 (en) 2023-05-03
CN111782656B (zh) 2024-04-12
CN111782656A (zh) 2020-10-16
JP2023532567A (ja) 2023-07-28
US20230251796A1 (en) 2023-08-10

Similar Documents

Publication Publication Date Title
WO2022001718A1 (zh) 数据读写方法及装置
JP6556911B2 (ja) 注釈付きアトミック書き込み操作を行う方法および装置
KR102147905B1 (ko) 어드레스 기반의 멀티-스트림 스토리지 장치 액세스
WO2021104380A1 (zh) 数据存储方法、装置及存储系统
US9817765B2 (en) Dynamic hierarchical memory cache awareness within a storage system
US10452562B2 (en) File access method and related device
JP5238685B2 (ja) フラッシュ・ストレージの離散領域を記述して問い合わせること
US9727452B2 (en) Distributing metadata across multiple different disruption regions within an asymmetric memory system
TWI804466B (zh) 擷取記憶體中儲存的資料的方法與去重複模組
US10891074B2 (en) Key-value storage device supporting snapshot function and operating method thereof
US7096342B2 (en) Flexible LUN/LBA interface for content addressable reference storage
US10296250B2 (en) Method and apparatus for improving performance of sequential logging in a storage device
US20200225882A1 (en) System and method for compaction-less key-value store for improving storage capacity, write amplification, and i/o performance
US9430492B1 (en) Efficient scavenging of data and metadata file system blocks
WO2021159711A1 (zh) 一种b+树的存取方法、装置和计算机可读存储介质
WO2015024406A1 (zh) 一种数据文件的管理方法及装置
US11449270B2 (en) Address translation method and system for KV storage device
WO2020192710A1 (zh) 基于lsm数据库的垃圾处理的方法、固态硬盘以及存储装置
US8516194B2 (en) Systems and methods for caching data with a nonvolatile memory cache
WO2020103468A1 (zh) 基于闪存的垃圾处理的方法、固态硬盘以及存储装置
WO2021232743A1 (zh) 缓存管理方法、装置、存储介质和固态非易失存储设备
WO2023071043A1 (zh) 文件聚合兼容方法、装置、计算机设备和存储介质
JP7462790B2 (ja) メタデータ記憶方法およびデバイス
US11782905B1 (en) Method and system for streaming data from portable storage devices
WO2024082702A1 (zh) 数据处理方法、装置、芯片以及计算机可读存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21832633

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022581690

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021832633

Country of ref document: EP

Effective date: 20230130