WO2024108939A1 - 一种多级映射框架、数据操作请求处理方法及系统 - Google Patents

一种多级映射框架、数据操作请求处理方法及系统 Download PDF

Info

Publication number
WO2024108939A1
WO2024108939A1 PCT/CN2023/095867 CN2023095867W WO2024108939A1 WO 2024108939 A1 WO2024108939 A1 WO 2024108939A1 CN 2023095867 W CN2023095867 W CN 2023095867W WO 2024108939 A1 WO2024108939 A1 WO 2024108939A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
layer
write
page layer
deletion
Prior art date
Application number
PCT/CN2023/095867
Other languages
English (en)
French (fr)
Inventor
李飞龙
许永良
孙明刚
Original Assignee
苏州元脑智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州元脑智能科技有限公司 filed Critical 苏州元脑智能科技有限公司
Publication of WO2024108939A1 publication Critical patent/WO2024108939A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/08—Addressing or allocation; Relocation in hierarchically structured memory systems, e.g. virtual memory systems
    • G06F12/10—Address translation
    • G06F12/1027—Address translation using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90—Details of database functions independent of the retrieved data types
    • G06F16/95—Retrieval from the web
    • G06F16/957—Browsing optimisation, e.g. caching or content distillation
    • G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Definitions

  • the present application relates to the technical field of data storage, and in particular to a multi-level mapping framework, a data operation request processing method and a system.
  • the industry has used multi-level cache methods to accelerate the speed of accessing I/O data through multi-level hash mapping, and used cluster technology composed of multiple control nodes.
  • the master node is responsible for processing the I/O requests of the host
  • the auxiliary node is responsible for the background tasks of the storage system (for example, RAID array initialization, inspection and reconstruction tasks, etc.), so as to improve the I/O performance of the storage system.
  • these two methods still cannot significantly improve I/O performance.
  • the multi-storage systems of multiple bank branches are usually shared by multiple host devices through a network, and the applications running on multiple host devices contain multiple application processes with diverse functions. These processes send I/O requests through the host to store data safely and reliably on the storage systems of each branch. In this case, data duplication and deletion are inevitable, resulting in a waste of storage resources.
  • the purpose of this application is to provide a multi-level mapping framework, a data operation request processing method, a data operation request processing system, an electronic device and a non-volatile storage medium, which can avoid the waste of storage resources caused by data duplication and deletion.
  • the technical problem solved by the present application is how to avoid the waste of storage resources caused by data duplication and deletion.
  • some embodiments of the present application provide a multi-level mapping framework, which is constructed based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes the root page layer, the top page layer, the middle page layer, the leaf page layer, the virtual block layer and the physical block layer from top to bottom; the data storage cluster is used to perform the operation corresponding to the data operation request according to the multi-level mapping framework;
  • the root page layer is used to implement the mapping of the top page layer
  • the top page layer is used to allocate volume identifiers and implement the mapping of the middle page layer.
  • the mapping of the page layer, the middle page layer is used to cache data and realize the mapping of the leaf page layer
  • the leaf page layer is used to set the data write count field
  • the virtual block layer is used to update the value of the data write count field
  • the physical block layer is used to store data
  • the data write count field is used to control whether to perform operations on the physical block layer.
  • Some embodiments of the present application further provide a data operation request processing method, which is applied to a data storage cluster.
  • the data storage cluster runs the above-mentioned multi-level mapping framework.
  • the data operation request processing method includes:
  • mapping table is written to the corresponding physical block layer, and the value of the data writing count field of the mapping table is set to 1;
  • the writing mode is logical writing
  • the data in the mapping table is written to the value of the count field plus 1.
  • the method before querying the inode index corresponding to the first host LBA using the inode table, the method further includes:
  • the inode table stores the correspondence between the inode index and the storage system.
  • the virtual block layer is used to determine the write mode according to the data write count field of the mapping table, including:
  • the virtual block layer is used to determine that the write mode of the data to be written is logical write
  • the virtual block layer is used to determine that the writing mode of the data to be written is physical writing.
  • using the intermediate page layer to cache the data to be written corresponding to the data write request includes:
  • the hot data area is used to store data with a read/write frequency greater than a preset value
  • the cold data area is used to store data with a read/write frequency less than or equal to a preset value.
  • using the top page layer to query the data read and write frequency of the host device that sends the data write request includes:
  • the page structure corresponding to the host device that sends the data write request in the top page layer is calculated according to the first host LBA, and the data read and write frequency of the host device that sends the data write request is queried in the page structure.
  • writing the data to be written into the leaf page layer corresponding to the volume identifier includes:
  • the data to be written cached in the hot data area or the cold data area of the middle page layer is written to the leaf page layer corresponding to the volume identifier.
  • the method after writing the mapping table to the physical block layer, the method further includes:
  • the data in the physical block layer is written to the RAID array.
  • a data read request is received, determining a second host LBA of the host device that sent the data read request, and querying a data storage location of the to-be-read data corresponding to the data read request in the multi-level mapping framework according to the second host LBA;
  • the method after querying the data storage location of the to-be-read data corresponding to the data read request in the multi-level mapping framework according to the second host LBA, the method further includes:
  • the data read and write frequency of the host device that sends the data read request is updated in the top page layer.
  • the present application also provides a data operation request processing method, which is applied to a data storage cluster.
  • the data storage cluster runs the above-mentioned multi-level mapping framework.
  • the data operation request processing method includes:
  • a data deletion request is received, determining a third host LBA of the host device that sent the data deletion request, and querying a data write calculation field of the to-be-deleted data corresponding to the data deletion request in the multi-level mapping framework according to the third host LBA;
  • the deletion method is physical deletion
  • the data to be deleted is deleted in the physical block layer, and the value of the data write count field of the data to be deleted is set to 0;
  • the deletion method is logical deletion, the data to be deleted is written into the count field with the value minus 1.
  • the deletion method is determined according to the data written into the calculation field of the data to be deleted, including:
  • the virtual block layer determines that the deletion method of the data to be deleted is logical deletion
  • the virtual block layer determines that the deletion method of the data to be deleted is physical deletion.
  • Some embodiments of the present application further provide a data operation request processing system, which is applied to a data storage cluster.
  • the data storage cluster runs the multi-level mapping framework as described above.
  • the data operation request processing system includes:
  • a write request receiving module for determining a first host LBA of a host device that sends the data write request if a data write request is received, and querying an inode index corresponding to the first host LBA using an inode table;
  • An identifier determination module used to map the first host LBA and inode index to the corresponding top page layer, and allocate a volume identifier for the data to be written in the top page layer;
  • a cache module used for caching the to-be-written data corresponding to the data write request using the intermediate page layer
  • the leaf page layer writing module is used to write the data to be written into the leaf page layer corresponding to the volume identifier
  • a virtual block layer writing module used to compress the data to be written into a mapping table using the leaf page layer corresponding to the volume identifier and add a data write count field, and write the mapping table and the corresponding data write count field into the virtual block layer;
  • a physical block layer write module is used to use the virtual block layer to determine the write mode according to the data write count field of the mapping table; wherein the write mode includes physical write and logical write; and is also used to write the mapping table to the corresponding physical block layer if the write mode is physical write, and set the value of the data write count field of the mapping table to 1; and is also used to add 1 to the value of the data write count field of the mapping table if the write mode is logical write.
  • Some embodiments of the present application further provide a data operation request processing system, which is applied to a data storage cluster.
  • the data storage cluster runs the above-mentioned multi-level mapping framework.
  • the data operation request processing system includes:
  • a query module for determining a third host LBA of a host device that sends a data deletion request if a data deletion request is received, and querying a data write calculation field of the to-be-deleted data corresponding to the data deletion request in a multi-level mapping framework according to the third host LBA;
  • a deletion method determination module is used to determine the deletion method according to the data written into the calculation field of the data to be deleted; wherein the deletion methods include physical deletion and logical deletion;
  • the deletion execution module is used to delete the data to be deleted in the physical block layer and set the value of the data write count field of the data to be deleted to 0 if the deletion method is physical deletion; it is also used to reduce the value of the data write count field of the data to be deleted by 1 if the deletion method is logical deletion.
  • Some embodiments of the present application further provide a non-volatile storage medium on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned data operation request processing method are implemented.
  • Some embodiments of the present application further provide an electronic device including a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps of executing the above-mentioned data operation request processing method are implemented.
  • Some embodiments of the present application provide a multi-level mapping framework, which is constructed based on the inode index and B-tree in a data storage cluster.
  • the multi-level mapping framework includes a root page layer, a top page layer, a middle page layer, a leaf page layer, a virtual block layer and a physical block layer from top to bottom; the data storage cluster is used to perform operations corresponding to data operation requests according to the multi-level mapping framework; wherein the root page layer is used to implement mapping of the top page layer, the top page layer is used to allocate volume identifiers and implement mapping of the middle page layer, the middle page layer is used to cache data and implement mapping of the leaf page layer, the leaf page layer is used to set a data write count field, the virtual block layer is used to update the value of the data write count field, the physical block layer is used to store data, and the data write count field is used to control whether to perform operations on the physical block layer.
  • the embodiment of the present application constructs a multi-level mapping framework based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes a root page layer, a top page layer, a middle page layer, a leaf page layer, a virtual block layer and a physical block layer from top to bottom. If the data storage cluster receives a data operation request, the corresponding operation is performed according to the multi-level mapping framework.
  • a data write count field is used to count the number of times the same data is stored in the data storage cluster, and then the data write count field is used to control whether to perform operations on the physical block layer to control data rewriting and deletion.
  • the embodiment of the present application also provides a data operation request processing method, a data operation request processing system, a non-volatile storage medium and an electronic device, which have the above-mentioned beneficial effects and are not repeated here.
  • FIG1 is a flow chart of a method for processing a data operation request provided by some embodiments of the present application.
  • FIG2 is a schematic diagram of the structure of a multi-level mapping framework provided in some embodiments of the present application.
  • FIG3 is a schematic diagram of a virtual block layer with a data write count of 1 provided by some embodiments of the present application.
  • FIG4 is a schematic diagram of a virtual block layer with a data write count of 2 provided by some embodiments of the present application.
  • FIG5 is a schematic diagram of the structure of a data operation request processing system provided by some embodiments of the present application.
  • FIG6 is a schematic diagram of the structure of an electronic device provided in some embodiments of the present application.
  • FIG. 7 is a schematic diagram of the structure of a non-volatile storage medium provided in some embodiments of the present application.
  • Some embodiments of the present application provide a multi-level mapping framework, which is constructed based on the inode index and B-tree within a data storage cluster.
  • the multi-level mapping framework includes a root page layer, a top page layer, a middle page layer, a leaf page layer, a virtual block layer and a physical block layer from top to bottom; the data storage cluster is used to execute operations corresponding to data operation requests according to the multi-level mapping framework.
  • the root page layer is used to realize the mapping of the top page layer
  • the top page layer is used to allocate volume identifiers and realize the mapping of the middle page layer
  • the middle page layer is used to cache data and realize the mapping of the leaf page layer
  • the leaf page layer is used to set the data write count field
  • the virtual block layer is used to update the value of the data write count field
  • the physical block layer is used to store data
  • the data write count field is used to control whether to perform operations on the physical block layer.
  • the above embodiment constructs a multi-level mapping framework based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes the root page layer, the top page layer, the middle page layer, the leaf page layer, the virtual block layer and the physical block layer from top to bottom. If the data storage cluster receives a data operation request, the corresponding operation is performed according to the multi-level mapping framework.
  • the data write count field is used to count the number of times the same data is stored in the data storage cluster, and then the data write count field is used to control whether to perform operations on the physical block layer to control data rewriting and deletion. Therefore, this embodiment can avoid the waste of storage resources caused by data duplication and deletion.
  • FIG. 1 is a flowchart of a data operation request processing method provided in some embodiments of the present application.
  • S101 construct a multi-level mapping framework based on the inode index and B-tree in the data storage cluster;
  • this embodiment can be applied to a data storage cluster, which can include multiple host devices.
  • a data storage cluster which can include multiple host devices.
  • a multi-level mapping framework can be constructed based on the above inode index and B-tree (also known as B-tree, which is a self-balancing tree that can keep data in order.
  • B-tree also known as B-tree, which is a self-balancing tree that can keep data in order.
  • This data structure enables the actions of searching data, sequentially accessing, inserting data, and deleting to be completed in logarithmic time).
  • the above multi-level mapping framework may include a root page layer, a top page layer, a middle page layer, a leaf page layer, a virtual block layer and a physical block layer from top to bottom.
  • the root page layer is used to implement the mapping of the top page layer.
  • the top page layer is used to allocate a volume (logical volume, a logical unit for host I/O access to data, composed of logical unit blocks) identifier and implement the mapping of the middle page layer.
  • the middle page layer is used to cache data and implement the mapping of the leaf page layer.
  • the leaf page layer is used to set the data write count field.
  • the virtual block layer is used to update the physical block layer. The value of the new data write count field, the data write count field is used to control whether to perform operations on the physical block layer.
  • the data operation request may be a data write request, a data delete request or a data read request, and the data operation request may be a request sent by a client to a data storage cluster.
  • the present application can determine the leaf page layer where the data to be written, deleted or read is located through a multi-level mapping framework, and then perform the corresponding data writing operation, data deletion operation or data reading operation.
  • This embodiment constructs a multi-level mapping framework based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes the root page layer, the top page layer, the middle page layer, the leaf page layer, the virtual block layer and the physical block layer from top to bottom. If the data storage cluster receives a data operation request, the corresponding operation is performed according to the multi-level mapping framework.
  • the data write count field is used to count the number of times the same data is stored in the data storage cluster, and then the data write count field is used to control whether to perform operations on the physical block layer to control data rewriting and deletion. Therefore, this embodiment can avoid the waste of storage resources caused by data duplication and deletion.
  • the operation corresponding to the data operation request is performed according to the multi-level mapping framework, including the following steps:
  • Step A1 Determine the first host LBA (logic block address) of the host device that sends the data write request, and use the inode table to query the inode index corresponding to the first host LBA;
  • the inode table Before using the inode table to query the inode index corresponding to the first host LBA, the inode table can also be read from the super block data structure of the Linux kernel; wherein the inode table stores the correspondence between the inode index and the storage system.
  • Step A2 Map the first host LBA and inode index to the corresponding top page layer, and allocate a volume identifier for the data to be written in the top page layer;
  • Step A3 Utilize the intermediate page layer to cache the data to be written corresponding to the data write request;
  • the process of using the middle page layer to cache data to be written includes: using the top page layer to query the data read and write frequency of the host device that sends the data write request; caching the data to be written to the hot data area or cold data area of the middle page layer according to the data read and write frequency; wherein the hot data area is used to store data with a read and write frequency greater than a preset value, and the cold data area is used to store data with a read and write frequency less than or equal to a preset value.
  • the process of using the top page layer to query the data read and write frequency of the host device that sends the data write request in the above process includes: calculating the page structure corresponding to the host device that sends the data write request in the top page layer according to the first host LBA, The data read and write frequency of the host device that sends the data write request is queried in the page structure.
  • Step A4 write the data to be written into the leaf page layer corresponding to the volume identifier
  • this embodiment may write the to-be-written data cached in the hot data area or the cold data area of the middle page layer into the leaf page layer corresponding to the volume identifier.
  • Step A5 compress the data to be written into a mapping table using the leaf page layer corresponding to the volume identifier and add a data write count field, and write the mapping table and the corresponding data write count field into the virtual block layer;
  • Step A6 using the virtual block layer to determine the write mode according to the data write count field of the mapping table; wherein the write mode includes physical write and logical write;
  • the process of the above-mentioned virtual block layer determining the writing method of the data to be written includes: if the data write count field of the mapping table is greater than or equal to 1, the virtual block layer determines that the writing method of the data to be written is logical writing; if the data write count field of the mapping table is 0, the virtual block layer determines that the writing method of the data to be written is physical writing.
  • Logical write refers to: an operation of only adding 1 to the data write count field but not writing the actual write data to the physical block layer
  • physical write refers to: an operation of writing the actual write data to the physical block layer
  • Step A7 If the writing mode is physical writing, the mapping table is written to the corresponding physical block layer, and the value of the data writing count field of the mapping table is set to 1;
  • Step A8 If the writing mode is logical writing, the data in the mapping table is written into the value of the count field plus 1.
  • the data in the physical block layer can also be written to the RAID array (Redundant Array of Independent Disks, a redundant array composed of independent disks).
  • RAID array Redundant Array of Independent Disks, a redundant array composed of independent disks.
  • the operation corresponding to the data operation request is performed according to the multi-level mapping framework, including the following steps:
  • Step B1 determining a third host LBA of the host device that sends the data deletion request, and querying a data write calculation field of the to-be-deleted data corresponding to the data deletion request in the multi-level mapping framework according to the third host LBA;
  • Step B2 Determine the deletion method according to the data to be deleted written into the calculation field
  • the deletion methods include physical deletion and logical deletion
  • the process of determining the deletion method based on the data write count field of the data to be deleted includes: if the data write count field of the data to be deleted is greater than or equal to 2, the virtual block layer determines that the deletion method of the data to be deleted is logical deletion; if the data write count field of the data to be deleted is 1, the virtual block layer determines that the deletion method of the data to be deleted is physical deletion.
  • Step B3 If the deletion method is physical deletion, delete the data to be deleted in the physical block layer and The value of the data write count field is set to 0;
  • Step B4 If the deletion method is logical deletion, the value of the data to be deleted is written into the count field minus 1.
  • the process of executing the operation corresponding to the data operation request according to the multi-level mapping framework includes:
  • Step C1 construct a multi-level mapping framework based on the inode index and B-tree in the data storage cluster;
  • Step C2 receiving a data reading request
  • Step C3 determining a second host LBA of the host device that sends the data read request, and querying a data storage location of the to-be-read data corresponding to the data read request in the multi-level mapping framework according to the second host LBA;
  • Step C4 Read and return the data to be read from the cache or memory according to the data storage location.
  • the data read and write frequency of the host device that sends the data read request can also be updated in the top page layer.
  • Hash also known as hash algorithm, uses a one-to-one mapping relationship such as key-value. It is mainly used for data query. Querying data based on a pre-set key value can significantly improve the data query speed.
  • this embodiment adds a multi-level mapping mechanism at the software level of the storage environment. Since multiple storage systems are usually shared by multiple host devices through a network, the design maps multiple storage systems to the same logical space area. Adding multi-level logical address space in the storage environment effectively solves the problem of storage resource waste in data duplication and deletion, thereby ensuring that the same data does not consume too much storage resources.
  • This embodiment combines the inode node in the Linux operating system with the B-tree data structure, and targets the pain point of storage resource waste in the storage field, and can effectively solve the problem of multiple waste of storage resources when data is copied and deleted in the storage field.
  • Figure 2 is a schematic diagram of the structure of a multi-level mapping framework provided by some embodiments of the present application, showing the root page layer corresponding to each storage system (including a logical address space of 0 to 16EB, inode[0] to inode[N]), the top leaf layer corresponding to each host, the middle page layer corresponding to hot and cold data, the leaf page layer corresponding to the volume, the virtual block layer, the physical block layer, and the super block.
  • the root page layer can map 512 B-trees to the top leaf layer, and each top leaf layer corresponds to 4KB of space.
  • the data to be written can be compressed into a mapping table and added. Data write count.
  • the 2MB user data in the physical block layer includes multiple 4KB data written.
  • EB (Exabyte) is a computer storage unit, mainly used in the storage field.
  • the maximum available virtual memory space of a 64-bit computer system is 16EB.
  • Logical page Memory management sets the lower 12 bits of the logical page field to 0, so that a logical page is 4K in size.
  • this embodiment uses multiple inode nodes to manage multiple storage systems, and then uses the super block data structure in the Linux kernel to manage an inode table. All inodes are managed through the inode table. Each inode in the inode table corresponds to each storage system in the storage environment in a one-to-one relationship, thereby realizing the management of multiple storage systems.
  • the inode table is shown in the dotted box in FIG2 .
  • the super block identifies an inode table, and each inode node in the inode table represents each storage system.
  • Each inode node can map up to 512 hosts, and a logical page of memory space is provided for the business processing of each host (i.e., 4KB of memory space is occupied).
  • the I/O requests initiated by multiple hosts to multiple storage systems correspond to the top page layer.
  • the hot data and cold data accessed by the host correspond to the middle page layer.
  • the first 2KB of each middle page stores frequently accessed hot data, and the last 2KB stores rarely accessed cold data.
  • the logical unit for host I/O access to data is volume, which corresponds to the leaf page layer in FIG2 .
  • Below the leaf page layer is the VB (virtual block) layer.
  • the data in the volume is compressed into a mapping table through a compression algorithm and a data write count field is added to the VB layer. All host I/O access requests are determined at the VB layer through the data write count field to determine whether the data is rewritten.
  • the data write count field is incremented by 1 and becomes 1. If the data write count field is greater than or equal to 1, it indicates that the data has been written. The method for handling this situation is to increment the data write count pointed to by the leaf page by 1, and the data will not be rewritten at this time.
  • VB stands for virtual block. This embodiment adds VB virtual blocks in the multi-level B-tree mapping.
  • PB stands for physical block. This embodiment adds PB physical blocks in the multi-level B-tree mapping.
  • Step D1 The data in the root page layer includes a key value composed of a host LBA and an inode index. Different inode indexes represent different storage systems (eg, the multiple storage systems of the above-mentioned multiple bank branches).
  • Step D2 Map the key value composed of the host LBA and inode index to the top page layer, store the read and write frequency of the data in the top page layer, determine whether the data is cold data or hot data based on the read and write frequency of the data, and regenerate a key value using the host LBA passed in from the upper layer and the volume identifier and hot and cold data identifiers generated by this layer.
  • the volume identifier can be automatically assigned by the system.
  • Step D3 Use the key value to map to the middle page layer, which corresponds to the hot and cold data layer. This layer will determine whether the data should be placed in the first 2KB hot data area or the last 2KB cold data area based on the hot and cold data identifiers passed in from the upper layer, and write the data to the leaf page layer based on the volume identifier passed in from the upper layer.
  • Step D4 The leaf page layer corresponds to the logical volume volume.
  • a key value is regenerated using the volume index and the data write count field.
  • the key value is used to map to the VB layer.
  • the data in the volume is compressed into a mapping table through a compression algorithm and a data write count field is added. The data is then written to the VB layer.
  • Step D5 The VB layer will perform a logical judgment on the data write count field and then map it to the lower PB layer.
  • the PB layer stores the data requested by the host I/O in units of 4KB logical pages. If the data write count is 0, it means that the data does not exist in the storage environment. After the data is written to the user-specified physical disk, the data write count field is incremented by 1 and becomes 1. If the data write count field is greater than or equal to 1, it means that the data has been written. The solution to this situation is to increment the data write count pointed to by the leaf page by 1, and the data will not be rewritten at this time.
  • Step D6 The user data organized at the PB layer is finally stored on the RAID array.
  • Figure 3 is a schematic diagram of a virtual block layer with a data write count of 1 provided in some embodiments of the present application
  • Figure 4 is a schematic diagram of a virtual block layer with a data write count of 2 provided in some embodiments of the present application.
  • volume[100] refers to the shaded data in the physical block PB layer through the virtual block VB layer. Since only volume[100] refers to the shaded data in the storage environment, the data write count field is 1. In the leaf page layer of Figure 4, volume[100+i] refers to the shaded data in the PB layer through the VB layer. Therefore, adding the data write count corresponding to volume[100], the new data write count field is 2. If the subsequent host I/O request is to write data to the same host LBA and inode index, it is judged as data rewriting, and no physical disk write operation will be performed.
  • this embodiment adds a multi-level mapping mechanism at the software level of the storage system, and innovatively combines the Linux super block, inode and B-tree to propose a new multi-level mapping mechanism.
  • a data write count field is added to the implementation details to control data rewriting and deletion, which well solves the problem of storage resource waste in data rewriting and deletion.
  • the multi-level mapping mechanism involved in the method proposed in this embodiment and the proposed data write count field are an algorithm and global variable for implementing a specified logical function, which can be implemented in C language or C++.
  • the multi-level mapping mechanism can be used not only in this business scenario, but also in other business scenarios such as big data, the Internet, and artificial intelligence.
  • This application adds a multi-level mapping mechanism at the software level of the storage environment, and innovatively combines the Linux super block, inode and B-tree to propose a new multi-level mapping mechanism. This mechanism is applied to this application, and data is added to the implementation details.
  • the write count field uses the data write count field to control the logical processing of data rewriting and deletion, which effectively solves the problem of storage resource waste in data rewriting and deletion.
  • This embodiment proposes a multi-level mapping mechanism in specific implementation details, and innovatively combines the Linux super block, inode and B-tree in specific implementation details to propose a new multi-level mapping mechanism, which can effectively solve the problem of storage resource waste in the storage environment.
  • FIG. 5 is a schematic diagram of the structure of a data operation request processing system provided by some embodiments of the present application, which is applied to a data storage cluster.
  • the data storage cluster runs the above-mentioned multi-level mapping framework.
  • the data operation request processing system includes:
  • a write request receiving module for determining a first host LBA of a host device that sends the data write request if a data write request is received, and querying an inode index corresponding to the first host LBA using an inode table;
  • An identifier determination module used to map the first host LBA and inode index to the corresponding top page layer, and allocate a volume identifier for the data to be written in the top page layer;
  • a cache module used for caching the to-be-written data corresponding to the data write request using the intermediate page layer
  • the leaf page layer writing module is used to write the data to be written into the leaf page layer corresponding to the volume identifier
  • a virtual block layer writing module used to compress the data to be written into a mapping table using the leaf page layer corresponding to the volume identifier and add a data write count field, and write the mapping table and the corresponding data write count field into the virtual block layer;
  • a physical block layer write module is used to use the virtual block layer to determine the write mode according to the data write count field of the mapping table; wherein the write mode includes physical write and logical write; and is also used to write the mapping table to the corresponding physical block layer if the write mode is physical write, and set the value of the data write count field of the mapping table to 1; and is also used to add 1 to the value of the data write count field of the mapping table if the write mode is logical write.
  • the above embodiment constructs a multi-level mapping framework based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes the root page layer, the top page layer, the middle page layer, the leaf page layer, the virtual block layer and the physical block layer from top to bottom. If the data storage cluster receives a data operation request, the corresponding operation is performed according to the multi-level mapping framework.
  • the data write count field is used to count the number of times the same data is stored in the data storage cluster, and then the data write count field is used to control whether to perform operations on the physical block layer to control data rewriting and deletion. Therefore, this embodiment can avoid the waste of storage resources caused by data duplication and deletion.
  • a table reading module is used to read the inode table from the super block data structure of the Linux kernel before using the inode table to query the inode index corresponding to the first host LBA;
  • the inode table stores the correspondence between the inode index and the storage system.
  • the virtual block layer write module uses the virtual block layer to determine the write direction according to the data write count field of the mapping table. The process includes: if the data write count field of the mapping table is greater than or equal to 1, the virtual block layer is used to determine that the write mode of the data to be written is logical write; if the data write count field of the mapping table is 0, the virtual block layer is used to determine that the write mode of the data to be written is physical write.
  • the process of the cache module using the middle page layer to cache the data to be written corresponding to the data write request includes: using the top page layer to query the data read and write frequency of the host device that sends the data write request; caching the data to be written to the hot data area or the cold data area of the middle page layer according to the data read and write frequency; wherein the hot data area is used to store data with a read and write frequency greater than a preset value, and the cold data area is used to store data with a read and write frequency less than or equal to a preset value.
  • the process of the cache module using the top page layer to query the data read and write frequency of the host device that sends the data write request includes: calculating the page structure corresponding to the host device that sends the data write request in the top page layer according to the first host LBA, and querying the data read and write frequency of the host device that sends the data write request in the page structure.
  • the process in which the leaf page layer writing module writes the data to be written into the leaf page layer corresponding to the volume identifier includes: writing the data to be written cached in the hot data area or the cold data area of the intermediate page layer into the leaf page layer corresponding to the volume identifier.
  • the disk placement module is used to place the data in the physical block layer to the disk of the RAID array after writing the mapping table to the physical block layer.
  • the data reading module is used to determine the second host LBA of the host device that sends the data reading request if a data reading request is received, and to query the data storage location of the data to be read corresponding to the data reading request in the multi-level mapping framework according to the second host LBA; and is also used to read and return the data to be read from the cache or memory according to the data storage location.
  • the frequency update module is used to update the data read and write frequency of the host device that sends the data read request in the top page layer after querying the data storage location of the to-be-read data corresponding to the data read request in the multi-level mapping framework according to the second host LBA.
  • Some embodiments of the present application further provide a data operation request processing system, which is applied to a data storage cluster.
  • the data storage cluster runs the above-mentioned multi-level mapping framework.
  • the data operation request processing system includes:
  • a query module for determining a third host LBA of a host device that sends a data deletion request if a data deletion request is received, and querying a data write calculation field of the to-be-deleted data corresponding to the data deletion request in a multi-level mapping framework according to the third host LBA;
  • a deletion method determination module is used to determine the deletion method according to the data written into the calculation field of the data to be deleted; wherein the deletion methods include physical deletion and logical deletion;
  • the deletion execution module is used to delete the data to be deleted in the physical block layer and set the value of the data write count field of the data to be deleted to 0 if the deletion method is physical deletion; it is also used to reduce the value of the data write count field of the data to be deleted by 1 if the deletion method is logical deletion.
  • the above embodiment constructs a multi-level mapping framework based on the inode index and B-tree in the data storage cluster.
  • the multi-level mapping framework includes the root page layer, the top page layer, the middle page layer, the leaf page layer, the virtual block layer and the physical block layer from top to bottom. If the data storage cluster receives a data operation request, the corresponding operation is performed according to the multi-level mapping framework.
  • the data write count field is used to count the number of times the same data is stored in the data storage cluster, and then the data write count field is used to control whether to perform operations on the physical block layer to control data rewriting and deletion. Therefore, this embodiment can avoid the waste of storage resources caused by data duplication and deletion.
  • the deletion method determination module determines the deletion method according to the data write count field of the data to be deleted, and the process includes: if the data write count field of the data to be deleted is greater than or equal to 2, the virtual block layer determines that the deletion method of the data to be deleted is logical deletion; if the data write count field of the data to be deleted is 1, the virtual block layer determines that the deletion method of the data to be deleted is physical deletion.
  • FIG6 is a schematic diagram of the structure of an electronic device provided in some embodiments of the present application. As shown in FIG6, the electronic device includes:
  • Communication interface 601 capable of exchanging information with other devices such as network devices;
  • the processor 602 is connected to the communication interface 601 to implement information exchange with other devices and is used to execute the data operation request processing method provided by one or more technical solutions when running the computer program.
  • the computer program is stored in the memory 603.
  • bus system 604. the various components in the electronic device are coupled together through the bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components.
  • the bus system 604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are marked as bus system 604 in FIG. 6.
  • the present application also provides a non-volatile storage medium on which a computer program is stored.
  • the non-volatile storage medium may include: a semiconductor storage chip, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (Random Access Memory), a memory card, ...
  • Figure 7 is a schematic diagram of the structure of a non-volatile storage medium provided by some embodiments of the present application.
  • the non-volatile storage medium can be a non-volatile or non-transient storage chip, specifically including a decoding driver, a storage matrix, a read/write circuit, an address line, a data line, a chip select line, and a read/write control line.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请公开了一种多级映射框架、数据操作请求处理方法及系统，所属的技术领域为数据存储技术。所述多级映射框架根据数据存储集群内的inode索引和B树构建得到，多级映射框架自上至下包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层；其中，根页层用于实现对顶页层的映射，顶页层用于分配volume标识符并实现对中间页层的映射，中间页层用于缓存数据并实现对叶页层的映射，叶页层用于设置数据写入计数字段，虚拟块层用于更新数据写入计数字段的值，物理块层用于存储数据，数据写入计数字段用于控制是否对物理块层执行操作。本申请能够避免数据复写和删除引起的存储资源浪费。

Description

一种多级映射框架、数据操作请求处理方法及系统
相关申请的交叉引用
本申请要求于2022年11月24日提交中国专利局，申请号为202211479284.8，申请名称为“一种多级映射框架、数据操作请求处理方法及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及数据存储技术领域，特别涉及一种多级映射框架、数据操作请求处理方法及系统。
背景技术
在大数据时代下，无时无刻不产生大量数据，这些数据不仅需要安全可靠的存储下来同时还需将它们以极高的I/O速率存储在后台存储环境中，这样每个互联网用户才能快速访问这些数据。
在提高I/O性能方面，业界已使用多级缓存cache方式，通过多级哈希映射方式加速存取I/O数据的速度，并且使用多控节点组成的集群技术，主节点负责处理主机的I/O请求，辅助节点负责存储系统的后台任务(例如，RAID阵列初始化、巡检和重构任务等等)，以此来提高存储系统的I/O性能。但是这两种方法仍不能显著提高I/O性能，例如多个银行分行的多存储系统通常由多个主机设备通过一个网络进行共享，在多个主机设备上运行的应用程序都包含多个功能多样的应用程序进程。这些进程通过主机发出I/O请求，以使数据安全可靠的存储于各分行的存储系统上，在这种情况下势必会有数据的复写和删除，导致存储资源浪费。
发明内容
本申请的目的是提供一种多级映射框架、一种数据操作请求处理方法、一种数据操作请求处理系统、一种电子设备及一种非易失性存储介质，能够避免数据复写和删除引起的存储资源浪费。
本申请所解决的技术问题为如何避免数据复写和删除引起的存储资源浪费。为解决上述技术问题，本申请一些实施例提供一种多级映射框架，多级映射框架根据数据存储集群内的inode索引和B树构建得到，多级映射框架自上至下包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层；数据存储集群用于根据多级映射框架执行数据操作请求对应的操作；
其中，根页层用于实现对顶页层的映射，顶页层用于分配volume标识符并实现对中间 页层的映射，中间页层用于缓存数据并实现对叶页层的映射，叶页层用于设置数据写入计数字段，虚拟块层用于更新数据写入计数字段的值，物理块层用于存储数据，数据写入计数字段用于控制是否对物理块层执行操作。
本申请一些实施例还提供一种数据操作请求处理方法，应用于数据存储集群，数据存储集群运行有上述的多级映射框架，数据操作请求处理方法，包括：
若接收到数据写入请求，则确定发送数据写入请求的主机设备的第一主机LBA，利用inode表查询第一主机LBA对应的inode索引；
利用第一主机LBA和inode索引映射到对应的顶页层，并在顶页层内为待写入数据分配volume标识符；
利用中间页层缓存数据写入请求对应的待写入数据；
将待写入数据写入volume标识符对应的叶页层；
利用volume标识符对应的叶页层将待写入数据压缩为映射表并添加数据写入计数字段，将映射表和对应的数据写入计数字段写入虚拟块层；
利用虚拟块层根据映射表的数据写入计数字段确定写入方式；其中，写入方式包括物理写入和逻辑写入；
若写入方式为物理写入，则将映射表写入至对应的物理块层，并将映射表的数据写入计数字段的值设置为1；
若写入方式为逻辑写入，则将映射表的数据写入计数字段的值加1。
在本申请的一些实施例中，在利用inode表查询第一主机LBA对应的inode索引之前，还包括：
从Linux内核的超级块super block数据结构中读取inode表；
其中，inode表中存储有inode索引与存储系统的对应关系。
在本申请的一些实施例中，利用虚拟块层根据映射表的数据写入计数字段确定写入方式，包括：
若映射表的数据写入计数字段大于或等于1，则利用虚拟块层判定待写入数据的写入方式为逻辑写入；
若映射表的数据写入计数字段为0，则利用虚拟块层判定待写入数据的写入方式为物理写入。
在本申请的一些实施例中，利用中间页层缓存数据写入请求对应的待写入数据，包括：
利用顶页层查询发送数据写入请求的主机设备的数据读写频率；
根据数据读写频率将待写入数据缓存至中间页层的热数据区域或冷数据区域；
其中，热数据区域用于存储读写频率大于预设值的数据，冷数据区域用于存储读写频率小于或等于预设值的数据。
在本申请的一些实施例中，利用顶页层查询发送数据写入请求的主机设备的数据读写频率，包括：
根据第一主机LBA计算发送数据写入请求的主机设备在顶页层中对应的页结构，在页结构中查询发送数据写入请求的主机设备的数据读写频率。
在本申请的一些实施例中，将待写入数据写入volume标识符对应的叶页层，包括：
将中间页层的热数据区域或冷数据区域中缓存的待写入数据写入volume标识符对应的叶页层。
在本申请的一些实施例中，在将映射表写入至物理块层之后，还包括：
将物理块层中的数据落盘至RAID阵列。
在本申请的一些实施例中，还包括：
若接收到数据读取请求，则确定发送数据读取请求的主机设备的第二主机LBA，根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置；
根据数据存储位置从缓存或内存中读取并返回待读取数据。
在本申请的一些实施例中，在根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置之后，还包括：
在顶页层中更新发送数据读取请求的主机设备的数据读写频率。
本申请还提供一种数据操作请求处理方法，应用于数据存储集群，数据存储集群运行有上述的多级映射框架，数据操作请求处理方法，包括：
若接收到数据删除请求，则确定发送数据删除请求的主机设备的第三主机LBA，根据第三主机LBA在多级映射框架中查询数据删除请求对应的待删除数据的数据写入计算字段；
根据待删除数据的数据写入计算字段确定删除方式；其中，删除方式包括物理删除和逻辑删除；
若删除方式为物理删除，则在物理块层中删除待删除数据，并将待删除数据的数据写入计数字段的值设置为0；
若删除方式为逻辑删除，则将待删除数据的数据写入计数字段的值减1。
在本申请的一些实施例中，根据待删除数据的数据写入计算字段确定删除方式，包括：
若待删除数据的数据写入计数字段大于或等于2，则虚拟块层判定待删除数据的删除方式为逻辑删除；
若待删除数据的数据写入计数字段为1，则虚拟块层判定待删除数据的删除方式为物理删除。
本申请一些实施例还提供了一种数据操作请求处理系统，应用于数据存储集群，数据存储集群运行有如上述的多级映射框架，数据操作请求处理系统，包括：
写入请求接收模块，用于若接收到数据写入请求，则确定发送数据写入请求的主机设备的第一主机LBA，利用inode表查询第一主机LBA对应的inode索引；
标识符确定模块，用于利用第一主机LBA和inode索引映射到对应的顶页层，并在顶页层内为待写入数据分配volume标识符；
缓存模块，用于利用中间页层缓存数据写入请求对应的待写入数据；
叶页层写模块，用于将待写入数据写入volume标识符对应的叶页层；
虚拟块层写入模块，用于利用volume标识符对应的叶页层将待写入数据压缩为映射表并添加数据写入计数字段，将映射表和对应的数据写入计数字段写入虚拟块层；
物理块层写入模块，用于利用虚拟块层根据映射表的数据写入计数字段确定写入方式；其中，写入方式包括物理写入和逻辑写入；还用于若写入方式为物理写入，则将映射表写入至对应的物理块层，并将映射表的数据写入计数字段的值设置为1；还用于若写入方式为逻辑写入，则将映射表的数据写入计数字段的值加1。
本申请一些实施例还提供了一种数据操作请求处理系统，应用于数据存储集群，数据存储集群运行有上述的多级映射框架，数据操作请求处理系统，包括：
查询模块，用于若接收到数据删除请求，则确定发送数据删除请求的主机设备的第三主机LBA，根据第三主机LBA在多级映射框架中查询数据删除请求对应的待删除数据的数据写入计算字段；
删除方式确定模块，用于根据待删除数据的数据写入计算字段确定删除方式；其中，删除方式包括物理删除和逻辑删除；
删除执行模块，用于若删除方式为物理删除，则在物理块层中删除待删除数据，并将待删除数据的数据写入计数字段的值设置为0；还用于若删除方式为逻辑删除，则将待删除数据的数据写入计数字段的值减1。
本申请一些实施例还提供了一种非易失性存储介质，其上存储有计算机程序，计算机程序执行时实现上述数据操作请求处理方法执行的步骤。
本申请一些实施例还提供了一种电子设备，包括存储器和处理器，存储器中存储有计算机程序，处理器调用存储器中的计算机程序时实现上述数据操作请求处理方法执行的步骤。
本申请一些实施例提供了一种多级映射框架，多级映射框架根据数据存储集群内的inode索引和B树构建得到，多级映射框架自上至下包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层；数据存储集群用于根据多级映射框架执行数据操作请求对应的操作；其中，根页层用于实现对顶页层的映射，顶页层用于分配volume标识符并实现对中间页层的映射，中间页层用于缓存数据并实现对叶页层的映射，叶页层用于设置数据写入计数字段，虚拟块层用于更新数据写入计数字段的值，物理块层用于存储数据，数据写入计数字段用于控制是否对物理块层执行操作。
本申请实施例根据数据存储集群内的inode索引和B树构建多级映射框架，多级映射框架自上至下依次包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，若数据存储集群接收到数据操作请求，则根据多级映射框架执行相应的操作。在上述多级映射框架中使用数据写入计数字段统计同一数据在数据存储集群中的存储次数，进而根据数据写入计数字段控制是否对物理块层执行操作，以控制数据重写和删除，因此本申请能够避免数据复写和删除引起的存储资源浪费。本申请实施例同时还提供了一种数据操作请求处理方法、一种数据操作请求处理系统、一种非易失性存储介质和一种电子设备，具有上述有益效果，在此不再赘述。
附图说明
为了更清楚地说明本申请的一些实施例，下面将对实施例中所需要使用的附图做简单的介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请的一些实施例所提供的一种数据操作请求处理方法的流程图；
图2为本申请的一些实施例所提供的一种多级映射框架的结构示意图；
图3为本申请的一些实施例所提供的一种数据写入计数为1的虚拟块层的示意图；
图4为本申请的一些实施例所提供的一种数据写入计数为2的虚拟块层的示意图；
图5为本申请的一些实施例所提供的一种数据操作请求处理系统的结构示意图；
图6为本申请的一些实施例所提供的一种电子设备的结构示意图；
图7为本申请的一些实施例所提供的一种非易失性存储介质的结构示意图。
具体实施方式
为使本申请的一些实施例的目的、技术方案和优点更加清楚，下面将结合本申请的一些 实施例中的附图，对本申请的一些实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的一些实施例提供了一种多级映射框架，多级映射框架根据数据存储集群内的inode索引和B树构建得到，多级映射框架自上至下包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层；数据存储集群用于根据多级映射框架执行数据操作请求对应的操作。
其中，根页层用于实现对顶页层的映射，顶页层用于分配volume标识符并实现对中间页层的映射，中间页层用于缓存数据并实现对叶页层的映射，叶页层用于设置数据写入计数字段，虚拟块层用于更新数据写入计数字段的值，物理块层用于存储数据，数据写入计数字段用于控制是否对物理块层执行操作。
上述实施例根据数据存储集群内的inode索引和B树构建多级映射框架，多级映射框架自上至下依次包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，若数据存储集群接收到数据操作请求，则根据多级映射框架执行相应的操作。在上述多级映射框架中使用数据写入计数字段统计同一数据在数据存储集群中的存储次数，进而根据数据写入计数字段控制是否对物理块层执行操作，以控制数据重写和删除，因此本实施例能够避免数据复写和删除引起的存储资源浪费。
下面请参见图1，图1为本申请的一些实施例所提供的一种数据操作请求处理方法的流程图。
具体步骤可以包括：
S101：根据数据存储集群内的inode索引和B树构建多级映射框架；
其中，本实施例可以应用于数据存储集群，该数据存储集群可以包括多个主机设备。在本步骤之前可以存在获取数据存储集群的inode索引和B树，进而基于上述inode索引和B树(又称B-tree，是一种自平衡的树，能够保持数据有序。这种数据结构能够让查找数据、顺序访问、插入数据及删除的动作，都在对数时间内完成)可以构建多级映射框架。
上述多级映射框架自上至下可以包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，根页层用于实现对顶页层的映射，顶页层用于分配volume(逻辑卷，主机I/O访问数据的逻辑单元，由逻辑单元block块组成)标识符并实现对中间页层的映射，中间页层用于缓存数据并实现对叶页层的映射，叶页层用于设置数据写入计数字段，虚拟块层用于更 新数据写入计数字段的值，数据写入计数字段用于控制是否对物理块层执行操作。
S102：接收数据操作请求；
其中，上述数据操作请求可以为数据写入请求、数据删除请求或数据读取请求，数据操作请求可以为客户端向数据存储集群发送的请求。
S103：根据多级映射框架执行数据操作请求对应的操作。
其中，在接收到数据操作请求之后，本申请可以通过多级映射框架确定待写入数据、待删除数据或待读取数据所在的叶页层，进而执行相应的数据写入操作、数据删除操作或数据读取操作。
本实施例根据数据存储集群内的inode索引和B树构建多级映射框架，多级映射框架自上至下依次包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，若数据存储集群接收到数据操作请求，则根据多级映射框架执行相应的操作。在上述多级映射框架中使用数据写入计数字段统计同一数据在数据存储集群中的存储次数，进而根据数据写入计数字段控制是否对物理块层执行操作，以控制数据重写和删除，因此本实施例能够避免数据复写和删除引起的存储资源浪费。
作为一种可行的实施方式，若数据操作请求为数据写入请求，则根据多级映射框架执行数据操作请求对应的操作，包括以下步骤：
步骤A1：确定发送数据写入请求的主机设备的第一主机LBA(logic block address，逻辑区块地址)，利用inode表查询第一主机LBA对应的inode索引；
其中，在利用inode表查询第一主机LBA对应的inode索引之前，还可以从Linux内核的超级块super block数据结构中读取inode表；其中，inode表中存储有inode索引与存储系统的对应关系。
步骤A2：利用第一主机LBA和inode索引映射到对应的顶页层，并在顶页层内为待写入数据分配volume标识符；
步骤A3：利用中间页层缓存数据写入请求对应的待写入数据；
作为一种可行的实施方式，利用中间页层缓存待写入数据的过程包括：利用顶页层查询发送数据写入请求的主机设备的数据读写频率；根据数据读写频率将待写入数据缓存至中间页层的热数据区域或冷数据区域；其中，热数据区域用于存储读写频率大于预设值的数据，冷数据区域用于存储读写频率小于或等于预设值的数据。
进一步的，上述过程中利用顶页层查询发送数据写入请求的主机设备的数据读写频率的过程包括：根据第一主机LBA计算发送数据写入请求的主机设备在顶页层中对应的页结构， 在页结构中查询发送数据写入请求的主机设备的数据读写频率。
步骤A4：将待写入数据写入volume标识符对应的叶页层；
具体的，本实施例可以将中间页层的热数据区域或冷数据区域中缓存的待写入数据写入volume标识符对应的叶页层。
步骤A5：利用volume标识符对应的叶页层将待写入数据压缩为映射表并添加数据写入计数字段，将映射表和对应的数据写入计数字段写入虚拟块层；
步骤A6：利用虚拟块层根据映射表的数据写入计数字段确定写入方式；其中，写入方式包括物理写入和逻辑写入；
作为一种可行的实施方式，上述虚拟块层确定待写入数据的写入方式的过程包括：若映射表的数据写入计数字段大于或等于1，则虚拟块层判定待写入数据的写入方式为逻辑写入；若映射表的数据写入计数字段为0，则虚拟块层判定待写入数据的写入方式为物理写入。
逻辑写入指：只对数据写入计数字段加1但不向物理块层写入实际写入数据的操作；物理写入指：向物理块层写入实际写入数据的操作。
步骤A7：若写入方式为物理写入，则将映射表写入至对应的物理块层，并将映射表的数据写入计数字段的值设置为1；
步骤A8：若写入方式为逻辑写入，则将映射表的数据写入计数字段的值加1。
进一步的，在将映射表写入至物理块层之后，还可以将物理块层中的数据落盘至RAID阵列(Redundant Array of Independent Disks独立磁盘组成的具有冗余性的阵列)。
作为一种可行的实施方式，若数据操作请求为数据删除请求，则根据多级映射框架执行数据操作请求对应的操作，包括以下步骤：
步骤B1：确定发送数据删除请求的主机设备的第三主机LBA，根据第三主机LBA在多级映射框架中查询数据删除请求对应的待删除数据的数据写入计算字段；
步骤B2：根据待删除数据的数据写入计算字段确定删除方式；
其中，删除方式包括物理删除和逻辑删除；
作为一种可行的实施方式，根据待删除数据的数据写入计算字段确定删除方式的过程包括：若待删除数据的数据写入计数字段大于或等于2，则虚拟块层判定待删除数据的删除方式为逻辑删除；若待删除数据的数据写入计数字段为1，则虚拟块层判定待删除数据的删除方式为物理删除。
步骤B3：若删除方式为物理删除，则在物理块层中删除待删除数据，并将待删除数据 的数据写入计数字段的值设置为0；
步骤B4：若删除方式为逻辑删除，则将待删除数据的数据写入计数字段的值减1。
作为一种可行的实施方式，若数据操作请求为数据读取请求，则根据多级映射框架执行数据操作请求对应的操作的过程包括：
步骤C1：根据数据存储集群内的inode索引和B树构建多级映射框架；
步骤C2：接收数据读取请求；
步骤C3：确定发送数据读取请求的主机设备的第二主机LBA，根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置；
步骤C4：根据数据存储位置从缓存或内存中读取并返回待读取数据。
进一步的，在根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置之后，还可以在顶页层中更新发送数据读取请求的主机设备的数据读写频率。
下面通过在实际应用中的实施例说明上述实施例描述的流程。
目前业界对于解决数据的复写和删除并没有很好的方法，常用的多级缓存哈希映射算法方式只是在一定程度上提高读取I/O(输入和输出)数据的速度，并没有解决在数据的复写和删除方面多次浪费存储资源的问题。哈希又称为hash算法，使用key-value这样一对一的映射关系，主要用于数据查询方面，根据事先设定好的key值查询数据，可以显著提高数据的查询速度。
针对数据的复写和删除问题，本实施例在存储环境的软件层面增加了多级映射机制，由于多存储系统通常由多个主机设备通过一个网络进行共享，所以设计将多存储系统映射到同一片逻辑空间区域。在存储环境中增加多级逻辑地址空间，新增的逻辑地址空间有效解决了数据复写和删除方面存在的存储资源浪费问题，以确保相同的数据不会过多消耗存储资源。
本实施例将Linux操作系统中的inode节点和B树这种数据结构结合在一起，针对存储领域中存在的存储资源浪费痛点，能够有效解决存储领域中数据复写和删除时多次浪费存储资源的问题。
请参见图2，图2为本申请的一些实施例所提供的一种多级映射框架的结构示意图，图中示出了对应各存储系统的根页层(包括0～16EB的逻辑地址空间、inode[0]～inode[N])、对应各主机的顶叶层、对应冷热数据的中间页层、对应volume卷的叶页层、virtual block(虚拟块)层、physical block(物理块)层、超级块super block。根页层可以映射512各B树至顶叶层，每个顶叶层对应4KB的空间。在virtual block层内可以将待写入数据压缩为映射表并添加 数据写入计数。physical block层内的2MB用户数据包括写入的多个4KB数据。EB(Exabyte，艾字节)计算机存储单位，主要用于存储领域，64位计算机系统的可用最大的虚拟内存空间为16EB，数据转换公式1EB＝1024PB＝1024TB。
逻辑页：内存管理都将逻辑页字段的低12位置为0，这样一个逻辑页就是4K大小。
如图2所示，本实施例使用多个inode节点管理多个存储系统，然后使用Linux内核中的超级块super block数据结构管理一个inode表，通过该inode表管理所有的inode，inode表中的每个inode和存储环境中的每个存储系统是一一对应关系，从而实现对多存储系统的管理，inode表如图2中虚线框所示。超级块标识一个inode表，inode表中的每个inode节点表示每个存储系统。每个inode节点最多可以映射512个主机，为每个主机的业务处理提供一个逻辑页的内存空间(即占用4KB的内存空间)，本实施例将多个主机向多个存储系统发起的I/O请求对应顶页层。主机访问的热数据和冷数据对应中间页层，每个中间页的前2KB存放频繁访问的热数据，后2KB存放很少访问的冷数据。主机I/O访问数据的逻辑单元是volume卷，volume卷对应图2中的叶页层。叶页层之下的是VB(virtual block)层，volume卷中的数据通过压缩算法压缩为映射表并添加数据写入计数字段放入VB层，所有主机的I/O访问请求都在VB层通过数据写入计数字段判断是否数据重写，如果数据写入计数为0则表明存储环境中没有该数据，将数据写入用户指定的物理磁盘后数据写入计数字段加1变为1。如果数据写入计数字段大于等于1则表明该数据已被写入过，这种情况的处理方法是将叶页指向的数据写入计数加1，此时就不会进行数据重写。VB即virtual block(虚拟块)，本实施例在多级B树映射中增加了VB虚拟块。PB即physical block(物理块)，本实施例在多级B树映射中增加了PB物理块。
基于图2所示的多级映射框架中实现数据写入的过程如下：
步骤D1：根页层中数据包括主机LBA和inode索引组成的key值，inode的不同索引代表不同的存储系统(例如上述多个银行分行的多存储系统)。
步骤D2：根据主机LBA和inode索引组成的key值映射到顶页层，在顶页层存放有关于数据的读写频率，根据数据的读写频率判断该数据是冷数据还是热数据，使用上层传入的主机LBA和本层自生成的volume标识符与冷热数据标识符重新生成一个key值。volume标识符可以由系统自动分配。
步骤D3：使用该key值映射到中间页层，中间页层对应冷热数据层，该层会根据上层传入的冷热数据标识符判断得出数据是放入前2KB的热数据区域还是后2KB的冷数据区域，并根据上层传入的volume标识符将数据写入叶页层。
步骤D4：叶页层对应逻辑卷volume，在该层使用volume索引和数据写入计数字段重新生成一个key值，使用该key值映射到VB层，并将volume卷中的数据通过压缩算法压缩为映射表并添加数据写入计数字段，然后写入VB层。
步骤D5：在VB层会对数据写入计数字段进行逻辑判断，然后映射到下层PB层。PB层以4KB逻辑页为单位存放主机I/O请求的数据，如果数据写入计数为0则表明存储环境中没有该数据，将数据写入用户指定的物理磁盘后数据写入计数字段加1变为1。如果数据写入计数字段大于等于1则表明该数据已被写入过，这种情况的处理方法是将叶页指向的数据写入计数加1，此时就不会进行数据重写。
步骤D6：在PB层被组织好的用户数据最终落盘到RAID阵列。
请参见图3和图4，图3为本申请的一些实施例提供的一种数据写入计数为1的虚拟块层的示意图，图4为本申请的一些实施例提供的一种数据写入计数为2的虚拟块层的示意图。
在图3的叶页层中的volume[100]通过虚拟块VB层引用物理块PB层中阴影部分数据，因为存储环境中只有volume[100]对该阴影部分中的数据进行引用，所以数据写入计数字段为1。在图4叶页层中的volume[100+i]通过VB层引用PB层中阴影部分数据。因此，加上volume[100]对应的数据写入计数得出新的数据写入计数字段为2。如果后续主机I/O请求是在同样的主机LBA和inode索引进行数据写入则被判断为数据重写，就不会进行物理磁盘写入操作。如果主机I/O请求是在同样的主机LBA和inode索引上进行数据删除，由于在其他存储系统上仍需该数据(例如其他银行分行的存储系统上仍需该数据)，就不会进行真实的物理操作删除操作。通过上述介绍的方法解决存储资源浪费问题。
本实施例针对存储环境出现概率较高的存储资源浪费问题，在存储系统的软件层面增加了多级映射机制，并创新性的将Linux超级块、inode与B树相结合提出一种全新的多级映射机制。同时在实现细节上加入数据写入计数字段用以控制数据重写和删除，很好地解决了在数据重写和删除方面存在的存储资源浪费问题。本实施例提出的方法中所涉及的多级映射机制以及提出的数据写入计数字段，是用于实现指定逻辑功能的一种算法和全局变量，可以由C语言或者C++实现。多级映射机制不仅可以在本业务场景下使用，还可以应用在大数据、互联网、人工智能等其他业务场景下使用。
在多存储系统的存储环境下，当多个主机通过同一个网络向多个存储系统发出主机I/O请求时会存在数据重写和删除的情况，在这种情况下，存在严重的存储资源浪费问题。本申请在存储环境的软件层面增加了多级映射机制，并创新性的将Linux超级块、inode与B树相结合提出一种全新的多级映射机制。将该机制运用到本申请中，同时在实现细节上加入数据 写入计数字段，使用数据写入计数字段控制数据重写和删除的逻辑处理，很好地解决了在数据重写和删除方面存在的存储资源浪费问题。本实施例在具体实现细节上提出了一种多级映射机制，并在具体实现细节上创新性的将Linux超级块、inode与B树相结合，提出一种全新的多级映射机制，能够有效解决存储环境中存在的存储资源浪费问题。
请参见图5，图5为本申请的一些实施例所提供的一种数据操作请求处理系统的结构示意图，应用于数据存储集群，数据存储集群运行有上述的多级映射框架，数据操作请求处理系统，包括：
写入请求接收模块，用于若接收到数据写入请求，则确定发送数据写入请求的主机设备的第一主机LBA，利用inode表查询第一主机LBA对应的inode索引；
标识符确定模块，用于利用第一主机LBA和inode索引映射到对应的顶页层，并在顶页层内为待写入数据分配volume标识符；
缓存模块，用于利用中间页层缓存数据写入请求对应的待写入数据；
叶页层写模块，用于将待写入数据写入volume标识符对应的叶页层；
虚拟块层写入模块，用于利用volume标识符对应的叶页层将待写入数据压缩为映射表并添加数据写入计数字段，将映射表和对应的数据写入计数字段写入虚拟块层；
物理块层写入模块，用于利用虚拟块层根据映射表的数据写入计数字段确定写入方式；其中，写入方式包括物理写入和逻辑写入；还用于若写入方式为物理写入，则将映射表写入至对应的物理块层，并将映射表的数据写入计数字段的值设置为1；还用于若写入方式为逻辑写入，则将映射表的数据写入计数字段的值加1。
上述实施例根据数据存储集群内的inode索引和B树构建多级映射框架，多级映射框架自上至下依次包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，若数据存储集群接收到数据操作请求，则根据多级映射框架执行相应的操作。在上述多级映射框架中使用数据写入计数字段统计同一数据在数据存储集群中的存储次数，进而根据数据写入计数字段控制是否对物理块层执行操作，以控制数据重写和删除，因此本实施例能够避免数据复写和删除引起的存储资源浪费。
进一步的，还包括：
读表模块，用于在利用inode表查询第一主机LBA对应的inode索引之前，从Linux内核的超级块super block数据结构中读取inode表；
其中，inode表中存储有inode索引与存储系统的对应关系。
进一步的，虚拟块层写入模块利用虚拟块层根据映射表的数据写入计数字段确定写入方 式的过程包括：若映射表的数据写入计数字段大于或等于1，则利用虚拟块层判定待写入数据的写入方式为逻辑写入；若映射表的数据写入计数字段为0，则利用虚拟块层判定待写入数据的写入方式为物理写入。
进一步的，缓存模块利用中间页层缓存数据写入请求对应的待写入数据的过程包括：利用顶页层查询发送数据写入请求的主机设备的数据读写频率；根据数据读写频率将待写入数据缓存至中间页层的热数据区域或冷数据区域；其中，热数据区域用于存储读写频率大于预设值的数据，冷数据区域用于存储读写频率小于或等于预设值的数据。
进一步的，缓存模块利用顶页层查询发送数据写入请求的主机设备的数据读写频率的过程包括：根据第一主机LBA计算发送数据写入请求的主机设备在顶页层中对应的页结构，在页结构中查询发送数据写入请求的主机设备的数据读写频率。
进一步的，叶页层写模块将待写入数据写入volume标识符对应的叶页层的过程包括：将中间页层的热数据区域或冷数据区域中缓存的待写入数据写入volume标识符对应的叶页层。
进一步的，还包括：
落盘模块，用于在将映射表写入至物理块层之后，将物理块层中的数据落盘至RAID阵列。
进一步的，还包括：
数据读取模块，用于若接收到数据读取请求，则确定发送数据读取请求的主机设备的第二主机LBA，根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置；还用于根据数据存储位置从缓存或内存中读取并返回待读取数据。
进一步的，还包括：
频率更新模块，用于在根据第二主机LBA在多级映射框架中查询数据读取请求对应的待读取数据的数据存储位置之后，在顶页层中更新发送数据读取请求的主机设备的数据读写频率。
本申请的一些实施例还提供一种数据操作请求处理系统，应用于数据存储集群，数据存储集群运行有上述的多级映射框架，数据操作请求处理系统，包括：
查询模块，用于若接收到数据删除请求，则确定发送数据删除请求的主机设备的第三主机LBA，根据第三主机LBA在多级映射框架中查询数据删除请求对应的待删除数据的数据写入计算字段；
删除方式确定模块，用于根据待删除数据的数据写入计算字段确定删除方式；其中，删除方式包括物理删除和逻辑删除；
删除执行模块，用于若删除方式为物理删除，则在物理块层中删除待删除数据，并将待删除数据的数据写入计数字段的值设置为0；还用于若删除方式为逻辑删除，则将待删除数据的数据写入计数字段的值减1。
上述实施例根据数据存储集群内的inode索引和B树构建多级映射框架，多级映射框架自上至下依次包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层，若数据存储集群接收到数据操作请求，则根据多级映射框架执行相应的操作。在上述多级映射框架中使用数据写入计数字段统计同一数据在数据存储集群中的存储次数，进而根据数据写入计数字段控制是否对物理块层执行操作，以控制数据重写和删除，因此本实施例能够避免数据复写和删除引起的存储资源浪费。
进一步的，删除方式确定模块根据待删除数据的数据写入计算字段确定删除方式的过程包括：若待删除数据的数据写入计数字段大于或等于2，则虚拟块层判定待删除数据的删除方式为逻辑删除；若待删除数据的数据写入计数字段为1，则虚拟块层判定待删除数据的删除方式为物理删除。
由于系统部分的实施例与方法部分的实施例相互对应，因此系统部分的实施例请参见方法部分的实施例的描述，这里暂不赘述。
本申请还提供了一种电子设备，可以包括存储器和处理器，存储器中存有计算机程序，处理器调用存储器中的计算机程序时，可以实现上述实施例所提供的步骤。当然电子设备还可以包括各种网络接口，电源等组件。图6为本申请的一些实施例所提供的一种电子设备的结构示意图，如图6所示，电子设备包括：
通信接口601，能够与其它设备比如网络设备等进行信息交互；
处理器602，与通信接口601连接，以实现与其它设备进行信息交互，用于运行计算机程序时，执行上述一个或多个技术方案提供的数据操作请求处理方法。而计算机程序存储在存储器603上。
当然，实际应用时，电子设备中的各个组件通过总线系统604耦合在一起。可理解，总线系统604用于实现这些组件之间的连接通信。总线系统604除包括数据总线之外，还包括电源总线、控制总线和状态信号总线。但是为了清楚说明起见，在图6中将各种总线都标为总线系统604。
本申请还提供了一种非易失性存储介质，其上存有计算机程序，该计算机程序被执行时可以实现上述实施例所提供的步骤。该非易失性存储介质可以包括：半导体存储芯片、U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random  Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。图7为本申请的一些实施例所提供的一种非易失性存储介质的结构示意图，该非易失性存储介质可以为非易失或非瞬时的存储芯片，具体包括译码驱动、存储矩阵、读写电路、地址线、数据线、片选线和读/写控制线。
说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似部分互相参见即可。对于实施例公开的系统而言，由于其与实施例公开的方法相对应，所以描述的比较简单，相关之处参见方法部分说明即可。应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以对本申请进行若干改进和修饰，这些改进和修饰也落入本申请权利要求的保护范围内。
还需要说明的是，在本说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的状况下，由语句“包括一个……”限定的要素，并不排除在包括要素的过程、方法、物品或者设备中还存在另外的相同要素。

Claims (20)

  1. 一种多级映射框架，其特征在于，所述多级映射框架根据数据存储集群内的inode索引和B树构建得到，所述多级映射框架自上至下包括根页层、顶页层、中间页层、叶页层、虚拟块层和物理块层；所述数据存储集群用于根据所述多级映射框架执行数据操作请求对应的操作；
    其中，所述根页层用于实现对所述顶页层的映射，所述顶页层用于分配volume标识符并实现对所述中间页层的映射，所述中间页层用于缓存数据并实现对所述叶页层的映射，所述叶页层用于设置数据写入计数字段，所述虚拟块层用于更新所述数据写入计数字段的值，所述物理块层用于存储数据，所述数据写入计数字段用于控制是否对所述物理块层执行操作。
  2. 根据权利要求1所述多级映射框架，其特征在于，所述根页层与所述数据存储集群内的存储系统对应，所述根页层中的数据包括主机LBA和inode索引组成的key值；所述顶页层与所述数据存储集群内的主机对应，所述中间页层包括用于缓存数据的热数据区域和冷数据区域，所述叶页层与主机的volume卷对应。
  3. 根据权利要求1或2所述多级映射框架，其特征在于，所述根页层实现对所述顶页层的映射过程包括：根据主机LBA和inode索引组成的key值映射到顶页层。
  4. 根据权利要求1或2所述多级映射框架，其特征在于，所述顶页层实现对所述中间页层的映射过程包括：使用所述根页层传入的主机LBA和所述顶页层自生成的volume标识符与冷热数据标识符重新生成一个key值，使用新的key值映射到所述中间页层。
  5. 根据权利要求1或2所述多级映射框架，其特征在于，所述中间页层实现对所述叶页层的映射过程包括：根据上层传入的volume标识符将数据写入叶页层。
  6. 一种数据操作请求处理方法，其特征在于，应用于数据存储集群，所述数据存储集群运行有如权利要求1所述的多级映射框架，所述数据操作请求处理方法，包括：
    若接收到数据写入请求，则确定发送所述数据写入请求的主机设备的第一主机LBA，利用inode表查询所述第一主机LBA对应的inode索引；
    利用所述第一主机LBA和所述inode索引映射到对应的顶页层，并在所述顶页层内为待写入数据分配volume标识符；
    利用所述中间页层缓存所述数据写入请求对应的所述待写入数据；
    将所述待写入数据写入所述volume标识符对应的叶页层；
    利用所述volume标识符对应的叶页层将所述待写入数据压缩为映射表并添加数据写入计数字段，将所述映射表和对应的数据写入计数字段写入所述虚拟块层；
    利用所述虚拟块层根据所述映射表的数据写入计数字段确定写入方式；其中，所述写入方式包括物理写入和逻辑写入；
    若所述写入方式为物理写入，则将所述映射表写入至对应的物理块层，并将所述映射表的数据写入计数字段的值设置为1；
    若所述写入方式为逻辑写入，则将所述映射表的数据写入计数字段的值加1。
  7. 根据权利要求6所述数据操作请求处理方法，其特征在于，在利用inode表查询所述第一主机LBA对应的inode索引之前，还包括：
    从Linux内核的超级块super block数据结构中读取所述inode表；
    其中，所述inode表中存储有inode索引与存储系统的对应关系。
  8. 根据权利要求6所述数据操作请求处理方法，其特征在于，利用所述虚拟块层根据所述映射表的数据写入计数字段确定写入方式，包括：
    若所述映射表的数据写入计数字段大于或等于1，则利用所述虚拟块层判定所述待写入数据的写入方式为逻辑写入；
    若所述映射表的数据写入计数字段为0，则利用所述虚拟块层判定所述待写入数据的写入方式为物理写入。
  9. 根据权利要求6所述数据操作请求处理方法，其特征在于，利用所述中间页层缓存所述数据写入请求对应的所述待写入数据，包括：
    利用所述顶页层查询发送所述数据写入请求的主机设备的数据读写频率；
    根据所述数据读写频率将所述待写入数据缓存至所述中间页层的热数据区域或冷数据区域；
    其中，所述热数据区域用于存储读写频率大于预设值的数据，所述冷数据区域用于存储读写频率小于或等于所述预设值的数据。
  10. 根据权利要求9所述数据操作请求处理方法，其特征在于，利用所述顶页层查询发送所述数据写入请求的主机设备的数据读写频率，包括：
    根据所述第一主机LBA计算发送所述数据写入请求的主机设备在所述顶页层中对应的页结构，在所述页结构中查询发送所述数据写入请求的主机设备的数据读写频率。
  11. 根据权利要求9所述数据操作请求处理方法，其特征在于，将所述待写入数据写入所述volume标识符对应的叶页层，包括：
    将所述中间页层的热数据区域或冷数据区域中缓存的待写入数据写入所述volume标识符对应的叶页层。
  12. 根据权利要求6所述数据操作请求处理方法，其特征在于，在将所述映射表写 入至所述物理块层之后，还包括：
    将所述物理块层中的数据落盘至RAID阵列。
  13. 根据权利要求6所述数据操作请求处理方法，其特征在于，还包括：
    若接收到数据读取请求，则确定发送所述数据读取请求的主机设备的第二主机LBA，根据所述第二主机LBA在所述多级映射框架中查询所述数据读取请求对应的待读取数据的数据存储位置；
    根据所述数据存储位置从缓存或内存中读取并返回所述待读取数据。
  14. 根据权利要求13所述数据操作请求处理方法，其特征在于，在根据所述第二主机LBA在所述多级映射框架中查询所述数据读取请求对应的待读取数据的数据存储位置之后，还包括：
    在所述顶页层中更新发送所述数据读取请求的主机设备的数据读写频率。
  15. 一种数据操作请求处理方法，其特征在于，应用于数据存储集群，所述数据存储集群运行有如权利要求1所述的多级映射框架，所述数据操作请求处理方法，包括：
    若接收到数据删除请求，则确定发送所述数据删除请求的主机设备的第三主机LBA，根据所述第三主机LBA在所述多级映射框架中查询所述数据删除请求对应的待删除数据的数据写入计算字段；
    根据所述待删除数据的数据写入计算字段确定删除方式；其中，所述删除方式包括物理删除和逻辑删除；
    若所述删除方式为物理删除，则在所述物理块层中删除所述待删除数据，并将所述待删除数据的数据写入计数字段的值设置为0；
    若所述删除方式为逻辑删除，则将所述待删除数据的数据写入计数字段的值减1。
  16. 根据权利要求15所述数据操作请求处理方法，其特征在于，根据所述待删除数据的数据写入计算字段确定删除方式，包括：
    若所述待删除数据的数据写入计数字段大于或等于2，则所述虚拟块层判定所述待删除数据的删除方式为逻辑删除；
    若所述待删除数据的数据写入计数字段为1，则所述虚拟块层判定所述待删除数据的删除方式为物理删除。
  17. 一种数据操作请求处理系统，其特征在于，应用于数据存储集群，所述数据存储集群运行有如权利要求1所述的多级映射框架，所述数据操作请求处理系统，包括：
    写入请求接收模块，用于若接收到数据写入请求，则确定发送所述数据写入请求的主机设备的第一主机LBA，利用inode表查询所述第一主机LBA对应的inode索引；
    标识符确定模块，用于利用所述第一主机LBA和所述inode索引映射到对应的顶页层，并在所述顶页层内为待写入数据分配volume标识符；
    缓存模块，用于利用所述中间页层缓存所述数据写入请求对应的所述待写入数据；
    叶页层写模块，用于将所述待写入数据写入所述volume标识符对应的叶页层；
    虚拟块层写入模块，用于利用所述volume标识符对应的叶页层将所述待写入数据压缩为映射表并添加数据写入计数字段，将所述映射表和对应的数据写入计数字段写入所述虚拟块层；
    物理块层写入模块，用于利用所述虚拟块层根据所述映射表的数据写入计数字段确定写入方式；其中，所述写入方式包括物理写入和逻辑写入；还用于若所述写入方式为物理写入，则将所述映射表写入至对应的物理块层，并将所述映射表的数据写入计数字段的值设置为1；还用于若所述写入方式为逻辑写入，则将所述映射表的数据写入计数字段的值加1。
  18. 一种数据操作请求处理系统，其特征在于，应用于数据存储集群，所述数据存储集群运行有如权利要求1所述的多级映射框架，所述数据操作请求处理系统，包括：
    查询模块，用于若接收到数据删除请求，则确定发送所述数据删除请求的主机设备的第三主机LBA，根据所述第三主机LBA在所述多级映射框架中查询所述数据删除请求对应的待删除数据的数据写入计算字段；
    删除方式确定模块，用于根据所述待删除数据的数据写入计算字段确定删除方式；其中，所述删除方式包括物理删除和逻辑删除；
    删除执行模块，用于若所述删除方式为物理删除，则在所述物理块层中删除所述待删除数据，并将所述待删除数据的数据写入计数字段的值设置为0；还用于若所述删除方式为逻辑删除，则将所述待删除数据的数据写入计数字段的值减1。
  19. 一种电子设备，其特征在于，包括存储器和处理器，所述存储器中存储有计算机程序，所述处理器调用所述存储器中的计算机程序时实现如权利要求6至14或者权利要求15至16任一项所述数据操作请求处理方法。
  20. 一种非易失性存储介质，其特征在于，所述非易失性存储介质中存储有计算机可执行指令，所述计算机可执行指令被处理器加载并执行时，实现如权利要求6至14或者权利要求15至16任一项所述数据操作请求处理方法。
PCT/CN2023/095867 2022-11-24 2023-05-23 一种多级映射框架、数据操作请求处理方法及系统 WO2024108939A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211479284.8A CN115576868B (zh) 2022-11-24 2022-11-24 一种多级映射框架、数据操作请求处理方法及系统
CN202211479284.8 2022-11-24

Publications (1)

Publication Number Publication Date
WO2024108939A1 true WO2024108939A1 (zh) 2024-05-30

Family

ID=84590644

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/095867 WO2024108939A1 (zh) 2022-11-24 2023-05-23 一种多级映射框架、数据操作请求处理方法及系统

Country Status (2)

Country Link
CN (1) CN115576868B (zh)
WO (1) WO2024108939A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115576868B (zh) * 2022-11-24 2023-03-03 苏州浪潮智能科技有限公司 一种多级映射框架、数据操作请求处理方法及系统

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326134A (zh) * 2015-06-30 2017-01-11 华为技术有限公司 Ftl地址映射的方法及装置
CN112506438A (zh) * 2020-12-14 2021-03-16 深圳大普微电子科技有限公司 一种映射表管理方法及固态硬盘
US20210342273A1 (en) * 2020-04-30 2021-11-04 EMC IP Holding Company LLC Mapping virtual block addresses to portions of a logical address space that point to the virtual block addresses
CN115576868A (zh) * 2022-11-24 2023-01-06 苏州浪潮智能科技有限公司 一种多级映射框架、数据操作请求处理方法及系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI480733B (zh) * 2012-03-29 2015-04-11 Phison Electronics Corp 資料寫入方法、記憶體控制器與記憶體儲存裝置
CN107016100A (zh) * 2017-04-10 2017-08-04 重庆大学 一种基于非易失性内存文件系统的元数据管理方法
CN109690522B (zh) * 2018-08-27 2024-02-27 袁振南 一种基于b+树索引的数据更新方法、装置及存储装置
CN110674051A (zh) * 2019-09-24 2020-01-10 中国科学院微电子研究所 一种数据存储方法及装置
CN115185858B (zh) * 2022-09-09 2023-01-24 北京特纳飞电子技术有限公司 地址映射表的处理方法、装置和存储设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106326134A (zh) * 2015-06-30 2017-01-11 华为技术有限公司 Ftl地址映射的方法及装置
US20210342273A1 (en) * 2020-04-30 2021-11-04 EMC IP Holding Company LLC Mapping virtual block addresses to portions of a logical address space that point to the virtual block addresses
CN112506438A (zh) * 2020-12-14 2021-03-16 深圳大普微电子科技有限公司 一种映射表管理方法及固态硬盘
CN115576868A (zh) * 2022-11-24 2023-01-06 苏州浪潮智能科技有限公司 一种多级映射框架、数据操作请求处理方法及系统

Also Published As

Publication number Publication date
CN115576868B (zh) 2023-03-03
CN115576868A (zh) 2023-01-06

Similar Documents

Publication Publication Date Title
US10067684B2 (en) File access method and apparatus, and storage device
US20230409473A1 (en) Namespace change propagation in non-volatile memory devices
US9342256B2 (en) Epoch based storage management for a storage device
US10831734B2 (en) Update-insert for key-value storage interface
KR20170008153A (ko) 비휘발성 장치에서 데이터 속성 기반 데이터 배치를 활용하기 위해 컴퓨터를 구동하는 경험적 인터페이스
US7415653B1 (en) Method and apparatus for vectored block-level checksum for file system data integrity
US11237980B2 (en) File page table management technology
Lee et al. An efficient index buffer management scheme for implementing a B-tree on NAND flash memory
WO2024108939A1 (zh) 一种多级映射框架、数据操作请求处理方法及系统
EP4336336A1 (en) Data compression method and apparatus
US11947419B2 (en) Storage device with data deduplication, operation method of storage device, and operation method of storage server
US7424574B1 (en) Method and apparatus for dynamic striping
US11675789B2 (en) Tracking utilization of data blocks in a storage system
US11662932B2 (en) Tiered storage system with defragmentation based on weighted flash fragmentation factor
US20240020014A1 (en) Method for Writing Data to Solid-State Drive
KR20210012085A (ko) 저널 리플레이를 개선하기 위한 스토리지 장치, 그것의 동작 방법, 및 스토리지 장치를 포함하는 전자 장치
CN115203079A (zh) 一种将数据写入固态硬盘的方法
US11366609B2 (en) Technique for encoding deferred reference count increments and decrements
CN110162268B (zh) 利用实时计算用于逐分块数据组织和放置的方法和系统
US20220269612A1 (en) Multi-value mapping for object store
US7533225B1 (en) Method and apparatus for enabling adaptive endianness
US11829341B2 (en) Space-efficient persistent hash table data structure
US20220164135A1 (en) Apparatus, method and computer program for managing memory page updates within non-volatile memory
US11803314B2 (en) Techniques for performing metadata updates
US20240176741A1 (en) Caching techniques using a two-level read cache