CN113505086A - Storage system capacity statistical method, device, storage medium and equipment - Google Patents
Storage system capacity statistical method, device, storage medium and equipment Download PDFInfo
- Publication number
- CN113505086A CN113505086A CN202111059752.1A CN202111059752A CN113505086A CN 113505086 A CN113505086 A CN 113505086A CN 202111059752 A CN202111059752 A CN 202111059752A CN 113505086 A CN113505086 A CN 113505086A
- Authority
- CN
- China
- Prior art keywords
- key
- value pair
- tree
- pba
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007619 statistical method Methods 0.000 title description 8
- 238000000034 method Methods 0.000 claims abstract description 41
- 230000004044 response Effects 0.000 claims description 45
- 238000003780 insertion Methods 0.000 claims description 39
- 230000037431 insertion Effects 0.000 claims description 39
- 230000009467 reduction Effects 0.000 claims description 12
- 238000012217 deletion Methods 0.000 claims description 10
- 230000037430 deletion Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 235000019580 granularity Nutrition 0.000 claims 7
- 238000007726 management method Methods 0.000 description 23
- 238000010586 diagram Methods 0.000 description 8
- 238000013507 mapping Methods 0.000 description 7
- 229920001485 poly(butyl acrylate) polymer Polymers 0.000 description 5
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000007787 solid Substances 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000005055 memory storage Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012005 ligant binding assay Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0238—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
- G06F12/0246—Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory in block erasable memory, e.g. flash memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F12/00—Accessing, addressing or allocating within memory systems or architectures
- G06F12/02—Addressing or allocation; Relocation
- G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
- G06F12/023—Free address space management
- G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9027—Trees
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a method, a device, a storage medium and equipment for counting the capacity of a storage system, wherein the method comprises the following steps: if a storage volume of the storage system receives a write request, writing corresponding data into a storage pool to obtain a PBA, respectively forming the PBA and the LBA corresponding to the data into a first key value pair and a second key value pair, and sending the first key value pair and the second key value pair to a metadata management module; sending, by the first B + tree, a first insert request to insert the first key-value pair and sending, by the second B + tree, a second insert request to insert the second key-value pair; judging whether the first B + tree has the LBA of the first key value pair; if the first key value pair does not exist, inserting the first key value pair into the first B + tree, and adding one to the first key number to count the capacity of the storage volume to increase one granularity; judging whether the PBA of the second key value pair exists in the second B + tree or not; if not, a second key-value pair is inserted into the second B + tree, and the second key-value pair is added by one to count the capacity of the storage pool increased by one granularity. The invention can accurately count the capacity of the storage volume and the storage pool.
Description
Technical Field
The present invention relates to the field of storage technologies, and in particular, to a method and an apparatus for counting a capacity of a storage system, a storage medium, and a device.
Background
Metadata (Mete data) refers to data (data about data) describing data, and can be understood as data with a wider scope than general meaning, and not only represents information such as type, name, value and the like of the data, but also further provides context information of the data, such as a domain to which the data belongs, a data source and the like. In a data storage system, metadata is the basis for information storage, being the smallest unit of data. In recent years, with the development of information technology, massive data is generated, but how to effectively manage and organize the massive data has become a prominent problem. For a large amount of stored data, the query analyzes the data content and data meaning in the stored data, so that the data can be used more effectively. In a storage system, efficient organization and management of metadata is an effective means for solving the problem, and can support management and maintenance of data by the system. Therefore, only if metadata is managed efficiently, data becomes more valuable.
The full flash memory is a storage system based on a full flash memory array, is an independent storage array or device completely composed of solid state storage media, and is mainly different from the traditional hard disk storage in that the performance is higher, and data processing is faster and more stable. In the full flash memory, the metadata management is crucial, and the metadata management mainly manages L-P (LBA → PBA) mapping and P-L (PBA → LBA) mapping relationships, where LBA (logical Block address) represents a logical Block address, and PBA (physical Block address) represents a physical Block address.
In a general storage system, statistics of storage volume usage capacity and storage pool usage capacity are calculated based on blocks (Block elements) as basic units, the size of each Block is more than one hundred megabytes, and the volume and pool capacity are calculated and counted according to the usage number of the Block.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a storage medium, and a device for counting a capacity of a storage system, so as to solve the problem in the prior art that there is no accurate method for counting storage volumes and storage pools in a full flash memory storage system.
Based on the above purpose, the present invention provides a storage system capacity statistical method, which comprises the following steps:
responding to a write request received by a storage volume of a storage system, writing data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively forming a first key value pair pointing to the PBA by the LBA and a second key value pair pointing to the LBA by the PBA and sending the first key value pair and the second key value pair to a metadata management module;
in response to the metadata management module receiving the first key-value pair and the second key-value pair, sending a first insertion request for inserting the first key-value pair to the first B + tree, and sending a second insertion request for inserting the second key-value pair to the second B + tree;
in response to the first B + tree receiving the first insertion request, judging whether the first B + tree has the LBA of the first key-value pair or not based on the first insertion request;
in response to the LBA of the first key-value pair not existing in the first B + tree, inserting the first key-value pair into the first B + tree, and adding one to the first key number to count the capacity of the storage volume to increase one granularity;
responding to the second B + tree receiving a second insertion request, and judging whether a PBA of a second key value pair exists in the second B + tree or not based on the second insertion request;
in response to the PBA for which the second key-value pair does not exist in the second B + tree, the second key-value pair is inserted into the second B + tree and the second key is incremented by one to count the capacity of the storage pool increased by one granularity.
In some embodiments, the method further comprises:
the second key-value pair is directly inserted into the second B + tree in response to the PBA of the second key-value pair already existing in the second B + tree.
In some embodiments, the method further comprises:
and in response to the first B + tree receiving a first deletion request for deleting the first key-value pair sent by the metadata management module, deleting the first key-value pair to be deleted from the first B + tree based on the first deletion request, and reducing the first key number by one to count the reduction of the capacity of the storage volume by one granularity.
In some embodiments, the method further comprises:
responding to a second B + tree receiving a second deleting request for deleting a second key value pair sent by a metadata management module, deleting the second key value pair to be deleted based on the second deleting request, and confirming whether other second key value pairs corresponding to the PBA of the second key value pair to be deleted are all deleted;
in response to all other second key-value pairs corresponding to the PBA having been deleted, the second key-value pair is decremented by one to count a reduction in the capacity of the storage pool by one granularity.
In some embodiments, the method further comprises:
in response to the second key being decremented by one to account for a reduction in the capacity of the storage pool by one granularity, the garbage collection module is notified, via the metadata management module, to garbage collect the PBA.
In some embodiments, writing the data corresponding to the write request to the storage pool of the storage system to obtain the PBA of the storage pool corresponding to the data comprises:
and writing the data corresponding to the write request into a storage pool of the storage system through the storage volume, and responding to the completion of the writing and sending the PBA corresponding to the data to the storage volume through the storage pool.
In some embodiments, the storage system comprises a full flash storage system, the storage volumes comprise thin volumes, and the storage pools comprise thin pools.
In another aspect of the present invention, a storage system capacity statistic apparatus is further provided, including:
the key value pair composing module is configured to respond to a write request received by a storage volume of the storage system, write data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively compose the PBA and the LBA of the storage volume corresponding to the data into a first key value pair pointing to the PBA from the LBA and a second key value pair pointing to the LBA from the PBA, and send the first key value pair and the second key value pair to the metadata management module;
the key value pair insertion module is configured to respond to the metadata management module receiving the first key value pair and the second key value pair, send a first insertion request for inserting the first key value pair to the first B + tree, and send a second insertion request for inserting the second key value pair to the second B + tree;
the first judgment module is configured to respond to the first B + tree receiving the first insertion request, and judge whether the first B + tree has the LBA of the first key-value pair or not based on the first insertion request;
a first statistics module configured to insert the first key-value pair into the first B + tree in response to the first B + tree not having the LBA of the first key-value pair, and add one to the first key-number to count an increase in capacity of the storage volume by one granularity;
the second judgment module is configured to respond to the second B + tree receiving a second insertion request, and judge whether a PBA of a second key-value pair exists in the second B + tree based on the second insertion request; and
and a second statistical module configured to insert the second key-value pair into the second B + tree in response to the PBA of the second key-value pair not existing in the second B + tree, and add one to the second key to count the increase in the capacity of the storage pool by one granularity.
In yet another aspect of the present invention, there is also provided a computer readable storage medium storing computer program instructions which, when executed by a processor, implement any one of the methods described above.
In yet another aspect of the present invention, a computer device is provided, which includes a memory and a processor, the memory storing a computer program, the computer program executing any one of the above methods when executed by the processor.
The invention has at least the following beneficial technical effects:
the method organizes the metadata of the storage system by using the data structure of the B + tree, and has high access efficiency; by associating the first key value pair and the second key value pair saved in the B + tree with the minimum granularity of data, the current use capacity of the storage volume and the physical use capacity of the storage pool can be respectively counted through the increased operation record of the first B + tree on the LBA and the increased operation record of the second B + tree on the PBA, the method is efficient and accurate, the concurrency degree of access can be improved, and efficient metadata access is obtained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained by using the drawings without creative efforts.
FIG. 1 is a diagram illustrating a method for counting capacity of a storage system according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a storage system capacity statistics apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a computer-readable storage medium for implementing a method for statistics of storage system capacity according to an embodiment of the present invention;
fig. 4 is a schematic hardware structure diagram of a computer device for performing a storage system capacity statistics method according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two non-identical entities with the same name or different parameters, and it is understood that "first" and "second" are only used for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprises" and "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements does not include all of the other steps or elements inherent in the list.
In view of the above-mentioned objects, a first aspect of the embodiments of the present invention provides an embodiment of a method for counting capacity of a storage system. Fig. 1 is a schematic diagram illustrating an embodiment of a storage system capacity statistical method provided by the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
step S10, responding to the fact that a storage volume of the storage system receives a write request, writing data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively forming a first key value pair pointing to the PBA from the LBA and a second key value pair pointing to the LBA from the PBA and sending the first key value pair and the second key value pair to the metadata management module;
step S20, in response to the metadata management module receiving the first key-value pair and the second key-value pair, sending a first insertion request for inserting the first key-value pair to the first B + tree, and sending a second insertion request for inserting the second key-value pair to the second B + tree;
step S30, in response to the first B + tree receiving the first insertion request, determining whether the LBA of the first key-value pair exists in the first B + tree based on the first insertion request;
step S40, in response to the first B + tree not having the LBA of the first key-value pair, inserting the first key-value pair into the first B + tree, and adding one to the first key-value to count the capacity of the storage volume to increase a granularity;
step S50, responding to the second B + tree receiving the second insertion request, and judging whether the PBA of the second key value pair exists in the second B + tree based on the second insertion request;
step S60, in response to the PBA of the second key-value pair not existing in the second B + tree, inserting the second key-value pair into the second B + tree, and adding one to the second key to count the capacity of the storage pool to increase by one granularity.
In this embodiment, an IO (data Input Output) flow of the full flash metadata includes: when a data query request needs to query data, firstly querying metadata, finding out an LBA → PBA mapping relation, firstly accessing a metadata cache, if corresponding metadata is found in the cache, directly checking the metadata and then returning to the query request, otherwise, accessing the metadata on an SSD (solid state disk), then returning to the query request, and finally accessing the data corresponding to the PBA by the query request; it can be seen that the access efficiency of metadata is very important for data reading and writing, and the organization method of metadata is very important for the access efficiency of metadata. In order to support efficient lookup of metadata objects within a full flash memory storage system, an advanced index structure support is required to achieve better lookup performance.
The B + tree is a tree data structure, belongs to a multi-branch balanced tree, and is characterized in that the data can be kept stable and orderly, and the insertion and modification of the B + tree have stable logarithmic time complexity. In particular, the B + tree index has a lookup time complexity of O (logn) and a spatial usage of 75% (non-leaf nodes as index nodes, not as nodes to hold data). O (logn) indicates that the time consumption increases logn times when the data increases n times, where log is base 2, e.g., only 8 times when the data increases 256 times. The B + tree search is performed through the root node and then downward traversal step by step until reaching the leaf node, so that the non-leaf node is an important node in the query process and is the most frequently accessed node, and the lower level node has higher access frequency, and the lower level non-leaf node is kept in the memory as much as possible. Due to the large amount of data access with high concurrency and short time delay, the full flash memory metadata is generally organized by using a data structure of a B + tree, the metadata is updated in real time, and the granularity of representation is more accurate, so that the storage system is more efficient. Therefore, the B + tree has better searching efficiency in combination, and is more suitable for organizing metadata objects such as LBAs and PBAs.
In this embodiment, the metadata management module mainly manages a mapping relationship (i.e., a first key value pair) between LBA → PBA and a mapping relationship (i.e., a second key value pair) between PBA → LBA, where LBA (logical Block address) represents a logical Block address, and PBA (physical Block address) represents a physical Block address. Specifically, the first key value pair represents mapping from a logical block address LBA of the storage volume to a physical block address PBA of the storage pool; the second key value pair represents mapping from a physical block address PBA of the storage pool to a logical block address LBA of the storage volume; the first B + tree comprises a plurality of first key value pairs and is used for reading and writing by a user host; the second B + tree comprises a plurality of second key value pairs for garbage collection to inquire whether the physical block address PBA is still used.
In this embodiment, the granularity (Grain) represents the minimum capacity unit of data, and the data is written for each granularity capacity.
The embodiment of the invention organizes the metadata of the storage system by using the data structure of the B + tree, thereby having high access efficiency; by associating the first key value pair and the second key value pair saved in the B + tree with the minimum granularity of data, the current use capacity of the storage volume and the physical use capacity of the storage pool can be respectively counted through the increased operation record of the first B + tree on the LBA and the increased operation record of the second B + tree on the PBA, the method is efficient and accurate, the concurrency degree of access can be improved, and efficient metadata access is obtained.
In some embodiments, the method further comprises: the second key-value pair is directly inserted into the second B + tree in response to the PBA of the second key-value pair already existing in the second B + tree.
In this embodiment, when the PBA of the second key-value pair already exists in the second B + tree, the second key-value pair is inserted into the second B + tree, but the second key-value is not incremented by one. When data in a storage volume is written into a storage pool, the data may be the same, and if two same data are written into the storage pool from the storage volume, the LBA corresponding to the two data are different, but the PBAs corresponding to the two data are the same because the data mapped onto the storage pool are the same. Thus, there may be a PBA in which the second key-value pair already exists discovered after the second B + tree receives the second insert request for the second key-value pair.
In some embodiments, the method further comprises: and in response to the first B + tree receiving a first deletion request for deleting the first key-value pair sent by the metadata management module, deleting the first key-value pair to be deleted from the first B + tree based on the first deletion request, and reducing the first key number by one to count the reduction of the capacity of the storage volume by one granularity.
In this embodiment, the deleted operation record of the LBA through the first B + tree may count that the capacity of the storage volume is reduced by one granularity, so that the capacity of the storage volume is counted more comprehensively.
In some embodiments, the method further comprises: responding to a second B + tree receiving a second deleting request for deleting a second key value pair sent by a metadata management module, deleting the second key value pair to be deleted based on the second deleting request, and confirming whether other second key value pairs corresponding to the PBA of the second key value pair to be deleted are all deleted; in response to all other second key-value pairs corresponding to the PBA having been deleted, the second key-value pair is decremented by one to count a reduction in the capacity of the storage pool by one granularity.
In this embodiment, because there may be a one-to-many condition between PBA and LBA, when deleting operation, it is necessary to delete all the second key-value pairs corresponding to PBA, and then count that the capacity of the storage pool is reduced by one granularity. This embodiment makes the capacity statistics of the storage pool more comprehensive.
In some embodiments, the method further comprises: in response to the second key being decremented by one to account for a reduction in the capacity of the storage pool by one granularity, the garbage collection module is notified, via the metadata management module, to garbage collect the PBA.
Through the embodiment, the storage volume and the storage pool regularly acquire the first key number of the first B + tree and the second key number of the second B + tree from the metadata management module, so that the current storage volume utilization capacity and the storage pool utilization capacity can be efficiently and accurately known, meanwhile, the garbage collection module can also accurately perform garbage collection on the PBA, and the garbage collection efficiency is improved conveniently.
In some embodiments, writing the data corresponding to the write request to the storage pool of the storage system to obtain the PBA of the storage pool corresponding to the data comprises: and writing the data corresponding to the write request into a storage pool of the storage system through the storage volume, and responding to the completion of the writing and sending the PBA corresponding to the data to the storage volume through the storage pool.
In some embodiments, the storage system comprises a full flash storage system and the storage volume comprises a thin volume.
In this embodiment, the thin volume uses space as needed, that is, only the storage pool space is occupied when data is written into the volume, and the thin volume can ensure efficient use of the storage space and is suitable for creating a plurality of volumes, so that the thin volume is selected as the storage volume.
In a second aspect of the embodiments of the present invention, a device for counting the capacity of a storage system is also provided. Fig. 2 is a schematic diagram illustrating an embodiment of a storage system capacity statistics apparatus provided by the present invention. As shown in fig. 2, a storage system capacity statistics apparatus includes: a key-value pair composing module 10 configured to, in response to a write request received by a storage volume of the storage system, write data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively compose the PBA and an LBA of the storage volume corresponding to the data into a first key-value pair pointing to the PBA from the LBA and a second key-value pair pointing to the LBA from the PBA, and send the first key-value pair and the second key-value pair to the metadata management module; a key-value pair insertion module 20 configured to, in response to the metadata management module receiving the first key-value pair and the second key-value pair, send a first insertion request for inserting the first key-value pair to the first B + tree, and send a second insertion request for inserting the second key-value pair to the second B + tree; a first determining module 30, configured to determine, in response to the first B + tree receiving the first insertion request, whether the LBA of the first key-value pair exists in the first B + tree based on the first insertion request; a first statistics module 40 configured to insert the first key-value pair into the first B + tree in response to the LBA of the first key-value pair not existing in the first B + tree, and add one to the first key number to count an increase in the capacity of the storage volume by one granularity; a second determining module 50, configured to determine, in response to the second B + tree receiving the second insertion request, whether there is a PBA of the second key-value pair in the second B + tree based on the second insertion request; and a second statistics module 60 configured to insert the second key-value pair into the second B + tree in response to the PBA of the second key-value pair not existing in the second B + tree, and to increase the capacity of the second key by one granularity by counting the storage pool.
In some embodiments, the apparatus further comprises a direct insertion module configured to directly insert the second key-value pair into the second B + tree in response to the PBA in which the second key-value pair already exists in the second B + tree.
In some embodiments, the apparatus further includes a first capacity reduction module configured to, in response to the first B + tree receiving a first delete request sent by the metadata management module to delete the first key-value pair, delete the first key-value pair to be deleted from the first B + tree based on the first delete request, and reduce the first key number by one to account for a capacity reduction of the storage volume by one granularity.
In some embodiments, the apparatus further includes a second capacity reduction module, configured to, in response to receiving, by the second B + tree, a second deletion request for deleting the second key-value pair sent by the metadata management module, delete the second key-value pair to be deleted based on the second deletion request, and confirm whether all other second key-value pairs corresponding to PBAs of the second key-value pair to be deleted have been deleted; in response to all other second key-value pairs corresponding to the PBA having been deleted, the second key-value pair is decremented by one to count a reduction in the capacity of the storage pool by one granularity.
In some embodiments, the apparatus further includes a garbage collection module configured to notify, by the metadata management module, the garbage collection module to garbage collect the PBAs in response to decreasing the second key by one to account for a decrease in the capacity of the storage pool by one granularity.
In some embodiments, the key-value pair composing module 10 includes a PBA sending module configured to write data corresponding to the write request to a storage pool of the storage system through the storage volume, and in response to completion of the write, send PBAs corresponding to the data to the storage volume through the storage pool.
In some embodiments, the storage system comprises a full flash storage system, the storage volumes comprise thin volumes, and the storage pools comprise thin pools.
In a third aspect of the embodiment of the present invention, a computer-readable storage medium is further provided, and fig. 3 is a schematic diagram of a computer-readable storage medium for implementing a capacity statistics method of a storage system according to an embodiment of the present invention. As shown in fig. 3, the computer-readable storage medium 3 stores computer program instructions 31. The computer program instructions 31, when executed by a processor, implement the method of any of the embodiments described above.
It should be understood that all the embodiments, features and advantages set forth above with respect to the storage system capacity statistical method according to the present invention are equally applicable to the storage system capacity statistical apparatus and the storage medium according to the present invention, without conflicting therewith.
In a fourth aspect of the embodiments of the present invention, there is further provided a computer device, including a memory 402 and a processor 401, where the memory stores a computer program, and the computer program, when executed by the processor, implements the method of any one of the above embodiments.
Fig. 4 is a schematic hardware structure diagram of an embodiment of a computer device for performing a storage system capacity statistics method according to the present invention. Taking the computer device shown in fig. 4 as an example, the computer device includes a processor 401 and a memory 402, and may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected by a bus or other means, and fig. 4 illustrates an example of a connection by a bus. The input device 403 may receive input numeric or character information and generate key signal inputs related to user settings and function control of the storage system capacity counting system. The output device 404 may include a display device such as a display screen.
The memory 402, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the storage system capacity statistical method in the embodiments of the present application. The memory 402 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by use of a storage system capacity statistical method, and the like. Further, the memory 402 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 402 may optionally include memory located remotely from processor 401, which may be connected to local modules via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The processor 401 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 402, that is, implements the storage system capacity statistical method of the above-described method embodiment.
Finally, it should be noted that the computer-readable storage medium (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items. The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.
Claims (10)
1. A method for counting the capacity of a storage system is characterized by comprising the following steps:
responding to a write request received by a storage volume of a storage system, writing data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively forming a first key value pair pointing to the PBA by the LBA and a second key value pair pointing to the LBA by the PBA and sending the first key value pair and the second key value pair to a metadata management module by the PBA and the PBA of the storage volume corresponding to the data;
in response to the metadata management module receiving the first key-value pair and the second key-value pair, sending a first insert request to insert the first key-value pair to a first B + tree and sending a second insert request to insert the second key-value pair to a second B + tree;
in response to the first B + tree receiving the first insertion request, determining whether the LBA of the first key-value pair exists in the first B + tree based on the first insertion request;
in response to the first B + tree not having the LBA of the first key-value pair, inserting the first key-value pair into the first B + tree and adding one to a first key number to count the increase in capacity of the storage volume by one granularity;
in response to the second B + tree receiving the second insertion request, determining whether a PBA of the second key-value pair exists in the second B + tree based on the second insertion request;
in response to the absence of a PBA of the second key-value pair in the second B + tree, inserting the second key-value pair into the second B + tree and incrementing a second key to count an increase in capacity of the storage pool by one of the granularities.
2. The method of claim 1, further comprising:
directly inserting the second key-value pair into the second B + tree in response to a PBA for which the second key-value pair already exists in the second B + tree.
3. The method of claim 1, further comprising:
in response to the first B + tree receiving a first deletion request for deleting a first key-value pair sent by the metadata management module, deleting the first key-value pair to be deleted from the first B + tree based on the first deletion request, and reducing the first key number by one to count the reduction of the capacity of the storage volume by one granularity.
4. The method of claim 1, further comprising:
responding to a second deletion request sent by the metadata management module and used for deleting a second key-value pair received by the second B + tree, deleting the second key-value pair to be deleted based on the second deletion request, and confirming whether other second key-value pairs corresponding to the PBA of the second key-value pair to be deleted are all deleted;
in response to all other second key-value pairs corresponding to the PBA having been deleted, decrementing the second key number by one to account for a reduction in the capacity of the storage pool by one of the granularities.
5. The method of claim 4, further comprising:
notifying, by the metadata management module, a garbage reclamation module to garbage reclaim the PBA in response to decrementing the second key by one to account for a decrease in capacity of the storage pool by one of the granularities.
6. The method of claim 1, wherein writing data corresponding to the write request to a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data comprises:
and writing the data corresponding to the writing request into a storage pool of the storage system through the storage volume, and responding to the writing completion and sending the PBA corresponding to the data to the storage volume through the storage pool.
7. The method of claim 1, wherein the storage system comprises a full flash storage system, wherein the storage volumes comprise thin volumes, and wherein the storage pools comprise thin pools.
8. A storage system capacity statistics apparatus, comprising:
a key-value pair composing module configured to, in response to a write request received by a storage volume of a storage system, write data corresponding to the write request into a storage pool of the storage system to obtain a PBA of the storage pool corresponding to the data, respectively compose the PBA and an LBA of the storage volume corresponding to the data into a first key-value pair pointing to the PBA from the LBA and a second key-value pair pointing to the LBA from the PBA, and send the first key-value pair and the second key-value pair to a metadata management module;
a key-value pair insertion module configured to send a first insertion request to insert the first key-value pair to a first B + tree and send a second insertion request to insert the second key-value pair to a second B + tree in response to the metadata management module receiving the first key-value pair and the second key-value pair;
a first determining module, configured to determine, in response to the first B + tree receiving the first insertion request, whether an LBA of the first key-value pair exists in the first B + tree based on the first insertion request;
a first statistics module configured to insert the first key-value pair into the first B + tree in response to the LBA of the first key-value pair not existing in the first B + tree, and add a first key number by one to count a capacity increase of the storage volume by one granularity;
a second determining module, configured to determine, in response to the second B + tree receiving the second insertion request, whether a PBA of the second key-value pair exists in the second B + tree based on the second insertion request; and
a second statistics module configured to insert the second key-value pair into the second B + tree in response to no PBA of the second key-value pair in the second B + tree, and to add a second key number by one to count an increase in the capacity of the storage pool by one of the granularities.
9. A computer-readable storage medium, characterized in that computer program instructions are stored which, when executed by a processor, implement the method according to any one of claims 1-7.
10. A computer device comprising a memory and a processor, characterized in that the memory has stored therein a computer program which, when executed by the processor, performs the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111059752.1A CN113505086B (en) | 2021-09-10 | 2021-09-10 | Storage system capacity statistical method, device, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111059752.1A CN113505086B (en) | 2021-09-10 | 2021-09-10 | Storage system capacity statistical method, device, storage medium and equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113505086A true CN113505086A (en) | 2021-10-15 |
| CN113505086B CN113505086B (en) | 2021-12-07 |
Family
ID=78017135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111059752.1A Active CN113505086B (en) | 2021-09-10 | 2021-09-10 | Storage system capacity statistical method, device, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113505086B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130275656A1 (en) * | 2012-04-17 | 2013-10-17 | Fusion-Io, Inc. | Apparatus, system, and method for key-value pool identifier encoding |
| US20160070480A1 (en) * | 2014-09-10 | 2016-03-10 | Netapp, Inc. | Technique for quantifying logical space trapped in an extent store |
| CN108897707A (en) * | 2018-06-26 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of capacity statistical method, device and the equipment of multi-controller |
| US20190163528A1 (en) * | 2017-11-30 | 2019-05-30 | Microsoft Technology Licensing, Llc | Automated capacity management in distributed computing systems |
| CN110532201A (en) * | 2019-08-23 | 2019-12-03 | 北京浪潮数据技术有限公司 | A kind of metadata processing method and device |
| CN111475508A (en) * | 2020-03-31 | 2020-07-31 | 浙江大学 | Efficient indexing method for optimizing leaf node merging operation |
| CN112181790A (en) * | 2020-09-15 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Capacity statistical method and system of storage equipment and related components |
| US20210255811A1 (en) * | 2020-02-13 | 2021-08-19 | International Business Machines Corporation | Automatic volume statistical and performance information tracking |
-
2021
- 2021-09-10 CN CN202111059752.1A patent/CN113505086B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130275656A1 (en) * | 2012-04-17 | 2013-10-17 | Fusion-Io, Inc. | Apparatus, system, and method for key-value pool identifier encoding |
| US20160070480A1 (en) * | 2014-09-10 | 2016-03-10 | Netapp, Inc. | Technique for quantifying logical space trapped in an extent store |
| US20190163528A1 (en) * | 2017-11-30 | 2019-05-30 | Microsoft Technology Licensing, Llc | Automated capacity management in distributed computing systems |
| CN108897707A (en) * | 2018-06-26 | 2018-11-27 | 郑州云海信息技术有限公司 | A kind of capacity statistical method, device and the equipment of multi-controller |
| CN110532201A (en) * | 2019-08-23 | 2019-12-03 | 北京浪潮数据技术有限公司 | A kind of metadata processing method and device |
| US20210255811A1 (en) * | 2020-02-13 | 2021-08-19 | International Business Machines Corporation | Automatic volume statistical and performance information tracking |
| CN111475508A (en) * | 2020-03-31 | 2020-07-31 | 浙江大学 | Efficient indexing method for optimizing leaf node merging operation |
| CN112181790A (en) * | 2020-09-15 | 2021-01-05 | 苏州浪潮智能科技有限公司 | Capacity statistical method and system of storage equipment and related components |
Non-Patent Citations (2)
| Title |
|---|
| 张娟娟等: "基于闪存阵列的缓存容量确定方法", 《电子测试》 * |
| 闫湖等: "基于分布式键值对存储技术的EMS数据库平台", 《电网技术》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113505086B (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11853549B2 (en) | Index storage in shingled magnetic recording (SMR) storage system with non-shingled region | |
| CN106874348B (en) | File storage and index method and device and file reading method | |
| CN111125447A (en) | Metadata access method, device and equipment and readable storage medium | |
| CN105468642A (en) | Data storage method and apparatus | |
| EP4174677A1 (en) | Data reading and writing method and device | |
| WO2023040200A1 (en) | Data deduplication method and system, and storage medium and device | |
| CN109407985B (en) | Data management method and related device | |
| US11449270B2 (en) | Address translation method and system for KV storage device | |
| WO2020192710A1 (en) | Method for processing garbage based on lsm database, solid state hard disk, and storage apparatus | |
| CN111352589B (en) | Distributed storage method, device, equipment and readable medium | |
| US20150212744A1 (en) | Method and system of eviction stage population of a flash memory cache of a multilayer cache system | |
| CN110647423B (en) | Method, device and readable medium for creating storage volume mirror image based on application | |
| KR20230026946A (en) | Key value storage device with hashing | |
| CN111930924A (en) | Data duplicate checking system and method based on bloom filter | |
| US10303655B1 (en) | Storage array compression based on the structure of the data being compressed | |
| CN111221475A (en) | Storage space management method, device, equipment and readable medium | |
| CN106354793B (en) | Method and device for monitoring hot spot object | |
| CN113505086B (en) | Storage system capacity statistical method, device, storage medium and equipment | |
| CN110781101A (en) | One-to-many mapping relation storage method and device, electronic equipment and medium | |
| CN111241090A (en) | Method and device for managing data index in storage system | |
| CN107315806B (en) | Embedded storage method and device based on file system | |
| CN115964002A (en) | Electric energy meter terminal file management method, device, equipment and medium | |
| CN111104435B (en) | Metadata organization method, device and equipment and computer readable storage medium | |
| CN113742378A (en) | Data query and storage method, related equipment and storage medium | |
| WO2020238750A1 (en) | Data processing method and apparatus, electronic device, and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |