CN111142803B - Metadata disk refreshing method, device, equipment and medium - Google Patents
Metadata disk refreshing method, device, equipment and medium Download PDFInfo
- Publication number
- CN111142803B CN111142803B CN201911386429.8A CN201911386429A CN111142803B CN 111142803 B CN111142803 B CN 111142803B CN 201911386429 A CN201911386429 A CN 201911386429A CN 111142803 B CN111142803 B CN 111142803B
- Authority
- CN
- China
- Prior art keywords
- tree
- directories
- brushing
- determining
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0673—Single storage device
- G06F3/0679—Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0652—Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
- G06F3/0656—Data buffering arrangements
Abstract
The application discloses a method, a device, equipment and a medium for refreshing metadata, which comprise the following steps: reading the water level of the current write cache; determining the number of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level; determining a target tree directory to be brushed of the lower tree directory quantity from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories; and flushing the metadata cached in the target tree to a solid state disk. Therefore, the corresponding quantity of the tree directories to be brushed and the data quantity threshold of the trees to be brushed are determined by utilizing the water level of the write cache, the data quantity of the tree directories and the data quantity of the trees are considered when the target trees to be brushed are determined, the data in the write cache can be brushed to the solid state disk in time, the problem that the data quantity to be brushed each time is too small is solved, and the performance of the storage system is improved.
Description
Technical Field
The present application relates to the field of storage technologies, and in particular, to a method, an apparatus, a device, and a medium for flushing metadata.
Background
In a full flash memory storage system, access and modification of metadata are generally random, so that the data volume required to be modified each time is small, and the characteristics of an SSD (Solid State Drive) require additional writing according to a certain block size, and if the metadata volume modified each time is small, the performance and the lifetime of the SSD disk are not facilitated. How to improve SSD disk writes of metadata is a current consideration.
Disclosure of Invention
In view of this, an object of the present application is to provide a method, an apparatus, a device, and a medium for flushing metadata to a disk, which can flush data in a write cache to a solid state disk in time, and avoid a problem of an excessively small data amount of each flushing, thereby improving performance of a storage system. The specific scheme is as follows:
in a first aspect, the present application discloses a metadata disk refreshing method applied to a full flash memory system, including:
reading the water level of the current write cache;
determining the number of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level;
determining a target tree directory to be brushed of the lower tree directory quantity from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories;
and flushing the metadata cached in the target tree to a solid state disk.
Optionally, the method further includes:
and generating a tree directory in the write cache according to the type and the data relation of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
Optionally, the determining the number of the tree-down directories and the data amount threshold of the tree-down according to the water level includes:
and if the water level is greater than a first preset threshold and less than or equal to a second preset threshold, determining that the number of the tree directories to be brushed is a first tree directory number, and the data volume threshold is a first data volume threshold.
Optionally, the determining the number of the tree-down directories and the data amount threshold of the tree-down according to the water level includes:
and if the water level is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the water level, and then determining the lower brushing tree directory quantity and the data quantity threshold by using the lower brushing proportion.
Optionally, the determining the number of the tree-down directories and the data amount threshold of the tree-down according to the water level includes:
and if the water level is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second tree directory number, and the data volume threshold is a second data volume threshold.
Optionally, the method further includes:
if the water level is smaller than a first preset threshold value, the metadata in the write cache is prohibited to be flushed to the solid state disk, or the metadata in the tree cache in the tree directory meeting preset conditions is periodically flushed to the solid state disk.
Optionally, the determining the number of target tree directories to be brushed from all tree directories includes
And determining the target tree directories to be brushed as the tree directories to be brushed from all the tree directories by using the small top heap.
In a second aspect, the present application discloses a metadata disk refreshing apparatus applied to a full flash memory system, including:
the cache water level reading module is used for reading the current write cache water level;
the lower brushing number determining module is used for determining the number of the lower brushing tree directories and the data quantity threshold of the lower brushing trees according to the water level;
the lower brushing target determining module is used for determining a target tree directory to be brushed, which is the number of the lower brushing tree directories, from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories;
and the target data downloading and brushing module is used for downloading and brushing the metadata cached in the target tree to the solid state disk.
In a third aspect, the application discloses a metadata disk brushing device, which comprises a processor and a memory; wherein the content of the first and second substances,
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the metadata flash method.
In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned metadata flashing method.
Therefore, the method comprises the steps of reading the water level of the current write cache, then determining the number of the tree directories to be brushed and the data quantity threshold of the tree to be brushed according to the water level, determining the target tree directories to be brushed, which are the number of the tree directories to be brushed, from all the tree directories, and determining the target tree to be brushed from the target tree directories by using the data quantity threshold; and finally, the metadata cached in the target tree is down-flushed to a solid state disk. Therefore, the corresponding tree directory amount to be refreshed and the data amount threshold value of the tree to be refreshed are determined by utilizing the water level of the write cache, the data amount of the tree directory and the data amount of the tree are considered when the target tree to be refreshed is determined, the data in the write cache can be timely refreshed to the solid state disk, the problem that the data amount refreshed each time is too small is avoided, and the performance of the storage system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only the embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of a metadata flashing method disclosed in the present application;
FIG. 2 is a schematic diagram of a metadata water-level swipe down according to the present disclosure;
FIG. 3 is a flow chart of a specific metadata flashing method disclosed herein;
FIG. 4 is a schematic structural diagram of a metadata disk brushing apparatus disclosed in the present application;
fig. 5 is a structural diagram of a metadata disk brushing apparatus disclosed in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an embodiment of the present application discloses a metadata disk refreshing method applied to a full flash memory system, including:
step S11: and reading the water level of the current write cache.
Step S12: and determining the quantity of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level.
In a specific implementation manner, in this embodiment, the metadata may be stored in the tree corresponding to the write cache first, and then the corresponding flush policy is determined according to the water level of the write cache, that is, the corresponding flush policy is determined according to the utilization rate of the write cache.
If the water level is smaller than a first preset threshold value, the metadata in the write cache is prohibited to be flushed to the solid state disk, or the metadata in the tree cache in the tree directory meeting preset conditions is periodically flushed to the solid state disk. That is, the utilization rate of the current write cache is low, for example, the first preset threshold is 35%, the water level of the current write cache is less than 35%, the data in the cache is few and can not be flushed, in order to prevent the data from not being flushed in the cache for a long time, whether a tree directory has no data entering beyond the preset time can be detected at regular time, and if the preset time is exceeded, the data of the tree in the tree directory is flushed.
And if the water level is greater than a first preset threshold and less than or equal to a second preset threshold, determining that the number of the tree directories to be brushed is a first tree directory number, and the data volume threshold is a first data volume threshold. For example, the current water level is greater than 35% and less than or equal to 75%, that is, the data amount in the current write cache is not very large, the data amount that the current water level can be below the current water level can be determined according to the total resource number, and then the disk is flushed, for example, only one tree directory with the largest data amount is selected each time, the under-tree flushing with the most data is screened from the trees in the selected tree directory at a very slow speed, and the specific data flushing amount is determined. For example, if the number of lockle (a value for determining the data amount of the lower brush when the lower brush is slowly made) is 1% and the total resource count is 1w, the data amount of the lower brush is 1w × 1% to 10 at the same time.
And if the water level is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the water level, and then determining the lower brushing tree directory quantity and the data quantity threshold by using the lower brushing proportion. For example, if the number is greater than 75% and less than or equal to 85%, the disk is flushed in a linear increasing manner, the number of tree directories selected increases linearly, and the threshold of the data size of the selected tree decreases linearly, such as:
the lower brush ratio is 10 (water _ level) -75) (75< water _ level < (85);
NUM _ TREE _ DIR (the number of lower-brush TREE directories) ═ MAX _ DESTAGE _ TREE _ DIR _ NUM _ DESTAGE _ ratio;
the TREE _ stage _ THRESHOLD is MIN _ TREE _ stage _ THRESHOLD 100/stage _ ratio, the TREE _ stage _ THRESHOLD is a data amount THRESHOLD of the lower-brushing TREE, and the MAX _ stage _ TREE _ DIR _ NUM is the maximum number of lower-brushing TREE directories determined at one time. MIN _ TREE _ stage _ THRESHOLD is the lowest data volume THRESHOLD for the lower brush TREE.
It is understood that, at this stage, when the water level rises, the lower-brushing ratio linearly increases, the number of tree directories selected at one time increases, the threshold value of the data amount of the selected tree decreases, and thus the number of selected trees increases to realize a linear increase in the lower-brushing data amount.
And if the water level is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second tree directory number, and the data volume threshold is a second data volume threshold. For example, when the water level is greater than 85%, the disk is fully flushed, for example, the number of the lower-brushing TREE directories is MAX _ depth _ TREE _ DIR _ NUM, and the data amount THRESHOLD of the lower-brushing TREE is MIN _ TREE _ depth _ THRESHOLD.
For example, referring to fig. 2, fig. 2 is a schematic diagram of a metadata brush-down by water level according to an embodiment of the present application. And if the write buffer water level is less than 35%, the lower brush is not performed, the water level is 35% -75% of the lower brush with a fixed lower brush value, namely the lower brush with a fixed lower brush amount, the lower brush is linearly increased by 75% -85%, and if the water level is more than 85%, the lower brush is performed with the maximum lower brush data amount of the lower brush. That is, different water levels correspond to different allowed data amounts to be brushed down, the tree brushing-down catalog and the tree brushing-down are determined according to different water levels, and the specific data amount to be brushed down is determined by the total resource number.
Step S13: determining target tree directories to be refreshed, which are the tree directories to be refreshed, in number from all the tree directories, and determining target trees to be refreshed from the target tree directories by using the data volume threshold; and the data volume of the target tree directory is greater than that of other tree directories in all the tree directories.
Step S14: and flushing the metadata cached in the target tree to a solid state disk.
In a specific embodiment, the target trees are added to a brushing-down queue, and the data cached in the trees are brushed down onto the solid state disk at the time of data brushing.
As can be seen, in the embodiment of the present application, the current write cache water level is read, then the number of the tree directories to be refreshed and the data amount threshold of the tree to be refreshed are determined according to the water level, the number of the target tree directories to be refreshed is determined from all the tree directories, and the target tree to be refreshed is determined from the target tree directories by using the data amount threshold; and finally, the metadata cached in the target tree is down-flushed to a solid state disk. Therefore, the corresponding quantity of the tree directories to be brushed and the data quantity threshold of the trees to be brushed are determined by utilizing the water level of the write cache, the data quantity of the tree directories and the data quantity of the trees are considered when the target trees to be brushed are determined, the data in the write cache can be brushed to the solid state disk in time, the problem that the data quantity to be brushed each time is too small is solved, and the performance of the storage system is improved.
Referring to fig. 3, the present application discloses a specific metadata disk refreshing method applied to a full flash memory system, including:
step S21: and generating a tree directory in the write cache according to the type and the data relation of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
In a specific implementation manner, in this embodiment, the write cache may be organized into a corresponding tree directory according to the metadata type and the data relationship, and then a plurality of trees are partitioned in the tree directory according to the data relationship of the metadata. And after the written metadata is acquired, caching the metadata into a corresponding tree.
Step S22: and reading the water level of the current write cache.
Step S23: and determining the quantity of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level.
Step S24: determining a target tree directory to be brushed in quantity from all tree directories by using a small top heap, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; and the data volume of the target tree directory is greater than that of other tree directories in all the tree directories.
In a specific implementation manner, a target tree directory to be flushed, which is the number of tree directories to be flushed, can be determined by using a data structure of a small top heap, it is noted that the number of the tree directories to be flushed is already determined, so that the size of the heap is fixed, the tree directory with the minimum data amount in the heap is placed on the top of the heap, the time complexity of o (logn) is only required for each adjustment, because the heap top is the minimum value, each time a new tree directory is found, and only comparison with the heap top element is required, and if the value is larger than the minimum value, the heap is eliminated, and the new tree directory is placed in the heap.
Step S25: and flushing the metadata cached in the target tree to a solid state disk.
Referring to fig. 4, an embodiment of the present application discloses a metadata disk refreshing apparatus, which is applied to a full flash memory system, and includes:
a cache level reading module 11, configured to read a current write cache level;
a lower-brushing quantity determining module 12, configured to determine, according to the water level, a lower-brushing tree directory quantity and a lower-brushing tree data quantity threshold;
a lower-brushing target determining module 13, configured to determine, from all tree directories, a number of target tree directories to be brushed, and determine, from the target tree directories, a target tree to be brushed by using the data amount threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories;
and the target data downloading module 14 is configured to download the metadata cached in the target tree to the solid state disk.
As can be seen, in the embodiment of the present application, the current write cache water level is read, then the number of the tree directories to be refreshed and the data amount threshold of the tree to be refreshed are determined according to the water level, the number of the target tree directories to be refreshed is determined from all the tree directories, and the target tree to be refreshed is determined from the target tree directories by using the data amount threshold; and finally, the metadata cached in the target tree is down-flushed to a solid state disk. Therefore, the corresponding quantity of the tree directories to be brushed and the data quantity threshold of the trees to be brushed are determined by utilizing the water level of the write cache, the data quantity of the tree directories and the data quantity of the trees are considered when the target trees to be brushed are determined, the data in the write cache can be brushed to the solid state disk in time, the problem that the data quantity to be brushed each time is too small is solved, and the performance of the storage system is improved.
The metadata disk-refreshing device further comprises a tree directory and tree generation module, and the tree directory and tree generation module is used for generating the tree directory in the write cache according to the type and data relationship of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
The lower-brushing-amount determining module 12 is specifically configured to determine that the lower-brushing-tree directory amount is a first tree directory amount and the data amount threshold is a first data amount threshold, if the water level is greater than a first preset threshold and is less than or equal to a second preset threshold. And if the water level is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the water level, and then determining the lower brushing tree directory quantity and the data quantity threshold by using the lower brushing proportion. And if the water level is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second tree directory number, and the data volume threshold is a second data volume threshold. If the water level is smaller than a first preset threshold, the writing cache is prohibited from being flushed to the solid state disk, or a tree directory meeting preset conditions is determined, and correspondingly, the target data flushing module 14 may be further configured to periodically flush the metadata cached in the tree directory meeting preset conditions to the solid state disk.
The down-brushing target determining module 13 is specifically configured to determine, from all the tree directories, a number of target tree directories to be down-brushed from the down-brushing tree directory by using the small top heap.
Referring to fig. 5, an embodiment of the present application discloses a metadata disk brushing apparatus, which includes a processor 21 and a memory 22; wherein, the memory 22 is used for saving computer programs; the processor 21 is configured to execute the computer program to implement the following steps:
reading the water level of the current write cache; determining the number of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level; determining a target tree directory to be brushed of the lower tree directory quantity from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories; and flushing the metadata cached in the target tree to a solid state disk.
As can be seen, in the embodiment of the present application, the current write cache water level is read, then the number of the tree directories to be refreshed and the data amount threshold of the tree to be refreshed are determined according to the water level, the number of the target tree directories to be refreshed is determined from all the tree directories, and the target tree to be refreshed is determined from the target tree directories by using the data amount threshold; and finally, the metadata cached in the target tree is down-flushed to a solid state disk. Therefore, the corresponding quantity of the tree directories to be brushed and the data quantity threshold of the trees to be brushed are determined by utilizing the water level of the write cache, the data quantity of the tree directories and the data quantity of the trees are considered when the target trees to be brushed are determined, the data in the write cache can be brushed to the solid state disk in time, the problem that the data quantity to be brushed each time is too small is solved, and the performance of the storage system is improved.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: and generating a tree directory in the write cache according to the type and the data relation of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: and if the water level is greater than a first preset threshold and less than or equal to a second preset threshold, determining that the number of the tree directories to be brushed is a first tree directory number, and the data volume threshold is a first data volume threshold.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: and if the water level is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the water level, and then determining the lower brushing tree directory quantity and the data quantity threshold by using the lower brushing proportion.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: and if the water level is greater than a third preset threshold, determining that the quantity of the tree directories to be brushed down is the quantity of the second tree directories, and the data quantity threshold is a second data quantity threshold.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: if the water level is smaller than a first preset threshold value, the metadata in the write cache is prohibited to be flushed to the solid state disk, or the metadata in the tree cache in the tree directory meeting preset conditions is periodically flushed to the solid state disk.
In this embodiment, when the processor 21 executes the computer subprogram stored in the memory 22, the following steps may be specifically implemented: and determining the target tree directories to be brushed as the tree directories to be brushed from all the tree directories by using the small top heap.
Further, an embodiment of the present application also discloses a computer readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the following steps:
reading the water level of the current write cache; determining the number of the tree-brushing directories and the data quantity threshold of the lower brushed trees according to the water level; determining a target tree directory to be brushed of the lower tree directory quantity from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories; and flushing the metadata cached in the target tree to a solid state disk.
As can be seen, in the embodiment of the present application, the current write cache water level is read, then the number of the tree directories to be refreshed and the data amount threshold of the tree to be refreshed are determined according to the water level, the number of the target tree directories to be refreshed is determined from all the tree directories, and the target tree to be refreshed is determined from the target tree directories by using the data amount threshold; and finally, the metadata cached in the target tree is down-flushed to a solid state disk. Therefore, the corresponding quantity of the tree directories to be brushed and the data quantity threshold of the trees to be brushed are determined by utilizing the water level of the write cache, the data quantity of the tree directories and the data quantity of the trees are considered when the target trees to be brushed are determined, the data in the write cache can be brushed to the solid state disk in time, the problem that the data quantity to be brushed each time is too small is solved, and the performance of the storage system is improved.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and generating a tree directory in the write cache according to the type and the data relation of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and if the water level is greater than a first preset threshold and less than or equal to a second preset threshold, determining that the number of the tree directories to be brushed is a first tree directory number, and the data volume threshold is a first data volume threshold.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and if the water level is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the water level, and then determining the lower brushing tree directory quantity and the data quantity threshold by using the lower brushing proportion.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and if the water level is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second tree directory number, and the data volume threshold is a second data volume threshold.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and if the water level is smaller than a first preset threshold value, forbidding to flush the metadata in the write cache to the solid state disk, or regularly flushing the metadata in the tree cache in the tree directory meeting preset conditions to the solid state disk.
In this embodiment, when the computer subprogram stored in the computer-readable storage medium is executed by the processor, the following steps may be specifically implemented: and determining the target tree directories to be brushed as the tree directories to be brushed from all the tree directories by using the small top heap.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The metadata disk-refreshing method, device, equipment and medium provided by the present application are introduced in detail above, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (5)
1. A method for refreshing metadata is applied to a full flash memory system and comprises the following steps:
reading the utilization rate of the current write cache;
determining the number of the tree-brushing directories and the data quantity threshold of the tree-brushing according to the utilization rate;
determining a target tree directory to be brushed of the lower tree directory quantity from all the tree directories, and determining a target tree to be brushed from the target tree directory by using the data volume threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories;
the metadata cached in the target tree is down-flushed to a solid state disk;
the determining the number of the tree-brushing directories and the data quantity threshold of the tree-brushing according to the utilization rate comprises the following steps:
if the utilization rate is greater than a first preset threshold and less than or equal to a second preset threshold, determining that the number of the tree directories to be brushed is a first tree directory number, and the data volume threshold is a first data volume threshold;
if the utilization rate is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding lower brushing proportion by using the utilization rate, and then determining the lower brushing tree directory number and the data volume threshold by using the lower brushing proportion;
if the utilization rate is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second number of tree directories, and the data volume threshold is a second data volume threshold;
wherein the method further comprises:
if the utilization rate is smaller than a first preset threshold value, forbidding to flush the metadata in the write cache to the solid state disk, or regularly flushing the metadata in the tree cache in the tree directory meeting preset conditions to the solid state disk;
the determining the number of target tree directories to be refreshed, which are the number of the tree directories to be refreshed, from all the tree directories comprises the following steps:
and determining the target tree directories to be brushed as the tree directories to be brushed from all the tree directories by using the small top heap.
2. The metadata flash method according to claim 1, further comprising:
and generating a tree directory in the write cache according to the type and the data relation of the metadata and generating a corresponding tree in the tree directory so as to cache the metadata into the corresponding tree.
3. A metadata disk refreshing device is applied to a full flash memory system and comprises:
the cache utilization rate reading module is used for reading the utilization rate of the current write cache;
the lower brushing number determining module is used for determining the number of the lower brushing tree directories and the data quantity threshold of the lower brushing trees according to the utilization rate;
the target-brushing determining module is used for determining target tree directories to be brushed in quantity from all the tree directories and determining target trees to be brushed from the target tree directories by using the data quantity threshold; the data volume of the target tree directory is larger than that of other tree directories in all the tree directories;
the target data downloading and brushing module is used for downloading and brushing the metadata cached in the target tree to a solid state disk;
the lower-brushing-quantity determining module is specifically configured to determine that the lower-brushing-tree directory quantity is a first tree directory quantity and the data volume threshold is a first data volume threshold if the utilization rate is greater than a first preset threshold and is less than or equal to a second preset threshold; if the utilization rate is greater than a second preset threshold and less than or equal to a third preset threshold, determining a corresponding under-brushing proportion by using the utilization rate, and then determining the under-brushing tree directory quantity and the data quantity threshold by using the under-brushing proportion; if the utilization rate is greater than a third preset threshold, determining that the number of the lower-brushing tree directories is a second number of tree directories, and the data volume threshold is a second data volume threshold; if the utilization rate is smaller than a first preset threshold, forbidding to flush the metadata in the write cache to the solid state disk, or determining a tree directory meeting preset conditions, and correspondingly, the target data flushing module is also used for regularly flushing the metadata in the tree cache in the tree directory meeting the preset conditions to the solid state disk;
and the lower-brushing target determining module is specifically used for determining the target tree directories to be brushed, the number of which is the number of the lower-brushing tree directories, from all the tree directories by using the small top heap.
4. The metadata disk brushing device is characterized by comprising a processor and a memory; wherein the content of the first and second substances,
the memory is used for storing a computer program;
the processor is configured to execute the computer program to implement the metadata flash method according to any one of claims 1 to 2.
5. A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the metadata flashing method according to any of claims 1 to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911386429.8A CN111142803B (en) | 2019-12-29 | 2019-12-29 | Metadata disk refreshing method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911386429.8A CN111142803B (en) | 2019-12-29 | 2019-12-29 | Metadata disk refreshing method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111142803A CN111142803A (en) | 2020-05-12 |
| CN111142803B true CN111142803B (en) | 2022-07-08 |
Family
ID=70521375
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911386429.8A Active CN111142803B (en) | 2019-12-29 | 2019-12-29 | Metadata disk refreshing method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111142803B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112181315B (en) * | 2020-10-30 | 2022-08-30 | 新华三大数据技术有限公司 | Data disk refreshing method and device |
| CN112905121B (en) * | 2021-02-20 | 2023-01-24 | 山东英信计算机技术有限公司 | Data refreshing method and system |
| CN113590547B (en) * | 2021-06-30 | 2024-02-23 | 济南浪潮数据技术有限公司 | Cache management method and system for ICFS |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103714016A (en) * | 2014-01-14 | 2014-04-09 | 贝壳网际（北京）安全技术有限公司 | Cache cleaning method, device and client side |
| CN103793181A (en) * | 2012-11-05 | 2014-05-14 | 腾讯科技（深圳）有限公司 | Data synchronizing method and data synchronizing system of multilevel association storage architecture |
| CN106294206A (en) * | 2015-05-21 | 2017-01-04 | 广州华多网络科技有限公司 | A kind of caching data processing method and device |
| CN108647151A (en) * | 2018-04-26 | 2018-10-12 | 郑州云海信息技术有限公司 | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely |
| CN108920616A (en) * | 2018-06-28 | 2018-11-30 | 郑州云海信息技术有限公司 | A kind of metadata access performance optimization method, system, device and storage medium |
| CN109033365A (en) * | 2018-07-26 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant device |
| CN109614344A (en) * | 2018-12-12 | 2019-04-12 | 浪潮(北京)电子信息产业有限公司 | A kind of spatial cache recovery method, device, equipment and storage system |
| CN110502457A (en) * | 2019-08-23 | 2019-11-26 | 北京浪潮数据技术有限公司 | A kind of metadata storing method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105607960B (en) * | 2015-10-26 | 2018-12-07 | 成都华为技术有限公司 | File system directory tree restorative procedure and device |
-
2019
- 2019-12-29 CN CN201911386429.8A patent/CN111142803B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103793181A (en) * | 2012-11-05 | 2014-05-14 | 腾讯科技（深圳）有限公司 | Data synchronizing method and data synchronizing system of multilevel association storage architecture |
| CN103714016A (en) * | 2014-01-14 | 2014-04-09 | 贝壳网际（北京）安全技术有限公司 | Cache cleaning method, device and client side |
| CN106294206A (en) * | 2015-05-21 | 2017-01-04 | 广州华多网络科技有限公司 | A kind of caching data processing method and device |
| CN108647151A (en) * | 2018-04-26 | 2018-10-12 | 郑州云海信息技术有限公司 | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely |
| CN108920616A (en) * | 2018-06-28 | 2018-11-30 | 郑州云海信息技术有限公司 | A kind of metadata access performance optimization method, system, device and storage medium |
| CN109033365A (en) * | 2018-07-26 | 2018-12-18 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant device |
| CN109614344A (en) * | 2018-12-12 | 2019-04-12 | 浪潮(北京)电子信息产业有限公司 | A kind of spatial cache recovery method, device, equipment and storage system |
| CN110502457A (en) * | 2019-08-23 | 2019-11-26 | 北京浪潮数据技术有限公司 | A kind of metadata storing method and device |
Non-Patent Citations (2)
| Title |
|---|
| RAF: A Random Access First Cache Management to Improve SSD-Based Disk Cache;Yang Liu;《IEEE》;20100916;全文 * |
| 嵌入式文件系统缓存管理机制研究;王冬等;《航空计算技术》;20190525(第03期);107-109 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111142803A (en) | 2020-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111142803B (en) | Metadata disk refreshing method, device, equipment and medium | |
| KR100809320B1 (en) | Apparatus and method for managing mapping information of non-volatile memory supporting separated cell type | |
| US8862810B2 (en) | Solid state device write operation management system | |
| EP3229142B1 (en) | Read cache management method and device based on solid state drive | |
| CN112631521B (en) | Method, system, equipment and medium for controlling water level of cache pool | |
| US11360672B2 (en) | Performing hybrid wear leveling operations based on a sub-total write counter | |
| JP5683023B2 (en) | Processing of non-volatile temporary data | |
| US20140181432A1 (en) | Priority-based garbage collection for data storage systems | |
| CN108647151A (en) | It is a kind of to dodge system metadata rule method, apparatus, equipment and storage medium entirely | |
| KR20120008079A (en) | Memory mapping techniques | |
| US10168940B2 (en) | Data storage using SLC and TLC memory banks and data maintenance method thereof | |
| JP2012033002A (en) | Memory management device and memory management method | |
| CN112506438B (en) | Mapping table management method and solid state disk | |
| CN111124304B (en) | Data migration method and device, electronic equipment and storage medium | |
| CN111158601A (en) | IO data flushing method, system and related device in cache | |
| CN109213450B (en) | Associated metadata deleting method, device and equipment based on flash memory array | |
| US20140195571A1 (en) | Fast new file creation cache | |
| CN110502457B (en) | Metadata storage method and device | |
| US11698742B2 (en) | Garbage collection in a memory component using an adjusted parameter | |
| CN113806087B (en) | Method and device for adjusting service speed based on brushing speed | |
| KR20150139383A (en) | Semiconductor device | |
| CN114675785A (en) | Distributed storage method, device, system and medium | |
| Gwak et al. | Reducing garbage collection overhead of log-structured file systems with GC journaling | |
| CN107544866B (en) | Log updating method and device | |
| Lee et al. | DTFS: exploiting the similarity of data versions to design a write-efficient file system in phase-change memory |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |