CN114356797A - Solid-state storage method, system and equipment based on file system optimization - Google Patents

Abstract

A solid-state storage method, a system and equipment based on file system optimization belong to the field of storage and are characterized in that: analyzing data, and acquiring metadata and a data LBA range from each file system; setting a relation table, and corresponding the analyzed LBA range with the data type; setting a fixed buffer area, and periodically writing the latest metadata back to the NAND; and when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data types. By analyzing the file systems, the LBA ranges of metadata and data in each file system are obtained and are correspondingly stored in different BLOCK domains, and cold and hot data are isolated, so that the triggering frequency and data moving amount of a background GC are effectively reduced; meanwhile, the latest metadata is written back to the NAND regularly by adding a fixed cache area aiming at the metadata of the file system, so that the duplication frequency of the hot data is reduced, the storage performance of the solid-state storage device is effectively improved, and the service life of the solid-state storage device is effectively prolonged.

Description

Solid-state storage method, system and equipment based on file system optimization

Technical Field

The invention belongs to the field of storage, and particularly relates to a solid-state storage method, a system and equipment based on file system optimization.

Background

Today, with the continuing development of the storage field, more and more applications use solid-state storage devices to install systems or store data. Based on the characteristics of the file system, a user needs to frequently update some management data when operating data, and due to the characteristic of NAND FLASH granules, such data can be frequently rewritten to cause a large number of invalid pages, which frequently triggers GC/WL operations in the background, and further affects the writing performance and the service life.

In the prior art, part of schemes require a file system to form characteristic information according to metadata, and a solid-state storage device performs differentiation processing on different types of data according to the characteristic information; and other schemes require that a specific file system is used and matched with the solid-state storage device before the file system can be normally used. The above schemes cannot be compatible with a common file system, and cannot meet the use requirements of users such as consumption level/workstation.

Disclosure of Invention

The present invention aims to solve the above problems and provide a solid-state storage method, system and device based on file system optimization.

In a first aspect, the present invention provides a solid-state storage method based on file system optimization, including: analyzing data, and acquiring metadata and a data LBA range from each file system;

setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type;

setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND; the duplication times of the metadata can be effectively reduced;

when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type; different types of data can be isolated at the BLOCK level;

when GC is involved, GC is respectively carried out according to the data types; the amount of unnecessary cold data transfer caused by mixing of cold and hot data can be effectively reduced.

Further, in the file system optimization-based solid-state storage method of the present invention, the analyzing data specifically includes:

firstly, analyzing an LBA0 sector through a primary analysis module, confirming a partition type through a partition mark, acquiring position information of a partition table through the partition type, finally analyzing the partition table to acquire complete partition table information of the whole solid-state storage device and all partition information contained in the complete partition table information, and acquiring an LBA range of the whole partition management data;

and analyzing the initial LBA of each partition according to different file system types through a secondary analysis module to obtain the metadata and the data LBA range of each partition file system.

Further, in the file system optimization-based solid-state storage method of the present invention, the analyzing data further includes: if the operating system partition exists in the analysis, analyzing the operating system type through the third-level analysis module to obtain the LBA range of the operating system management data and the user data and the LBA range of part of the system large file, wherein the LBA range of the system large file comprises but is not limited to a hibernation file and a virtual memory file.

Further, in the file system optimization-based solid-state storage method, the relation table m is used for corresponding the LBA range obtained by analysis to a plurality of data types; the method specifically comprises the following steps: the LBA range of the partition management data corresponds to the type of the management data separately; the file system metadata and the operating system management data correspond to the hot data type; the file system data and the operating system user data correspond to the cold data type; after receiving the HOST write command, the data types are distinguished according to the command LBA, and then the FTL carries out differentiated processing.

Further, in the solid-state storage method based on file system optimization, the BLOCK is divided into a plurality of domains, and the domains correspond to data types one by one; when a HOST write command is received, the data is written into an idle BLOCK corresponding to a BLOCK domain according to the data type, and different types of data can be isolated on a BLOCK layer.

Further, in the solid-state storage method based on file system optimization, the fixed cache region is used for caching the metadata of the file system; periodically updating the NAND in the normal use process; forcibly updating the NAND before shutdown/power failure; the duplication times of the metadata can be effectively reduced by setting the fixed cache region, and the storage performance is improved.

Furthermore, according to the solid-state storage method based on the file system optimization, after the analysis data and the relation table m are updated, the partition information x is divided before shutdown/power-off₁Storing the corresponding relation table into FLASH specific BLOCK, and checking the actual partition information x when powering on next time₂And the aforementioned x₁And directly reading the BLOCK data for reconstruction after the consistency is verified.

Further, the solid-state storage method based on file system optimization of the present invention, the partition types include but are not limited to MBR, GPT; the partition information includes but is not limited to each partition file system type, each partition starting position, and partition size; the file system types include, but are not limited to, FAT32, exFAT, NTFS, Ext2, Ext3, Ext4, reiserFS, VFAT, brtfs.

In a second aspect, the present invention provides a solid-state storage system based on file system optimization, which includes an analysis module and a processing module;

the analysis module is used for analyzing the data and acquiring metadata and a data LBA range from each file system;

the processing module is used for setting a relation table m, corresponding the LBA range obtained by analysis to the data type and maintaining the corresponding relation between the LBA range and the data type; the fixed cache region is used for caching the metadata of the file system and regularly writing the latest metadata back to the NAND;

when the processing module carries out write command operation, the processing module is used for correspondingly storing data into different BLOCK fields according to data types; when GC is involved, the control system respectively performs GC processing according to the data types.

In a third aspect, the present invention provides a solid-state storage device based on file system optimization, where the storage device stores an analysis and processing program, and when HOST performs read-write operation on the storage device, the storage device stores data in a storage medium by performing the analysis and processing program; the method specifically comprises the following steps:

analyzing data, and acquiring metadata and a data LBA range from each file system;

setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type;

setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND;

when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type;

when GC is involved, GC is performed separately according to the data type.

According to the solid-state storage method, system and device based on the file system optimization, the LBA ranges of metadata and data in each file system are obtained by analyzing the file systems, and the metadata and the LBA ranges are correspondingly stored in different BLOCK domains, so that cold and hot data are isolated, and the triggering frequency and data moving amount of a background GC are effectively reduced; meanwhile, the latest metadata is written back to the NAND regularly by adding a fixed cache area aiming at the metadata of the file system, so that the duplication frequency of the hot data is reduced, the storage performance of the solid-state storage device is effectively improved, and the service life of the solid-state storage device is effectively prolonged.

Drawings

FIG. 1 is a schematic diagram illustrating a data parsing process according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a partition structure according to an embodiment of the present invention;

fig. 3 is a schematic diagram illustrating a write command processing flow according to an embodiment of the present invention.

Detailed Description

The following describes the solid-state storage method, system and device based on file system optimization in detail through the accompanying drawings and embodiments.

The Solid State storage device used in the embodiments of the present disclosure is a Solid State Drive (SSD), which is a hard disk made of a Solid State electronic storage chip array, and generally includes a controller and a memory (e.g., NAND FLASH chip, DRAM chip, etc.).

A file system refers to a method and data structure used by an operating system to clarify and manage files on a storage device (mechanical hard disk, solid state disk, etc.) or partition. The software mechanism in the operating system that is responsible for managing and storing file information is called a file management system, referred to as a file system for short. From a system perspective, a file system is a system that organizes and allocates space of a file storage device, is responsible for file storage, and manages stored files. In particular, it is responsible for creating, storing, reading, modifying, dumping, deleting files, etc. for the user.

The file system mainly includes two data segments, namely metadata (metadata) and data, and the metadata is data for describing and managing data information and is mainly responsible for indicating data storage positions, history information, file state records and the like. Any manipulation of the data may result in an update of the metadata, so the update frequency of the metadata is much higher than that of the data.

Garbage recovery (GC): and collectively moving the effective data in a plurality of blocks with less effective data amount to a new idle block, and releasing the plurality of moved invalid blocks into the idle block by erasing.

The solid state disk adopted by the embodiment of the disclosure comprises a controller, a NAND and a DRAM;

the partition type used in the solid state disk in the embodiment of the present disclosure is GPT.

Example one

The solid-state storage method based on the file system optimization disclosed by the embodiment of the disclosure comprises the following steps: analyzing data, and acquiring metadata and a data LBA range from each file system; setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type; setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND; the duplication times of the metadata can be effectively reduced; when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type; different types of data can be isolated at the BLOCK level; when GC is involved, GC is respectively carried out according to the data types; the amount of unnecessary cold data transfer caused by mixing of cold and hot data can be effectively reduced.

The solid-state storage method based on the file system optimization in the embodiment of the disclosure comprises the following specific steps:

a first part: parsing module workflow, as shown in fig. 1;

step S101, after the host is powered on and subjected to basic initialization, reading partition management data (including boot sectors and a hard disk partition table DPT) of LBAs 0-LBA 33, judging whether a partition table exists, and if the partition table does not exist, jumping to step S102, jumping to step S103; the structure of the partition is shown in FIG. 2;

s102, scanning a specific position of the NAND to confirm whether backup analysis data exist or not, if yes, jumping to S104, and if not, jumping to S105;

step S103, waiting for host to update the partition management data, and jumping to step S105 after the area is updated;

step S104, checking whether the check code of the partition header data is consistent with the check code in the backup analysis data, if so, skipping to step S107, and if not, skipping to step S105;

s105, starting to analyze the partition header data by primary analysis to form partition type information and information of each partition file system, and jumping to S106 if at least one partition file system exists, or jumping to S103 if at least one partition file system does not exist;

step S106, the second-level analysis module begins to analyze a segment of LBA of the initial position of each file system, the LBA fields of the metadata and the data of the file system are divided into different data types, and the LBA fields of the metadata part of the file system are separately divided into a segment of fixed cache area to correspond to the LBA fields. Meanwhile, whether an operating system partition exists is judged, if yes, the step S108 is skipped, and if not, the step S109 is skipped;

step S107, rebuilding a corresponding table of the LBA domain and the data type and a fixed cache region corresponding to the file system metadata through backup analysis data; skipping to step S109;

step S108, the third-level analysis module further analyzes information of an operating system (including but not limited to Windows and Linux systems), divides management data of the operating system, LBA (logical block addressing) domains of user data and LBA (logical block addressing) segments of system large files into different data types, and jumps to step S109;

step S109, maintaining a corresponding relation table of the LBA domain and the data type, wherein the file system metadata/operating system management data correspond to relatively hot data, and the data/user data correspond to relatively cold data; skipping to step S110;

step S110, dividing a plurality of BLOCK domains for one-to-one correspondence according to the data types divided by the analysis module, judging the data types through the command LBA after a host issues a write command, and jumping to step S111 when the write command belongs to the file system metadata type, wherein other data types are processed by the FTL and written into the idle BLOCK of the corresponding BLOCK domain;

step S111, updating data in a fixed cache area, periodically handing the data to FTL for processing and updating the data to an idle BLOCK corresponding to a BLOCK domain; if normal/abnormal power-down occurs, the data in the fixed buffer area is preferentially updated to the NAND.

A second part: write command processing flow, as shown in FIG. 3;

step S201, after the SSD receives a host write command, the SSD judges the specific type of the write data according to the LBA domain and data type corresponding relation table maintained by the analysis module, if the metadata is written, the step S202 is skipped, and if not, the step S203 is skipped;

step S202, updating metadata in a fixed cache region corresponding to the metadata and ending the command, and writing the cache region data needing to be updated into a BLOCK domain corresponding to the metadata type periodically or when the SSD is idle;

step S203, judging whether the written data is partition table information, if not, writing the data into a BLOCK domain corresponding to the data according to a normal flow and ending the command, and if so, jumping to the step S204;

step S204, the analysis management module re-analyzes the latest partition table and file system information, updates the corresponding relation table of the LBA domain and the data type, and writes the latest analysis data and the reprint relation table into the specific position of the NAND.

In the solid-state storage method based on file system optimization according to this embodiment, the file system is analyzed by the analysis module inside the solid-state hard disk to obtain different LBA ranges of metadata and data in each file system, and the metadata and the data are correspondingly stored in different BLOCK domains to isolate cold and hot data, so that the frequency of background GC triggering and the workload of data migration are reduced; meanwhile, a fixed buffer area aiming at the LBA range of the file system metadata is added to periodically write the latest metadata back to the NAND, so that the frequency of the rewriting of the hot data is reduced. Therefore, the performance and the service life of the solid state disk are improved.

Example two

The embodiment of the disclosure discloses a solid-state storage system based on file system optimization, which comprises an analysis module and a processing module; the analysis module is used for analyzing the data and acquiring metadata and a data LBA range from each file system; the processing module is used for setting a relation table m, corresponding the LBA range obtained by analysis to the data type and maintaining the corresponding relation between the LBA range and the data type; the fixed cache region is used for caching the metadata of the file system and regularly writing the latest metadata back to the NAND; when the processing module carries out write command operation, the processing module is used for correspondingly storing data into different BLOCK fields according to data types; when GC is involved, the control system respectively performs GC processing according to the data types. When the solid state disk storage system performs data storage, the method includes the following steps:

step one, solid state disk is electrified and started; when a partition/file system (including changing the size, the type and the initial position) is newly built or modified in the using process, the primary analysis module analyzes the LBA0 sector, the partition type is confirmed through the partition mark in the LBA0, the position information of the partition table is obtained through the partition type, finally the partition table is analyzed to obtain the complete partition table information of the whole solid state disk and all the partition information contained in the complete partition table information, and the LBA range of the whole partition management data is obtained;

step two, the secondary analysis module analyzes the initial LBA of each partition according to different file system types to obtain the metadata and the data LBA range of each partition file system;

step three, if an operating system partition is found in the analysis process, the third-level analysis module further analyzes the operating system type to obtain the LBA ranges of the operating system management data and the user data, and analyzes the LBA ranges of some system large files;

maintaining a corresponding relation table m of the LBA ranges and the data types, and corresponding the plurality of LBA ranges acquired by the analysis module to the plurality of data types, wherein the LBA ranges of the partition management data can be individually corresponding to the management data types, the file system metadata and the operating system management data can be corresponding to the hot data types, and the file system data and the operating system user data can be corresponding to the cold data types; after receiving the HOST write command, distinguishing data types according to the command LBA, and then handing over to the FTL for differential processing;

step five, the BLOCK divides a plurality of domains, the domains correspond to the data types one by one, when the solid state disk receives the host write command, the host write command is written into the idle BLOCK corresponding to the BLOCK domain according to the data types, and different types of data can be isolated on the BLOCK layer

Step six, a section of space is specially marked out in the cache for caching the metadata of the file system, the metadata is regularly updated into the NAND in the normal use process, and the metadata is forcibly updated into the NAND before shutdown/power failure; this operation aims to reduce the number of times of copying of metadata;

seventhly, when the GC is involved, the solid state disk respectively carries out GC on cold/hot data according to the condition of the internal data, so that the cold/hot data are prevented from being mixed and unnecessary cold data moving is avoided; when the data analysis process and the table m are updated, the partition information x is transmitted before the power-off/power-off₁Storing the corresponding relation table into FLASH specific BLOCK, and verifying actual partition information x in next power-on₂And x₁And after the check is consistent, the BLOCK data can be directly read for reconstruction.

The partition types include, but are not limited to, MBR, GPT; the partition information includes but is not limited to each partition file system type, each partition starting position, and partition size; the file system types include, but are not limited to, FAT32, exFAT, NTFS, Ext2, Ext3, Ext4, reiserFS, VFAT, brtfs; the LBA range of the system large file includes but is not limited to a hibernation file and a virtual memory file; the method can be selected and applied according to actual conditions in specific applications.

EXAMPLE III

The embodiment of the disclosure discloses a solid-state storage device based on file system optimization, wherein an analysis and processing program is stored in the storage device, and when HOST performs read-write operation on the storage device, the storage device classifies and stores data in a storage medium by executing the analysis and processing program; the method specifically comprises the following steps: analyzing data, and acquiring metadata and a data LBA range from each file system; setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type; setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND; when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type; when GC is involved, GC is performed separately according to the data type.

The specific storage steps are the same as those described in the first embodiment or the second embodiment, and are not described herein for brevity.

The above embodiments are only a part of embodiments of the technical solution of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Claims (10)

1. A solid-state storage method based on file system optimization is characterized in that:

analyzing data, and acquiring metadata and a data LBA range from each file system;

setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type;

setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND;

when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type;

when GC is involved, GC is performed separately according to the data type.

2. The file system optimization-based solid-state storage method according to claim 1, wherein the parsing the data specifically comprises:

firstly, analyzing an LBA0 sector through a primary analysis module, confirming a partition type through a partition mark, acquiring position information of a partition table through the partition type, finally analyzing the partition table to acquire complete partition table information of the whole solid-state storage device and all partition information contained in the complete partition table information, and acquiring an LBA range of the whole partition management data;

and analyzing the initial LBA of each partition according to different file system types through a secondary analysis module to obtain the metadata and the data LBA range of each partition file system.

3. The file system optimization-based solid-state storage method according to claim 2, wherein the parsing the data further comprises: if the operating system partition exists in the analysis, analyzing the operating system type through the three-level analysis module to obtain the LBA range of the operating system management data and the user data and the LBA range of part of the system large files.

4. The file system optimization-based solid-state storage method according to claim 3, wherein the relationship table m is used for corresponding the LBA range obtained by analysis to a plurality of data types; the method specifically comprises the following steps: the LBA range of the partition management data corresponds to the type of the management data separately; the file system metadata and the operating system management data correspond to the hot data type; the file system data and the operating system user data correspond to the cold data type; after receiving the HOST write command, the data types are distinguished according to the command LBA, and then the FTL carries out differentiated processing.

5. The file system optimization-based solid-state storage method according to claim 1, wherein the BLOCK is divided into a plurality of domains, corresponding to data types one to one; and writing the data into the idle BLOCK of the corresponding BLOCK domain according to the data type when the HOST write command is received.

6. The file system optimization-based solid-state storage method according to claim 1, wherein the fixed cache region is used for caching file system metadata; periodically updating the NAND in the normal use process; forced update into NAND before shutdown/power down.

7. The file system optimization-based solid-state storage method according to claim 2, wherein: after the analysis data and the relation table m are updated, the partition information x is sent to the power-off/power-off unit before the power-off/power-off unit is powered off₁Storing the corresponding relation table into FLASH specific BLOCK, and checking the actual partition information x when powering on next time₂And the aforementioned x₁And directly reading the BLOCK data for reconstruction after the consistency is verified.

8. The file system optimization-based solid-state storage method according to claim 2, wherein: the partition types include, but are not limited to, MBR, GPT; the partition information includes but is not limited to each partition file system type, each partition starting position, and partition size; the file system types include, but are not limited to, FAT32, exFAT, NTFS, Ext2, Ext3, Ext4, reiserFS, VFAT, brtfs.

9. A solid-state storage system based on file system optimization, characterized by: the system comprises an analysis module and a processing module;

the analysis module is used for analyzing the data and acquiring metadata and a data LBA range from each file system;

the processing module is used for setting a relation table m, corresponding the LBA range obtained by analysis to the data type and maintaining the corresponding relation between the LBA range and the data type; the fixed cache region is used for caching the metadata of the file system and regularly writing the latest metadata back to the NAND;

when the processing module carries out write command operation, the processing module is used for correspondingly storing data into different BLOCK fields according to data types; when GC is involved, the control system respectively performs GC processing according to the data types.

10. A solid-state storage device based on file system optimization, characterized by: when HOST carries out read-write operation on the storage device, the storage device classifies and stores data in a storage medium by executing the analysis and processing program; the method specifically comprises the following steps:

analyzing data, and acquiring metadata and a data LBA range from each file system;

setting a relation table m, corresponding the LBA range obtained by analysis to the data type, and maintaining the corresponding relation between the LBA range and the data type;

setting a fixed cache region for caching the metadata of the file system and periodically writing the latest metadata back to the NAND;

when the write command is operated, correspondingly storing the data into different BLOCK fields according to the data type;

when GC is involved, GC is performed separately according to the data type.

Images