CN113568868B

CN113568868B - File system management method, system, electronic equipment and medium

Info

Publication number: CN113568868B
Application number: CN202110859193.6A
Authority: CN
Inventors: 袁东平
Original assignee: Chongqing Unisinsight Technology Co Ltd
Current assignee: Chongqing Unisinsight Technology Co Ltd
Priority date: 2021-07-28
Filing date: 2021-07-28
Publication date: 2024-02-06
Anticipated expiration: 2041-07-28
Also published as: CN113568868A

Abstract

The invention provides a file system management method, a system, electronic equipment and a medium, wherein the method is used for obtaining the file creation request of a file to be created, configuring the space size required by pre-allocation for the file to be created, obtaining the idle information of the file system, obtaining the tag information comprising idle indication positions, and distributing the space of the file to be created for the file to be created according to the space size required by pre-allocation from the idle starting positions and the ordering of the idle intervals, so that file fragments can be reduced or avoided during writing, the continuous storage capacity of the same file is improved, and the read-write performance of the file system is improved.

Description

File system management method, system, electronic equipment and medium

Technical Field

The present invention relates to the field of image processing technology in the field of steel, and in particular, to a file system management method, system, electronic device, and medium.

Background

The level ldb (multi-layer key value database) is a very efficient open source key-value database realized by google, the data organization mode based on LSM (Log Structured Merge Trees, LSM tree) has very high writing efficiency when writing sequentially, merging operation becomes writing bottleneck when writing randomly in a large amount, and the number of level0 files is expanded seriously, so that the usability is affected.

The file system functions to implement the access function of binary data. The EXT4 (Fourth extended filesystem, fourth generation extended file system) and XFS of the common file system are not good in performance under the resource-limited scene of embedded type and the like, and file fragments are easy to appear when multimedia files are written in parallel, so that the continuous storage capacity of the same file is reduced, and then the read-write performance of the file system is affected.

Disclosure of Invention

In view of the above-mentioned drawbacks of the prior art, an object of the present invention is to provide a file system management method, system, electronic device and medium, which are used for solving the technical problems that file fragments are easy to occur when multimedia files are written in parallel, so that the continuous storage capacity of the same file is reduced, and then the read-write performance of the file system is affected.

In view of the foregoing, the present invention provides a file system management method, including:

acquiring a file creation request of a file to be created, and configuring a pre-allocation required space size for the file to be created;

acquiring idle information of a file system, wherein the idle information comprises idle position identifiers of a plurality of idle intervals;

acquiring tag information, wherein the tag information comprises idle starting positions of all the idle intervals remained after the file system performs space allocation last time, and an idle position mark of one idle interval comprises the idle starting position;

And from the idle starting position, distributing a file space to be created for the file to be created according to the size of the space required by the pre-allocation according to the ordering of the idle intervals.

Optionally, the idle position identifier includes an idle interval initial position and an idle interval end position, and the allocating the file space to be created for the file to be created according to the size of the space required for pre-allocation from the idle initial position includes any one of the following:

and acquiring the size of the idle interval and the number of the idle position identifiers in the idle information, and distributing the file space to be created with the size of the space to be created for the file to be created according to one or more of the size of the idle interval, the size of the space to be pre-distributed and the idle starting position, wherein the size of the idle interval is determined according to the initial position of the idle interval and the end position of the idle interval.

Optionally, the obtaining the size of the idle interval and the number of the idle position identifiers in the idle information, and allocating the file space to be created with the size of the space to be created in advance according to one or more of the size of the idle interval, the size of the space to be pre-allocated and the idle starting position includes any one of the following:

If the idle information only comprises an idle position mark of the idle interval, the idle starting position is the same as the idle interval initial position, and if the idle interval size is larger than or equal to the pre-allocation required space size, the file space to be created with the pre-allocation required space size is allocated for the file to be created from the idle interval initial position;

if the idle information comprises idle position identifiers of a plurality of idle intervals, determining a target interval from each idle interval according to the idle starting position, determining the size of the target interval according to the size of each idle interval of the target interval, and if the size of the target interval is larger than or equal to the size of the space required by pre-allocation, distributing a file space to be created with the size of the space required by pre-allocation for the file to be created in the target interval from the idle starting position;

if the idle information comprises a plurality of idle position identifiers of the idle sections, determining a target section from each idle section according to the idle starting position, determining the size of the target section according to the size of each idle section of the target section, if the size of the target section is smaller than the size of the space required by pre-allocation, acquiring the size of a difference space between the size of the target section and the size of the space required by pre-allocation, determining a supplementary section according to the difference space, wherein the supplementary section comprises a plurality of idle sections which are positioned behind the target section and select the size of the difference space according to a sorting order, and taking the target section and the supplementary section as the file space to be created.

Optionally, the determining manner of the supplementary interval includes at least one of the following:

acquiring the first idle interval size of a first idle interval after the target interval, and if the difference space size is smaller than or equal to the first idle interval size, distributing a space with the difference space size in the first idle interval as a supplementary interval;

sequentially acquiring the first combination size of the idle interval first combination after the target interval, and if the difference space size is smaller than or equal to the first combination size of the idle interval, distributing the space with the difference space size in the first combination of the idle interval as a supplementary interval;

and sequentially acquiring the first combination size of the idle interval of the first combination of the idle interval after the target interval, if the difference space size is larger than the first combination size of the idle interval, sequentially acquiring the second combination size of the idle interval of the second combination of the idle interval before the target interval, and if the first combination size of the idle interval and the second combination size of the idle interval are larger than the difference space size, sequentially distributing the space with the difference space size in the first combination of the idle interval and the second combination of the idle interval as a supplementary interval.

Optionally, the method further comprises:

acquiring deletion information of a deletion file, wherein the deletion information comprises an occupied position identifier of the deletion file;

and merging the occupied interval corresponding to the deleted file occupied position identification with the idle interval, and updating the idle information.

Optionally, the deletion file occupying position identifier includes an occupying initial position and an occupying end position, the idle position identifier includes an idle interval initial position and an idle interval end position, and updating the idle information includes:

if the initial position and the end position of each idle interval in the idle information are different from the initial position and the end position, adding the initial position and the end position to the idle information;

if the idle interval initial position of one idle interval exists in the idle information and is the same as the occupied end position, replacing the idle interval initial position with the occupied end position;

if the idle interval ending position of one idle interval exists in the idle information and is the same as the occupied initial position, replacing the idle interval ending position with the occupied initial position.

Optionally, the method further comprises:

opening the file to be created, and acquiring a file space to be created of the file to be created, wherein the file space to be created comprises a plurality of idle intervals;

and reading and writing data of the file to be created, and sequentially loading each idle interval into a cache according to the file offset of the file to be created.

Optionally, the method further comprises:

and recording the modification time and the file size of the file to be created after data reading and writing.

Optionally, the caching module caches the full path and all node information of the file system, and the metadata storage module stores metadata in the file system, and the method further comprises at least one of the following steps:

when node information is inquired, the full paths are matched in the cache module step by step, if the matching fails, the node information is searched in the metadata storage module, and the node information is cached in the cache module;

when updating node information, updating the node information in the cache module, and then storing the node information into the metadata storage module;

And updating the metadata storage module through batch operation writing.

Optionally, the method further comprises: if the power failure restarting occurs, a pre-written log is obtained and analyzed, and data recovery is carried out on the file system according to the analyzed pre-written log.

The invention also provides a file system management system, which comprises a file system module and a metadata storage module for storing metadata of the file system, wherein the file system module comprises an access interface of the metadata storage module, and the metadata storage module comprises an idle unit and a system recording unit, wherein:

the idle unit is used for recording idle information of idle space which is not allocated to the file, and the idle information comprises idle position identifiers of a plurality of idle intervals;

the system recording unit is used for recording label information, and comprises idle starting positions of the idle intervals remained after the file system performs space allocation last time, wherein the idle position identification of one idle interval comprises the idle starting position;

if a file creation request of a file to be created is received through the access interface, using the idle starting position as a starting point, and distributing the idle space with the size of the space required by pre-distribution as the file space to be created according to the ordering of the idle intervals.

Optionally, the metadata storage module includes at least one of:

the node unit comprises a file unit and a catalog unit and is used for recording attribute information of files and catalogs, wherein the attribute information comprises at least one of node identity identification information, father identity identification information, creation time, modification time, used size, file space size to be created and name;

the index unit is used for searching files and scanning catalogues and comprises at least one of father identity identification information, names, identity identification information and types;

the active node unit is used for recording the modification time and the file size of the file to be created after data reading and writing;

and the allocation unit is used for recording the occupation information of the free space allocated to the file.

Optionally, the file system management system further includes a cache module, where the cache module is configured to cache path and node information of the file system;

when node information is inquired, the paths are matched in the cache module step by step, if the matching fails, the node information is searched in the metadata storage module, and the node information is cached in the cache module;

and updating the metadata storage module through batch operation writing.

Optionally, the file system management system further comprises at least one of:

acquiring a parent directory of a file to be created, if the file system comprises the parent directory, distributing the node identification information of the directory to be created for the directory to be created, and writing the node identification information of the directory to be created into the node unit, the index unit and the distribution unit respectively;

writing the occupation information of the file space to be created into the distribution unit;

updating the idle unit and/or the system recording unit according to the occupation information of the file space to be created;

acquiring a catalog to be deleted, and deleting the catalog to be deleted in the node unit and the index unit if the catalog to be deleted in the index unit is an empty catalog;

acquiring deletion information of a deletion file, wherein the deletion information comprises occupation position identifiers of the deletion file, merging an occupation interval corresponding to the occupation position identifiers of the deletion file with the idle interval, and updating the idle information in the idle unit and the occupation information in the distribution unit.

Optionally, the file system management system further includes an input/output module, where the input/output module is configured to manage an opened file handle of the file to be created, where the file handle is associated with the node information, obtain a file space to be created of the file to be created after the file to be created is opened, where the file space to be created includes a plurality of idle sections, read and write data of the file to be created, and sequentially load each idle section into the cache module according to a file offset of the file to be created.

Optionally, the metadata storage module includes a database, the free unit includes a free table, the system recording unit includes a system table, the node unit includes a node table, the index unit includes an index table, the active node unit includes an active node table, and the allocation unit includes an allocation table, where each table includes a header identifier and a footer identifier, and a primary key prefix between each table is different.

The invention also provides an electronic device, which comprises a processor, a memory and a communication bus;

the communication bus is used for connecting the processor and the memory;

The processor is configured to execute a computer program stored in the memory to implement the method according to any one of the embodiments described above.

The invention also provides a computer readable storage medium having stored thereon a computer program for causing the computer to perform the method according to any of the embodiments described above.

As described above, the file system management method, system, electronic device and medium provided by the invention have the following beneficial effects:

the file creation request of the file to be created is obtained, the size of the space required by pre-allocation is configured for the file to be created, the idle information of the file system is obtained, the tag information comprising the idle starting position is obtained, the space of the file to be created is allocated for the file to be created according to the size of the space required by pre-allocation from the idle starting position according to the sequence of each idle interval, so that file fragments can be reduced or avoided during writing, the continuous storage capacity of the same file is improved, and the read-write performance of the file system is improved.

Drawings

FIG. 1 is a flowchart illustrating a method for managing a file system according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a file system management system according to a second embodiment of the present invention;

FIG. 3-1 is a diagram illustrating an Index table Index according to one embodiment;

FIG. 3-2 is a schematic diagram of a Node table Node according to an embodiment;

3-3 are schematic diagrams of Active node table activities according to one embodiment;

FIGS. 3-4 are schematic diagrams of Free list provided by one embodiment;

FIGS. 3-5 are schematic diagrams of allocation table Alloc according to an embodiment;

FIGS. 3-6 are schematic diagrams of a system record table SYS according to an embodiment;

FIG. 4 is a schematic diagram of a cache module according to an embodiment;

FIGS. 5-1, 5-2, and 5-3 are exemplary diagrams of allocations of file space to be created provided by an embodiment;

FIGS. 6-1, 6-2 and 6-3 are exemplary diagrams of spatial merging after deleting files according to one embodiment;

FIG. 7 is a schematic diagram of another file system management system according to an embodiment;

FIG. 8 is a schematic diagram of an exemplary embedded file system scenario set forth in one embodiment;

fig. 9 is a schematic structural diagram of a terminal according to an embodiment.

Detailed Description

Other advantages and effects of the present invention will become apparent to those skilled in the art from the following disclosure, which describes the embodiments of the present invention with reference to specific examples. The invention may be practiced or carried out in other embodiments that depart from the specific details, and the details of the present description may be modified or varied from the spirit and scope of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other without conflict.

It should be noted that the illustrations provided in the following embodiments merely illustrate the basic concept of the present invention by way of illustration, and only the components related to the present invention are shown in the drawings and are not drawn according to the number, shape and size of the components in actual implementation, and the form, number and proportion of the components in actual implementation may be arbitrarily changed, and the layout of the components may be more complicated.

Example 1

Referring to fig. 1, a file system management method provided by an embodiment of the present invention includes:

s101: and acquiring a file creation request of the file to be created, and configuring the pre-allocated required space size for the file to be created.

The file creation request may include, but is not limited to, at least one of a file type, a file format, a use, a request source (information of a requester), and the like, and a corresponding pre-allocated required space size may be configured for a file to be created according to the file creation request, that is, a reserved file size of the file to be created is determined according to the file creation request. For example, when the file type in the file creation request is a video file, the pre-allocation required space size is configured to be 1G, and when the file type in the file creation request is an audio file, the pre-allocation required space size is configured to be 500M. The configuration rules for pre-allocation of the required amount of space may also be set by those skilled in the art as desired.

Optionally, before step S101, the method further includes:

and acquiring parent directory information of the file to be created, and if a parent directory corresponding to the parent directory information exists in the file system, distributing corresponding node identification information (node ID) for the file to be created.

Optionally, the file system in this embodiment includes a leveldb-based embedded virtual file system.

S102: and acquiring idle information of the file system.

Wherein the idle information includes, but is not limited to, idle location identifications of a number of idle intervals.

Alternatively, the idle information may be position information of a blank block. For example, the data blocks capable of storing the virtual file are preconfigured with corresponding data block serial numbers, the data block serial numbers of the data blocks in idle intervals of a plurality of data block strokes are used as idle position identifiers, for example, the data blocks 10-20 form an idle interval, and the idle position identifiers of the idle intervals are (10, 20).

The status of the available data blocks in the current file system can be known from this free information.

S103: and acquiring tag information.

The label information comprises idle starting positions of all idle intervals remained after the file system performs space allocation last time, and the idle position identification of one idle interval comprises the idle starting position.

Alternatively, each free space remaining after the file system performs space allocation last time may be understood as that one or more free spaces (i.e. a plurality of free spaces in the free information in step S102) remain after the file system has last received the file creation request to complete space allocation for the file to be created last time (after the space allocation last time). The idle starting position is the idle interval initial position of one idle interval in the plurality of idle intervals. In other words, the idle start position is the idle section initial position of the one of the idle sections that is currently ranked most forward.

Alternatively, the idle start position may be an idle interval initial position of a first idle interval after an occupied space end position occupied by a previous space allocation. It should be noted that, the determination manners of the first idle interval after the occupied space end position include, but are not limited to, the following manners:

if the occupied space end position is not the end position of the initial allocable space, the first idle interval is the idle interval sequenced after the occupied space end position;

if the occupied space end position is the end position of the initial allocatable space, the first idle interval is the idle interval with the highest sequence from the initial position of the initial allocatable space.

In other words, the ordering of the unoccupied intervals begins at the end position of the occupied space, and is recirculated back to the start position of the initial allocatable space through the end position of the initial allocatable space to the end position of the occupied space.

Therefore, the method can sequentially occupy the free intervals, realizes sequential storage of metadata based on a database, has higher reading and writing efficiency, has small IO pressure on the hard disk by metadata operation, and can be separated from file data reading and writing to different devices (such as different disks and the like).

For example, the number of data blocks is 1-100, the last data block occupied by the last space allocation is 30, the current blank data block is 0-15, 31-50,75-100, the occupied space end position is 30, the first idle interval after the occupied space end position is 31-50, and the idle start position is 31. For another example, the number of the data block is 1-100, the last data block occupied by the last space allocation is 30, the current blank data block is 0-15, 45-50,75-100, the occupied space end position is 30, the first idle interval after the occupied space end position is 45-50, and the idle start position is 45. For another example, the number of the data block is 1-100, the last data block occupied by the last space allocation is 100, the current blank data block is 0-15, 45-50, the occupied space end position is 100, and the first idle interval after the occupied space end position is 0-15, so the idle start position is 0. For another example, the number of the data block is 1-100, the last data block occupied by the last space allocation is 100, the current blank data block is 5-15, 45-50, the occupied space end position is 100, the first idle interval after the occupied space end position is 5-15, and the idle start position is 5. For example, the number of the data block is 1-100, the last data block occupied by the last space allocation is 100, the current blank data block is 0,5-15, 45-50, the occupied space end position is 100, the first idle interval after the occupied space end position is 0, and the idle start position is 0.

Alternatively, the free start position may be a free interval initial position of a first free interval from a start position of the initial allocatable space.

For example, the data blocks have data block numbers of 1-100, the currently blank data blocks of 0-15, 31-50,75-100, and the idle starting position of 0. For another example, the data blocks have data block numbers 1-100, the currently blank data blocks 31-50,75-100, and the idle starting position 31.

For the two cases that the idle starting position can be the idle interval initial position of the first idle interval from the starting position of the initial allocatable space and the idle starting position can be the idle interval initial position of the first idle interval after the occupied space ending position occupied by the previous space allocation, the label information can be recorded after the previous space allocation is completed.

Alternatively, the idle start position may be an idle interval initial position of an idle interval with the best matching degree with the size of the space required for pre-allocation. At this time, after the size of the space required by the pre-allocation is obtained, the space is matched according to the current size of each idle interval. The matching degree can be determined by calculating the relative error between the size of the space required by pre-allocation and the size of each idle interval, and the smaller the relative error is, the better the matching degree is. If the matching degree of at least two idle intervals is the same, the idle interval initial position of the first idle interval positioned behind the occupied space end position in each idle interval with the same matching degree can be selected as the idle initial position. If there are at least two idle intervals with the same matching degree, the idle interval initial position of which idle interval is selected as the idle initial position can be determined according to other preset modes by the person skilled in the art.

For example, the number of data blocks is 1-100, the number of currently blank data blocks is 0-15, 31-50,75-100, the size of the pre-allocation space is 15 data blocks, and the idle starting position is 0. For another example, the number of data blocks is 1-100, the number of currently blank data blocks is 0-15, 31-50,75-100, the size of the pre-allocation space is 20 data blocks, and the idle starting position is 31. For another example, the number of data blocks is 1-100, the number of currently blank data blocks is 0-15, 35-50,75-100, the size of the pre-allocated required space is 15 data blocks, the occupied space end position of the last space allocation is 34, and the idle start position is 35.

Therefore, a proper idle interval can be selected for the file to be created, the file to be created is set in the same continuous idle interval as much as possible in a 'filling' mode, and file fragments can be effectively avoided. Meanwhile, if the occupied space end position of the last space allocation can be considered, space allocation is performed after the occupied space end position, metadata can be stored sequentially based on a database, the reading and writing efficiency is higher, the IO pressure of the metadata operation on a hard disk is low, and the metadata operation and the file data reading and writing can be separated into different devices (such as different magnetic disks and the like).

S104: and from the idle starting position, distributing the space of the file to be created for the file to be created according to the size of the space required by the pre-distribution.

Optionally, since the free start position is the free interval initial position of one of the free intervals a, that is, the file space to be created includes at least a part of the free interval a.

Optionally, the idle interval includes one or more data blocks, and since the data blocks corresponding to the idle interval are pre-ordered, the file space to be created includes a plurality of data blocks selected according to the ordering of the data blocks from the data block where the idle start position is located. In other words, the file section to be created may select, from the idle start position, a space of a size required for pre-allocation according to the order of the idle sections.

By pre-configuring the size of the space required by pre-allocation for the file to be created and configuring the space of the file to be created according to the size of the space required by pre-allocation, the phenomenon that a plurality of file fragments appear in multi-file parallel writing can be effectively avoided, and the generation of fragments in a file system is effectively reduced.

Optionally, when the file to be created subsequently occupies only the file space to be created when reading and writing is performed, when the data of the file to be created exceeds the file space to be created, the data coverage can be started from the starting position of the file space to be created (the original data is deleted and the new data is written), and the new file to be created can be created again to record the exceeding data.

Optionally, from the idle starting position, allocating the file space to be created for the file to be created according to the size of the space required by the pre-allocation includes:

and from the idle starting position, distributing a file space to be created for the file to be created according to the size of the space required by pre-distribution according to the ordering of the idle intervals.

Wherein the ordering of the idle intervals comprises at least one of:

the cyclic ordering of the free space, for example, from the end position of the occupied space, through the end position of the initial allocatable space, and back to the start position of the initial allocatable space, to the end position of the occupied space.

The matching degree sequencing of the idle intervals, for example, the idle starting position may be the idle interval initial position of an idle interval with the best matching degree with the size of the space required by pre-allocation, the idle interval corresponding to the idle starting position is used as the idle interval of the first name of sequencing, the differential space is determined, an idle interval with the best matching degree with the differential space is used as the idle interval of the second name of sequencing, and the like, and optionally, when a plurality of idle interval matching degrees are consistent, a certain rule can be formulated by a person skilled in the art to select an idle interval (such as combining natural sequencing or cyclic sequencing);

Natural ordering of the idle intervals, e.g., natural ordering of individual data blocks within the idle intervals from front to back.

In some embodiments, the idle position identifier includes an idle interval initial position and an idle interval end position, and from the idle initial position, allocating a file space to be created for the file to be created according to the size of the space required for pre-allocation includes any one of the following:

the method comprises the steps of obtaining the size of an idle interval and the number of idle position identifiers in idle information, distributing a file space to be created with the size of the space to be created according to one or more idle intervals, the size of the space to be pre-allocated and the idle starting position, and determining the size of the idle interval according to the initial position of the idle interval and the end position of the idle interval.

Optionally, acquiring the size of the idle interval and the number of the idle position identifiers in the idle information, and distributing the file space to be created with the size of the space to be created pre-distributed according to one or more of the size of the idle interval, the size of the space to be pre-distributed and the idle starting position, wherein the file space to be created with the size of the space to be pre-distributed comprises any one of the following:

if the idle information only comprises an idle position mark of an idle interval, the idle initial position is the same as the idle interval initial position, the size of the idle interval is determined according to the idle interval initial position and the idle interval end position, and if the size of the idle interval is larger than or equal to the size of the space required by pre-allocation, the space of the file to be created with the size of the space required by pre-allocation is allocated for the file to be created from the idle interval initial position;

If the idle information comprises idle position identifiers of a plurality of idle intervals, determining a target interval from each idle interval according to idle starting positions, determining the size of the target interval according to the size of each idle interval of the target interval, and if the size of the target interval is larger than or equal to the size of the space required by pre-allocation, distributing the space of the file to be created with the size of the space required by pre-allocation in the target interval from the idle starting positions of the target interval;

if the idle information comprises idle position identifiers of a plurality of idle sections, determining a target section from each idle section according to idle starting positions, determining the size of the target section according to the size of each idle section of the target section, if the size of the target section is smaller than the size of a space required by pre-allocation, acquiring the size of a difference space between the size of the target section and the size of the space required by pre-allocation, determining a supplementary section according to the difference space, wherein the supplementary section comprises a plurality of idle sections which are positioned in the target section and then select the size of the difference space according to a sorting order, and taking the target section and the supplementary section as file spaces to be created.

Optionally, since the target interval includes at least two idle intervals, the idle interval sizes of the idle intervals may be accumulated to obtain the target interval size. Since each idle section has natural ordering (from front to back, and is irrelevant to idle starting positions), if the target section size of the target section obtained from the idle starting position to the idle section ending position of the last idle section in each idle section is still smaller than the pre-allocation required space size, the supplementary section can be determined from the natural ordering starting position of the idle section. Optionally, each idle section further includes a current sequence, where the current sequence is from an idle start position to an idle section end position of a last idle section in the natural sequence, and receives an idle section initial position of a first idle section in the natural sequence to an idle section end position of an idle section before the idle start position in the natural sequence.

If the idle information only includes an idle position identifier of an idle interval, that is, only one idle interval exists in the current file system. If the idle information only includes idle position identifiers of a plurality of idle intervals, that is, a plurality of idle intervals exist in the current file system.

If the idle information only comprises an idle position mark of an idle interval, the idle initial position is the same as the idle interval initial position, the size of the idle interval is determined according to the idle interval initial position and the idle interval end position, and if the size of the idle interval is smaller than the size of the space required by pre-allocation, the feedback creation fails. If the size of the idle interval is larger than or equal to the size of the space required by pre-allocation, configuring continuous space with the size of the space required by pre-allocation for the file to be created as the space of the file to be created from the initial position of the idle interval.

If the idle information comprises idle position identifiers of a plurality of idle sections, a target section can be determined from the idle sections through tag information, and then space allocation is performed according to the corresponding relation between the size of the target section and the size of the space required by pre-allocation. Specifically, if the size of the target interval meets the size of the space required by pre-allocation, only the target interval is occupied, and from the initial position of the idle interval, a continuous space with the size of the space required by pre-allocation is configured for the file to be created in the target interval as the space of the file to be created. If the size of the target interval cannot meet the size of the space required by the pre-allocation, a supplementary interval needs to be determined, and then file spaces to be created are configured in the target interval and the supplementary interval. Optionally, the ordering of the unprecedented intervals includes starting from the end position of the occupied space, recirculating back to the start position of the initial allocatable space via the end position of the initial allocatable space, and ending at the end position of the occupied space.

The supplementary interval is determined by at least one of the following ways:

acquiring the first idle interval size of a first idle interval after the target interval, and if the difference space size is smaller than or equal to the first idle interval size, distributing the space with the difference space size in the first idle interval as a supplementary interval;

and sequentially acquiring the first combination size of the idle interval after the target interval, sequentially acquiring the second combination size of the idle interval before the target interval if the difference space size is larger than the first combination size of the idle interval, and sequentially distributing the space with the difference space size in the first combination and the second combination of the idle interval as a supplementary interval if the first combination size of the idle interval and the second combination of the idle interval are larger than the difference space size.

For ease of understanding, the idle interval location identifier is identified by a data block sequence number as an example, and a number of examples of determining the file space to be created are as follows:

If the size of the pre-allocation required space is 10 data blocks, the current idle interval is (2, 8), (11, 19), (50, 100), the idle starting position is 2, the difference space is 3, the size of the first idle interval (11, 19) of the first idle interval after the target interval is 9, the target interval is (2, 8), the supplementary interval is (11, 13), and the file space to be created is [ 2, 8), (11, 13 ].

If the size of the space required for pre-allocation is 10 data blocks, the current free interval is (2, 8), (11, 19), (98, 100), the free starting position is 97, the target interval is (98, 100), the supplementary interval is (2, 8), and the space of the file to be created is [ 2, 8), (98, 100 ], wherein the storage sequence of the file to be created is that the file to be created is stored (98, 100) first and then (2, 8).

If the size of the pre-allocation space is 10 data blocks, the current idle interval is (2, 8), (11, 12), (98, 100), the idle starting position is 2, the first combination of the idle intervals is (11, 12), (98, 100), and the size of the first combination of the idle intervals is: 2+3=5, the difference space size is 3, the target interval is (2, 8), the supplementary interval is (11, 12), (98), the file space to be created is [ 2, 8), (11, 12), (98 ], wherein the storage sequence of the file to be created is that the file to be created is stored firstly (2, 8) then (11, 12) and finally (98).

If the size of the space required for pre-allocation is 10 data blocks, the current idle interval is (2, 8), (11, 12), (98, 100), the idle starting position is 11, the first combination of the idle interval is (98, 100), the size of the first combination of the idle interval is 3, the second combination of the idle interval is (2, 8), the size of the first combination of the idle interval is 7, and the difference space size is: 10-2=8, the target interval is (11, 12), the supplementary interval is (98, 100), (2, 6), the file space to be created is [ 2, 6), (11, 12), (98, 100) ], wherein the storage sequence of the file to be created is that the file to be created is stored first (11, 12) and then (98, 100) and finally (2, 6).

It should be noted that the first combination of idle intervals and the second combination of idle intervals may include one or more idle intervals.

In the process of storing metadata of a file to be created, the metadata can be stored sequentially by starting from an idle starting position, so that, for example, the idle space is merged after deleting the data later.

In some embodiments, the file management method further comprises:

and merging the occupied interval corresponding to the deleted file occupied position identifier with the idle interval, and updating the idle information.

Optionally, the occupied area and the idle area are combined, that is, if the occupied area is continuous with the idle area, the occupied area is combined with the idle area.

Optionally, deleting the file occupation location identifier includes occupying an initial location and occupying an end location, the idle location identifier includes an idle interval initial location and an idle interval end location, and updating the idle information includes:

if the initial position and the end position of each idle interval in the idle information are different from the initial position and the end position, the initial position and the end position are added to the idle information, namely an idle interval is added;

if the idle interval initial position of one idle interval exists in the idle information and the occupied end position is the same, replacing the idle interval initial position with the occupied end position, namely increasing one idle interval;

if the idle interval end position of one idle interval exists in the idle information is the same as the occupied initial position, the idle interval end position is replaced by the occupied initial position, namely, the idle interval is increased.

In some embodiments, the file management method further comprises:

Combining the deleted file occupation position identification with the idle information, and updating the idle information;

deleted files in the deleted file system occupy a location identifier.

By merging the idle intervals, updating the idle information of the merged idle intervals and deleting the occupied position mark of the deleted file, the data volume of stored data can be reduced.

In some embodiments, after completing the allocation of the file space to be created, the file management method further comprises:

update the idle information and/or the tag information.

Optionally, the idle interval includes a plurality of idle blocks (data blocks), and the manner of updating the tag information includes any one of the following:

if the idle information only comprises an idle position identifier of an idle interval, and the size of the idle interval is equal to the size of the space required by pre-allocation, the updated position identifier comprises a null value;

if the idle information only comprises an idle position identifier of one idle interval, and the size of the idle interval is larger than the size of the space required by pre-allocation, updating the position identifier comprises updating an initial position and an idle interval end position of the idle interval, and the updating initial position comprises the position of a first idle block of the idle interval left after the last idle block in the file space to be created;

If the idle information only comprises idle position identifiers of a plurality of idle intervals and the size of the target interval is equal to the size of the space required by the pre-allocation, the updated position identifiers comprise idle position identifiers of other idle intervals except the target interval;

if the idle information only comprises idle position identifiers of a plurality of idle intervals and the size of the target interval is larger than the size of the space required by pre-allocation, the update position identifiers comprise idle position identifiers of the idle intervals except the target interval, the update position identifiers comprise update initial positions and idle interval end positions of the idle intervals, and the update initial positions comprise positions of first idle blocks of the idle intervals left after the last idle block in the file space to be created.

In some embodiments, after completing step S104, the file management method further includes:

opening a file to be created, and acquiring a file space to be created of the file to be created, wherein the file space to be created comprises a plurality of idle intervals;

and reading and writing data of the file to be created, and respectively and sequentially loading each idle interval into the cache according to the file offset of the file to be created.

Optionally, loading each idle interval into the cache in turn includes:

And acquiring idle starting positions corresponding to the file space to be created, sequentially acquiring each idle interval from the idle starting positions according to the sequence, and loading the idle intervals into a cache.

After the previous idle interval is fully written, the next idle interval is acquired, so that excessive occupation of memory by the cache allocation records can be avoided.

For example, when a file (such as a file to be created after a space is allocated) in a file system is opened, an allocation record (information of the file space to be created) of the file is partially loaded into a memory, and data reading and writing are performed through calculating a reading and writing position of the allocation record of a cache during reading and writing, and when a reading and writing offset exceeds a cache record, related allocation records are reloaded from a database, so that the cache allocation record is prevented from occupying excessive memory.

In some embodiments, the file management method further comprises:

and recording the modification time and the file size of the file to be created after the data is read and written.

Optionally, the modification time and the file size of the file to be created are recorded in real time, so that the fact that the file size is ensured due to power failure can be avoided, and meanwhile, database expansion caused by performance degradation after the IO model is degraded into random reading and writing due to frequent updating of file node information can be avoided.

In some embodiments, the full path and all node information of the file system is cached by the caching module, metadata in the file system is stored by the metadata storage module, and the file management method further comprises at least one of:

when node information is inquired, firstly, step-by-step matching the full path in a cache module, if the matching fails, searching the node information in a metadata storage module, and caching the node information in the cache module, so that the inquiry times from the metadata storage module can be reduced;

when updating the node information, firstly updating the node information in the cache module, and then storing the node information into the metadata storage module, so that the information in the cache module is consistent with the data in the metadata storage module;

and updating the metadata storage module through batch operation writing, so that data inconsistency can be avoided.

In some embodiments, the file management method further comprises:

if the power failure restarting occurs, the pre-written log is obtained and analyzed, and the data recovery is carried out on the file system according to the analyzed pre-written log.

Metadata storage modules include, but are not limited to, databases, such as the leveldb database, and the like.

In the data recovery process, the update of a plurality of tables of the database can be performed through batch operation writing, so that data inconsistency caused by power failure can be avoided.

In some embodiments, the metadata is stored in the leveldb database in K-V (key-value) form.

Alternatively, the metadata may be stored in other relational databases.

In some embodiments, the cache module includes a red-black tree for querying node information via a path and an LRU (Least Recently Used ) linked list for aging the cache.

Optionally, when the cache is inserted, the cache node is added to the red black tree and the LRU linked list simultaneously. The LRU linked list is updated when the cache is accessed. And releasing the oldest buffer after the buffer number reaches the upper limit. The cache is moved out of the LRU linked list when it is referenced by io, managed using a reference count, and rejoined into the LRU linked list when the reference count is 0.

The file system management method in this embodiment may be used to control the file system management system in any of the following embodiments, and specific implementation manners may refer to the following embodiments, which are not described herein.

The embodiment provides a file system management method, which comprises the steps of obtaining a file creation request of a file to be created, configuring a pre-allocation required space size for the file to be created, obtaining idle information of a file system, obtaining tag information, and allocating the file to be created for the file to be created according to the pre-allocation required space size from an idle starting position, so that file fragments can be reduced or avoided during writing, the continuous storage capacity of the same file is improved, and the read-write performance of the file system is improved.

Optionally, metadata of the file system in the embodiment is stored sequentially based on the leveldb, so that the read-write efficiency is high, the IO pressure of metadata operation on the hard disk is small, and the metadata operation and the file data read-write operation can be separated into different devices.

Optionally, by recording the modification time and the file size of the file to be created after data reading and writing, file size loss caused by power failure is avoided, and database expansion caused by large amount of repeated data of level0 in the level ldb due to performance degradation after the IO model is degraded into random reading and writing caused by frequent updating of file node information can be avoided.

Optionally, the file space to be created is sequentially allocated from the idle starting position, and stored, so that the file space to be created is conveniently combined with adjacent data (idle interval) when the space is released (the occupied interval of the file is deleted when the file is deleted), and meanwhile, a mechanism of pre-allocation (pre-allocation of the required space size) +cyclic allocation (sequential allocation of the file space to be created from the idle starting position, especially when the sequencing of the idle interval is from the occupied space ending position, the file space to be created is recycled back to the initial position of the initial allocatable space through the ending position of the initial allocatable space, and the occupied space ending position) can effectively avoid file fragments.

Optionally, the node cache can greatly reduce the disk reading operation when inquiring the file catalogue, reduce the hard disk pressure and improve the efficiency.

Optionally, the embodiment provides a single-process file system construction method and a fragment processing mechanism based on a leveldb database and block equipment, and an efficient power-down protection mechanism, so that the generation of fragments is effectively avoided, and the power-down protection is effectively realized.

Example two

Referring to fig. 2, the present embodiment provides a file system management system 200, which includes a file system module 201 and a metadata storage module 202 for storing metadata of the file system, wherein the file system module 201 includes an access interface of the metadata storage module, and the metadata storage module 202 includes a free unit 2021 and a system recording unit 2022, and the method includes:

the idle unit 2021 is configured to record idle information of an idle space not allocated to a file, where the idle information includes idle location identifiers of a plurality of idle intervals;

the system recording unit 2022 is configured to record tag information, including idle start positions of idle intervals remaining after the file system performs space allocation last time, where an idle position identifier of one idle interval includes an idle start position;

If a file creation request of a file to be created is received through an access interface, an idle starting position is taken as a starting point, and an idle space with a pre-allocation required space size is distributed to serve as a file space to be created according to the ordering of all idle intervals.

In some embodiments, the idle position identifier includes an idle interval initial position and an idle interval end position, and allocating, with the idle initial position as a starting point, an idle space of a size required for pre-allocation as a file space to be created according to the ordering of each idle interval includes any one of the following:

Optionally, the determination manner of the supplementary interval includes at least one of the following:

In some embodiments, the metadata storage module comprises at least one of:

The node unit comprises a file unit and a catalog unit and is used for recording attribute information of files and catalogs, wherein the attribute information comprises at least one of node identity identification information, father identity identification information, creation time, modification time, used size, file space size to be created and name to be created, the node identity identification information is globally unique, and the current maximum value is loaded from the metadata storage module during reloading through accumulation allocation;

Optionally, the index unit records are managed from top to bottom.

Optionally, the active node unit records real-time information of the first-layer directory and the opened file, including node identification information, modification time, file size, and pre-allocation required space size.

Optionally, the allocation unit is further adapted to record the offset in a file; the system recording unit is also used for recording configuration and state information of the file system module, such as information of the hard disk, partition information, allocated (occupied) condition and the like.

Optionally, if the file system is restarted after power failure, the pre-written log is obtained and analyzed, and data recovery is performed on the file system according to the analyzed pre-written log. The pre-written log may be recorded and parsed in a manner known to those skilled in the art, and is not limited herein. After the power-down restarting, the modification time and the file size of the file to be created after the data reading and writing can be read in the active node unit, so that the data inconsistency caused by the power-down prevention can be avoided.

Optionally, the metadata storage module includes a database, the idle unit includes an idle table, the system record unit includes a system table, the node unit includes a node table, the index unit includes an index table, the active node unit includes an active node table, the allocation unit includes an allocation table, each table includes a header identifier and a footer identifier, and the main key prefixes between the tables are different.

Optionally, the metadata storage module includes, but is not limited to, a database, such as a leveldb database, etc., where each unit may exist in the form of a data table, and each data table is distinguished by a KEY prefix, and prepix@and prepix@are additionally inserted as a header and a footer for quick positioning.

Referring to fig. 3-1, fig. 3-1 is a schematic diagram of an Index table Index, which includes parent identification information pid, name, identification information id, type; referring to fig. 3-2, fig. 3-2 is a schematic diagram of a Node table Node, which includes Node id, pid, ctime, mtiem, size used, file space size to be created allocsize, name; referring to fig. 3-3, fig. 3-3 is a schematic diagram of an Active node table, including node identification information id, modification time mtiem, used size, and file space size to be created allocsize; referring to fig. 3-4, fig. 3-4 are schematic diagrams of Free table Free, including a Free interval initial position blkstart and a Free interval end position blkend; referring to fig. 3-5, fig. 3-5 are schematic diagrams of an allocation table Alloc, including node identification information id, offset in a file, an occupied initial position blkstart, and an occupied end position blkend; referring to fig. 3-6, fig. 3-6 are schematic diagrams of a system record table SYS, which includes a hard disk size disc, an allocated (occupied) condition unisize, tag information lastblk, and system information SYS;

In some embodiments, the file system management system further includes a caching module for caching path and node information of the file system;

when inquiring node information, firstly matching paths step by step in a cache module, if matching fails, searching the node information in a metadata storage module, and caching the node information in the cache module;

when updating the node information, firstly updating the node information in the cache module, and then storing the node information into the metadata storage module;

and updating the metadata storage module through batch operation writing.

Fig. 4 is a schematic structural diagram of a cache module, where, as shown in fig. 4, the cache module includes a Tree of red-black trees and an LRU linked list, the red-black trees are used for querying node information through paths, and the LRU linked list is used for performing aging processing on a cache, where:

and when the buffer is inserted, the buffer node TreeNode is added into the red black tree and the LRU chain table simultaneously, when the buffer is accessed, the LRU chain table is updated, the oldest buffer is released after the buffer quantity reaches the upper limit, the buffer is removed from the LRU chain table when the buffer is referenced by io, the management is carried out by using the reference count, and when the reference count is 0, the LRU chain table is added again. FCache is file cache data, DCache is data cache data, and IOnode is a node for caching quilt. The LRU linked list comprises a plurality of ListNodes.

In some embodiments, the file system management system further comprises at least one of:

acquiring a parent directory of a file to be created, if the file system comprises the parent directory, distributing the identity identification information of the node of the directory to be created for the directory to be created, and writing the identity identification information of the node of the directory to be created into a node unit, an index unit and a distribution unit respectively;

writing the occupation information of the file space to be created into an allocation unit;

acquiring a to-be-deleted directory, and deleting the to-be-deleted directory in the node unit and the index unit if the to-be-deleted directory in the index unit is an empty directory;

acquiring deletion information of a deletion file, wherein the deletion information comprises occupation position identifiers of the deletion file, merging an occupation interval corresponding to the occupation position identifiers of the deletion file with an idle interval, and updating the idle information in the idle unit and the occupation information in the allocation unit.

In some embodiments, the file system management system further includes an input/output module (io module), where the input/output module is configured to manage a file handle of the opened file to be created, the file handle is associated with node information, obtain a file space to be created of the file to be created after the file to be created is opened, the file space to be created includes a plurality of idle sections, read and write data of the file to be created, and sequentially load each idle section into the buffer module according to a file offset of the file to be created.

In some embodiments, the file offset may be obtained in a manner known to those skilled in the art.

Optionally, the io module manages file handles that have been opened. The handle directly associates with the cache node to reduce memory usage and to share file real-time status. When the file is opened, the allocation record part is loaded to the memory, and when the file is read and written, the data is read and written through the calculation and writing positions of the allocation record of the cache, and when the read and writing offset exceeds the cache record, the relevant allocation record is reloaded from the database, so that the cache allocation record is prevented from occupying excessive memory. Updating the file modification time and size after writing the data, and writing the active record. Metadata is stored in the leveldb database in K-V form. The metadata may be written over in some manner and stored in other relational databases.

An example of allocation of the file space to be created is shown in fig. 5-1, fig. 5-2 and fig. 5-3, in which only the Free information is deleted when the size of the Free space of the scene one shown in fig. 5-1 is equal to the size of the space required for pre-allocation, as can be seen from fig. 5-1, the occupied space end position is 0, the size of the space required for pre-allocation is 50, the node ID is 2, the Free space is (0, 50), after the space allocation, the Free information is deleted, at this time, free information Free is empty, the new occupied space end position is 50, and the occupied space is (0, 50). After the sum of the sizes of the free intervals of the scene two shown in fig. 5-2 exceeds the size of the space required for pre-allocation, the old record is deleted and the rest record is written, as can be seen from fig. 5-2, the occupied space end position is 0, the size of the space required for pre-allocation is 50, the node ID is 2, the free interval is (0, 100), after space allocation, the new free information is (50, 100), the new occupied space end position is 50, and the occupied space is (0, 50). The scenario three idle record query shown in fig. 5-3 continues looking up from the header when it reaches the end of the table. As can be seen from fig. 5-3, the occupied space ending position is 75, the size of the pre-allocation required space is 50, the node ID is 2, the free intervals are (0, 50) and (75,100), after the space allocation, the new free information is (25, 50), the new occupied space ending position is 25, and the occupied space is formed by two free intervals of (75,100) and (0, 25).

As shown in fig. 6-1, fig. 6-2 and fig. 6-3, the space merging example after deleting the file is that the scene one occupation information shown in fig. 6-1 is not adjacent to the idle information and is directly inserted into the idle information, and as can be seen from fig. 6-1, when the idle information Free is empty, the occupation information Alloc is directly inserted into the idle information and the occupation information is deleted. The second and third allocation records of the scenes shown in fig. 6-2 and fig. 6-3 delete adjacent record writing merging records when the idle information is adjacent. As can be seen from fig. 6-2, when there is "0" adjacent to the occupancy information Alloc in the Free of the idle information, the idle information (0, 100) is updated after the idle information and the occupancy information are combined, and the occupancy information is deleted. As can be seen from fig. 6-3, when there is "25" adjacent to the partial occupation information Alloc in the Free information, the Free information is combined with the occupation information, the Free information (0, 50), (75,100) is updated, and the occupation information is deleted.

In this embodiment, the system is essentially provided with a plurality of modules for executing the method in any of the above embodiments, and specific functions and technical effects are only needed with reference to the above embodiment, and are not described herein again.

Referring to fig. 7, fig. 7 is a schematic structural diagram of another file system management system, which includes a file system module fs, a database db, an input/output module io, and a cache module cache, wherein the database module is used for storing metadata meta, and the io module processes data.

Referring to fig. 8, fig. 8 is a schematic diagram of a typical embedded file system scenario, where in the scenario shown in fig. 8, the scenario mainly includes a hard disk, a partition, data, a cache, an IO, and a file system interface fs, and the description is as follows:

the hard disk is divided into an index partition, a data partition and a backup partition. The data partition is divided in units of 1 MiB. The update of db0 is asynchronously written into db1 for backup in a batch operation mode, so that the reliability of the database is increased. And after loading, checking the integrity of the main library and the back library, and preferentially using the main library to synchronize the standby library under the condition that the main library is complete, and otherwise, using the standby library to synchronize the main library.

Referring to fig. 9, an embodiment of the present invention also provides an electronic device 600 comprising a processor 601, a memory 602, and a communication bus 603;

a communication bus 603 for connecting the processor 601 and the memory 602;

the processor 601 is configured to execute a computer program stored in the memory 602 to implement the method as described in one or more of the above embodiments.

An embodiment of the invention also provides a computer-readable storage medium, characterized in that it has stored thereon a computer program,

the computer program is for causing a computer to execute the method according to any one of the above embodiments.

The embodiment of the present application further provides a non-volatile readable storage medium, where one or more modules (programs) are stored, where the one or more modules are applied to a device, and the device may be caused to execute instructions (instructions) of a step included in the embodiment one of the embodiment of the present application.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The above embodiments are merely illustrative of the principles of the present invention and its effectiveness, and are not intended to limit the invention. Modifications and variations may be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the invention. Accordingly, it is intended that all equivalent modifications and variations of the invention be covered by the claims, which are within the ordinary skill of the art, be within the spirit and scope of the present disclosure.

Claims

1. A method of file system management, the method comprising:

starting from the idle starting position, distributing a file space to be created for the file to be created according to the size of the space required by the pre-distribution according to the sequence of the idle intervals; the idle position identification comprises an idle interval initial position and an idle interval end position, and the allocating the file space to be created for the file to be created according to the size of the space required by the pre-allocation from the idle initial position comprises the following steps:

Acquiring the size of the idle interval and the number of the idle position identifiers in the idle information, and distributing a file space to be created with the size of the space to be created in advance for the file to be created according to one or more of the size of the idle interval, the size of the space to be pre-distributed and the idle starting position, wherein the size of the idle interval is determined according to the initial position of the idle interval and the end position of the idle interval;

the step of obtaining the size of the idle interval and the number of the idle position identifiers in the idle information, and allocating the file space to be created with the size of the space to be created in advance for the file to be created according to one or more of the size of the idle interval, the size of the space to be pre-allocated and the idle starting position comprises any one of the following steps:

2. The file system management method according to claim 1, wherein the manner of determining the supplemental interval includes at least one of:

3. The file system management method according to any one of claims 1 to 2, wherein said method further comprises:

4. The file system management method according to claim 3, wherein said deletion file occupation location identification includes an occupation initial location and an occupation end location, said free location identification includes a free interval initial location and a free interval end location, and updating said free information includes:

5. The file system management method according to any one of claims 1 to 2, wherein said method further comprises:

6. The file system management method according to claim 5, wherein said method further comprises:

7. The file system management method according to any one of claims 1-2, wherein full path and all node information of the file system is cached by a caching module, metadata in the file system is stored by a metadata storage module, the method further comprising at least one of:

and updating the metadata storage module through batch operation writing.

8. The file system management method according to any one of claims 1 to 2, further comprising: if the power failure restarting occurs, a pre-written log is obtained and analyzed, and data recovery is carried out on the file system according to the analyzed pre-written log.

9. A file system management system, comprising a file system module and a metadata storage module for storing file system metadata, the file system module comprising an access interface for the metadata storage module, the metadata storage module comprising an idle unit and a system recording unit, wherein:

if a file creation request of a file to be created is received through the access interface, taking the idle starting position as a starting point, and distributing idle space with the size of a pre-distributed required space as a file space to be created according to the ordering of the idle intervals;

The idle position mark comprises an idle interval initial position and an idle interval end position, and from the idle initial position, allocating a file space to be created for the file to be created according to the size of the space required by the pre-allocation comprises:

10. The file system management system of claim 9, wherein the metadata storage module comprises at least one of:

11. The file system management system of claim 9, further comprising a caching module for caching path and node information of the file system;

and updating the metadata storage module through batch operation writing.

12. The file system management system of claim 10, wherein the file system management system further comprises at least one of:

13. The file system management system according to claim 11, further comprising an input/output module, wherein the input/output module is configured to manage an opened file handle of the file to be created, the file handle is associated with the node information, the file to be created space of the file to be created is obtained after the file to be created is opened, the file to be created space includes a plurality of free sections, data read and write are performed on the file to be created, and each free section is sequentially loaded into the buffer module according to a file offset of the file to be created.

14. A file system management system according to any of claims 10 or 12, wherein the metadata storage module comprises a database, the free unit comprises a free table, the system record unit comprises a system table, the node unit comprises a node table, the index unit comprises an index table, the active node unit comprises an active node table, and the allocation unit comprises an allocation table, wherein each table comprises a header identifier and a footer identifier, respectively, and the primary key prefix differs between each table.

15. An electronic device comprising a processor, a memory, and a communication bus;

the communication bus is used for connecting the processor and the memory;

the processor is configured to execute a computer program stored in the memory to implement the method of any one of claims 1-8.

16. A computer-readable storage medium, having a computer program stored thereon,

the computer program for causing the computer to perform the method of any one of claims 1-8.