CN110018998B - File management method and system, electronic equipment and storage medium - Google Patents
File management method and system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN110018998B CN110018998B CN201910295315.6A CN201910295315A CN110018998B CN 110018998 B CN110018998 B CN 110018998B CN 201910295315 A CN201910295315 A CN 201910295315A CN 110018998 B CN110018998 B CN 110018998B
- Authority
- CN
- China
- Prior art keywords
- address
- file
- access
- mapping relation
- file system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/13—File access structures, e.g. distributed indices
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The application discloses a file management method, a file management system, an electronic device and a computer readable storage medium, wherein the method comprises the following steps: when an access request of a target file is received, an access address in the access request and an index value of the target file are acquired; wherein the target file is the common file; searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node; and determining an address mapping relation item corresponding to the access address according to the index field, and determining a physical address of a data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address, thereby improving the performance of managing the fragmented file by the file system.
Description
Technical Field
The present invention relates to the field of storage technology, and more particularly, to a file management method, a system, an electronic device, and a computer readable storage medium.
Background
For the storage of fragmented data, the prior art is to directly employ a general file system, such as EXT4 (chinese full name: fourth generation extended file system, english full name: fourth extended filesystem). A shard corresponds to a common file stored in a common file system.
The general file system has powerful functions, including file read-write function, directory, link and other functions. However, the storage of the fragmented data only uses the file read-write function, and the function utilization rate of the general file system is low and the performance is poor.
Therefore, how to improve the performance of file system management of fragmented files is a technical problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The present application is directed to a file management method, system, electronic device, and computer readable storage medium, which improve the performance of managing fragmented files in a file system.
In order to achieve the above object, the present application provides a file management method applied to a file system, where the file system is a file system for managing common files in a general file system, the method includes:
when an access request of a target file is received, an access address in the access request and an index value of the target file are acquired; wherein the target file is the common file;
searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node;
and determining an address mapping relation item corresponding to the access address according to the index field, and determining a physical address of a data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
Wherein the determining, according to the index field, an address mapping relation item corresponding to the access address includes:
searching the index field in a red and black tree structure in the memory to obtain an address mapping relation item corresponding to the access address.
If the address mapping relation item corresponding to the access address does not exist in the red-black tree structure, before determining the physical address of the data block in the address mapping relation item, the method further comprises the following steps:
searching the index field in an address mapping table corresponding to the index node to obtain an address mapping relation item corresponding to the access address, and loading the address mapping relation item into the red-black tree structure.
After loading the address mapping relation item into the red-black tree structure, the method further comprises:
and loading the address mapping relation item into a global LRU table.
Wherein, the management granularity of the metadata area in the file system is at least 4KB, and the management granularity of the data area is at least 1MB.
Before determining the physical address of the data block corresponding to the access address in the address mapping relation item, the method further comprises the following steps:
judging whether written data exists in the data block corresponding to the access address according to the target field in the address mapping relation item;
if yes, executing the step of determining the physical address of the data block corresponding to the access address in the address mapping relation item.
Wherein if the access request is a read request, further comprising:
and when the data block corresponding to the access address does not have the written data, directly returning the data block to be empty.
Wherein if the access request is a write request, further comprising:
distributing a physical address for the data to be written in the write request, and writing the physical address into the address mapping relation item;
and writing the data to be written into the data block corresponding to the physical address.
To achieve the above object, the present application provides a file management system,
the method is applied to a file system, wherein the file system is used for managing common files in a general file system, and comprises the following steps:
the acquisition module is used for acquiring an access address in the access request and an index value of the target file when the access request of the target file is received; wherein the target file is the common file;
the determining module is used for searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node;
and the response module is used for determining an address mapping relation item corresponding to the access address according to the index field, and determining the physical address of the data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
To achieve the above object, the present application provides an electronic device, including:
a memory for storing a computer program;
and a processor for implementing the steps of the file management method as described above when executing the computer program.
To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the file management method as described above.
According to the scheme, the file management method provided by the application comprises the following steps: when an access request of a target file is received, an access address in the access request and an index value of the target file are acquired; wherein the target file is the common file; searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node; and determining an address mapping relation item corresponding to the access address according to the index field, and determining a physical address of a data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
The file management method is applied to the file system, the file system is used for managing the common files in the common file system, namely, the metadata only comprises the metadata of the common files, the data management function of the files in the common file system is realized through the index node and the address mapping relation item, one file fragment is one common file stored on the file system, the functions of catalogues, soft and hard links, special files and the like in the common file system are eliminated, the design of the whole file system is simplified, the function utilization rate is high, and the performance of managing the fragmented files of the file system is further improved. The application also discloses a file management system, an electronic device and a computer readable storage medium, and the technical effects can be achieved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art. The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:
FIG. 1 is a flowchart of a file management method according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of metadata according to an embodiment of the present application;
FIG. 3 is a schematic diagram illustrating a physical space division provided in an embodiment of the present application;
FIG. 4 is a flowchart of another file management method according to an embodiment of the present disclosure;
FIG. 5 is a flowchart of another file management method according to an embodiment of the present disclosure;
FIG. 6 is a block diagram of a file management system according to an embodiment of the present disclosure;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
The embodiment of the application discloses a file management method which is applied to a file system, wherein the file system is a file system for managing common files in a general file system, and the performance of managing fragmented files by the file system is improved.
Referring to fig. 1, a flowchart of a file management method according to an exemplary embodiment is shown, as shown in fig. 1, including:
s101: when an access request of a target file is received, an access address in the access request and an index value of the target file are acquired; wherein the target file is the common file;
the execution main body of the embodiment is a file system, the file system operates in a user space and can be used for managing fragmented files, and compared with the scheme that a general file system operates in a kernel layer in the prior art, the file system is more convenient to maintain and better in performance. The file system of the embodiment can provide all function interfaces to the outside in a manner of a C language API, for example, a host program of a server side of glumerfs (english holonomy: gluster File System, a free software, which is mainly applied in a cluster system and has good expandability) can implement logics such as creation, deletion, reading and writing of the fragmented files through the function interfaces, and can directly access the logic block device or the disk through the Linux block device file, wherein each fragmented file is stored in a data block of the logic block device or the disk. The file type of the file system is a file in the common sense, such as a data file, an executable file and the like. Therefore, unlike the general file system, the file system of the present embodiment only implements logic such as creation, deletion, reading and writing of files, and does not implement other functions such as directory, hard and soft links, and special files.
When the file system is formatted, a fixed-length area is divided at the head of the block device for storing metadata of the fragmented files, wherein the metadata only comprises metadata of common files in the general file system, and at least comprises an index node table for storing basic information of all files in the file system and an address mapping table for storing address mapping relations.
Specifically, an Inode table (for example, inode table) includes a plurality of inodes (for example, inodes), each Inode stores basic information of a file, such as a file size, a modification time, and the like, an address mapping table is used as an supplement to the Inode table, and stores an address mapping relationship between an access address of a file and a physical address of a data block storing the file, as shown in fig. 2, each Inode records metadata of a file, and may include an i_check field (check field), an i_mode field (file type and access right), an i_flags field (file flag), an i_uid field (file owner ID), an i_gid field (user group ID), and a plurality of i_extension fields. As can be seen from fig. 1, the check bit does not exist in the data block (data block), so that all metadata in this embodiment has self-checking capability, but the reliability and performance of the file system can be considered by not checking the stored file data.
The i_existence field in each inode points to a special 1MB data block (i.e., an existence block in the figure), and the address mapping relation of the file is stored. The data block includes a plurality of address mapping relation items (i.e. the existence items in the figure), each address mapping relation item may include a check bit (crc), a field (block) pointing to a physical address of the data block, a reserved bit (reserved), and a target field (i.e. the bitmap field in the figure) representing whether written data exists in each data block, where the target field is mainly used to implement a sparse file with smaller granularity and improve the read-write performance, and detailed description will be given in the following embodiments.
Of course, super blocks for storing global information of the file system, block bitmaps for recording physical space usage of all data blocks, etc. may also be included, which are not particularly limited herein. The bit block diagram can be obtained according to the target fields in all the address mapping relation items.
The physical space of a logical block device or disk is divided into a metadata area and a data area. As shown in fig. 3, the front 1058MB is used as a metadata area, and the remaining portion is used as a data area. The super block (superblock), inode table (Inode table) and bit block map (block bitmap) described above may be stored in the metadata area, and two super blocks in the map are used for redundancy backup. And an address mapping table as a supplement to the Inode table may be stored in the data area by an algorithm.
Preferably, the management granularity of the metadata area is at least 4KB, namely, various metadata are cached, read-written and self-checked according to at least 4KB, the management granularity of the data area is at least 1MB, namely, the allocation granularity of the data blocks is at least 1MB. The size of the data block divided by the general file system is generally 512 bytes, but the size of the data block of the file system provided by the embodiment is at least 1MB, that is, the management granularity of the data area is far greater than that of the general file system, so that the required metadata space is smaller, even most metadata can reside in the memory, and the data access performance is ensured. The management granularity of the metadata in the general file system is generally 512 bytes, but the management granularity of the metadata of the file system provided by the embodiment is at least 4KB, which is far greater than that of the general file system, and the management efficiency of the metadata is improved.
In this step, the access request may be a read request or a write request, and at least includes an index value of the target file and an access address of the current access, where the index value may be a group number or a UUID (chinese full name: universal unique identifier, english full name: universally Unique Identifier), and is not specifically limited herein. If the access request is a write request, the data to be written should also be included.
S102: searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node;
in a specific implementation, when the file system is started, the inode table is loaded into the memory hash table. When an access request is received, searching an index node table in the memory hash table for the index value of the target file obtained in the previous step to obtain an index node corresponding to the target file, and determining an index field of an address mapping relation item corresponding to the access address in the index node, for example, an extension 0 in fig. 2.
S103: and determining an address mapping relation item corresponding to the access address according to the index field, and determining a physical address of a data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
In this step, an address mapping relationship item corresponding to the access address, for example, an existence item in fig. 2, may be determined according to the correspondence between the index field and the address mapping relationship item determined in the previous step. Preferably, since the red-black tree structure can accelerate the searching efficiency, when a certain address mapping relation item is accessed for the first time, the address mapping relation item can be loaded into the red-black tree structure, and then the index field is searched in the red-black tree structure, namely the step can comprise the following steps: searching the index field in a red and black tree structure in the memory to obtain an address mapping relation item corresponding to the access address. If the address mapping relation item corresponding to the access address does not exist in the red-black tree structure, searching the index field in the address mapping table corresponding to the index node to obtain the address mapping relation item corresponding to the access address, and loading the address mapping relation item into the red-black tree structure.
When the address mapping relation item corresponding to the index field is not searched in the red and black number structure, the address mapping relation item corresponding to the access address is obtained by searching the index field in the address mapping table in the metadata area when the data of the access address is accessed for the first time, and the address mapping relation item corresponding to the access address is loaded into the memory red and black tree structure. Preferably, the address mapping relation item can also be loaded into a global LRU (English full name: least Recently Used, a page replacement algorithm of memory management) table. Since if a data has not been accessed in the last period of time, it is less likely to be accessed in the future. The step can eliminate the data which is not accessed for the longest time when the limited space is full of data, so that the execution efficiency of the process is improved, and the response efficiency of the access request is further improved. And searching the index field in the red-black tree structure of the memory to obtain a mapping relation item corresponding to the access address when the data corresponding to the access address is not accessed for the first time.
After the address mapping relation item is obtained, the physical address of the data block corresponding to the access request can be determined, and the access request is responded according to the physical address. And if the access request is a read request, returning the data in the object data block, and if the access request is a write request, writing the data to be written in the write request into the data block.
The file management method provided by the embodiment of the invention is applied to a file system, the file system is used for managing common files in a general file system, namely metadata only comprises metadata of the common files, the data management function of the files in the general file system is realized through index nodes and address mapping relation items, one file fragment is one common file stored on the file system, the functions of catalogues, soft and hard links, special files and the like in the general file system are eliminated, the design of the whole file system is simplified, the function utilization rate is high, and the performance of managing the fragmented files of the file system is further improved.
When the above access request is a read request, referring to fig. 4, another flowchart of a file management method according to an exemplary embodiment is shown, as shown in fig. 4, including:
s201: when a read request of a target file is received, a read address in the read request and an index value of the target file are obtained;
in this embodiment, the access request is defined as a read request, where the read request includes an index value of the target file and a read address corresponding to the current request.
S202: searching an index node table according to the index value to obtain an index node corresponding to the target file, and obtaining an index field of an address mapping relation item corresponding to the reading address in the index node;
s203: searching the index field in a red-black tree structure in a memory to obtain an address mapping relation item corresponding to the reading address;
s204: judging whether the data block has written data or not according to a target field in the address mapping relation item, if so, entering S205; if not, directly returning the data block to be empty;
because the file system in this embodiment may provide sparse file characteristics, there may be a data block with empty write data, so before determining the physical address, it may be determined whether the data block has write data through the target field in the address mapping relation item, if yes, the write data is returned, if not, the data block is directly returned to be empty, and the subsequent step of determining the physical address is not required, thereby improving the reading performance of the file system.
S205: and determining the physical address of the data block in the address mapping relation item, and returning the writing data in the data block according to the physical address.
It can be seen that, in this embodiment, if the access request is a read request, before determining the physical address, a step of determining whether the data block corresponding to the read address has written data is added, and if not, the data is returned to be empty directly, so that the data reading process is simplified, and the reading performance of the file system is improved.
When the above access request is a write request, referring to fig. 5, a flowchart of still another file management method according to an exemplary embodiment is shown, as shown in fig. 5, including:
s301: when a write request of a target file is received, a write address in the write request and an index value of the target file are acquired;
in this embodiment, the access request is defined as a write request, where the read request includes an index value of the target file, a write address corresponding to the current request, and data to be written.
S302: searching an index node table according to the index value to obtain an index node corresponding to the target file, and obtaining an index field of an address mapping relation item corresponding to the writing address in the index node;
s303: searching the index field in a red-black tree structure in a memory to obtain an address mapping relation item corresponding to the writing address;
s304: judging whether the data block has written data or not according to a target field in the address mapping relation item, if so, entering S305; if not, entering S306;
s305: determining the physical address of the data block in the address mapping relation item, and entering S307;
s306: distributing a physical address for the data to be written in the write request, writing the physical address into the address mapping relation item, and entering S307;
s307: and writing the data to be written into the data block corresponding to the physical address.
In this embodiment, whether the data block has the written data is determined according to the target field in the address mapping relation item, if yes, the physical address of the data block is determined, the data to be written is written into the data block in an overwriting manner, if not, the physical address field in the address mapping relation item is empty, firstly, the physical address is allocated to the data to be written, the physical address is added to the physical address field, and the data to be written is written into the data block corresponding to the physical address.
A document management system according to an embodiment of the present application is described below, and a document management system described below and a document management method described above may be referred to each other.
Referring to FIG. 6, a block diagram of a file management system is shown according to an exemplary embodiment, as shown in FIG. 6, comprising:
an obtaining module 601, configured to obtain, when an access request of a target file is received, an access address in the access request and an index value of the target file; wherein the target file is the common file;
a determining module 602, configured to search an index node table according to the index value to obtain an index node corresponding to the target file, and determine an index field corresponding to the access address in the index node;
and the response module 603 is configured to determine an address mapping relation item corresponding to the access address according to the index field, and determine a physical address of the data block corresponding to the access address in the address mapping relation item, so as to respond to the access request according to the physical address.
The file management system provided by the embodiment of the invention is applied to a file system, the file system is used for managing common files in a general file system, namely metadata only comprises metadata of the common files, the data management function of the files in the general file system is realized through index nodes and address mapping relation items, one file fragment is one common file stored on the file system, the functions of catalogues, soft and hard links, special files and the like in the general file system are eliminated, the design of the whole file system is simplified, the function utilization rate is high, and the performance of managing the fragmented files of the file system is further improved.
On the basis of the above embodiment, as a preferred implementation manner, the response module 603 includes:
the first searching unit is used for searching the index field in the red and black tree structure in the memory to obtain an address mapping relation item corresponding to the access address;
and the response unit is used for determining the physical address of the data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
On the basis of the foregoing embodiment, as a preferred implementation manner, if the address mapping relation item corresponding to the access address does not exist in the red-black tree structure, the response module 603 further includes:
and the second searching unit is used for searching the index field in the address mapping table corresponding to the index node to obtain an address mapping relation item corresponding to the access address, and loading the address mapping relation item into the red-black tree structure.
On the basis of the above embodiment, as a preferred implementation manner, the response module 603 further includes:
and the loading unit is used for loading the address mapping relation item into the global LRU table.
Based on the above embodiment, as a preferred implementation manner, the management granularity of the metadata area in the file system is at least 4KB, and the management granularity of the data area is at least 1MB.
On the basis of the above embodiment, as a preferred implementation manner, the response module 603 further includes:
the judging unit is used for judging whether the data block corresponding to the access address has written data or not according to the target field in the address mapping relation item; if yes, starting the workflow of the response unit;
on the basis of the foregoing embodiment, as a preferred implementation manner, if the access request is a read request, the response module 603 further includes:
and the return unit is used for directly returning the empty data block when the data block corresponding to the access address does not have the written data.
On the basis of the foregoing embodiment, as a preferred implementation manner, if the access request is a write request, the response module 603 further includes:
the allocation unit is used for allocating a physical address for the data to be written in the write request and writing the physical address into the address mapping relation item;
and the writing unit is used for writing the data to be written into the data block corresponding to the physical address.
The specific manner in which the various modules perform the operations in relation to the systems of the above embodiments have been described in detail in relation to the embodiments of the method and will not be described in detail herein.
The present application also provides an electronic device, referring to fig. 7, and a block diagram of an electronic device 700 provided in an embodiment of the present application, as shown in fig. 7, may include a processor 11 and a memory 12. The electronic device 700 may also include one or more of a multimedia component 13, an input/output (I/O) interface 14, and a communication component 15.
The processor 11 is configured to control the overall operation of the electronic device 700 to perform all or part of the steps in the file management method described above. The memory 12 is used to store various types of data to support operation on the electronic device 700, which may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, messages sent and received, pictures, audio, video, and so forth. The Memory 12 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 13 may include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 12 or transmitted through the communication component 15. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 14 provides an interface between the processor 11 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 15 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G or 4G, or a combination of one or more thereof, the corresponding communication component 15 may thus comprise: wi-Fi module, bluetooth module, NFC module.
In an exemplary embodiment, the electronic device 700 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated ASIC), digital signal processor (Digital Signal Processor, abbreviated DSP), digital signal processing device (Digital Signal Processing Device, abbreviated DSPD), programmable logic device (Programmable Logic Device, abbreviated PLD), field programmable gate array (Field Programmable Gate Array, abbreviated FPGA), controller, microcontroller, microprocessor, or other electronic components for performing the file management methods described above.
In another exemplary embodiment, a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the above-described file management method is also provided. For example, the computer readable storage medium may be the memory 12 described above including program instructions executable by the processor 11 of the electronic device 700 to perform the file management method described above.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the claims of the present application.
It should also be noted that in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Claims (11)
1. The file management method is characterized by being applied to a file system, wherein the file system is a file system for managing common files in a general file system, and the file system realizes the functions of data management of the general file system, removing catalogues, soft and hard links and special files of the general file system; the method comprises the following steps:
when an access request of a target file is received, an access address in the access request and an index value of the target file are acquired; wherein the target file is the common file;
searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node;
and determining an address mapping relation item corresponding to the access address according to the index field, and determining a physical address of a data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
2. The method of claim 1, wherein determining the address mapping relation item corresponding to the access address according to the index field includes:
searching the index field in a red and black tree structure in the memory to obtain an address mapping relation item corresponding to the access address.
3. The method according to claim 2, wherein if the address mapping relation item corresponding to the access address does not exist in the red-black tree structure, before determining the physical address of the data block in the address mapping relation item, further comprising:
searching the index field in an address mapping table corresponding to the index node to obtain an address mapping relation item corresponding to the access address, and loading the address mapping relation item into the red-black tree structure.
4. The method of claim 3, further comprising, after loading the address mapping relationship item into the red-black tree structure:
and loading the address mapping relation item into a global LRU table.
5. The file management method according to claim 1, wherein the management granularity of the metadata area in the file system is at least 4KB, and the management granularity of the data area is at least 1MB.
6. The file management method according to any one of claims 1 to 5, further comprising, before determining a physical address of a data block corresponding to the access address in the address mapping relation item:
judging whether written data exists in the data block corresponding to the access address according to the target field in the address mapping relation item;
if yes, executing the step of determining the physical address of the data block corresponding to the access address in the address mapping relation item.
7. The method of claim 6, further comprising, if the access request is a read request:
and when the data block corresponding to the access address does not have the written data, directly returning the data block to be empty.
8. The method of claim 6, further comprising, if the access request is a write request:
distributing a physical address for the data to be written in the write request, and writing the physical address into the address mapping relation item;
and writing the data to be written into the data block corresponding to the physical address.
9. The file management system is characterized by being applied to a file system, wherein the file system is a file system for managing common files in a general file system, and the file system realizes the functions of data management of the general file system, removing catalogues, soft and hard links and special files of the general file system; the system comprises:
the acquisition module is used for acquiring an access address in the access request and an index value of the target file when the access request of the target file is received; wherein the target file is the common file;
the determining module is used for searching an index node table according to the index value to obtain an index node corresponding to the target file, and determining an index field corresponding to the access address in the index node;
and the response module is used for determining an address mapping relation item corresponding to the access address according to the index field, and determining the physical address of the data block corresponding to the access address in the address mapping relation item so as to respond to the access request according to the physical address.
10. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the file management method according to any one of claims 1 to 8 when executing said computer program.
11. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the file management method according to any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295315.6A CN110018998B (en) | 2019-04-12 | 2019-04-12 | File management method and system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295315.6A CN110018998B (en) | 2019-04-12 | 2019-04-12 | File management method and system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110018998A CN110018998A (en) | 2019-07-16 |
| CN110018998B true CN110018998B (en) | 2023-05-12 |
Family
ID=67191280
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910295315.6A Active CN110018998B (en) | 2019-04-12 | 2019-04-12 | File management method and system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110018998B (en) |
Families Citing this family (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559485A (en) * | 2019-09-26 | 2021-03-26 | 伊姆西Ip控股有限责任公司 | Method, apparatus and computer program product for managing a storage system |
| CN110795626A (en) * | 2019-10-28 | 2020-02-14 | 南京弹跳力信息技术有限公司 | Big data processing method and system |
| CN111274051B (en) * | 2020-01-19 | 2023-08-22 | 中信银行股份有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111309376A (en) * | 2020-02-13 | 2020-06-19 | 无锡华云数据技术服务有限公司 | Data processing method, device, equipment and storage medium |
| CN111881466B (en) * | 2020-08-06 | 2023-05-30 | 中电科网络安全科技股份有限公司 | File output method and device, electronic equipment and storage medium |
| CN112380176A (en) * | 2020-11-17 | 2021-02-19 | 浙江大华技术股份有限公司 | Method and device for monitoring file system abnormity in real time and storage medium |
| CN112650477A (en) * | 2020-12-31 | 2021-04-13 | 深圳软牛科技有限公司 | Project data batch transfer method and device and electronic equipment |
| CN113656078A (en) * | 2021-07-28 | 2021-11-16 | 北京车和家信息技术有限公司 | Equipment starting method and device, storage medium and electronic equipment |
| CN116820787B (en) * | 2023-08-31 | 2023-12-15 | 湖北芯擎科技有限公司 | Load balancing method, load balancing device, electronic equipment and computer readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5557674A (en) * | 1993-07-22 | 1996-09-17 | Kok-Wah Yeow | Absolute static lock of files and directories on magnetic disk storage media in single machine personal microcomputers |
| WO1998022890A1 (en) * | 1996-11-22 | 1998-05-28 | Mangosoft Corporation | System for tracking data |
| WO2015131403A1 (en) * | 2014-03-07 | 2015-09-11 | 华为技术有限公司 | File accessing method and related device |
| WO2016082196A1 (en) * | 2014-11-28 | 2016-06-02 | 华为技术有限公司 | File access method and apparatus and storage device |
| US9471595B1 (en) * | 2013-09-30 | 2016-10-18 | Emc Corporation | Recovering file mapping information in file systems |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103019953B (en) * | 2012-12-28 | 2015-06-03 | 华为技术有限公司 | Construction system and construction method for metadata |
| US20150248443A1 (en) * | 2014-03-02 | 2015-09-03 | Plexistor Ltd. | Hierarchical host-based storage |
| WO2016033718A1 (en) * | 2014-09-01 | 2016-03-10 | 华为技术有限公司 | File access method and apparatus, and storage system |
| CN109376122A (en) * | 2018-09-25 | 2019-02-22 | 深圳市元征科技股份有限公司 | A kind of file management method, system and block chain node device and storage medium |
-
2019
- 2019-04-12 CN CN201910295315.6A patent/CN110018998B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5557674A (en) * | 1993-07-22 | 1996-09-17 | Kok-Wah Yeow | Absolute static lock of files and directories on magnetic disk storage media in single machine personal microcomputers |
| WO1998022890A1 (en) * | 1996-11-22 | 1998-05-28 | Mangosoft Corporation | System for tracking data |
| US9471595B1 (en) * | 2013-09-30 | 2016-10-18 | Emc Corporation | Recovering file mapping information in file systems |
| WO2015131403A1 (en) * | 2014-03-07 | 2015-09-11 | 华为技术有限公司 | File accessing method and related device |
| WO2016082196A1 (en) * | 2014-11-28 | 2016-06-02 | 华为技术有限公司 | File access method and apparatus and storage device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110018998A (en) | 2019-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110018998B (en) | File management method and system, electronic equipment and storage medium | |
| US11947489B2 (en) | Creating snapshots of a storage volume in a distributed storage system | |
| KR101977575B1 (en) | Apparatus and method for directory entry look up, and recording medium recording the directory entry look up program thereof | |
| US8423733B1 (en) | Single-copy implicit sharing among clones | |
| US8583893B2 (en) | Metadata management for virtual volumes | |
| CN108090168B (en) | Universal F2FS file system parsing method, terminal device and storage medium | |
| US11580162B2 (en) | Key value append | |
| US10891074B2 (en) | Key-value storage device supporting snapshot function and operating method thereof | |
| WO2017041654A1 (en) | Method and apparatus for writing and acquiring data in distributed storage system | |
| US9384201B2 (en) | Method of managing data of file system using database management system | |
| US9262313B2 (en) | Provisioning in heterogenic volume of multiple tiers | |
| CN108121813B (en) | Data management method, device, system, storage medium and electronic equipment | |
| CN111381779B (en) | Data processing method, device, equipment and storage medium | |
| US9116904B2 (en) | File system operation on multi-tiered volume | |
| WO2020093501A1 (en) | File storage method and deletion method, server, and storage medium | |
| CN113568582B (en) | Data management method, device and storage equipment | |
| CN111177143B (en) | Key value data storage method and device, storage medium and electronic equipment | |
| CN111400334B (en) | Data processing method, data processing device, storage medium and electronic device | |
| US10430292B2 (en) | Snapshot deletion in a distributed storage system | |
| CN110955665A (en) | Cache query method and device and electronic equipment | |
| US10289345B1 (en) | Contention and metadata write amplification reduction in log structured data storage mapping | |
| KR20140042431A (en) | Computing system and method for managing data in the system | |
| US10210067B1 (en) | Space accounting in presence of data storage pre-mapper | |
| CN112115521B (en) | Data access method and device | |
| US11782629B2 (en) | Data processing method, device, and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |