WO2015116125A1 - File system analysis in user daemon - Google Patents

File system analysis in user daemon Download PDF

Info

Publication number
WO2015116125A1
WO2015116125A1 PCT/US2014/013987 US2014013987W WO2015116125A1 WO 2015116125 A1 WO2015116125 A1 WO 2015116125A1 US 2014013987 W US2014013987 W US 2014013987W WO 2015116125 A1 WO2015116125 A1 WO 2015116125A1
Authority
WO
WIPO (PCT)
Prior art keywords
file system
analysis
metadata
user
file
Prior art date
Application number
PCT/US2014/013987
Other languages
French (fr)
Inventor
Anoop KUMAR R.
Anand A. GANJIHAL
Santigopal MONDAL
Rajagopal Chellam
Sandya Srivilliputtur Mannarswamy
Kumarswamy SUBBAKRISHNA
Original Assignee
Hewlett-Packard Development Company, L.P.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hewlett-Packard Development Company, L.P. filed Critical Hewlett-Packard Development Company, L.P.
Priority to PCT/US2014/013987 priority Critical patent/WO2015116125A1/en
Publication of WO2015116125A1 publication Critical patent/WO2015116125A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers

Definitions

  • FIG. 1 illustrates a block diagram of a computing system for file system analysis in a user daemon according to examples of the present disclosure
  • FIG. 2 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure
  • FIG. 3 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure
  • FIG. 4 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure.
  • file systems running on enterprise and user computing systems have also increased in size. These file systems act as containers of data, and their sizes may be in the gigabytes, terabytes, and/or petabytes and may support millions or billions of file system objects.
  • the increases in file system sizes, along with the corresponding increases in the number of file system objects burden system resources and negatively impact users' experiences when the file systems are checked for file system consistency.
  • a file system consistency check (also referred to as “checker,” “file system checker,” or “fsck”) refers to a software program or instructions that checks the consistency of a file system on a computing system.
  • the file system check analyzes a file system to discover and identify inconsistent states.
  • the file system check may also correct or fix the identified inconsistencies, problems, errors, etc.
  • file systems are checked for errors and inconsistencies while the file system is in an offline state (that is, while the file system is inaccessible to users, applications, etc.).
  • offline file system checks depends on the number of file system objects that need to be examined. Depending on the size of the file system and the number of related objects, the file system check may take hours or days to execute, rendering the computing system unusable and/or unavailable. For example, in a typical offline file system check on a large file system such as a 32 terabyte with 50 million objects may take thirty or more hours to perform the checking. Similarly, a 32 terabyte file system with 100 million objects may take more than fifty hours to perform file system checking, for example. During this time, the file system remains offline and unavailable.
  • Such large downtimes are typically unacceptable in enterprise storage environments, and the large downtimes make offline file system checking non-scalable as file system sizes and number of objects continues to grow.
  • a file system may be taken offline in order to perform the file system check. However, this may take many hours to complete, rendering the file system otherwise unusable, as discussed above.
  • Another solution provides for online file system checking by creating a snapshot of the file system and its associated metadata. The checking is then performed on the snapshot such that errors are identified and corrected. Once the checking is completed including fixing any errors in the snapshot of the file system, the fixes are then applied back to the original file system from the snapshot file system.
  • a file system check may be performed on the snapshot with errors logged to a separate file. The file system may then be taken offline in order to apply any necessary fixes.
  • a method may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system.
  • the method may further include performing a background file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system.
  • the method may then include returning a result of the background file system analysis from the file system user daemon to the kernel of the operating system.
  • file system consistency checking occurs in user space (such as a user daemon of the operating system) while the file system is online and accepting normal user file system operations. In this way, excessive downtime due to offline file system checking is avoided. Moreover, the file system may be corrected and "updated in place” rather than having to transition the file system into an offline state. Additionally, performing the file system consistency checking in the user space (as opposed to a purely kernel- based approach) enables the mounted file system to actively satisfy file input/output (IO) operations. By performing the file system consistency checking in the user space, complex code may be avoided, and the file system consistency checking is isolated from active kernel operations with well-defined communication interfaces, instead of embedding the file system consistency checking within the kernel.
  • Metadata includes file tags (traditionally known as inode numbers in an example) which act as indirection mechanisms between directory entries and mcells.
  • the mcells represent traditional inodes.
  • the metadata provides versatility in that it facilitates dynamic volume addition, removal, and expansion.
  • a first type of metadata is file system metadata relating to metadata utilized in mounting the file system.
  • a second type of metadata is metadata relating to the operation of the file system after it is mounted and is analyzed or checked while the file system is mounted, online, and operational.
  • FIG. 1 illustrates a block diagram of a computing system 100 for file system (FS) analysis in a user daemon according to examples of the present disclosure.
  • FIG. 1 includes particular components, modules, etc. according to various examples. However, in different embodiments, more, fewer, and/or other components, modules, arrangements of components/modules, etc. may be used according to the teachings described herein.
  • various components, modules, etc. described herein may be implemented as one or more software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, application specific integrated circuits (ASICs), embedded controllers, hardwired circuitry, etc.), or some combination of these.
  • special-purpose hardware e.g., application specific hardware, application specific integrated circuits (ASICs), embedded controllers, hardwired circuitry, etc.
  • the computing system 100 may include any appropriate type of computing device, including for example smartphones, tablets, desktops, laptops, workstations, servers, smart monitors, smart televisions, digital signage, scientific instruments, retail point of sale devices, video walls, imaging devices, peripherals, or the like, including any suitable combinations thereof.
  • the computing system 100 may include a processing resource 102 that represents generally any suitable type or form of processing unit or units capable of processing data or interpreting and executing instructions.
  • the instructions may be stored on a non-transitory tangible computer-readable storage medium, such as a memory resource or on a separate device (not shown), or on any other type of volatile or non-volatile memory that stores instructions to cause a programmable processor to perform the techniques described herein.
  • the computing system 100 may include dedicated hardware, such as one or more integrated circuits, Application Specific Integrated Circuits (ASICs), Application Specific Special Processors (ASSPs), Field Programmable Gate Arrays (FPGAs), or any combination of the foregoing examples of dedicated hardware, for performing the techniques described herein.
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Special Processors
  • FPGAs Field Programmable Gate Arrays
  • multiple processors and/or processing resources may be used, as appropriate, along with multiple memories, memory resources, and/or types of memory.
  • the computing system 100 may include a file system user daemon analysis module 110 and a result module 112.
  • the modules described herein may be a combination of hardware and programming.
  • the programming may be processor executable instructions stored on a tangible memory resource such as a memory resource (not shown), and the hardware may include processing resource 102 for executing those instructions.
  • the memory resource can be said to store program instructions that when executed by the processing resource 102 implement the modules described herein.
  • Other modules may also be utilized as will be discussed further below in other examples.
  • the file system user daemon analysis module 110 performs, using a user daemon, a file system analysis to check for inconsistencies within the metadata and/or metadata pages of the file system. During this analysis, the user daemon forks threads to perform consistency checking of the metadata and/or metadata pages. In one example, the file system user daemon analysis module 110 analyzes file system metadata of the file system while the file system is online (i.e., after the file system is mounted) to identify any corrupt file system metadata. A variety of consistency checks and analyses of file system metadata and/or associated metadata pages can be performed.
  • the operating system kernel communicates with the file system user daemon analysis module 110 (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects.
  • the file system operation may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
  • any inconsistencies found in the page header of a metadata page may be fixed. If page header inconsistencies are corrected at this stage, a free mcell list within the corrected pages is maintained and the links to the respective pages are also corrected. After verifying and correcting these free mcells, the corresponding mcells are cleaned on the BMT bitmap, which may be maintained per volume.
  • the file system user daemon analysis module 110 maintains a page header state of verified and fixed on an auxiliary data structure, which may be maintained per volume. This state change is protected through a page-bit-lock so that a parallel on-demand analysis thread context does not perform any corrections on the same page for header verification purposes.
  • the file system user daemon analysis module 110 may also log the metadata identified as corrupt file system metadata in a corruption log, such as a tag corruption log or a mcell corruption log.
  • a corruption log such as a tag corruption log or a mcell corruption log.
  • the logging functionality may be performed by other suitable modules, as described more fully below. Corruptions relating to tags are logged in the tag corruption log while corruptions relating to mcells are logged in the mcell corruption log.
  • the file system user daemon analysis module 110 may check the tag entry for the mcell to determine whether the tag entry also points to the particular mcell. If not, the tag is identified as corrupt and is logged in the tag corrupt list. If the tag on a particular mcell in not valid, then the invalid mcell is logged in the mcell corruption list. Further, if the link from the particular mcell to the corresponding tag and back from the tag to the mcell is valid, the tag is added to an in-process verification list. Once this tag is added to the list, other parallel threads (on-demand threads) may be prevented from acting on the same tag.
  • the records of the mcell ⁇ i.e., file statistics, extent maps of a file, a file block map table, etc.) are then verified, and if the records are not consistent, the mcell is logged in the mcell corruption list as being corrupt.
  • the auxiliary mcell bitmap that is based on the state of each of the primary mcells is updated, as are the corresponding mcells under each of the primary mcells. Once a primary mcell and its chain is verified, the corresponding tag may be marked as valid in the tag bitmap, and the tag may be removed from the in-process verification list.
  • the file system user daemon analysis module 110 may also detect orphan tags by analyzing the tag directory for each mountable entity (i.e., file system, file set, etc.) and checking to determine whether each tag is verified by analyzing the tag bitmap. If any tag is determined to be an orphan (that is, not associated with a particular mcell), it is added to the tag corruption list. Similarly, the file system user daemon analysis module 110 may also detect orphan mcells.
  • the file system user daemon analysis module 110 may handle any active file system operations that arrive during the analysis process (i.e., an on-demand analysis). For example, if a file system operation arrives during the analysis process, the file system user daemon analysis module 110 checks whether the relevant metadata has already been verified using a verification status bit, which signifies whether the metadata has been verified. If verified, normal file system operation may continue. However, if the verification status bit indicates that the metadata has not been verified yet, the metadata is verified first in the context of the active file system thread and then the file system operation continues. In this way, the normal analysis flow is interrupted to process the new file system operation. The analysis of the remainder of the file system will then continue. User actions are therefore accommodated upon request rather than being delayed.
  • a verification status bit which signifies whether the metadata has been verified. If verified, normal file system operation may continue. However, if the verification status bit indicates that the metadata has not been verified yet, the metadata is verified first in the context of the active file system thread and then the file system operation continues. In this way, the normal
  • the functionality of the file system user daemon analysis module 110 may be divided into separate modules.
  • the computing system may include a background file system analysis module that performs the functionality of the file system user daemon analysis module 110 as a background process. This enables users to continue to access the file system and its data while the analysis is being performed.
  • the computing system 100 may include an on-demand file system user daemon analysis module. When a user requests information that has not yet been verified by the background file system analysis module, the on-demand file system user daemon analysis module performs an immediate check on the requested metadata so that the user request (or data relating to the user request) may be made available to the user.
  • the functionality of these modules may include the functionality of the file system user daemon analysis module and/or additional functionality as discussed below regarding FIG. 2.
  • the computing system 100 also includes a result module 112, which is responsible for returning the results of the file system analysis performed by the file system user daemon analysis module 110 from the user daemon to the kernel of the operating system. This may occur, for example, through a remote procedure call, an input/output control command, or via another suitable communication between the user daemon and the kernel. Periodically the file result module 112 may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon.
  • the file system user daemon analysis module 110 completes the background checking on a volume of the file system
  • the file system user daemon informs the kernel, via the result module 112, of the completion through an input/output control command, and the volume is returned to active service class.
  • the computing system 100 include others additional modules.
  • the computing system 100 may include a correction module responsible for correcting any metadata that is identified as corrupt during the analysis performed by the file system user daemon analysis module 110.
  • the file system is temporarily placed in a hold state (or frozen).
  • the corrections are applied to the file system metadata.
  • the correction module may hold or freeze the online file system only once to apply necessary changes and corrections, rather than bringing it offline during the analysis phase as inconsistencies are detected.
  • the correction module freezes or holds the file system and performs a file system flush.
  • the correction module then corrects the tag corruption log entries and the mcell corruption log entries.
  • the correction module may also perform a correction of the storage bitmap that tracks allocated disk space.
  • the computing system 100 includes a pre-mount FS verification module to verify the FS metadata necessary for mounting the file system.
  • the pre-mount FS verification module analyzes the operating system metadata of the file system while the file system is offline (i.e., before the file system is mounted on the operating system of the computing system) to identify any corrupt file system metadata for mounting the file system.
  • the pre-mount FS verification module may also assign the volumes of the file system into appropriate service classes.
  • the pre-mount FS verification module may validate and fix per volume superblock, metadata describing the file system metadata, root directory metadata, and reserved bitfile metadata table (RBMT) pages.
  • the pre-mount FS verification module may also check and/or correct domain attributes, domain mutable attributes, and volume attributes. These corrections are then synchronized across different volumes of the file system.
  • the pre-mount FS verification module provides hooks into file system functions to ensure that metadata is checked before it is accessed by the operating system.
  • the pre-mount FS verification module 110 may also perform metadata verification for RBMT pages, including those RBMT pages that are accessed during the mounting process.
  • the pre-mount FS verification module may also assign certain file system volumes into a special service class that accepts read/write/update file requests.
  • a special volume designated as "FSCK volume” may be created ant put into the active service class where new creates occur.
  • the FSCK volume separates existing metadata that is yet to be checked from new metadata that is created during active file system operation. While the this example provides for a separate FSCK volume, other examples may use reserve space on each volume itself for online file system checking to be utilized by active file system operations such as creates/writes/updates, which might utilize storage space to be allocated while the online file system checking is in process.
  • FIG. 2 illustrates a flow diagram of a method 200 for file system analysis in a user daemon according to examples of the present disclosure.
  • the method 300 may be executed by a computing system or a computing device such as computing system 100 FIG. 1.
  • method 200 may include: validating file system metadata for a file system prior to mounting the file system on an operating system (block 202), analyzing in a user daemon of the operating system the file system to identify any inconsistencies (block 204); logging the identified corrupt file system metadata into a corruption list (block 206); and correcting the identified corrupt file system metadata while freezing the file system (block 208).
  • the method 200 includes validating file system metadata prior to mounting the file system.
  • a computing system e.g., computing system 100 of FIG. 1
  • a variety of consistency checks of the file system metadata and/or associated metadata pages can be performed. Any inconsistencies in the FS metadata may be corrected prior to mounting the file system and bringing it online. Moreover, during the pre-mount FS verification, volumes of the file system may also be assigned into appropriate service classes. The method 200 continues to block 204.
  • the method 200 may include analyzing in a user daemon of the operating system the file system to identify any inconsistencies.
  • a computing system e.g., computing system 100 of FIG. 1
  • the operating system kernel communicates with the file system user daemon (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects.
  • the file system may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
  • the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon.
  • the file system user daemon completes the background checking on a volume of the file system, the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.
  • the analysis may occur responsive to a user initiated file system operation, such as a request by a user to access certain file system data.
  • the analysis may also occur while the file system is online so that the file system remains accessible to users during the analysis process.
  • the method 200 continues to block 206.
  • the method 200 includes logging the identified corrupt file system metadata into a corruption list.
  • the computing system e.g., computing system 100 of FIG. 1
  • the corruption list for example, includes multiple corruption lists such as a tag corruption list (such as inode bit map) and a mcell corruption list (such as inode corruptions). In this case, corrupt tag metadata are logged in the tag corruption list, while corrupt mcell metadata are logged in the mcell corruption list.
  • the method 200 continues to block 208.
  • the method 200 includes correcting the identified corrupt file system metadata.
  • the computing system e.g., computing system 100 of FIG. 1
  • the file system may be frozen or placed in a state of temporary hold in an example, during which time corrections may be applied to the metadata as indicated in the corruption log lists.
  • correcting the identified corrupt file system metadata includes correcting the identified corrupt file system metadata logged into the tag corruption list and also correcting the identified corrupt file system metadata logged into the mcell corruption list.
  • FIG. 3 illustrates a flow diagram of a method 300 for file system analysis in a user daemon according to examples of the present disclosure.
  • the method 300 may be executed by a computing system or a computing device such as computing system 100 of FIG. 1.
  • the method 300 may include: transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system (block 302); performing a background analysis of metadata or volumes of a file system in a file system user daemon (block 304); and returning file system analysis result to the kernel (block 306).
  • the method 300 may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system.
  • a computing system e.g., computing system 100 of FIG. 1
  • transitioning the analysis from the kernel to the user daemon decreases the complexity, risk, and redundancy needed to execute the analysis at the kernel level.
  • the method 300 continues at block 304.
  • the method 300 may include performing a background analysis of disk volumes of a file system in a file system user daemon.
  • a computing system e.g., computing system 100 of FIG. 1
  • performs a background file system analysis using, for example, the file system user daemon analysis module 110 of FIG. 1 ) of a plurality of disk volumes of the file system in the file system user daemon of the operating system.
  • the operating system kernel communicates with the file system user daemon (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects.
  • the file system operation may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
  • analyzing the file system metadata includes performing, by the computing system, a background analysis of the file system metadata as described above while the file system is online in the user daemon (that is, after it is mounted). This includes checking the consistency of metadata and metadata pages of the file system using the user daemon. While the background analysis is being performed, if the computing system is interrupted, such as by a user action or request for data from the file system, an on-demand analysis of the file system metadata relating to the user action or request may be performed. In this way, the user action causes the computing system to suspend performing the background analysis on the metadata objects undergoing the on- demand analysis of the file system metadata relating to the user action.
  • the background analysis of the file system metadata resumes following the completion of the on-demand analysis of the file system metadata relating to the user action.
  • the background analysis can continue in parallel with the on- demand analysis being performed relating to a user action if the metadata objects being analyzed are different.
  • any corrupt page headers for bitfile metadata table (BMT) (traditionally inode table) pages associated with the file system metadata may be corrected during the background analysis.
  • BMT bitfile metadata table
  • the method 300 may include returning the file system analysis result to the kernel.
  • a computing system e.g., computing system 100 of FIG. 1
  • the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon.
  • the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.
  • the method 300 includes performing an on-demand file system analysis in the file system daemon of the operating system responsive to a user action, and returning a result of the on-demand file system analysis from the file system user daemon to the kernel of the operating system.
  • the method 300 includes validating file operating system metadata for a file system prior to mounting the file system on the operating system of a computing system.
  • the method 300 may include logging any returned results of the background file system analysis in one of a tag corruption log and a mcell corruption log, and correcting the identified errors in the file system while the file system is in a temporary hold. It should be understood that the processes depicted in FIG. 3 represent illustrations, and that other processes may be added or existing processes may be removed, modified, or rearranged without departing from the scope and spirit of the present disclosure.
  • FIG. 4 illustrates a flow diagram of a method 400 for file system analysis in a user daemon according to examples of the present disclosure.
  • the method 400 may be executed by a computing system or a computing device such as computing system 100 of FIG. 1.
  • method 400 may include: transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system (block 402); performing a background analysis of disk volumes of a file system in a file system user daemon (block 404); determining whether to perform an on-demand file system analysis (block 406); performing an on-demand file system analysis in the file system user daemon (block 408); returning the on-demand file system analysis results to the operating system kernel (block 410); continuing the background analysis of disk volumes of the file system (block 412); and returning the file system analysis result to the kernel (block 414).
  • the method may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system.
  • a computing system e.g., computing system 100 of FIG. 1
  • the method 400 continues at block 404.
  • the method 400 may include performing a background analysis of disk volumes of a file system in a file system user daemon.
  • a computing system e.g., computing system 100 of FIG. 1
  • performing the background analysis may include analyzing metadata of a file system.
  • the method may include determining whether to perform an on-demand file system analysis.
  • a computing system e.g., computing system 100 of FIG. 1 .
  • a positive determination may be made when a user or the file system requests data that is in a portion of the file system not yet analyzed during the background analysis.
  • the method 400 continues at block 408. However, if no such request is made, or if the requested data has already been analyzed, the method 400 continues to block 412.
  • the method may include performing an on-demand file system analysis in the file system user daemon.
  • a computing system e.g., computing system 100 of FIG. 1
  • an on-demand analysis of the file system metadata relating to the user action or request may be performed.
  • the user action causes the computing system to suspend performing the background analysis on the metadata objects undergoing the on-demand analysis of the file system metadata relating to the user action.
  • the background analysis of the file system metadata resumes following the completion of the on-demand analysis of the file system metadata relating to the user action.
  • the background analysis can continue in parallel with the on-demand analysis being performed relating to a user action if the metadata objects being analyzed are different (for example, if the data are in different volumes).
  • BMT bitfile metadata table
  • the method 400 continues at block 410, and the results of the on- demand file system analysis are returned to the kernel of the operating system, such as by the file system result module 112 of FIG. 1.
  • the method 400 may include returning the file system analysis result to the kernel.
  • a computing system e.g., computing system 100 of FIG. 1
  • the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon.
  • the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Examples of online file system analysis and correction are disclosed. In one example implementation according to aspects of the present disclosure, a method may be implemented on a computing system. The method may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system. The method may further include performing a background file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system. The method may then include returning a result of the background file system analysis from the file system user daemon to the kernel of the operating system.

Description

FILE SYSTEM ANALYSIS IN USER DAEMON
BACKGROUND
[0001] As computing devices such as laptops, smart phones, tablets, and other similar computing devices become more popular, increasing amounts of data is generated. These devices often rely on various remote and/or cloud based computing environments for access to data, applications, and other information, which may be stored on servers or other similar computing systems accessible via the Internet or another suitable network. As the amount of data has increased, so too have the demands for performance and system capabilities increased. Consequently, computing systems and their related file systems have grown in size and complexity.
BRIEF DESCRIPTION OF THE DRAWINGS
[0002] The following detailed description references the drawings, in which:
[0003] FIG. 1 illustrates a block diagram of a computing system for file system analysis in a user daemon according to examples of the present disclosure;
[0004] FIG. 2 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure;
[0005] FIG. 3 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure; and
[0006] FIG. 4 illustrates a flow diagram of a method for file system analysis in a user daemon according to examples of the present disclosure.
DETAILED DESCRIPTION
[0007] With the rapid growth of big data, cloud based computing, and user generated content, file systems running on enterprise and user computing systems have also increased in size. These file systems act as containers of data, and their sizes may be in the gigabytes, terabytes, and/or petabytes and may support millions or billions of file system objects. However, the increases in file system sizes, along with the corresponding increases in the number of file system objects burden system resources and negatively impact users' experiences when the file systems are checked for file system consistency.
[0008] A file system consistency check (also referred to as "checker," "file system checker," or "fsck") refers to a software program or instructions that checks the consistency of a file system on a computing system. In one example, the file system check analyzes a file system to discover and identify inconsistent states. The file system check may also correct or fix the identified inconsistencies, problems, errors, etc.
[0009] Typically, file systems are checked for errors and inconsistencies while the file system is in an offline state (that is, while the file system is inaccessible to users, applications, etc.). However, such offline file system checks depends on the number of file system objects that need to be examined. Depending on the size of the file system and the number of related objects, the file system check may take hours or days to execute, rendering the computing system unusable and/or unavailable. For example, in a typical offline file system check on a large file system such as a 32 terabyte with 50 million objects may take thirty or more hours to perform the checking. Similarly, a 32 terabyte file system with 100 million objects may take more than fifty hours to perform file system checking, for example. During this time, the file system remains offline and unavailable. Such large downtimes are typically unacceptable in enterprise storage environments, and the large downtimes make offline file system checking non-scalable as file system sizes and number of objects continues to grow.
[0010] Various file system consistency checking solutions exist. In one example, a file system may be taken offline in order to perform the file system check. However, this may take many hours to complete, rendering the file system otherwise unusable, as discussed above. Another solution provides for online file system checking by creating a snapshot of the file system and its associated metadata. The checking is then performed on the snapshot such that errors are identified and corrected. Once the checking is completed including fixing any errors in the snapshot of the file system, the fixes are then applied back to the original file system from the snapshot file system. Alternatively, a file system check may be performed on the snapshot with errors logged to a separate file. The file system may then be taken offline in order to apply any necessary fixes.
[0011] Various embodiments will be described below by referring to several examples of online file system (FS) analysis in a user daemon. In one example implementation according to aspects of the present disclosure, a method may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system. The method may further include performing a background file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system. The method may then include returning a result of the background file system analysis from the file system user daemon to the kernel of the operating system.
[0012] In some implementations, file system consistency checking occurs in user space (such as a user daemon of the operating system) while the file system is online and accepting normal user file system operations. In this way, excessive downtime due to offline file system checking is avoided. Moreover, the file system may be corrected and "updated in place" rather than having to transition the file system into an offline state. Additionally, performing the file system consistency checking in the user space (as opposed to a purely kernel- based approach) enables the mounted file system to actively satisfy file input/output (IO) operations. By performing the file system consistency checking in the user space, complex code may be avoided, and the file system consistency checking is isolated from active kernel operations with well-defined communication interfaces, instead of embedding the file system consistency checking within the kernel. These and other advantages will be apparent from the description that follows.
[0013] It should be understood that, throughout the disclosure, reference is made to a two tiered metadata scheme with two distinct types of metadata. Generally, metadata includes file tags (traditionally known as inode numbers in an example) which act as indirection mechanisms between directory entries and mcells. The mcells represent traditional inodes. The metadata provides versatility in that it facilitates dynamic volume addition, removal, and expansion. A first type of metadata is file system metadata relating to metadata utilized in mounting the file system. A second type of metadata is metadata relating to the operation of the file system after it is mounted and is analyzed or checked while the file system is mounted, online, and operational.
[0014] FIG. 1 illustrates a block diagram of a computing system 100 for file system (FS) analysis in a user daemon according to examples of the present disclosure. FIG. 1 includes particular components, modules, etc. according to various examples. However, in different embodiments, more, fewer, and/or other components, modules, arrangements of components/modules, etc. may be used according to the teachings described herein. In addition, various components, modules, etc. described herein may be implemented as one or more software modules, hardware modules, special-purpose hardware (e.g., application specific hardware, application specific integrated circuits (ASICs), embedded controllers, hardwired circuitry, etc.), or some combination of these.
[0015] It should be understood that the computing system 100 may include any appropriate type of computing device, including for example smartphones, tablets, desktops, laptops, workstations, servers, smart monitors, smart televisions, digital signage, scientific instruments, retail point of sale devices, video walls, imaging devices, peripherals, or the like, including any suitable combinations thereof.
[0016] The computing system 100 may include a processing resource 102 that represents generally any suitable type or form of processing unit or units capable of processing data or interpreting and executing instructions. The instructions may be stored on a non-transitory tangible computer-readable storage medium, such as a memory resource or on a separate device (not shown), or on any other type of volatile or non-volatile memory that stores instructions to cause a programmable processor to perform the techniques described herein. Alternatively or additionally, the computing system 100 may include dedicated hardware, such as one or more integrated circuits, Application Specific Integrated Circuits (ASICs), Application Specific Special Processors (ASSPs), Field Programmable Gate Arrays (FPGAs), or any combination of the foregoing examples of dedicated hardware, for performing the techniques described herein. In some implementations, multiple processors and/or processing resources may be used, as appropriate, along with multiple memories, memory resources, and/or types of memory.
[0017] In addition to the processing resource 102, the computing system 100 may include a file system user daemon analysis module 110 and a result module 112. In one example, the modules described herein may be a combination of hardware and programming. The programming may be processor executable instructions stored on a tangible memory resource such as a memory resource (not shown), and the hardware may include processing resource 102 for executing those instructions. Thus the memory resource can be said to store program instructions that when executed by the processing resource 102 implement the modules described herein. Other modules may also be utilized as will be discussed further below in other examples.
[0018] The file system user daemon analysis module 110 performs, using a user daemon, a file system analysis to check for inconsistencies within the metadata and/or metadata pages of the file system. During this analysis, the user daemon forks threads to perform consistency checking of the metadata and/or metadata pages. In one example, the file system user daemon analysis module 110 analyzes file system metadata of the file system while the file system is online (i.e., after the file system is mounted) to identify any corrupt file system metadata. A variety of consistency checks and analyses of file system metadata and/or associated metadata pages can be performed.
[0019] In an example, when an active file system operation is initiated by the file system user daemon analysis module 110 for an object whose metadata has not been checked, the operating system kernel communicates with the file system user daemon analysis module 110 (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects. The file system operation may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
[0020] In one example, during the analysis, no corrections to the metadata are made. However, in one example, any inconsistencies found in the page header of a metadata page may be fixed. If page header inconsistencies are corrected at this stage, a free mcell list within the corrected pages is maintained and the links to the respective pages are also corrected. After verifying and correcting these free mcells, the corresponding mcells are cleaned on the BMT bitmap, which may be maintained per volume.
[0021] In an example, the file system user daemon analysis module 110 maintains a page header state of verified and fixed on an auxiliary data structure, which may be maintained per volume. This state change is protected through a page-bit-lock so that a parallel on-demand analysis thread context does not perform any corrections on the same page for header verification purposes.
[0022] The file system user daemon analysis module 110 may also log the metadata identified as corrupt file system metadata in a corruption log, such as a tag corruption log or a mcell corruption log. In other examples, the logging functionality may be performed by other suitable modules, as described more fully below. Corruptions relating to tags are logged in the tag corruption log while corruptions relating to mcells are logged in the mcell corruption log.
[0023] For each mcell, the file system user daemon analysis module 110 may check the tag entry for the mcell to determine whether the tag entry also points to the particular mcell. If not, the tag is identified as corrupt and is logged in the tag corrupt list. If the tag on a particular mcell in not valid, then the invalid mcell is logged in the mcell corruption list. Further, if the link from the particular mcell to the corresponding tag and back from the tag to the mcell is valid, the tag is added to an in-process verification list. Once this tag is added to the list, other parallel threads (on-demand threads) may be prevented from acting on the same tag. The records of the mcell {i.e., file statistics, extent maps of a file, a file block map table, etc.) are then verified, and if the records are not consistent, the mcell is logged in the mcell corruption list as being corrupt.
[0024] After the tags and mcells are analyzed, the auxiliary mcell bitmap that is based on the state of each of the primary mcells is updated, as are the corresponding mcells under each of the primary mcells. Once a primary mcell and its chain is verified, the corresponding tag may be marked as valid in the tag bitmap, and the tag may be removed from the in-process verification list.
[0025] In one example, the file system user daemon analysis module 110 may also detect orphan tags by analyzing the tag directory for each mountable entity (i.e., file system, file set, etc.) and checking to determine whether each tag is verified by analyzing the tag bitmap. If any tag is determined to be an orphan (that is, not associated with a particular mcell), it is added to the tag corruption list. Similarly, the file system user daemon analysis module 110 may also detect orphan mcells.
[0026] In one example, the file system user daemon analysis module 110 may handle any active file system operations that arrive during the analysis process (i.e., an on-demand analysis). For example, if a file system operation arrives during the analysis process, the file system user daemon analysis module 110 checks whether the relevant metadata has already been verified using a verification status bit, which signifies whether the metadata has been verified. If verified, normal file system operation may continue. However, if the verification status bit indicates that the metadata has not been verified yet, the metadata is verified first in the context of the active file system thread and then the file system operation continues. In this way, the normal analysis flow is interrupted to process the new file system operation. The analysis of the remainder of the file system will then continue. User actions are therefore accommodated upon request rather than being delayed.
[0027] In one example, the functionality of the file system user daemon analysis module 110 may be divided into separate modules. For example, the computing system may include a background file system analysis module that performs the functionality of the file system user daemon analysis module 110 as a background process. This enables users to continue to access the file system and its data while the analysis is being performed. Additionally, the computing system 100 may include an on-demand file system user daemon analysis module. When a user requests information that has not yet been verified by the background file system analysis module, the on-demand file system user daemon analysis module performs an immediate check on the requested metadata so that the user request (or data relating to the user request) may be made available to the user. The functionality of these modules may include the functionality of the file system user daemon analysis module and/or additional functionality as discussed below regarding FIG. 2.
[0028] The computing system 100 also includes a result module 112, which is responsible for returning the results of the file system analysis performed by the file system user daemon analysis module 110 from the user daemon to the kernel of the operating system. This may occur, for example, through a remote procedure call, an input/output control command, or via another suitable communication between the user daemon and the kernel. Periodically the file result module 112 may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon. Once the file system user daemon analysis module 110 completes the background checking on a volume of the file system, the file system user daemon informs the kernel, via the result module 112, of the completion through an input/output control command, and the volume is returned to active service class.
[0029] In other examples, the computing system 100 include others additional modules. For instance, the computing system 100 may include a correction module responsible for correcting any metadata that is identified as corrupt during the analysis performed by the file system user daemon analysis module 110. During the correction phase performed by the correction module, the file system is temporarily placed in a hold state (or frozen). During this brief time, the corrections are applied to the file system metadata. By applying all of the corrections after the file system user daemon analysis module 110 has completed the analysis of the file system, the correction module may hold or freeze the online file system only once to apply necessary changes and corrections, rather than bringing it offline during the analysis phase as inconsistencies are detected.
[0030] During the correction phase, the correction module freezes or holds the file system and performs a file system flush. The correction module then corrects the tag corruption log entries and the mcell corruption log entries. The correction module may also perform a correction of the storage bitmap that tracks allocated disk space.
[0031] In another example, the computing system 100 includes a pre-mount FS verification module to verify the FS metadata necessary for mounting the file system. For example, the pre-mount FS verification module analyzes the operating system metadata of the file system while the file system is offline (i.e., before the file system is mounted on the operating system of the computing system) to identify any corrupt file system metadata for mounting the file system. The pre-mount FS verification module may also assign the volumes of the file system into appropriate service classes.
[0032] A variety of consistency checks of the FS metadata and/or associated metadata pages can be performed on the metadata for mounting the file system. For example, the pre-mount FS verification module may validate and fix per volume superblock, metadata describing the file system metadata, root directory metadata, and reserved bitfile metadata table (RBMT) pages. The pre-mount FS verification module may also check and/or correct domain attributes, domain mutable attributes, and volume attributes. These corrections are then synchronized across different volumes of the file system. In one example, the pre-mount FS verification module provides hooks into file system functions to ensure that metadata is checked before it is accessed by the operating system. The pre-mount FS verification module 110 may also perform metadata verification for RBMT pages, including those RBMT pages that are accessed during the mounting process.
[0033] The pre-mount FS verification module may also assign certain file system volumes into a special service class that accepts read/write/update file requests. In this way, a special volume designated as "FSCK volume" may be created ant put into the active service class where new creates occur. In this case, the FSCK volume separates existing metadata that is yet to be checked from new metadata that is created during active file system operation. While the this example provides for a separate FSCK volume, other examples may use reserve space on each volume itself for online file system checking to be utilized by active file system operations such as creates/writes/updates, which might utilize storage space to be allocated while the online file system checking is in process.
[0034] FIG. 2 illustrates a flow diagram of a method 200 for file system analysis in a user daemon according to examples of the present disclosure. The method 300 may be executed by a computing system or a computing device such as computing system 100 FIG. 1.
[0035] In one example, method 200 may include: validating file system metadata for a file system prior to mounting the file system on an operating system (block 202), analyzing in a user daemon of the operating system the file system to identify any inconsistencies (block 204); logging the identified corrupt file system metadata into a corruption list (block 206); and correcting the identified corrupt file system metadata while freezing the file system (block 208).
[0036] At block 202, the method 200 includes validating file system metadata prior to mounting the file system. For example, a computing system (e.g., computing system 100 of FIG. 1 ) validates file system metadata for a file system prior to mounting the file system on an operating system of a computing system.
[0037] A variety of consistency checks of the file system metadata and/or associated metadata pages can be performed. Any inconsistencies in the FS metadata may be corrected prior to mounting the file system and bringing it online. Moreover, during the pre-mount FS verification, volumes of the file system may also be assigned into appropriate service classes. The method 200 continues to block 204.
[0038] At block 204, the method 200 may include analyzing in a user daemon of the operating system the file system to identify any inconsistencies. In one example, a computing system (e.g., computing system 100 of FIG. 1 ) analyzes in a user daemon of the operating system the file system to identify any inconsistencies in the file system metadata.
[0039] For example, when an active file system operation is initiated (such as by the file system user daemon analysis module 110 of FIG. 1) for an object whose metadata has not been checked, the operating system kernel communicates with the file system user daemon (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects. The file system may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
[0040] Periodically the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon. Once the file system user daemon completes the background checking on a volume of the file system, the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.
[0041] In one example, the analysis may occur responsive to a user initiated file system operation, such as a request by a user to access certain file system data. The analysis may also occur while the file system is online so that the file system remains accessible to users during the analysis process. The method 200 continues to block 206. [0042] At block 206, the method 200 includes logging the identified corrupt file system metadata into a corruption list. For example, the computing system (e.g., computing system 100 of FIG. 1 ) logs the identified corrupt file system metadata into a corruption list. The corruption list, for example, includes multiple corruption lists such as a tag corruption list (such as inode bit map) and a mcell corruption list (such as inode corruptions). In this case, corrupt tag metadata are logged in the tag corruption list, while corrupt mcell metadata are logged in the mcell corruption list. The method 200 continues to block 208.
[0043] At block 208, the method 200 includes correcting the identified corrupt file system metadata. For example, the computing system (e.g., computing system 100 of FIG. 1 ) corrects the identified corrupt file system metadata logged in the corruption list. During the correcting phase, the file system may be frozen or placed in a state of temporary hold in an example, during which time corrections may be applied to the metadata as indicated in the corruption log lists. In one example, correcting the identified corrupt file system metadata includes correcting the identified corrupt file system metadata logged into the tag corruption list and also correcting the identified corrupt file system metadata logged into the mcell corruption list.
[0044] Additional processes also may be included, and it should be understood that the processes depicted in FIG. 2 represent illustrations, and that other processes may be added or existing processes may be removed, modified, or rearranged without departing from the scope and spirit of the present disclosure.
[0045] FIG. 3 illustrates a flow diagram of a method 300 for file system analysis in a user daemon according to examples of the present disclosure. The method 300 may be executed by a computing system or a computing device such as computing system 100 of FIG. 1.
[0046] In one example, the method 300 may include: transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system (block 302); performing a background analysis of metadata or volumes of a file system in a file system user daemon (block 304); and returning file system analysis result to the kernel (block 306). [0047] At block 302, the method 300 may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system. For example, a computing system (e.g., computing system 100 of FIG. 1 ) transitions a file system analysis from a kernel of an operating system to a file system user daemon of the operating system. In essence, transitioning the analysis from the kernel to the user daemon decreases the complexity, risk, and redundancy needed to execute the analysis at the kernel level. The method 300 continues at block 304.
[0048] At block 304, the method 300 may include performing a background analysis of disk volumes of a file system in a file system user daemon. For example, a computing system (e.g., computing system 100 of FIG. 1 ) performs a background file system analysis (using, for example, the file system user daemon analysis module 110 of FIG. 1 ) of a plurality of disk volumes of the file system in the file system user daemon of the operating system.
[0049] For example, when an active file system operation is initiated (such as by the file system user daemon analysis module 110 of FIG. 1) for an object whose metadata has not been checked, the operating system kernel communicates with the file system user daemon (such as through a remote procedure call) to initiate the analysis and checking of the metadata, metadata pages, and/or associated metadata objects. The file system operation may halt or pause its operation until the remote procedure call is returned with validation results from the analysis of the metadata, metadata pages, and/or associated metadata objects. If the validation is successful, the file system operation proceeds. However, if the validation is not successful, an error is returned to the user, and the metadata, metadata pages, and/or associated metadata objects are quarantined until the inconsistency is corrected. Once corrected, the metadata, metadata pages, and/or associated metadata objects may be removed from the quarantine. In another example, rather than quarantining the metadata identified as inconsistent, the metadata may be logged in a consistency log to be corrected later by the kernel as discussed below.
[0050] In one example, analyzing the file system metadata includes performing, by the computing system, a background analysis of the file system metadata as described above while the file system is online in the user daemon (that is, after it is mounted). This includes checking the consistency of metadata and metadata pages of the file system using the user daemon. While the background analysis is being performed, if the computing system is interrupted, such as by a user action or request for data from the file system, an on-demand analysis of the file system metadata relating to the user action or request may be performed. In this way, the user action causes the computing system to suspend performing the background analysis on the metadata objects undergoing the on- demand analysis of the file system metadata relating to the user action. The background analysis of the file system metadata resumes following the completion of the on-demand analysis of the file system metadata relating to the user action. The background analysis can continue in parallel with the on- demand analysis being performed relating to a user action if the metadata objects being analyzed are different. In an example, any corrupt page headers for bitfile metadata table (BMT) (traditionally inode table) pages associated with the file system metadata may be corrected during the background analysis. The method 300 continues at block 306.
[0051] At block 306, the method 300 may include returning the file system analysis result to the kernel. For example, a computing system (e.g., computing system 100 of FIG. 1 ) returns a result of the background file system analysis from the file system user daemon to the kernel of the operating system (such as via the result module 112 of FIG. 1 ). Periodically the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon. Once the file system user daemon completes the background checking on a volume of the file system, the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.
[0052] Additional processes also may be included. For example, the method 300 includes performing an on-demand file system analysis in the file system daemon of the operating system responsive to a user action, and returning a result of the on-demand file system analysis from the file system user daemon to the kernel of the operating system. In another example, the method 300 includes validating file operating system metadata for a file system prior to mounting the file system on the operating system of a computing system. Further, the method 300 may include logging any returned results of the background file system analysis in one of a tag corruption log and a mcell corruption log, and correcting the identified errors in the file system while the file system is in a temporary hold. It should be understood that the processes depicted in FIG. 3 represent illustrations, and that other processes may be added or existing processes may be removed, modified, or rearranged without departing from the scope and spirit of the present disclosure.
[0053] FIG. 4 illustrates a flow diagram of a method 400 for file system analysis in a user daemon according to examples of the present disclosure. The method 400 may be executed by a computing system or a computing device such as computing system 100 of FIG. 1. In one example, method 400 may include: transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system (block 402); performing a background analysis of disk volumes of a file system in a file system user daemon (block 404); determining whether to perform an on-demand file system analysis (block 406); performing an on-demand file system analysis in the file system user daemon (block 408); returning the on-demand file system analysis results to the operating system kernel (block 410); continuing the background analysis of disk volumes of the file system (block 412); and returning the file system analysis result to the kernel (block 414).
[0054] At block 402, the method may include transitioning a file system analysis from a kernel of an operating system to a file system user daemon of the operating system. For example, a computing system (e.g., computing system 100 of FIG. 1 ) transitions a file system analysis from a kernel of an operating system to a file system user daemon of the operating system as described above. The method 400 continues at block 404.
[0055] At block 404, the method 400 may include performing a background analysis of disk volumes of a file system in a file system user daemon. For example, a computing system (e.g., computing system 100 of FIG. 1 ) performs a background file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system as described above. In an example, performing the background analysis may include analyzing metadata of a file system. The method 400 continues at block 406.
[0056] At block 406, the method may include determining whether to perform an on-demand file system analysis. For example, a computing system (e.g., computing system 100 of FIG. 1 ). A positive determination may be made when a user or the file system requests data that is in a portion of the file system not yet analyzed during the background analysis. In this case, the method 400 continues at block 408. However, if no such request is made, or if the requested data has already been analyzed, the method 400 continues to block 412.
[0057] At block 408, the method may include performing an on-demand file system analysis in the file system user daemon. For example, a computing system (e.g., computing system 100 of FIG. 1 ) performs an on-demand analysis of the file system metadata as described above while the file system is online in the user daemon (that is, after it is mounted). This includes checking the consistency of metadata and metadata pages of the file system using the user daemon responsive to a user action or user request or a request to access a portion of the file system not yet checked or analyzed.
[0058] While the background analysis is being performed, if the computing system is interrupted, such as by a user action or request for data from the file system, an on-demand analysis of the file system metadata relating to the user action or request may be performed. In this way, the user action causes the computing system to suspend performing the background analysis on the metadata objects undergoing the on-demand analysis of the file system metadata relating to the user action. The background analysis of the file system metadata resumes following the completion of the on-demand analysis of the file system metadata relating to the user action. The background analysis can continue in parallel with the on-demand analysis being performed relating to a user action if the metadata objects being analyzed are different (for example, if the data are in different volumes). In an example, any corrupt page headers for bitfile metadata table (BMT) (traditionally inode table) pages associated with the file system metadata may be corrected during the background analysis.
[0059] The method 400 continues at block 410, and the results of the on- demand file system analysis are returned to the kernel of the operating system, such as by the file system result module 112 of FIG. 1.
[0060] If it is determined at block 406 that no on-demand file system analysis is to be performed, or if the on-demand file system analysis is complete, the method 400 continues at block 412. In this case, the background analysis of the disk volumes of the file system resumes.
[0061] Once the disk volume or volumes being checked are complete, the method 400 may include returning the file system analysis result to the kernel. For example, a computing system (e.g., computing system 100 of FIG. 1 ) returns a result of the background file system analysis from the file system user daemon to the kernel of the operating system as described above such as via the file system result module 112 of FIG. 1 as described above. Periodically the file system user daemon may update the results of the analysis by making an input/output control command to the kernel and an object state database in the kernel records regarding which metadata have been validated and can be served responsive to file system or user requests without contacting the user daemon. Once the file system user daemon completes the background checking on a volume of the file system, the file system user daemon informs the kernel of the completion through an input/output control command, and the volume is returned to active service class.
[0062] Additional processes also may be included, and it should be understood that the processes depicted in FIG. 4 represent illustrations, and that other processes may be added or existing processes may be removed, modified, or rearranged without departing from the scope and spirit of the present disclosure.
[0063] It should be emphasized that the above-described examples are merely possible examples of implementations and set forth for a clear understanding of the present disclosure. Many variations and modifications may be made to the above-described examples without departing substantially from the spirit and principles of the present disclosure. Further, the scope of the present disclosure is intended to cover any and all appropriate combinations and sub-combinations of all elements, features, and aspects discussed above. All such appropriate modifications and variations are intended to be included within the scope of the present disclosure, and all possible claims to individual aspects or combinations of elements or steps are intended to be supported by the present disclosure.

Claims

WHAT IS CLAIMED IS: 1. A system comprising:
a processing resource;
a file system user daemon analysis module to transition a file system analysis from a kernel of an operating system to a file system user daemon of the operating system and to perform a file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system; and
a result module to return the results of the file system analysis from the file system user daemon to the kernel of the operating system.
2. The system of claim 1 , further comprising:
a pre-mount file system verification module, executable by the processing resource, to analyze file system metadata of the file system while the file system is offline to identify any corrupt operating file metadata and to correct the identified corrupt file system metadata while the file system is offline
3. The system of claim 1 , further comprising:
a logging module, executable by the processing resource, to log the identified corrupt file system metadata into a corruption list, wherein the corruption list includes a tag corruption list and a mcell corruption list.
4. The system of claim 1 , further comprising:
a correction module, executable by the processing resource, to correct the identified corrupt file system metadata while freezing the file system
5. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:
transition a file system analysis from a kernel of an operating system to a file system user daemon of the operating system;
perform a background file system analysis of a plurality of disk volumes of the file system in the file system user daemon of the operating system; and return a result of the background file system analysis from the file system user daemon to the kernel of the operating system.
6. The non-transitory computer-readable storage medium of claim 5, wherein the instructions further cause the processor to:
perform an on-demand file system analysis in the file system daemon of the operating system responsive to a user action; and
returning a result of the on-demand file system analysis from the file system user daemon to the kernel of the operating system.
7. The non-transitory computer-readable storage medium of claim 5, wherein the instructions further cause the processor to:
validate file system metadata for a file system prior to mounting the file system on the operating system of a computing system.
8. The non-transitory computer-readable storage medium of claim 5, wherein the instructions further cause the processor to:
log any returned results of the background file system analysis in one of a tag corruption log and a mcell corruption log; and
correct the identified errors in the file system while the file system is in a temporary hold.
9. The non-transitory computer-readable storage medium of claim 6, wherein the instructions further cause the processor to:
log any returned results of the on-demand file system analysis in one of a tag corruption log and a mcell corruption log; and
correct the identified errors in the file system while the file system is in a temporary hold.
10. A method comprising:
validating, by a computing system, file system metadata for a file system prior to mounting the file system on an operating system of a computing system; analyzing, by the computing system, in a user daemon of the operating system the file system to identify any inconsistencies within the file system;
logging, by the computing system, any identified inconsistencies in a corruption list; and
correcting, by the computing system, the identified inconsistencies in the file system while the file system is in a temporary hold.
11. The method of claim 10, wherein analyzing in the user daemon of the operating system the file system to identify any inconsistencies within the file system occurs responsive to a user initiated file system operation.
12. The method of claim 10, wherein analyzing in the user daemon of the operating system the file system to identify any inconsistencies within the file system occurs while the file system is online.
13. The method of claim 10, further comprising:
mounting, by the computing system, the file system on an operating system of the computing device to cause the file system to be online after validating the file system metadata.
14. The method of claim 10, wherein analyzing in the user daemon the file system further comprises:
performing, by the computing system, a background analysis of the file system; and
responsive to a user action, performing, by the computing system, an on- demand analysis of the file system relating to a user action, wherein the user action causes the computing system to suspend performing the background analysis while performing the on-demand analysis of the file system relating to the user action.
15. The method of claim 10, wherein identifying any inconsistencies within the file system includes identifying any inconsistencies within the metadata of the file system.
PCT/US2014/013987 2014-01-31 2014-01-31 File system analysis in user daemon WO2015116125A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/US2014/013987 WO2015116125A1 (en) 2014-01-31 2014-01-31 File system analysis in user daemon

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/US2014/013987 WO2015116125A1 (en) 2014-01-31 2014-01-31 File system analysis in user daemon

Publications (1)

Publication Number Publication Date
WO2015116125A1 true WO2015116125A1 (en) 2015-08-06

Family

ID=53757526

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/013987 WO2015116125A1 (en) 2014-01-31 2014-01-31 File system analysis in user daemon

Country Status (1)

Country Link
WO (1) WO2015116125A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10467205B2 (en) 2016-03-21 2019-11-05 International Business Machines Corporation Online file system check using file system clone
CN110532032A (en) * 2019-07-31 2019-12-03 华为技术有限公司 A kind of booting file system detection method and relevant device
US10712957B2 (en) 2018-09-21 2020-07-14 International Business Machines Corporation Disk storage capacity reorganization
US11269746B1 (en) * 2021-01-22 2022-03-08 EMC IP Holding Company LLC Recovery of page description blocks based on context
US20220138151A1 (en) * 2020-11-04 2022-05-05 Netapp Inc. Sibling object generation for storing results of operations performed upon base objects
US11755590B2 (en) 2020-11-04 2023-09-12 Netapp, Inc. Data connector component for implementing integrity checking, anomaly detection, and file system metadata analysis

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030182389A1 (en) * 2002-03-22 2003-09-25 Edwards John K. System and method for performing an on-line check of a file system
US20050246612A1 (en) * 2004-04-30 2005-11-03 Microsoft Corporation Real-time file system repairs
US20060282471A1 (en) * 2005-06-13 2006-12-14 Mark Timothy W Error checking file system metadata while the file system remains available
WO2010050944A1 (en) * 2008-10-30 2010-05-06 Hewlett-Packard Development Company, L.P. Online checking of data structures of a file system
US20120095971A1 (en) * 2010-10-19 2012-04-19 Symantec Corporation Online file system consistency check

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030182389A1 (en) * 2002-03-22 2003-09-25 Edwards John K. System and method for performing an on-line check of a file system
US20050246612A1 (en) * 2004-04-30 2005-11-03 Microsoft Corporation Real-time file system repairs
US20060282471A1 (en) * 2005-06-13 2006-12-14 Mark Timothy W Error checking file system metadata while the file system remains available
WO2010050944A1 (en) * 2008-10-30 2010-05-06 Hewlett-Packard Development Company, L.P. Online checking of data structures of a file system
US20120095971A1 (en) * 2010-10-19 2012-04-19 Symantec Corporation Online file system consistency check

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10467205B2 (en) 2016-03-21 2019-11-05 International Business Machines Corporation Online file system check using file system clone
US11210273B2 (en) 2016-03-21 2021-12-28 International Business Machines Corporation Online file system check using file system clone
US10712957B2 (en) 2018-09-21 2020-07-14 International Business Machines Corporation Disk storage capacity reorganization
CN110532032A (en) * 2019-07-31 2019-12-03 华为技术有限公司 A kind of booting file system detection method and relevant device
US20220138151A1 (en) * 2020-11-04 2022-05-05 Netapp Inc. Sibling object generation for storing results of operations performed upon base objects
US11755590B2 (en) 2020-11-04 2023-09-12 Netapp, Inc. Data connector component for implementing integrity checking, anomaly detection, and file system metadata analysis
US11269746B1 (en) * 2021-01-22 2022-03-08 EMC IP Holding Company LLC Recovery of page description blocks based on context

Similar Documents

Publication Publication Date Title
EP3125120B1 (en) System and method for consistency verification of replicated data in a recovery system
US8356148B2 (en) Snapshot metadata management in a storage system
US9639429B2 (en) Creating validated database snapshots for provisioning virtual databases
US9892005B2 (en) System and method for object-based continuous data protection
US9367598B2 (en) Merging an out of synchronization indicator and a change recording indicator in response to a failure in consistency group formation
US8577855B2 (en) Online file system consistency check
WO2015116125A1 (en) File system analysis in user daemon
US20080222152A1 (en) System and Method for Performing File System Checks on an Active File System
US10678653B2 (en) Recovery of in-memory state in a log-structured filesystem using fuzzy checkpoints
US10007548B2 (en) Transaction system
US9519545B2 (en) Storage drive remediation in a raid system
US20170371916A1 (en) Database management device, database management method, and storage medium
US10452644B2 (en) Computer system, method for verifying data, and computer
US20170212902A1 (en) Partially sorted log archive
WO2015015339A1 (en) A method for a logging process in a data storage system
EP3264254B1 (en) System and method for a simulation of a block storage system on an object storage system
US9898468B2 (en) Single pass file system repair with copy on write
KR102437777B1 (en) Methods and systems to detect silent corruptionof data
US10599530B2 (en) Method and apparatus for recovering in-memory data processing system
WO2015116023A1 (en) Online file system metadata analysis and correction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14880919

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14880919

Country of ref document: EP

Kind code of ref document: A1